Files
promptkit/docs/roadmap/implementation.md

1289 lines
76 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Computational Efficiency Implementation Plan
Implement the remaining local work in the [performance roadmap](performance.md)
in order. Each sequential stage is scoped for one Luna coding-agent prompt.
Update status and evidence as each work unit lands.
Stages 1–14 are complete. The completed
inspection plan is preserved at
`481e2d4:docs/roadmap/implementation.md` in Git history.
## Baseline And Scope
- Planning baseline: `481e2d494c5a4e8b7f3a2a6f1cceaeb3f7c71d05`.
- Roadmap item 1 is complete. ConfigKit received the performance request in its
`docs/roadmap/promptkit-performance-request.md`, commit `d53021e`. Do not
resubmit it, implement upstream changes, wait for a release, or upgrade the
dependency as part of this plan. A later ordinary dependency update will
pick up available upstream improvements.
- Roadmap items 2–5 are localized runtime improvements. Item 7 concerns
identity/metadata inspection; item 6 is conditional on ownership evidence
and measured benefit. The latter condition is part of the accepted scope,
not permission to leave the review unfinished.
- No public declarations, configuration semantics, dependencies, release
tooling, or version tags need changing. This is an internal optimization
effort, not a new consumer performance contract.
## Working Rules
Start with [development](../development.md) and select applicable policies.
Read the [agent](../policy/agents.md),
[architecture](../policy/architecture.md),
[testing](../policy/testing.md), and
[documentation](../policy/documentation.md) policies; reuse unchanged policies
already read. Consult the [component overview](../internal/overview.md) and the
focused documents named below.
Preserve the [configuration handoff ADR](../adr/0001-configuration-boundary.md).
Use captured final configuration; operator provenance stays out of ordinary
execution. Keep request/result ownership, error identities, exact selection,
prepared-handle lifecycle, cancellation, and generation/repair behavior intact.
Retain indexed lookup, captured parses, per-render text reuse, operation-local
schema reuse, frozen transport retries, and constant-time queue removal.
Use standard-library mechanisms. Add no global cache, writer pool, borrowed
public result, new execution/input limit, or performance release gate. These
are scope boundaries, not new permanent product prohibitions. Internal helper
names and allocation choices below are implementation decisions, not public
invariants. Tests must not freeze a particular capacity, clone count, internal
call sequence, or exact allocation/timing number.
For each stage, inspect the current tree, preserve unrelated work, implement
only that stage, format changed Go files, run its focused checks, review the
diff, and commit the completed unit with a plain-English message. Update owning
internal documentation when its mechanics change. Do not widen scope to
unrelated cleanup or retire other roadmaps. Publication is a separate task.
## Measurement And Evidence Method
Add maintained benchmarks before changing the behavior they measure. Record
the benchmark-only baseline commit so the same fixture can be rerun later.
Keep setup, source acquisition, template compilation, fixture creation, and
engine construction outside timing unless the benchmark explicitly measures
that operation. Use `ReportAllocs`, retain a result in a sink where necessary,
and check representative output outside timing. Run benchmark comparisons
without the race detector.
The default comparison command is:
```sh
GOTOOLCHAIN=local GOWORK=off GOPROXY=off \
go test -mod=readonly -run '^$' -bench 'BENCHMARK_PATTERN' \
-benchmem -benchtime=500ms -count=5 PACKAGE
```
Replace the pattern and package with each stage's values. Dependencies and the
selected Go toolchain must already be installed for this offline invocation.
Use the same environment and unchanged fixture for before/after comparisons.
Record revisions, Go/module versions, processor, parallelism, command, repeated
results, and a short interpretation in this plan's execution evidence. Compare
ranges and medians; no benchmark-analysis dependency is required. Keep raw
profiles and generated binaries outside the repository.
For changes affecting buffer/string/handle lifetimes, also compare retained
heap with a documented, fixed synthetic workload: hold a small batch of
results or handles, force GC, compare live heap against an equivalent setup-only
baseline, and use `runtime.KeepAlive` for the retained objects. An opt-in
benchmark is sufficient; do not add heap/timing assertions to ordinary tests.
Distinguish immutable shared strings from mutable deep copies. Report noisy
retained-heap evidence honestly rather than deriving it from `B/op`.
Probes from the original review are preliminary evidence. Maintained fixtures
and recorded comparisons own the implementation evidence. Retain improvements
when measurements support the intended reduction without a material correctness,
retention, or maintenance trade-off; the original percentages are not targets.
## Stage 1 — Establish Validation And Captured-Rendering Benchmarks
**Status: Complete (2026-10-09).** Maintained basic-validation, compiled
rendering, and retained-output benchmarks are committed. The benchmark-only
baseline is `0eb009ad5a3248e2a1e12cad9b3a05906d6b0850`; no production behavior
changed.
**Outcome:** Reproducible baselines for roadmap items 2 and 3; no production
change in this stage.
**Read:** `internal/validate/standard_validator.go`, its tests,
`internal/prompt/go_renderer.go`, `template.go`, renderer tests, and
[internal sources](../internal/sources.md).
**Work:**
1. Add `BenchmarkBasicValidation` with short text, 1 MiB nonempty text,
whitespace-only text, and Unicode whitespace. Construct bodies/contracts
outside timing and exercise the real basic validator.
2. Add `BenchmarkCompiledRendering` using `CompileTemplate` and a small
`TemplateLookup` fixture built before timing. Exercise
`NewCompiledRenderer`, not only `NewGoRenderer`'s reparsing fallback.
3. Cover a small message, an artifact-backed message around 256 KiB, and
repeated references across session/messages. Keep session output within the
existing session contract, e.g. render an input length rather than its body.
Retain the existing fallback benchmark where useful; label the paths clearly.
4. Add opt-in `BenchmarkRetainedRenderedOutput` using the evidence method
above. Capture baselines and record the benchmark commit for later comparison.
**Verify:** `go test ./internal/validate ./internal/prompt`; compare the two new
benchmark families with the standard command. Benchmarks have no provider
calls, allocation assertions, or source parsing hidden in the compiled path.
## Stage 2 — Check Basic Output Emptiness Without A String Copy
**Status: Complete (2026-10-09).** Basic validation now checks the response
bytes directly. Behavioral coverage includes empty bytes, ASCII and Unicode
whitespace, ordinary text, and arbitrary non-UTF-8 bytes. The comparison with
the Stage 1 benchmark fixture is recorded below.
**Outcome:** Basic validation avoids a response-sized temporary string.
**Work:**
1. Replace `strings.TrimSpace(string(artifact.Body)) == ""` in the basic branch
with `len(bytes.TrimSpace(artifact.Body)) == 0`. Remove imports only if unused.
2. Preserve statuses, `IsValid`, repair input, error handling, and the existing
before/after cancellation checks. Do not change the other validation modes.
3. Reuse existing behavioral tests. Add only missing meaningful cases for empty
bytes, ASCII/Unicode whitespace, and nonempty bytes, preserving behavior for
arbitrary bytes rather than adding a UTF-8 requirement.
4. Compare `BenchmarkBasicValidation` against Stage 1 and record the result.
**Verify:** `go test ./internal/validate ./internal/usecase`; run the benchmark
family. Existing empty-output repair behavior remains unchanged.
## Stage 3 — Render Messages And Sessions With Local String Builders
**Status: Complete (2026-10-09).** Message and session template execution now
uses per-call `strings.Builder` values. Exact rendering, cancellation, and
concurrent compiled-template behavior pass; benchmark evidence shows lower
allocation without a measurable retained-output increase.
**Outcome:** Template output becomes a string without a final buffer-to-string
copy.
**Work:**
1. Replace the local message `bytes.Buffer` in `Render` and the session buffer
in `executeSessionTemplate` with local `strings.Builder` values. Execute
templates into their addresses and obtain `String()` afterward.
2. Keep template cloning/binding, error checks, cache metadata, ordering, and
cancellation placement. Leave artifact-text conversion/cache unchanged.
Do not introduce eager growth based on all artifacts or shared writers.
3. Run existing exact-content, session, cancellation, and compiled-template
concurrency coverage. Add a behavioral case only if the changed path lacks
protection; do not test builder internals.
4. Compare Stage 1's compiled-rendering and retained-output measurements.
Check both allocated bytes and live output storage; explain any retention
trade-off before retaining the change. Update internal sources if needed.
**Verify:** `go test -race ./internal/prompt`; run compiled-rendering benchmarks
without `-race`. Different request inputs must remain independent.
## Stage 4 — Reserve A Useful Initial File-Read Capacity
**Status: Complete (2026-10-09).** File reads now use the opened regular
file's size only as a capped initial-capacity hint, continue reading to EOF,
and compact a completed body when a stale hint leaves disproportionate unused
capacity. The exact-size and overreported-size benchmark comparisons are
recorded below.
**Outcome:** Ordinary file reads avoid early geometric buffer growth without
trusting metadata as the amount of content to return.
**Read:** `internal/artifact/reader.go`, `reader_test.go`, and internal sources.
**Work:**
1. Add `BenchmarkArtifactFileRead` using real temporary regular files of small,
1 MiB, and larger representative sizes. Create files outside timing, warm
content-type lookup, then measure the real reader. Record a baseline before
production edits.
2. Reserve initial capacity from the opened file's size. Use zero for a
nonpositive hint and `min(size, 1 MiB)` for a positive hint before conversion
to `int`. The 1 MiB reservation ceiling is an internal allocation heuristic,
not a file/content limit; data continues to grow normally beyond it. This
avoids an enormous eager allocation from stale or unusually large metadata
and keeps integer conversion safe on supported architectures.
3. Keep the read-until-EOF loop and existing regular-file, error, hash, metadata,
closure, and cancellation behavior. Do not size the result from `Stat`, use
`ReadFull` to enforce its size, impose an input limit, or add background I/O.
4. Extend the existing opened-file seam to test an overreported and an
underreported size, including a very large reported hint with small actual
content. Assert returned bytes/size/hash and failures, not backing capacity.
Add opt-in `BenchmarkRetainedArtifactFileRead` for representative live-buffer
retention, including overreported size. Compare both allocation and retention
and update internal sources.
**Verify:** `go test ./internal/artifact`; run `BenchmarkArtifactFileRead`.
Existing progress-cancellation and nonregular-file tests remain authoritative.
## Stage 5 — Establish Encoding Benchmarks And Error-Boundary Coverage
**Status: Complete (2026-10-09).** `PrepareGeneration` now has an offline
benchmark baseline for no extras, small extras, a 1 MiB string extra, nested
JSON extras, and structured output. Tests protect nested invalid-extra context
and request-error classification when the schema cannot be encoded. No
production behavior changed.
**Outcome:** A repeatable request-preparation baseline and focused protection
for the classifications that Stage 6 must preserve. No production change.
**Read:** `internal/llm/openai_compatible_client.go`, client/prepared-generation
tests, [integration contract](../integrations/openai-compatible-chat.md), and
[internal model client](../internal/llm.md).
**Work:**
1. Add `BenchmarkPrepareGeneration`, measuring actual `PrepareGeneration`
without HTTP dispatch: no extras, small extras, a 1 MiB string extra, and
nested JSON-compatible extras. Build inputs and the client outside timing;
use a direct synthetic credential or no credential so environment changes
do not affect results. Include a representative structured-output request.
2. Extend existing invalid-extra tests only as needed for a nested unsupported
value or cycle, checking `ErrInvalidRequest`, useful key context, and zero
transport calls. Do not freeze full diagnostic wording or all invalid-key
combinations.
3. Cover a serializable-extra request whose schema cannot be encoded: the
client currently classifies full-request encoding failure as
`ErrRequestFailed`. Also cover invalid extras plus an unencodable schema:
invalid extras retain `ErrInvalidRequest`. This protects classification
when validation moves to a failure-only path.
4. Capture the baseline. Existing body omission, explicit-zero, cache-control,
structured-output, and frozen-attempt tests own their established behavior.
**Verify:** `go test ./internal/llm`; run `BenchmarkPrepareGeneration`.
## Stage 6 — Encode Valid Extra Parameters Only In The Complete Request
**Status: Complete (2026-10-09).** Successful request preparation now encodes
the assembled payload once. Extra-parameter classification remains covered on
the failure path, including precedence when the schema also cannot be encoded.
The Stage 5 benchmark comparison is recorded below.
**Outcome:** Successful request preparation does not serialize extras twice.
**Work:**
1. Keep `openAIChatRequestPayload` responsible for body assembly and empty/
reserved-key checks. Remove its successful-path per-value `json.Marshal`
calls; place accepted values directly into the payload map.
2. Marshal the complete assembled map once in `PrepareGeneration`. On success,
continue authentication resolution and construction of the frozen payload
exactly as before. Do not change body shapes or omission/presence semantics.
3. On complete-request marshal failure only, examine the extras with the
existing per-value marshal check to identify an invalid extra. Return the
existing `ErrInvalidRequest` classification and `extra_params.<key>` context
for that failure. If extras are serializable, retain `ErrRequestFailed` for
the original full-request encoding failure. Cold-path repeated work is
acceptable; successful requests get the optimization. No new reflective
validator or stricter accepted-value rules are needed.
4. Adjust any helper-only tests whose incidental assembly boundary changed,
retaining protection at `PrepareGeneration`/`Generate`. Preserve frozen
request bytes across attempts and record Stage 5 comparisons. Update internal
model-client mechanics; change integration prose only enough to describe
validation before dispatch without prescribing two encodings.
**Verify:** `go test ./internal/llm ./internal/generation`; run encoding
benchmarks. Invalid requests still dispatch nothing, and retry tests still
exercise one frozen body per generation.
## Stage 7 — Establish Inspection Benchmarks
**Status: Complete (2026-10-09).** Captured configuration lookup and public
requirement-inspection benchmarks are committed. Repeated benchmark-only
baseline measurements are recorded below; no production behavior changed.
**Outcome:** Owned lookup and assembled requirement-inspection baselines, with
no production change. This separates fixture work from the selection refactor.
**Work:**
1. Add `BenchmarkInspectionSelection` in configuration tests using a captured
snapshot constructed outside timing. Measure existing prompt/profile lookups
with small and representative mutable nested-extra payloads.
2. Add root `BenchmarkRequirementInspection` using a real engine with a simple
prompt and the same small/large-extra distinction. Build the engine and
request outside timing, supply needed caller variables, and exercise only
the public inspection call. No artifacts, schemas, or providers are accessed.
3. Check selected identities and inventories outside timing. Record the
benchmark-only baseline commit and repeated measurements before routing
changes. Do not treat long immutable strings as deep-copy workload.
**Verify:** `go test ./internal/configuration .`; run the two new benchmark
families in their owning packages.
## Stage 8 — Add Shared Lightweight Configuration Lookups
**Status: Complete (2026-10-09).** Captured prompt and profile selection now
share their lookup rules, with owned metadata-only results available for
inspection. The Stage 7 benchmark comparison is recorded below; root
requirement-inspection routing remains unchanged.
**Outcome:** Captured configuration can provide identity/inspection metadata
without copying execution payloads. Existing runtime behavior stays unchanged
until Stages 9 and 10.
**Read:** `internal/configuration/prompts.go`, `targets.go`, their tests,
`internal/usecase/prompt_inspection.go`, `profile_inspection.go`, and public
inspection GoDoc.
**Work:**
1. Extract existing prompt ID/version selection into a private
`lookupPromptEntry` helper. Preserve literal keys, blank checks, omitted
version ambiguity, not-found mapping, and construction-time hashes. Use it
from existing owned-definition/inspection construction; do not duplicate
selection rules.
2. Add internal `GetPromptInspection(context.Context, string, string)
(*domain.PromptInspection, error)` returning owned cached inspection data
with copied inputs,
without copying template structures. Keep `GetPromptSnapshot` independently
owned, but stop copying an unused inspection-input slice just to obtain its
cached hash. Retain the private `prompt` helper where callers need both
values, composing it from shared lookup/copy helpers.
3. Extract a shared private indexed-profile lookup used by `GetProfile` and
new `GetProfileIdentity(ctx, id) (string, error)`. The new method returns the
selected lookup ID after successful lookup, with no `ExtraParams` copy or
readiness/credential validation. `GetProfile` retains its owned deep copy.
Use existing context/not-found handling and the captured target invariant.
4. Test selection parity, owned returned prompt inputs, missing IDs, and partial
profiles through configuration APIs. Extend Stage 7's configuration benchmark
with lightweight lookup cases so old/new paths can be compared with the same
payload. Root requirement-inspection routing remains unchanged in this stage.
**Verify:** `go test ./internal/configuration ./internal/usecase .`; run the two
benchmark families in their owning packages. No new public methods or ConfigKit
queries occur during lookup.
## Stage 9 — Use Profile Identity Lookup During Requirement Inspection
**Status: Complete (2026-10-09).** Requirement inspection now uses the
identity-only profile capability when available and keeps the full-profile
fallback for other internal repositories. Exact identity, error classification,
and partial-profile behavior are covered; benchmark comparison with the Stage 7
baseline is recorded below.
**Outcome:** Requirement inspection no longer deep-copies a target it does not
use.
**Work:**
1. Add a narrow optional repository capability in `internal/usecase` matching
`GetProfileIdentity`. Add `resolveProfileIdentity` beside the existing
profile selection helper, sharing basic ID/repository checks where useful.
2. Use the capability for the assembled configuration adapter. Wrap repository
failures with existing `ErrProfileLoad` and keep exact requested identities.
For internal repositories lacking the capability, fall back to
`resolveProfileSelection`, preserving its nil-profile/nil-target checks.
Do not change existing repository interfaces or public extension contracts.
3. Route only the identity requirement in `InspectRequirements` through this
helper. Keep default selection, partial-profile support, cancellation/error
order, direct-session handling, and analyzer behavior. `InspectProfile` and
preparation still need complete owned targets and keep their existing path.
4. Reuse existing selection tests, add focused capability/fallback parity
coverage, and compare `BenchmarkRequirementInspection` with Stage 7,
especially the mutable nested-extra case. Update internal runner mechanics.
**Verify:** `go test ./internal/configuration ./internal/usecase .`; run the
requirement-inspection benchmark. No model, credential, admission, schema, or
provenance work is introduced.
## Stage 10 — Select Prompt Inspection Metadata Without Execution-Definition Copies
**Status: Complete (2026-10-09).** Prompt and requirement inspections now use
captured metadata when available, preserving the definition-based fallback.
Exact snapshot selection, hash parity, session bypass, and caller-owned results
are covered; race verification passed. The repeated inspection benchmark
comparison is recorded below.
**Outcome:** Prompt and requirement inspection avoid copying template execution
structures they do not consume.
**Work:**
1. Add an optional `GetPromptInspection` repository capability in the runner.
Factor a shared inspection-selection helper from `InspectPrompt`: use the
captured metadata capability when available, otherwise derive the same
result from `resolvePromptDefinition` as today. Preserve guards, not-found/
ambiguity categories, and cached-hash versus fallback-hash behavior.
2. Let `InspectPrompt` use that helper. Let `InspectRequirements` select the
same prompt metadata, use its default profile when needed, and request the
exact requirement snapshot by selected ID/version.
3. The analyzer currently reads only `definition.ID`/`Version`; pass a local
identity-only `domain.PromptDefinition` constructed from that metadata rather
than loading a full execution definition. Do not change analysis semantics
or enlarge its scope. Keep snapshot metadata/template ownership as today.
4. Protect exact selectors, default selection, session bypass, caller-owned
results, and hash equality with preparation through existing tests. Cover
both optional-capability and fallback paths without pinning helper call
counts. Compare inspection benchmarks and update runner/source documentation.
**Verify:** `go test -race ./internal/configuration ./internal/usecase .`;
run inspection benchmarks without `-race`. Requirement inventories and public
prompt metadata remain identical for the same captured context.
## Stage 11 — Establish Assembled Workflow Benchmarks
**Status: Complete (2026-10-09).** The benchmark-only baseline is committed at
`743e78d`; it covers public preparation, prepared-execution creation, `Run`,
and repeated `Details()` with small, large-message, and mutable-payload
fixtures. Fixture checks and the repeated measurements are recorded below.
**Outcome:** Reproducible end-to-end local benchmarks, including the baseline
for conditional snapshot optimization. No production change.
**Read:** `internal/usecase/runner.go`, `prepared_execution.go`, root
`prepared_execution.go`, `convert.go`, client adapters, and
`prepared_execution_contract_test.go`. Locate the actual adapter file rather
than assuming an adapter filename.
**Work:**
1. Add root `BenchmarkEnginePreparation`, `BenchmarkEngineRun`, and
`BenchmarkPreparedExecutionDetails` using a real assembled engine and an
injected deterministic client. Use no live HTTP, environment credentials,
or source mutation in timing. Keep engine setup outside timing; discard
unused prepared handles so the benchmark does not retain every iteration.
2. Cover a small case and separate large-message and mutable-payload cases.
Use a few hundred optional schema properties and nested extra-map entries
within existing JSON-value limits, not merely long immutable strings.
Use the real schema validator for the assembled structured-output case.
Separate creation cost from repeated `Details()` cost.
3. Capture the benchmark-only baseline commit and repeated results. Verify
that the large-message case measures rendering/output storage while the
mutable-payload case exercises deep copying. Record the specific schemas,
input sizes, retained sinks, and handle lifetimes used by the fixtures.
**Verify:** `go test .`; run the three new families. Check representative
content/validation outside timing. Keep preparation and `Run` measurements
separate from engine construction and provider latency.
## Stage 12 — Establish Snapshot Ownership And Live-Heap Evidence
**Status: Complete (2026-10-09).** Internal snapshot and fixed-batch retained-
heap benchmarks are committed at `11ecb92`. The ownership inventory confirms
that message/cache metadata, hashes, and schema are preparation-owned, while
nested `ExtraParams` still need a deep copy before any transfer. Repeated
measurements and lifecycle assumptions are recorded below; no production
behavior changed.
**Outcome:** A measured ownership inventory for Stage 13's conditional change.
No production change; keep this analysis separate from creating the assembled
workflow benchmarks.
**Read:** The preparation, handle, conversion, and adapter files listed in
Stage 11, plus `internal/domain/message.go` and `internal/jsonvalue/jsonvalue.go`.
**Work:**
1. Add a focused internal `BenchmarkPreparedSnapshot` using existing fast
runner collaborators to expose snapshot cost without schema-compilation
noise. Add opt-in `BenchmarkRetainedPreparedHandles` with ready, completed,
and discarded subcases, holding a fixed batch under the common live-heap
evidence method. These measurements are not lifecycle assertions.
2. Record the ownership inventory and baseline in execution evidence: message
slices/cache pointers are copied by concatenation; hashes are newly allocated;
schema is copied from the validation plan; nested extras can still alias an
internal repository/request after shallow target merging; details must remain
separate from execution and survive completion/discard; public conversion
and injected-client adapters create caller-owned values. Confirm these facts
against current code, including fake/internal collaborators, before Stage 13.
3. Record readiness/run/discard lifetime assumptions and repeated observations.
Check whether the candidate below can save mutable schema/message metadata
copies without transferring an externally accessible map. Do not remove a
production copy during this analysis stage.
**Verify:** `go test ./internal/usecase .`; run the new families. Benchmarks must
check representative content/validation outside timing, avoid counting immutable
message length as deep-copied bytes, and document lifetime assumptions.
## Stage 13 — Conditionally Consolidate Prepared Creation's First Snapshot Copy
**Status: Complete (2026-10-09).** The selected transfer was retained. Prepared
execution now reuses preparation-owned messages, hashes, and schema, while
deep-copying nested provider extras and keeping an independent details
snapshot. Source-mutation and invalid-copy coverage pass; benchmark evidence
shows meaningful reductions for mutable payloads without a material increase
in retained heap.
**Outcome:** The ownership inventory supports transferring preparation-owned
values. The simplified creation path is retained with its nested-extra copy,
independent details snapshot, and existing cleanup and caller-ownership
boundaries.
**Selected candidate:** Transfer preparation's already-owned messages, hashes,
and schema into its private execution payload instead of deep-copying them
again in creation. Still detach nested extras and still make a separate,
complete details snapshot.
**Work:**
1. Use Stage 12's inventory to confirm that `operation.run`'s message/cache
structures, input-hash map, and copied schema have no mutable external owner.
Immutable string sharing is safe; no read-only promise should be imposed on
public or injected values to justify removing a copy.
2. If the inventory supports the candidate, replace the first general
`clonePreparedRun(operation.run)` with transfer of `operation.run`, after
deeply copying its `EffectiveModelParams.ExtraParams` using existing
`jsonvalue.CopyMap` and preserving the invalid-request error boundary.
The schema already passed through `jsonvalue.Copy` during preparation;
keep that validation. Leave the subsequent full clone for `details` intact.
3. Keep `Details()`'s defensive clone and facade conversion, claim/discard
synchronization, payload cleanup, observers, and generated-client copies.
Further conversion or borrowed-snapshot redesigns are not this stage's
candidate. Preserve public availability of details after execution and
discard, and the separation of mutable details from execution state.
4. Verify independence from retained internal source-profile, renderer, and
validation-plan data using existing tests plus missing meaningful nested-
mutation coverage. Compare Stages 11–12 creation and live-heap evidence. Retain
the candidate only if ownership remains correct and savings justify it
without a material retention/complexity trade-off. Otherwise restore the
candidate edit and record why the current design is retained; keep useful
benchmarks. Do not substitute a more elaborate refactor merely to claim a
reduction. Update runner mechanics if the change is retained.
**Verify:** `go test -race ./internal/usecase .`; run snapshot/assembled/retained
benchmarks without `-race`. Existing single-claim, discard races, frozen-source,
redaction, repair, and details-after-completion behavior must continue to pass.
## Stage 14 — Consolidate Evidence And Verify The Complete Local Scope
**Status: Complete (2026-10-09).** The complete local roadmap has been
implemented, ownership decisions and before/after evidence are recorded, and
the final suite, vet, example, formatting, and changed-document link checks
passed. No release or publication work was performed.
**Outcome:** The local roadmap scope is complete with clear measurements and
unchanged consumer contracts.
**Work:**
1. Record a concise before/after summary and revision/command references for
each changed path in execution evidence. Keep any Stage 13 retain decision
explicit. Include cumulative-allocation versus retained-heap distinctions
and small-input trade-offs; do not imply provider-latency improvements.
2. Review canonical internal source/runner/model-client descriptions against
the implemented flow. Consumer GoDoc and format semantics should remain
unchanged; remove incidental duplicated encoding mechanics where Stage 6
required it. Update the inventory only if responsibilities actually changed.
3. Verify roadmap coverage: item 1 remains complete without dependency work;
items 2–5 have implemented local improvements; item 7 uses lightweight
inspection; item 6 has the selected conditional outcome and evidence;
production compiled rendering and assembled offline workflows have maintained
benchmarks. Do not leave a requested item silently unexamined.
4. Run assembled verification below once on final code, review the complete
diff and tree for artifacts, and mark stages/roadmap progress accurately.
Keep this plan as the implementation record until ordinary roadmap
retirement; do not add release gates or publish a release.
**Verify:**
```sh
go test -race -vet=off -mod=readonly ./...
go vet -mod=readonly ./...
python3 scripts/check-examples.py
git diff --check
```
Check local Markdown links and changed-file formatting using existing tooling
or an appropriate focused check. Reuse successful checks for unchanged content
rather than rerunning merely to create the final commit.
## Execution Evidence
### Inspection Baseline
The benchmark-only fixture is committed at `04d5d695e0201a97c1070e1ff49ef4f0c9a7f5ba`
(`Add requirement inspection benchmarks`). The configuration benchmarks build
one captured snapshot per case before timing and measure the existing prompt
snapshot and owned-profile lookups. The public benchmark builds one real Engine
and both requests before timing, then calls only `InspectRequirements` with
the caller variable supplied. None of the paths reads artifacts or schemas or
contacts a provider. The small fixture has two provider-option entries; the
larger fixture has 64 nested provider maps, each with mutable label/route slices
and nested limit/route maps. Prompt lookup allocation is independent of profile
extra size because it does not copy the selected profile. The profile lookup
and assembled requirement inspection expose the mutable-map copy cost being
measured. Environment: Linux/amd64, Intel Core i5-7260U @ 2.20 GHz, benchmark
parallelism 4, Go 1.26.8, ConfigKit `v0.5.0`, and JSON Schema `v6.0.2`.
Command (five 500 ms samples per case, without the race detector):
```sh
GOTOOLCHAIN=local GOWORK=off GOPROXY=off GONOPROXY=none GOSUMDB=off \
go test -mod=readonly -run '^$' \
-bench 'BenchmarkInspectionSelection|BenchmarkRequirementInspection' \
-benchmem -benchtime=500ms -count=5 ./internal/configuration .
```
Times list all five `ns/op` samples in command order, followed by median and
range. Allocation figures are `B/op; allocs/op`.
| Configuration lookup | `ns/op` samples (median; range) | Allocations |
| --- | --- | --- |
| Prompt, small extras | 356.8, 354.2, 370.1, 352.8, 364.5 (356.8; 352.8–370.1) | 400; 4 |
| Prompt, nested extras | 423.2, 478.8, 456.2, 436.6, 460.4 (456.2; 423.2–478.8) | 400; 4 |
| Profile, small extras | 18785, 21077, 18952, 19701, 18682 (18952; 18682–21077) | 9538; 211 |
| Profile, nested extras | 624104, 635513, 600281, 600667, 620228 (620228; 600281–635513) | 261024–261028; 5979 |
| Public requirement inspection | `ns/op` samples (median; range) | Allocations |
| --- | --- | --- |
| Small extras | 23022, 25193, 24687, 24974, 23157 (24687; 23022–25193) | 11922; 240 |
| Nested extras | 635726, 642297, 591992, 605001, 590312 (605001; 590312–642297) | 263370–263372; 6008 |
The prompt snapshot lookup remained at 400 B and four allocations in both
fixtures, while owned profile lookup rose from about 9.5 KB/211 allocations to
about 261 KB/5979 allocations. Public requirement inspection rose from about
11.9 KB/240 allocations to about 263 KB/6008 allocations. These results
establish a baseline in which unrelated nested provider extras dominate the
large profile-selection and requirement-inspection cases; they are local
allocation measurements, not end-to-end provider-latency claims. Fixture
identity and prompt/requirement inventories were checked outside timing.
### Lightweight Inspection Lookups Comparison
The before measurements are the Stage 7 benchmark-only baseline at `04d5d69`;
the after measurements use the same fixtures at `b7ff371` (`Add lightweight
configuration lookups`). The command, environment, and five-sample procedure
are those in the Inspection Baseline subsection. Each after-case also checks
the selected identity or requirement/input inventory outside timing. The new
prompt-inspection cases return copied metadata without execution templates;
the profile-identity cases use the same two-entry and 64-entry nested-extra
snapshots as the full profile cases.
| Configuration path | Before `ns/op` (median; range) | Before allocations | After `ns/op` (median; range) | After allocations |
| --- | --- | --- | --- | --- |
| Prompt snapshot, small extras | 356.8, 354.2, 370.1, 352.8, 364.5 (356.8; 352.8–370.1) | 400; 4 | 292.6, 272.5, 332.9, 277.4, 302.3 (292.6; 272.5–332.9) | 336; 3 |
| Prompt snapshot, nested extras | 423.2, 478.8, 456.2, 436.6, 460.4 (456.2; 423.2–478.8) | 400; 4 | 359.0, 357.1, 388.8, 356.4, 365.7 (359.0; 356.4–388.8) | 336; 3 |
| Prompt inspection, small extras | — | — | 279.0, 194.3, 199.2, 187.8, 182.3 (194.3; 182.3–279.0) | 208; 2 |
| Prompt inspection, nested extras | — | — | 277.0, 236.8, 253.4, 222.5, 230.3 (236.8; 222.5–277.0) | 208; 2 |
| Full profile, small extras | 18785, 21077, 18952, 19701, 18682 (18952; 18682–21077) | 9538; 211 | 18887, 18555, 18517, 18761, 20126 (18761; 18517–20126) | 9538; 211 |
| Full profile, nested extras | 624104, 635513, 600281, 600667, 620228 (620228; 600281–635513) | 261024–261028; 5979 | 616937, 612152, 666875, 639239, 641841 (639239; 612152–666875) | 261024–261029; 5979 |
| Profile identity, small extras | — | — | 18.62, 18.62, 17.89, 18.57, 20.24 (18.62; 17.89–20.24) | 0; 0 |
| Profile identity, nested extras | — | — | 20.69, 19.83, 18.86, 18.65, 19.22 (19.22; 18.65–20.69) | 0; 0 |
The prompt snapshot now avoids the unused inspection-input copy, saving 64 B
and one allocation in these fixtures. The dedicated metadata lookup uses
208 B/two allocations for either payload size, and identity lookup uses no
allocation despite the nested provider data. Full owned-profile allocation
counts are unchanged and their before/after time ranges overlap.
The root benchmark still uses the existing inspection route; these small
changes are from its prompt snapshot dependency, not profile-identity routing:
| Public requirement inspection | Before `ns/op` (median; range) | Before allocations | After `ns/op` (median; range) | After allocations |
| --- | --- | --- | --- | --- |
| Small extras | 23022, 25193, 24687, 24974, 23157 (24687; 23022–25193) | 11922; 240 | 24320, 24715, 23919, 23911, 23885 (23919; 23885–24715) | 11858; 239 |
| Nested extras | 635726, 642297, 591992, 605001, 590312 (605001; 590312–642297) | 263370–263372; 6008 | 595187, 647166, 675430, 595950, 626551 (626551; 595187–675430) | 263304–263309; 6007 |
Root requirement-inspection allocation decreased by 64–66 B and one allocation;
the timing ranges overlap. The large profile extras still dominate until the
next routing change. No public method or ConfigKit query was added. Focused
verification passed: `go test ./internal/configuration ./internal/usecase .`;
both benchmark families were rerun without the race detector.
### Requirement-Inspection Identity Comparison
The before values use the Stage 7 benchmark-only baseline at `04d5d69`; after
values use the same root fixture at `6c5dec7` (`Inspect requirements by
profile identity`). The command and environment are those in the Inspection
Baseline subsection. The small case has two nested provider-option entries and
the larger case has 64; neither benchmark case accesses a provider, artifact,
or schema. Times list all five `ns/op` samples in command order, followed by
median and range.
| Requirement-inspection case | Stage 7 `ns/op` (median; range) | Stage 7 allocations | Identity lookup `ns/op` (median; range) | Identity lookup allocations |
| --- | --- | --- | --- | --- |
| Small extras | 23022, 25193, 24687, 24974, 23157 (24687; 23022–25193) | 11922; 240 | 2895, 2794, 2807, 3047, 2876 (2876; 2794–3047) | 2264; 26 |
| Nested extras | 635726, 642297, 591992, 605001, 590312 (605001; 590312–642297) | 263370–263372; 6008 | 2917, 2730, 2797, 2804, 2787 (2797; 2730–2917) | 2264; 26 |
The 64-entry case reduced cumulative allocation by about 261 KB and 5982
allocations per inspection; its median fell from about 605 microseconds to
2.8 microseconds. The small case fell from about 11.9 KB/240 allocations to
2.3 KB/26 allocations. These fixture-local results support using identity-only
selection for the information inspection consumes; they do not imply provider
latency improvements. The optional-capability and complete-profile fallback
paths preserve exact requested IDs, profile-load error wrapping, and
nil-profile/nil-target validation. Focused verification passed:
`go test ./internal/configuration ./internal/usecase .`.
### Prompt-Metadata Inspection Comparison
The before values use the Stage 9 implementation at `6c5dec7`; after values
use `c5f733a` (`Select prompt metadata for inspection`) with the same inspection
fixtures. The command and environment are those in the Inspection Baseline
subsection: Linux/amd64, Intel Core i5-7260U @ 2.20 GHz, benchmark parallelism
4, Go 1.26.8, ConfigKit `v0.5.0`, JSON Schema `v6.0.2`, and backend modules
`v3.20260930.0`. Five 500 ms samples were collected without the race detector.
Times list all five samples, followed by median and range; allocations are
`B/op; allocs/op`.
| Public requirement inspection | Stage 9 `ns/op` (median; range) | Stage 9 allocations | Metadata inspection `ns/op` (median; range) | Metadata inspection allocations |
| --- | --- | --- | --- | --- |
| Small extras | 2895, 2794, 2807, 3047, 2876 (2876; 2794–3047) | 2264; 26 | 2681, 2837, 2852, 2978, 2847 (2847; 2681–2978) | 2112; 24 |
| Nested extras | 2917, 2730, 2797, 2804, 2787 (2797; 2730–2917) | 2264; 26 | 2898, 3004, 2955, 4766, 4410 (3004; 2898–4766) | 2112; 24 |
The metadata route saves 152 cumulative bytes and two allocations per public
requirement inspection in both fixtures. Timing ranges overlap for the small
case; the larger after-run is noisy and does not show a reliable timing change.
The result supports the targeted copy reduction, not a provider-latency claim.
Configuration metadata inspection separately measured 208 B/two allocations
for both small and nested-extra fixtures, independent of their profile payload.
The benchmark command was:
```sh
GOTOOLCHAIN=local GOWORK=off GOPROXY=off GONOPROXY=none GOSUMDB=off \
go test -mod=readonly -run '^$' \
-bench 'BenchmarkInspectionSelection|BenchmarkRequirementInspection' \
-benchmem -benchtime=500ms -count=5 ./internal/configuration .
```
Verification passed on `c5f733a`:
`go test -race -mod=readonly ./internal/configuration ./internal/usecase .`.
### Assembled Workflow Benchmark Baseline
The benchmark-only fixture is committed at `743e78d` (`Add assembled workflow
benchmark baseline`). The same committed source produced the measurements
below. Environment: Linux/amd64, Intel Core i5-7260U @ 2.20 GHz, benchmark
parallelism 4, Go 1.26.8, ConfigKit `v0.5.0`, JSON Schema `v6.0.2`, and backend
modules `v3.20260930.0`.
Command (five 500 ms samples per case, without the race detector):
```sh
GOTOOLCHAIN=local GOWORK=off GOPROXY=off GONOPROXY=none GOSUMDB=off \
go test -mod=readonly -run '^$' \
-bench '^(BenchmarkEnginePreparation|BenchmarkEngineRun|BenchmarkPreparedExecutionDetails)$' \
-benchmem -benchtime=500ms -count=5 .
```
All cases use one assembled engine constructed before timing and a deterministic
injected client; there is no live HTTP or credential lookup. The small prompt
renders `Hello Ada`. The large-message prompt has a 256 KiB literal prefix and
renders that prefix plus a short interpolated suffix. The mutable-payload
prompt uses 64 nested provider-option entries, each with labels, limits, and
route maps/slices, plus a local JSON Schema with 256 optional string
properties and `additionalProperties: false`. Preflight confirms that the
injected `{}` response passes the real schema validator; the `Run` benchmark
also measures that validation. The schema, configuration, and request fixtures
are created before each timer starts.
`BenchmarkEnginePreparation` measures ordinary `Prepare` separately from
`PrepareExecution` creation. The prepared-execution case discards each unused
handle inside its timed iteration and retains only the most recent discarded
handle pointer. `BenchmarkPreparedExecutionDetails` creates and validates one
ready handle outside timing, measures repeated public `Details()` calls on it,
retains only the latest caller-owned result, then discards the handle. Ordinary
prepare and run sinks are overwritten per iteration and cleared after timing.
The large-message result shares immutable string content, so its allocation
figures are not treated as deep-copy bytes. No benchmark retains every
iteration or includes engine construction/provider latency.
Times list the five `ns/op` samples in command order, followed by median and
range. Allocation figures are `B/op; allocs/op` (ranges where they varied).
| Benchmark case | `ns/op` samples (median; range) | Allocations |
| --- | --- | --- |
| Prepare, small | 10935, 10121, 13018, 10288, 8485 (10288; 8485–13018) | 4513; 50 |
| Prepare, large message | 2064223, 1627980, 1588515, 1008441, 911123 (1588515; 911123–2064223) | 872964–872988; 50 |
| Prepare, mutable payload | 6430306, 6519806, 5970344, 5854621, 6456738 (6430306; 5854621–6519806) | 2914750–2914824; 41134–41135 |
| Prepared execution creation + discard, small | 8194, 8016, 8095, 8073, 7939 (8073; 7939–8194) | 5210; 56 |
| Prepared execution creation + discard, large message | 937857, 955672, 919810, 1058503, 1094114 (955672; 919810–1094114) | 873679–873681; 56 |
| Prepared execution creation + discard, mutable payload | 8114785, 8394320, 10387083, 8418604, 8528766 (8418604; 8114785–10387083) | 3711282–3711329; 60727 |
| Run, small | 12157, 11921, 12128, 13762, 12491 (12157; 11921–13762) | 6091; 66 |
| Run, large message | 1082064, 1058363, 1040185, 996377, 951818 (1040185; 951818–1082064) | 874562–874577; 66–67 |
| Run, mutable payload | 6072294, 6338179, 6386632, 7167500, 6805715 (6386632; 6072294–7167500) | 2993599–2993661; 41808–41809 |
| Details, small | 1104, 1039, 1010, 1045, 1058 (1045; 1010–1104) | 1056; 6 |
| Details, large message | 975.1, 2480, 4734, 3402, 1008 (2480; 975.1–4734) | 1056; 6 |
| Details, mutable payload | 2275096, 3281027, 1504724, 1365349, 1329800 (1504724; 1329800–3281027) | 670315–670339; 11548 |
The 256 KiB message does not make `Details()` allocate 256 KiB per call;
current public conversion shares immutable message strings while copying the
message slice and other metadata. By contrast, `Details()` over the mutable
payload costs about 670 KB and 11.5 thousand allocations per call, providing a
specific snapshot-copy baseline for later ownership analysis. Large-message
timings, especially `Details()`, are noisy and are retained as ranges rather
than point claims. Focused verification passed: `go test -mod=readonly .`;
representative rendered messages, prepared details, generated output, and
structured validation were checked outside benchmark timing.
### Prepared Snapshot Ownership And Live-Heap Baseline
The benchmark-only baseline is committed at `11ecb92` (`Add prepared snapshot
ownership benchmarks`). `BenchmarkPreparedSnapshot` exercises the internal
runner with the existing fake prompt/profile repositories, renderer, artifact
reader, a stateless injected client, and a fast prepared validator. It measures
`PrepareExecution` and discard without compiling a schema. Its small fixture
uses one rendered message, no provider extras, and a one-property schema. The
mutable fixture uses eight rendered messages plus one appended message, cache
metadata on each, one input/hash, 64 nested provider-option entries, and a
256-property schema. Fixture checks run outside timing; each measured handle is
discarded and only the latest discarded handle is retained as a sink.
`BenchmarkRetainedPreparedHandles` uses the mutable fixture and a fixed batch
of four handles. After fixture setup and checks, it preallocates an empty handle
slice, forces GC, and records a setup-only `HeapAlloc` baseline. It then retains
four ready, completed, or discarded handles, forces GC again, and reports the
heap delta. The retained benchmark command uses `-benchtime=1x`; the single
timed operation is followed by filling the remaining batch slots outside the
timer. Both samples call `runtime.KeepAlive` for the runner, request, and
retained batch. The metric is a live-heap difference, not a derivation from
`B/op`.
Environment: Linux/amd64, Intel Core i5-7260U @ 2.20 GHz, benchmark parallelism
4, Go 1.26.8, ConfigKit `v0.5.0`, JSON Schema `v6.0.2`, and backend modules
`v3.20260930.0`.
Snapshot command (five 500 ms samples, without the race detector):
```sh
GOTOOLCHAIN=local GOWORK=off GOPROXY=off GONOPROXY=none GOSUMDB=off \
go test -mod=readonly -run '^$' -bench '^BenchmarkPreparedSnapshot$' \
-benchmem -benchtime=500ms -count=5 ./internal/usecase
```
Times list the five `ns/op` samples followed by median and range; allocation
figures are `B/op; allocs/op` (ranges where they varied).
| Snapshot case | `ns/op` samples (median; range) | Allocations |
| --- | --- | --- |
| Small | 45776, 51524, 46933, 46834, 44663 (46834; 44663–51524) | 13014–13016; 216 |
| Mutable payload | 5343223, 5234718, 4213550, 4419520, 5046181 (5046181; 4213550–5343223) | 1211931–1212104; 25243–25244 |
Retained-heap command (five fixed-batch samples):
```sh
GOTOOLCHAIN=local GOWORK=off GOPROXY=off GONOPROXY=none GOSUMDB=off \
go test -mod=readonly -run '^$' -bench '^BenchmarkRetainedPreparedHandles$' \
-benchmem -benchtime=1x -count=5 ./internal/usecase
```
| Retained handle state | `retained-B` samples (median; range) | Timed-operation allocations |
| --- | --- | --- |
| Ready | 1458352, 1458256, 1459320, 1459048, 1459160 (1459048; 1458256–1459320) | 1211896–1213256 B; 25242–25251 |
| Completed | 731040, 730768, 729352, 729448, 729624 (729624; 729352–731040) | 1213024–1214368 B; 25255–25264 |
| Discarded | 728560, 730928, 730544, 730496, 730496 (730496; 728560–730928) | 1211896–1213256 B; 25242–25251 |
The ready four-handle batch retains about 1.46 MB above setup; completed and
discarded batches retain about 0.73 MB. The roughly 0.73 MB difference is
consistent with the extra execution snapshot held only while a handle is
ready. After claim/run or discard, `payload.clear` removes that execution
payload while the independent details snapshot remains available. Heap deltas
include runtime/GC noise; the five observed ranges are reported rather than
treated as an exact object-size guarantee.
The source ownership inventory is:
- `completePreparationWithStructuredOutput` builds a fresh input-hash map and
obtains rendered messages through `domain.ConcatRenderedMessages`, which
copies message slices and each `CacheControl` pointer while sharing immutable
string contents.
- `structuredOutputFromValidationPlan` copies the validation plan's schema
document with `jsonvalue.Copy` before storing it in the prepared run.
- `resolveExecutionTarget` and `mergeExecutionTargetOverride` use the
shallow-copy helper `copyExtraParams`. The configuration repository's full
`GetProfile` and the public request conversion deep-copy extra values, but
internal profile fakes return only shallow struct copies and internal
`RunRequest` override maps may be source-owned. Nested extras can therefore
remain aliased through the runner's outer-map copy. Stage 13 must keep a
`jsonvalue.CopyMap` boundary before transferring the prepared run.
- `PrepareExecution` deep-clones `operation.run` for execution, then clones
that snapshot again for details. `Details()` deep-clones details per call;
the root facade conversion also creates caller-owned maps, schema, messages,
and cache metadata. Claim completion and `Discard` clear only the private
payload, so the details copy survives both lifecycles.
- The injected-client adapter converts generated requests to caller-owned
prompt messages/cache metadata, execution-target extra maps, and structured
schema values. Its client cannot mutate the private prepared snapshot through
a generation request.
The assembled public `Details()` benchmark independently measured 670315–670339
B and 11548 allocations for one mutable-payload details call. The internal
snapshot benchmark isolates creation with fakes; together these baselines show
that mutable schema/extras copies dominate the small message-slice cost. The
Stage 13 candidate can remove the execution clone of preparation-owned
messages, hashes, and schema, but must retain a deep copy of extras and the
separate details clone. Focused verification passed: `go test -mod=readonly
./internal/usecase .`; representative content, copied schema, fake validation
status, and post-completion details were checked outside timing.
### Prepared Creation Snapshot Comparison
The before measurements use the Stage 11 assembled benchmark fixture committed
at `743e78d` and the Stage 12 ownership fixture committed at `11ecb92`. The
after measurements use the Stage 13 candidate in this completion. Both runs
used the same fixtures and environment: Linux/amd64, Intel Core i5-7260U @
2.20 GHz, benchmark parallelism 4, Go 1.26.8, ConfigKit `v0.5.0`, JSON Schema
`v6.0.2`, and backend modules `v3.20260930.0`. Benchmarks ran without the race
detector. The commands are those in the Assembled Workflow Benchmark Baseline
and Prepared Snapshot Ownership And Live-Heap Baseline subsections above.
The assembled creation benchmark measures `PrepareExecution` followed by
discarding its unused handle. Times list five `ns/op` samples, then median and
range; allocations are `B/op; allocs/op`.
| Case | Before `ns/op` (median; range) | Before allocations | After `ns/op` (median; range) | After allocations |
| --- | --- | --- | --- | --- |
| Small | 8194, 8016, 8095, 8073, 7939 (8073; 7939–8194) | 5210; 56 | 11028, 11059, 11756, 12279, 11089 (11089; 11028–12279) | 4665; 53 |
| Large message | 937857, 955672, 919810, 1058503, 1094114 (955672; 919810–1094114) | 873679–873681; 56 | 1751124, 1647596, 1761392, 1556071, 1584325 (1647596; 1556071–1761392) | 873135–873144; 53 |
| Mutable payload | 8114785, 8394320, 10387083, 8418604, 8528766 (8418604; 8114785–10387083) | 3711282–3711329; 60727 | 12157300, 12363416, 12185745, 11868982, 21225784 (12185745; 11868982–21225784) | 3483089–3483252; 56324–56326 |
Small and large cases each save about 0.5 KB and three allocations, while the
mutable case saves about 228 KB and 4,401–4,403 allocations. The after timing
samples in this run are higher than the recorded before samples, while other
repetitions on the same machine varied substantially; the timing evidence
does not establish a reliable latency improvement or regression. The ordinary
`Prepare`, `Run`, and public `Details()` allocation counts are unchanged within
their recorded baseline ranges; this candidate only changes prepared-execution
creation. The `B/op` and `allocs/op` values are cumulative allocation per
operation; the separate `retained-B` table is a post-GC live-heap delta for four
handles. They measure different effects and are not inferred from one another.
The focused internal snapshot benchmark isolates the same creation path using
the Stage 12 fake-collaborator fixture. Times again list five samples, median,
and range.
| Snapshot case | Before `ns/op` (median; range) | Before allocations | After `ns/op` (median; range) | After allocations |
| --- | --- | --- | --- | --- |
| Small | 45776, 51524, 46933, 46834, 44663 (46834; 44663–51524) | 13014–13016; 216 | 50907, 79812, 62186, 65569, 62599 (62599; 50907–79812) | 9405–9406; 156 |
| Mutable payload | 5343223, 5234718, 4213550, 4419520, 5046181 (5046181; 4213550–5343223) | 1211931–1212104; 25243–25244 | 5991662, 5508812, 6244981, 5176459, 4365122 (5508812; 4365122–6244981) | 983033–983309; 20837–20839 |
The internal fixture saves about 3.6 KB/60 allocations for the small case and
about 229 KB/4,404–4,407 allocations for the mutable case. Timing samples vary
between repeated runs and do not establish a consistent speedup or regression.
The retained-heap comparison uses the documented fixed batch of four mutable-
payload handles. These are five independently repeated `retained-B` samples
after the candidate; before samples are recorded in the baseline subsection.
| Handle state | Before samples (median; range) | After samples (median; range) |
| --- | --- | --- |
| Ready | 1458352, 1458256, 1459320, 1459048, 1459160 (1459048; 1458256–1459320) | 1458936, 1459048, 1460592, 1460464, 1460192 (1460192; 1458936–1460592) |
| Completed | 731040, 730768, 729352, 729448, 729624 (729624; 729352–731040) | 731312, 730496, 730768, 730544, 731040 (730768; 730496–731312) |
| Discarded | 728560, 730928, 730544, 730496, 730496 (730496; 728560–730928) | 731040, 731088, 730720, 730768, 730608 (730768; 730608–731088) |
All before/after retained ranges overlap. The median shifts are about 1.1 KB
for ready and completed handles and 272 B for discarded handles, all small
relative to their live sets and within observed run-to-run variation. This is
not evidence of a material retention increase, nor an exact object-size
guarantee. Together with the measured allocation reduction and mutation tests,
it supports retaining the simple ownership transfer while keeping the nested
extra-map copy, details clone, lifecycle cleanup, and caller-owned conversions.
Focused validation passed: `go test -race -mod=readonly ./internal/usecase .`.
### Validation And Compiled-Rendering Baseline
The benchmark-only baseline is commit
`0eb009ad5a3248e2a1e12cad9b3a05906d6b0850` (`Add validation and rendering
benchmarks`), on top of planning baseline
`3fde1f6cbbc2cc5b3dc17779ba44e0af6cd8a836`. The measured source is unchanged
from that commit. No production files changed.
Environment: Linux/amd64, Intel Core i5-7260U @ 2.20 GHz, benchmark parallelism
4; `go1.26.8`, module directive `go 1.26.0`, ConfigKit `v0.5.0`, JSON Schema
`v6.0.2`, and the backend modules `v3.20260930.0`.
Command (five 500 ms samples per benchmark, without the race detector):
```sh
GOTOOLCHAIN=local GOWORK=off GOPROXY=off GONOPROXY=none GOSUMDB=off \
go test -mod=readonly -run '^$' \
-bench 'Benchmark(BasicValidation|CompiledRendering|RetainedRenderedOutput|GoRendererArtifactReferences)$' \
-benchmem -benchtime=500ms -count=5 \
./internal/validate ./internal/prompt
```
Times below list all five `ns/op` samples in command order, followed by median
and range. Allocation figures are `B/op; allocs/op` (ranges where they varied).
| Benchmark case | `ns/op` samples (median; range) | Allocations |
| --- | --- | --- |
| Basic validation, short text | 46.59, 44.98, 43.31, 46.23, 43.19 (44.98; 43.19–46.59) | 0; 0 |
| Basic validation, 1 MiB nonempty | 193859, 329884, 418896, 386126, 378173 (378173; 193859–418896) | 1048576–1048577; 1 |
| Basic validation, ~1 MiB whitespace | 1112498, 1100439, 1175913, 1290678, 1108746 (1112498; 1100439–1290678) | 1048592–1048595; 2 |
| Basic validation, Unicode whitespace | 170.8, 164.2, 158.6, 159.2, 165.2 (164.2; 158.6–170.8) | 16; 1 |
| Compiled rendering, small message | 3931, 8488, 2041, 1902, 2181 (2181; 1902–8488) | 1632; 20 |
| Compiled rendering, 256 KiB artifact message | 285812, 380128, 655925, 664387, 537684 (537684; 285812–664387) | 1050656–1050713; 25 |
| Compiled rendering, repeated across session/messages | 1870972, 1362368, 1142268, 1073474, 1173588 (1173588; 1073474–1870972) | 3945983–3946034; 74 |
| `NewGoRenderer` fallback, one artifact reference | 407143, 629149, 522549, 562200, 470393 (522549; 407143–629149) | 1056409–1056477; 69–70 |
| `NewGoRenderer` fallback, repeated references | 1439883, 1535813, 1300176, 1541795, 1416267 (1439883; 1300176–1541795) | 2897171–2897255; 157 |
| Retained output, four 256 KiB rendered results | 398753, 392206, 384503, 380386, 406691 (392206; 380386–406691) | 1050702–1050724; 25–26 |
For the retained-output benchmark, post-GC live-heap deltas above the same
setup-only fixture were 1048272, 1050176, 1046032, 1049856, and 1049016 bytes
(median 1049016; range 1046032–1050176). The batch contains four separately
rendered 256 KiB messages. This is retained-heap evidence, not an inference
from `B/op`; it includes result structure and allocator/runtime noise. The
initial exploratory run exposed cross-count sink retention in the measurement
baseline; the benchmark now clears prior sinks before measuring setup, and the
recorded values are from the corrected committed fixture.
The short validator case allocates nothing, while the 1 MiB nonempty body has
about one response-sized allocation; the whitespace-only case also retains a
small failure-result allocation. Compiled rendering provides separate small,
artifact-sized, and repeated-reference baselines, with the existing parsing
fallback reported for context. Several render timings show wide ranges on this
machine, so the median/range—not any single run—are the comparison baseline.
The retained-output results consistently remain near 1 MiB for four outputs.
Focused verification passed: `go test -mod=readonly ./internal/validate
./internal/prompt`. Record later before/after comparisons, ownership analysis,
conditional decisions, and final verification in this section as the remaining
stages complete. Keep the feature roadmap focused on target state and progress
rather than duplicating the staged procedure.
### Basic-Validation Copy Comparison
The before measurement uses the unchanged Stage 1 benchmark fixture at commit
`05fe13d` (`Record validation and rendering baselines`), immediately before the
production change. The after measurement uses the same fixture and benchmark
implementation at `ed7c273` (`Avoid copying output during basic validation`).
Environment and module versions match the baseline above: Linux/amd64, Intel
Core i5-7260U @ 2.20 GHz, parallelism 4, Go 1.26.8, ConfigKit `v0.5.0`, JSON
Schema `v6.0.2`, and backend modules `v3.20260930.0`.
Command for both revisions (five 500 ms samples, without the race detector):
```sh
GOTOOLCHAIN=local GOWORK=off GOPROXY=off GONOPROXY=none GOSUMDB=off \
go test -mod=readonly -run '^$' -bench '^BenchmarkBasicValidation$' \
-benchmem -benchtime=500ms -count=5 ./internal/validate
```
Times list all five `ns/op` samples in command order, followed by median and
range. Allocation figures are `B/op; allocs/op`.
| Benchmark case | Before `ns/op` (median; range) | Before allocations | After `ns/op` (median; range) | After allocations |
| --- | --- | --- | --- | --- |
| Short text | 45.39, 45.10, 42.20, 44.19, 44.21 (44.21; 42.20–45.39) | 0; 0 | 41.80, 41.19, 39.90, 41.02, 39.18 (41.02; 39.18–41.80) | 0; 0 |
| 1 MiB nonempty text | 216006, 220902, 226243, 228846, 184748 (220902; 184748–228846) | 1048576–1048577; 1 | 39.76, 42.16, 41.61, 40.79, 40.92 (40.92; 39.76–42.16) | 0; 0 |
| ~1 MiB whitespace-only text | 766108, 709398, 731226, 915411, 787797 (766108; 709398–915411) | 1048592–1048594; 2 | 558919, 562202, 561004, 555487, 564593 (561004; 555487–564593) | 16; 1 |
| Unicode whitespace | 126.6, 113.2, 116.7, 111.6, 115.2 (115.2; 111.6–126.6) | 16; 1 | 115.1, 116.9, 106.1, 111.2, 110.9 (111.2; 106.1–116.9) | 16; 1 |
The 1 MiB nonempty case's median fell from 220902 ns/op to 40.92 ns/op while
response-sized allocation fell from 1048576–1048577 bytes to zero. The
whitespace case still scans the body and returns a small validation failure;
its response-sized temporary string is gone, reducing allocation from about
1 MiB and two allocations to 16 bytes and one allocation. Small-input timings
and the Unicode case remain in the same order of magnitude. The benchmark no
longer reports bytes per second because the nonempty fast path can classify a
body from its boundary bytes without scanning the full body; latency and
allocation comparisons remain meaningful.
Focused verification passed on the implementation commit:
`go test -mod=readonly ./internal/validate ./internal/usecase`. The benchmark
family above was rerun on that commit with the recorded command.
### Compiled-Rendering And Retained-Output Comparison
The maintained Stage 1 benchmark fixture was measured before the rendering
change at `dd89be3` (after the basic-validation change, which does not affect
rendering), then again after replacing the message and session buffers in the
working tree based on that revision. Setup and fixture contents are unchanged.
Environment: Linux/amd64, Intel Core i5-7260U @ 2.20 GHz, benchmark parallelism
4, Go 1.26.8, ConfigKit `v0.5.0`, JSON Schema `v6.0.2`, and backend modules
`v3.20260930.0`.
Command for both runs (five 500 ms samples per benchmark, without the race
detector):
```sh
GOTOOLCHAIN=local GOWORK=off GOPROXY=off GONOPROXY=none GOSUMDB=off \
go test -mod=readonly -run '^$' \
-bench '^(BenchmarkCompiledRendering|BenchmarkRetainedRenderedOutput)$' \
-benchmem -benchtime=500ms -count=5 ./internal/prompt
```
Times list all five `ns/op` samples in command order, followed by median and
range. Allocation figures are `B/op; allocs/op`.
| Compiled-render case | Before `ns/op` (median; range) | Before allocations | After `ns/op` (median; range) | After allocations |
| --- | --- | --- | --- | --- |
| Small message | 2066, 1906, 2003, 2027, 2010 (2010; 1906–2066) | 1632; 20 | 1912, 2030, 1877, 2045, 1926 (1926; 1877–2045) | 1568; 20 |
| Artifact-backed 256 KiB message | 279116, 341635, 285472, 348345, 341764 (341635; 279116–348345) | 1050713–1050718; 25 | 165860, 239221, 255435, 223641, 214475 (223641; 165860–255435) | 788485–788489; 24 |
| Repeated across session and messages | 966012, 1073914, 1098700, 1154416, 1064160 (1073914; 966012–1154416) | 3946015–3946038; 74 | 778584, 777091, 888988, 739473, 802395 (778584; 739473–888988) | 2446632–2446651; 71–72 |
The retained-output workload holds four separately rendered 256 KiB results
and reports post-GC live-heap change over the same setup-only fixture. Its
allocation and retained-heap samples were:
| Measurement | Before | After |
| --- | --- | --- |
| `ns/op` (median; range) | 331625, 333337, 343766, 302864, 329355 (331625; 302864–343766) | 244284, 239994, 238773, 239356, 264424 (239994; 238773–264424) |
| `B/op; allocs/op` | 1050696–1050708; 25 | 788486–788495; 24 |
| Retained heap (`retained-B`, median; range) | 1049744, 1046160, 1052768, 1044928, 1047264 (1047264; 1044928–1052768) | 1045824, 1054000, 1048288, 1047504, 1047168 (1047504; 1045824–1054000) |
The artifact-backed case removes about 256 KiB of allocation per render, and
the repeated case removes about 1.5 MiB; both also lose one allocation. Small
rendering changes by 64 bytes and 0 allocations. The retained-heap ranges
overlap, with medians differing by only 240 bytes for four 256 KiB outputs,
so the returned strings keep approximately the same live output storage rather
than retaining an additional builder buffer. This is expected: the builder's
backing storage is the returned string instead of being copied into a second
allocation. Measurements support retaining the simpler local-builder change
without a material retention trade-off; the existing per-render artifact-text
conversion and cache remain unchanged.
Focused verification passed: `go test -race -mod=readonly ./internal/prompt`,
including exact message/session output, cancellation, artifact-text lifecycle,
and a new concurrent-render case that verifies independent request variables,
artifact text, and session IDs against shared compiled templates.
### Artifact File-Read Capacity Comparison
The benchmark-only fixture was committed as `996295d` (`Add artifact file-read
benchmarks`) before the reader change. Before measurements use that revision;
after measurements use the same fixture after the bounded reservation and
post-read compaction were implemented. The real-file cases contain 4 KiB,
1 MiB, and 4 MiB of repeated `x` bytes. The retained case reads a real 256 KiB
file whose opened-file seam reports 16 MiB, retaining four results. Fixture
creation and MIME lookup are outside timing. Environment: Linux/amd64, Intel
Core i5-7260U @ 2.20 GHz, benchmark parallelism 4, Go 1.26.8, ConfigKit
`v0.5.0`, JSON Schema `v6.0.2`, and backend modules `v3.20260930.0`.
Command for both runs (five 500 ms samples per benchmark, without the race
detector):
```sh
GOTOOLCHAIN=local GOWORK=off GOPROXY=off GONOPROXY=none GOSUMDB=off \
go test -mod=readonly -run '^$' \
-bench '^(BenchmarkArtifactFileRead|BenchmarkRetainedArtifactFileRead)$' \
-benchmem -benchtime=500ms -count=5 ./internal/artifact
```
Times list all five `ns/op` samples in command order, followed by median and
range. Allocation figures are `B/op; allocs/op`.
| File-read case | Before `ns/op` (median; range) | Before allocations | After `ns/op` (median; range) | After allocations |
| --- | --- | --- | --- | --- |
| 4 KiB | 37904, 38759, 37073, 36437, 36930 (37073; 36437–38759) | 70513–70514; 12 | 37222, 37414, 36521, 37867, 39988 (37414; 36521–39988) | 70513; 12 |
| 1 MiB | 3576064, 4010278, 3919714, 3918643, 3844238 (3918643; 3576064–4010278) | 4752589–4752677; 23–24 | 3105775, 3055909, 3160450, 3268202, 3079892 (3105775; 3055909–3268202) | 1115066–1115090; 12 |
| 4 MiB | 16818098, 15977605, 16465923, 15753739, 15589001 (15977605; 15589001–16818098) | 20244106–20244130; 33 | 17060165, 16462547, 15995322, 17206316, 16180077 (16462547; 15995322–17206316) | 21177936–21178008; 24–25 |
The 1 MiB case reduced cumulative allocation from about 4.75 MB to 1.12 MB
and from 23–24 to 12 allocations, with a lower median time. For 4 MiB, the
reservation reduced allocation count from 33 to 24–25 but increased cumulative
allocated bytes by about 0.93 MB; timing ranges overlap. The small-file case
was effectively unchanged. These results support the specified capped hint as
a way to avoid repeated growth for ordinary 1 MiB reads, while showing that
the 1 MiB initial reserve is not a cumulative-byte win for every larger file.
| Overreported-size retained workload | Before | After |
| --- | --- | --- |
| `ns/op` (median; range) | 1144696, 1073206, 1060135, 1059339, 1073194 (1073194; 1059339–1144696) | 1188822, 1201670, 1199592, 1175553, 1173590 (1188822; 1173590–1201670) |
| `B/op; allocs/op` | 779427–779463; 18 | 1377557–1377574; 17 |
| Retained heap (`retained-B`, median; range) | 1082272, 1082496, 1082944, 1082384, 1082496 (1082496; 1082272–1082944) | 1049728, 1049392, 1049280, 1049504, 1049168 (1049392; 1049168–1049728) |
For the stale 16 MiB hint and 256 KiB actual body, the bounded reserve adds
about 0.6 MB of cumulative allocation and roughly 0.1 ms to the read, but the
post-read compaction prevents it from increasing live retained storage; the
four-result retained-heap median is about 33 KiB lower than the baseline. This
is intentionally a stale-metadata path, not a normal exact-size read. The
ordinary reader still derives returned bytes, size, and hash from content read
through EOF. Focused verification passed: `go test -mod=readonly
./internal/artifact`; the existing cancellation and nonregular-file checks
remain in the passing suite.
### Request-Preparation Baseline
The benchmark and boundary-test fixture is committed at `3f15715` (`Add
request preparation baseline and boundary tests`). The benchmark measures
`PrepareGeneration` without dispatch, with client construction, request
fixtures, and the structured schema created before timing. It uses a synthetic
endpoint and no credential or provider calls. Environment: Linux/amd64, Intel
Core i5-7260U @ 2.20 GHz, benchmark parallelism 4, Go 1.26.8, ConfigKit
`v0.5.0`, JSON Schema `v6.0.2`, and backend modules `v3.20260930.0`.
Command (five 500 ms samples per case, without the race detector):
```sh
GOTOOLCHAIN=local GOWORK=off GOPROXY=off GONOPROXY=none GOSUMDB=off \
go test -mod=readonly -run '^$' -bench '^BenchmarkPrepareGeneration$' \
-benchmem -benchtime=500ms -count=5 ./internal/llm
```
Times list all five `ns/op` samples in command order, followed by median and
range. Allocation figures are `B/op; allocs/op`.
| Preparation case | `ns/op` samples (median; range) | Allocations |
| --- | --- | --- |
| No extras | 2771, 2873, 2872, 2907, 2885 (2873; 2771–2907) | 1632; 23 |
| Small extras | 4216, 4255, 4440, 4090, 4245 (4245; 4090–4440) | 1928; 32 |
| 1 MiB string extra | 3108655, 3046197, 3165484, 3115831, 3074278 (3108655; 3046197–3165484) | 2147898–2213942; 26–27 |
| Nested JSON extras | 5933, 5803, 5675, 5490, 5611 (5675; 5490–5933) | 2304; 40 |
| Structured output | 6421, 6018, 6479, 6153, 6712 (6421; 6018–6712) | 2632–2633; 47 |
The baseline covers payload construction and complete JSON encoding for both
ordinary and representative structured requests. The 1 MiB case's higher
allocation volume is expected to include the large string and encoded body;
these measurements are the comparison reference for the following request
encoding change, not a provider-latency estimate. Focused verification passed:
`go test -mod=readonly ./internal/llm`.
### Request-Preparation Copy Comparison
The before results above use the Stage 5 benchmark fixture committed at
`3f15715`; the after results below use the unchanged fixture after the client
change in the working tree based on `c63ca53`. Both measurements use the same
offline environment and command recorded in the baseline subsection. `ns/op`
values list all five samples, followed by median and range. Allocation figures
are `B/op; allocs/op`.
| Preparation case | Before `ns/op` (median; range) | Before allocations | After `ns/op` (median; range) | After allocations |
| --- | --- | --- | --- | --- |
| No extras | 2771, 2873, 2872, 2907, 2885 (2873; 2771–2907) | 1632; 23 | 2849, 2841, 2872, 2746, 2902 (2849; 2746–2902) | 1632; 23 |
| Small extras | 4216, 4255, 4440, 4090, 4245 (4245; 4090–4440) | 1928; 32 | 3772, 3865, 3888, 3804, 3931 (3865; 3772–3931) | 1904; 29 |
| 1 MiB string extra | 3108655, 3046197, 3165484, 3115831, 3074278 (3108655; 3046197–3165484) | 2147898–2213942; 26–27 | 1524191, 1516315, 1537340, 1565157, 1538894 (1537340; 1516315–1565157) | 1140425–1197988; 25 |
| Nested JSON extras | 5933, 5803, 5675, 5490, 5611 (5675; 5490–5933) | 2304; 40 | 4440, 4446, 4389, 4302, 4481 (4440; 4302–4481) | 2016; 32 |
| Structured output | 6421, 6018, 6479, 6153, 6712 (6421; 6018–6712) | 2632–2633; 47 | 6253, 6137, 6267, 6315, 6232 (6253; 6137–6315) | 2632–2633; 47 |
The 1 MiB extra reduced median preparation time by about half and cumulative
allocation by approximately 1 MB. Small and nested extras also used fewer
bytes and allocations; no-extras and structured-output allocations were
unchanged. Timing ranges are machine-local evidence, not a consumer guarantee.
Invalid nested extras retain `ErrInvalidRequest` and key context before
transport. With an unencodable schema, serializable extras retain
`ErrRequestFailed`; invalid extras retain `ErrInvalidRequest`. Prepared retry
attempts continue to use the frozen body. Verification passed:
`go test -mod=readonly ./internal/llm ./internal/generation`, and the benchmark
family above was rerun without the race detector.
### Final Coverage And Verification
Stage 13's ownership-transfer implementation, tests, runner documentation, and
decision evidence are committed at `80aecb6` (`Reuse prepared execution
snapshot ownership`). The completed local scope maps to the accepted roadmap as
follows:
- Item 1's ConfigKit performance request remains recorded at `d53021e`; no
upstream implementation or dependency update was included.
- Items 2–5 are implemented in validation, rendering, request encoding, and
artifact reading. Their before/after fixtures, measurements, and focused
verification appear in the corresponding execution-evidence sections.
- Item 6's ownership review selected the simpler private-payload transfer in
Stage 13. It retains nested-extra and details copies; source isolation,
invalid-copy classification, allocation savings, and fixed-batch heap
observations are recorded above.
- Item 7's lightweight prompt/profile selection and requirement inspection are
implemented and measured in the inspection evidence sections. Production
compiled rendering and assembled offline preparation/execution benchmarks
are also maintained and recorded.
The canonical runner document describes the new snapshot ownership mechanics.
The source/validation and model-client documents remain consistent with their
implemented flows; no package responsibility, public API, or format semantics
changed. This plan remains the implementation record, and the performance
roadmap's status now reflects completion rather than proposed work.
Final verification passed on the Stage 13 code: `go test -race -vet=off
-mod=readonly ./...`, `go vet -mod=readonly ./...`, and
`python3 scripts/check-examples.py` (prepare, run, observe). The changed Go
files were `gofmt`-clean, and changed-file Markdown local targets were checked
separately. These checks are local correctness and documentation validation;
they do not add a release gate or imply provider-latency improvement.
## Open Questions
No consumer-policy decisions remain open. File reservation and encoding error
handling have concrete approaches above. The prepared-copy candidate was
evaluated and retained under the documented ownership and evidence criteria.
No implementation-scope questions remain open.