Verify startup loading improvements

docs/roadmap/startup-benchmark.md

@@ -1,7 +1,8 @@
 # Startup construction measurements and focused experiments
 
-Status: Stages 1–3 complete; Stage 4 is in progress with two retained experiments.
+Status: Complete. Both focused optimizations are retained; final consumer
-Complete consumer-workload verification remains outstanding.
+verification and handoff are recorded below. Acceptance rests on allocation
+reductions and behavioral checks; busy-host timing remains inconclusive.
 
 Measured on 2026-10-09 under the [startup roadmap](startup.md). The
 [consumer request](promptkit-performance-request.md) owns the submitted workload
@@ -725,7 +726,199 @@ inheritance, explanation, sensitivity, secrets, and independence passed.
 formatting, whitespace, the offline default suite, vet, and builds.
 Documentation changes passed the local-link and whitespace checks.
 
-This focused experiment is complete. Stage 4 has two retained changes, while
+At completion of this focused experiment, Stage 4 had two retained changes and
-Stage 5's complete consumer verification remains outstanding: PromptKit's
+Stage 5's complete consumer verification remained outstanding. The final pass
-file-backed, shared-base, and layered fixtures and its applicable configuration
+below completes those checks. No consumer dependency upgrade or release is
-tests. No consumer dependency upgrade or release is published.
+published.
+
+## Final consumer verification and handoff
+
+Status: Complete; retain both optimizations and conclude this performance
+effort. Measured on 2026-10-09. The maintainer confirmed that the host was very
+busy and selected allocations as the better comparison metric. This pass
+therefore establishes allocation improvements and behavioral compatibility,
+not an elapsed-time or CPU-speedup claim.
+
+### Comparison and reproduction
+
+- Before: `91ae4fa7eff1571e7c05de01f9e922d75df7453d`, the maintained benchmark
+  commit before either optimization. Its production code matches the original
+  current-code baseline `f1722274c02c08ff28556dd66f29e0da07a9f2c3`.
+- After: `c546124029645b0635a9babe2d97a74ac77f7f8f`, containing
+  `673faea`'s path-buffer sizing and `c546124`'s combined final metadata walk.
+  These remain the only production changes relative to the before revision.
+- PromptKit: `eb830bb61346672f0f6c5bce6119b76ccc6d61ac` for both comparisons,
+  with its benchmark, adapter, tests, and other dependencies unchanged.
+- Separate disposable ConfigKit clones hold the exact before/after commits;
+  only a disposable PromptKit clone's module replacement selects between them.
+  Original consumer files and committed dependencies remain unchanged.
+- Toolchain, hardware, runtime settings, and offline/readonly module settings
+  match the recorded environment above. Timed runs were sequential and did not
+  overlap our correctness checks or profile analysis. Other host work was not
+  controlled.
+- All logs and disposable checkouts are outside the repositories at
+  `/tmp/configkit-startup-final.jO52hW`. This pass adds no production code,
+  dependencies, benchmark fixtures, or release gates.
+
+The primary tables use per-metric medians of five longer observations, run
+after, then before. Initial batches ran before, then after with
+`-benchtime=500ms -count=5`; their allocation reductions were reproduced in
+the longer batches. No outliers were removed from the completed paired batches.
+
+Run from each ConfigKit copy:
+
+```sh
+GOTOOLCHAIN=local GOWORK=off GOPROXY=off GOMAXPROCS=4 \
+  go test -mod=readonly -run '^$' -bench '^BenchmarkStartupConstruction$' \
+  -benchmem -benchtime=1s -count=5
+```
+
+In the disposable PromptKit copy, select the relevant ConfigKit copy using
+the Stage 1 `go mod edit -replace` command, then run:
+
+```sh
+GOTOOLCHAIN=local GOWORK=off GOPROXY=off GOMAXPROCS=4 \
+  go test -mod=readonly -run '^$' -bench '^BenchmarkConfigurationConstruction$' \
+  -benchmem -benchtime=1s -count=5 ./internal/configuration
+```
+
+### Combined local construction results
+
+The maintained fixture and timing boundaries are unchanged. These are total
+allocated bytes and allocation counts per construction, not retained heap.
+
+| Catalog | Before B | After B | Byte reduction | Before allocations | After allocations | Allocation reduction |
+| --- | ---: | ---: | ---: | ---: | ---: | ---: |
+| 32 pairs | 11,999,452 | 10,887,026 | 9.27% | 129,136 | 78,605 | 39.13% |
+| 128 pairs | 47,264,285 | 42,917,578 | 9.20% | 513,719 | 311,704 | 39.32% |
+| 512 pairs | 189,216,372 | 171,145,796 | 9.55% | 2,051,983 | 1,244,004 | 39.38% |
+
+The reductions hold across all three selected sizes. This is scaling evidence
+for these fixtures, not an asymptotic guarantee. At 128 pairs, the combined
+changes remove approximately 202,000 allocations and 4.35 MB per construction.
+
+### Broader PromptKit workloads
+
+All fixture preparation, reusable source capture, and fixture files are outside
+timing. Every construction still includes the real adapter's ConfigKit load,
+callbacks, validation, and post-return prompt materialization, compilation,
+hashing, and execution-target capture.
+
+- `independent`: 128 profiles and 128 prompt versions; 256 input/effective
+  entries in one tier.
+- `file_backed`: 64 profiles and 64 prompt versions; 128 input/effective
+  entries in one tier. Message bodies are read through a captured
+  `fstest.MapFS` namespace, not a physical-disk benchmark. Native-file behavior
+  is separately covered by the configuration tests.
+- `shared_base`: one base profile, 128 sibling-derived profiles, and one prompt
+  version; 130 input/effective entries in one tier. Each child uses `extends`
+  and overrides temperature while inheriting endpoint, model, and nested
+  provider parameters.
+- `layered`: three underlays and main input; 513 input entries become 129
+  effective entries across four tiers. Profile fields come from different
+  precedence tiers; one prompt selects the final profile.
+
+| Workload | Before B | After B | Byte reduction | Before allocations | After allocations | Allocation reduction |
+| --- | ---: | ---: | ---: | ---: | ---: | ---: |
+| independent | 50,158,875 | 45,512,786 | 9.26% | 538,542 | 330,484 | 38.63% |
+| file_backed | 27,045,487 | 24,715,761 | 8.61% | 280,679 | 175,854 | 37.35% |
+| shared_base | 30,860,505 | 28,240,215 | 8.49% | 383,695 | 243,357 | 36.58% |
+| layered | 28,415,777 | 25,542,623 | 10.11% | 342,923 | 208,686 | 39.14% |
+
+The broader consumer cases reproduce reductions through the existing public
+API. These totals include PromptKit work and must not be presented as isolated
+ConfigKit costs. No consumer source or dependency upgrade was needed.
+
+### Complementary allocation evidence
+
+The original complementary comparison used
+`-benchtime=200ms -count=3` with the Stage 4 selector for composition,
+broad underlays, inheritance loading, and rendered history. Per-metric medians:
+
+| Benchmark / size | Before B | After B | Before allocations | After allocations |
+| --- | ---: | ---: | ---: | ---: |
+| ComposeWideMappings / 128 | 848,509 | 797,562 | 9,658 | 5,962 |
+| ComposeWideMappings / 256 | 1,703,587 | 1,595,291 | 19,652 | 11,860 |
+| ComposeWideMappings / 512 | 3,626,286 | 3,403,288 | 39,643 | 23,659 |
+| InheritanceInspectedLoad / 16 | 876,217 | 832,232 | 11,050 | 7,990 |
+| InheritanceInspectedLoad / 32 | 1,545,193 | 1,460,268 | 19,943 | 14,011 |
+| InheritanceInspectedLoad / 64 | 2,906,050 | 2,739,206 | 37,778 | 26,098 |
+| UnderlayBroadCatalog / 128 | 1,265,034 | 1,218,918 | 15,904 | 12,370 |
+| UnderlayBroadCatalog / 256 | 2,581,008 | 2,437,650 | 33,473 | 24,307 |
+| UnderlayBroadCatalog / 512 | 5,269,009 | 4,931,073 | 68,581 | 48,151 |
+| UnderlayRenderedHistory / 32 | 504,551 | 503,463 | 3,294 | 3,221 |
+| UnderlayRenderedHistory / 64 | 1,006,071 | 1,003,952 | 6,529 | 6,392 |
+| UnderlayRenderedHistory / 128 | 2,031,859 | 2,027,696 | 12,997 | 12,732 |
+
+Every case allocates less. Composition exercises the retained path-formatting
+change but not final metadata indexing. History rendering loads outside timing;
+its small allocation reduction comes from path formatting, not the fused load
+traversal.
+
+Apparent timing regressions prompted five-observation reverse-order repeats of
+broad-underlay / 128 and / 256 and rendered-history / 32, / 64, and / 128 using
+`-benchtime=500ms -count=5` and selector
+`^(BenchmarkUnderlayBroadCatalog|BenchmarkUnderlayRenderedHistory)$/^(32|64|128|256)$`.
+Their allocation-count medians exactly match the corresponding initial rows,
+and allocated-byte reductions recur. No persistent cache or result-lifetime
+change was introduced; retained heap was not measured separately.
+
+### Timing limitation
+
+Timing varied greatly without source changes. For example, the final
+candidate's local / 32 median was 45.098 ms in the initial batch and 17.362 ms
+in the longer batch. Both showed approximately 78,605 allocations. The initial
+baseline's / 512 observations ranged from 574.500 to 2,027.077 ms, whereas its
+longer repeat ranged from 261.986 to 269.806 ms.
+
+Complementary timing repeats also produced large apparent regressions. Before
+the maintainer's busy-host notice, one further candidate-only diagnostic
+completed: broad-underlay / 128 returned from a 7.556 ms median to 1.589 ms,
+and rendered-history / 32 from 2.386 ms to 0.306 ms, with unchanged source and
+allocation counts. The subsequent baseline diagnostic was intentionally
+interrupted, and further timing comparisons stopped. Its partial output is not
+included in matched results; the interrupted diagnostic is not a test failure.
+
+Raw logs retain elapsed-time observations, but this pass establishes neither
+a general speedup nor a reproducible timing regression attributable to the
+changes. Optional quiet-host timing work can refine that assessment. It is not
+a remaining requirement for this allocation-focused outcome or a release gate.
+
+### Behavioral verification and final decision
+
+Both ConfigKit selections passed PromptKit's complete configuration-package
+tests and 15 selected public contract tests:
+
+```sh
+GOTOOLCHAIN=local GOWORK=off GOPROXY=off \
+  go test -mod=readonly -count=1 ./internal/configuration
+
+GOTOOLCHAIN=local GOWORK=off GOPROXY=off \
+  go test -mod=readonly -count=1 \
+  -run '^Test(ConfigurationInspection|PublicConstruction|ProfileInheritance|EngineCapturesPromptContent|PreparationStillReads)' .
+```
+
+Those checks exercise final effective values, explicit zero/null/omission,
+normalization history, strict conflicts and tier precedence, imported and
+native file content, source-context ownership and closing, public inheritance,
+required credentials, error identities, captured content, and redacted
+inspection without provider calls. Expectations and fixtures were unchanged.
+
+ConfigKit's applicable semantic, ownership, race, facade, and everyday-check
+results from Stage 4 apply to the identical production candidate. This pass
+reuses those successful checks; its documentation updates passed local-link
+and whitespace validation. Component responsibilities and documented
+internal flow remain accurate, so no architecture, ADR, public-contract, or
+internal-document change is needed.
+
+Retain both changes. They remove redundant buffer growth and final-tree
+traversal while preserving separately owned metadata indexes and the existing
+configuration model. Allocation reductions reproduce locally and across the
+recorded consumer's independent, content-file, inheritance, and layered
+workloads, with existing behavioral tests passing. No further optimization is
+selected for this effort.
+
+Stages 4–6 are complete. No required consumer verification remains for this
+scope; timing and retained-heap limits are explicit. Release preparation,
+roadmap retirement, publication, and any consumer dependency update remain
+separate work.

docs/roadmap/startup.md

@@ -1,7 +1,7 @@
 # Startup Loading Performance Roadmap
 
-Status: Stages 1–3 complete; Stage 4 is in progress with two optimizations
+Status: Complete. Two optimizations are retained, with final consumer
-retained. Findings and reproduction commands are in
+verification and handoff recorded. Findings and reproduction commands are in
 [startup-benchmark.md](startup-benchmark.md).
 
 Audience: ConfigKit maintainers implementing and reviewing this work.
@@ -122,11 +122,11 @@ finding and recommend whether further investigation is justified.
 
 ### 4. Implement and measure focused changes
 
-Status: In progress. `Path.String` buffer sizing and combined final metadata
+Status: Complete. `Path.String` buffer sizing and combined final metadata
 traversal are retained after fresh construction, consumer, inspection, and
 profile comparisons. See the [path-formatting experiment](startup-benchmark.md#focused-experiment-pre-size-pathstring)
 and [metadata-traversal experiment](startup-benchmark.md#focused-experiment-combine-final-metadata-traversal).
-Complete consumer-workload verification remains outstanding.
+No further optimization is selected for this effort.
 
 1. Implement the selected internal change while preserving the supported API
    and established behavior. Audit any reduced copying or reused data for
@@ -152,7 +152,12 @@ justify their complexity; a predetermined optimization count is unnecessary.
 
 ### 5. Verify the complete result against consumer workloads
 
-Status: Not started.
+Status: Complete. The combined candidate was compared with the pre-optimization
+baseline across all local sizes, complementary benchmarks, and all four recorded
+PromptKit workloads. Consumer configuration and public contract tests passed.
+Acceptance rests on reproducible allocation reductions; the maintainer confirmed
+that busy-host timing is not a reliable comparison here. See the
+[final verification](startup-benchmark.md#final-consumer-verification-and-handoff).
 
 1. Compare the final candidate with the unoptimized current-code baseline using
    the local fixture and relevant existing benchmarks. Confirm that gains hold
@@ -175,7 +180,11 @@ benefit has been verified.
 
 ### 6. Record the outcome and hand off
 
-Status: Not started.
+Status: Complete. Retained changes, revisions, allocation comparisons,
+reproduction commands, behavioral checks, and timing/retained-heap limitations
+are recorded in the [final handoff](startup-benchmark.md#final-consumer-verification-and-handoff).
+No required verification remains for this scope. Release preparation and
+publication remain separate work.
 
 Summarize the retained changes, before/after results, revisions, reproduction
 commands, validation, tradeoffs, and unresolved questions in this roadmap.