Retire completed roadmaps for release preparation
This commit is contained in:
@@ -94,6 +94,17 @@ preserved in Git history at `73ce6e9` under `docs/roadmap/`. Current overlay
|
||||
behavior belongs to the [configuration contract](config.md#selected-overlays),
|
||||
and release operations belong to the [release procedure](release.md).
|
||||
|
||||
The completed ReleaseKit adoption, startup performance request and experiments,
|
||||
benchmark evidence, performance audit, implementation plan, and remediation
|
||||
record are preserved in Git history at `8e91a2b` under `docs/roadmap/`. For example:
|
||||
|
||||
```sh
|
||||
git show 8e91a2b:docs/roadmap/startup-benchmark.md
|
||||
```
|
||||
|
||||
Remaining optimization candidates belong to
|
||||
[deferred scope](roadmap/deferred.md#deferred-performance-experiments).
|
||||
|
||||
Install the Go version required by `go.mod` or a compatible newer toolchain.
|
||||
Acquire declared dependencies explicitly before entering an offline
|
||||
development or validation environment:
|
||||
|
||||
@@ -32,3 +32,20 @@ concrete consumer evidence, compatibility review, and an explicit design change.
|
||||
Built-in sibling inheritance using `extends` is implemented as described in the
|
||||
[configuration contract](../config.md#sibling-inheritance). It is distinct from
|
||||
inheritance between selected overlay files, which remains excluded above.
|
||||
|
||||
## Deferred performance experiments
|
||||
|
||||
The completed performance audit left these internal optimization candidates for
|
||||
separate investigation:
|
||||
|
||||
- branch-local explanation replay instead of the current whole-tree replay;
|
||||
- redesigning the private metadata-copy handoff while preserving detached public
|
||||
records;
|
||||
- caching schema descriptors across loads; and
|
||||
- indexing complex contributor-trace membership while preserving its equality
|
||||
rules and compact linked traces.
|
||||
|
||||
These are not committed implementation work. Reconsider them with evidence that
|
||||
the benefit justifies the ownership, retained-memory, or equivalence risks. The
|
||||
completed audit and implementation evidence are preserved in Git history at
|
||||
`8e91a2b` under `docs/roadmap/`.
|
||||
|
||||
@@ -1,501 +0,0 @@
|
||||
# Implement the performance audit revisions
|
||||
|
||||
Status: Complete. Audience: a Luna coding agent implementing one stage at a time,
|
||||
in numerical order. The [performance audit roadmap](performance-audit.md) owns
|
||||
the target and rationale. This plan replaces the completed ReleaseKit plan,
|
||||
which remains available in Git history at `0c77dee`.
|
||||
|
||||
Planning baseline: `0c77deed05417edb32f42c7bf24df2fdc2581a25`.
|
||||
The production baseline is unchanged from the audit's `48619de` revision.
|
||||
Keep the earlier Path.String sizing and combined final-metadata traversal
|
||||
improvements. This sprint changes private algorithms, not public API,
|
||||
configuration semantics, dependencies, Go minimum, or release gates.
|
||||
|
||||
## Working instructions
|
||||
|
||||
- Read [development.md](../development.md), the applicable agent, architecture,
|
||||
testing, and documentation policies, and owning contract/test files before
|
||||
each stage. Use the graph tools for code discovery as instructed by AGENTS.md.
|
||||
- Preserve user changes. Implement only the current stage and its regressions;
|
||||
update its status/evidence here and commit the validated unit with a short
|
||||
plain-English message. Do not publish, push, or modify consumer dependencies.
|
||||
- Operation indexes are private phase-local data. Inspection indexes are fully
|
||||
built before publication and only read afterward. Returned records remain
|
||||
detached. Keep effective and historical indexes separate with independently
|
||||
owned mutable path storage.
|
||||
- Preserve the existing [configuration contract](../config.md): presence,
|
||||
null/empty distinctions, composition, diagnostics, chronology, sensitivity,
|
||||
callback lifetime, secret isolation, and handle closure. Do not add rejection
|
||||
rules, limits, stricter callbacks, concurrency promises, or complexity guarantees.
|
||||
- Run each stage's focused checks and `./scripts/check.sh` before handing off
|
||||
production changes. Run `go test -race ./internal/engine` after retained-index
|
||||
or ownership changes, and the full race suite in Stage 18. Reuse checks for
|
||||
unchanged content. No new CI wait, release gate, or performance threshold.
|
||||
- Prefer algorithm reasoning for this sprint. Benchmark runs are not prerequisites.
|
||||
Optional measurements use the same prepared local Go, offline dependencies,
|
||||
sequential runs, and artifacts outside repositories. Favor B/op and allocs/op
|
||||
on the busy host; do not infer CPU improvements from noisy timings.
|
||||
- Extend existing behavioral tests for missing risks, not traversal counts.
|
||||
Small test-only baseline algorithms may support bounded differential coverage.
|
||||
Do not keep alternate production engines or switches. If the selected algorithm
|
||||
cannot preserve behavior, record the discrepancy and seek review rather than
|
||||
strengthening the contract or silently dropping work.
|
||||
|
||||
## Resolved implementation choices
|
||||
|
||||
1. Containment and path deduplication use canonical token spelling, as today.
|
||||
Metadata/document queries additionally enforce contextual key/index shape.
|
||||
These predicates remain distinct.
|
||||
2. Use a small private engine prefix trie with canonical **escaped** pointer
|
||||
tokens and integer IDs into owner-held slices. Root is zero tokens; `/`
|
||||
is one empty token. Scan separator boundaries without decoding/rejoining
|
||||
tokens. Escapes contain no slash. The trie does not validate document shape.
|
||||
3. The helper returns IDs, not values/provenance. Owners select deepest ancestor,
|
||||
earliest inserted ancestor/intersection, or all relevant IDs, explicitly
|
||||
restoring owner order. No general path framework or public hook is introduced.
|
||||
4. Sensitivity uses a worklist: ordinary dependencies activate on intersection,
|
||||
while directed inheritance links activate on a classified ancestor/equal
|
||||
path of their dependency. Preserve both rules.
|
||||
5. Inspection retains full replay and on-demand safe rendering. Prepare immutable
|
||||
coalesced resolution descriptors and inheritance lookup data once, with no
|
||||
per-field tree or rendered-explanation cache.
|
||||
6. Defer branch-local replay, public metadata-copy redesign, global schema caching,
|
||||
and complex contributor-trace membership indexing. Their ownership/memory/
|
||||
equality risks exceed the selected scope. Existing clones and linked traces
|
||||
stay in place. No new dependency, pool, or package-global mutable cache is needed.
|
||||
|
||||
## Stage 1: Confirm the baseline and semantic evidence — Complete
|
||||
|
||||
Scope: orientation and test-only evidence, with no optimization yet.
|
||||
|
||||
1. Inspect status/revisions. Read the audit and configuration contract sections
|
||||
for metadata, sensitivity, explanation, resolution, and overrides, plus their
|
||||
implementation/test owners.
|
||||
2. Run `./scripts/check.sh`. Record revision, local `go version`, result, and
|
||||
environmental limitations in an evidence subsection here. Do not repeat or
|
||||
revise the completed startup timing campaign.
|
||||
3. Identify existing coverage for numeric shapes, removed paths, escaped keys,
|
||||
inheritance, callback detachment, and independent loads. Add test-only reference
|
||||
helpers if useful for later containment, coverage/coalescing, overlap, and
|
||||
sensitivity equivalence tests. Keep fixtures bounded and deterministic; do not
|
||||
copy the loader. Helpers may instead accompany their first consuming stage.
|
||||
|
||||
Done: baseline checks recorded and semantic test owners identified. Existing
|
||||
contracts/tests are authoritative; reference algorithms add equivalence evidence,
|
||||
not new public guarantees. A documentation-only stage needs documentation checks.
|
||||
|
||||
### Evidence
|
||||
|
||||
- Current revision: `52793301ba80eb49f91956009e60cd812f92e94c`; planning baseline:
|
||||
`0c77deed05417edb32f42c7bf24df2fdc2581a25`. The working tree was clean, and
|
||||
production Go files match the audit baseline `48619de024d794cf41e90b171b7fcbe609dcd437`.
|
||||
- `GOTOOLCHAIN=local go version`: `go1.26.8 linux/amd64`.
|
||||
`GOTOOLCHAIN=local ./scripts/check.sh`: passed (`go test ./...`, `go vet ./...`,
|
||||
and `go build ./...`, in addition to formatting and whitespace checks). No
|
||||
environmental limitations encountered; no benchmark campaign was repeated.
|
||||
- Existing semantic coverage includes numeric/key-index and escaped path shapes
|
||||
(`path_test.go`, `provenance_test.go`, `override_test.go`), removed/history
|
||||
paths (`metadata_test.go`, `resolution_test.go`, `inheritance_history_test.go`),
|
||||
nested and sequence-contained inheritance (`inheritance_test.go`,
|
||||
`inheritance_walk_test.go`), detached callback/metadata records
|
||||
(`callback_ownership_test.go`, `resolution_test.go`, `metadata_test.go`), and
|
||||
independent loads (`provenance_test.go`). Differential reference helpers are
|
||||
deferred to the first stage that consumes each algorithm so they can remain
|
||||
small and bounded.
|
||||
|
||||
## Stage 2: Make presence-only queries lightweight — Complete
|
||||
|
||||
Roadmap candidate 1. Owner: `internal/engine/metadata.go`.
|
||||
|
||||
1. Make `Metadata.IsExplicit` validate effective path/shape and read the stored
|
||||
boolean, without calling `Lookup`, cloning origins, or projecting inheritance.
|
||||
A shared private effective-field lookup may serve both methods; `Lookup`
|
||||
continues returning complete detached records.
|
||||
2. Format each query pointer once per lookup. Preserve root, zero metadata,
|
||||
invalid/unknown/removed paths, parsed numeric tokens, and explicit key/index
|
||||
distinctions. Leave `Resolution.IsExplicit` entry-state semantics unchanged.
|
||||
3. Extend metadata/presence tests only where agreement with `Lookup` lacks
|
||||
coverage: direct/inherited input, defaults, and explicit false/zero/null.
|
||||
|
||||
Validate: `go test ./internal/engine -run 'Metadata|Presence|Explicit|Inherited'`,
|
||||
relevant root facade tests, and the everyday check. No allocation assertion.
|
||||
|
||||
## Stage 3: Detect inheritance without constructing paths — Complete
|
||||
|
||||
Roadmap candidate 9. Owner: `internal/engine/inheritance_load.go`.
|
||||
|
||||
1. Replace detection's full path-building walk with a nil-safe boolean visitor:
|
||||
inspect mapping keys for `extends`, recurse into values/sequence children,
|
||||
and return immediately on a match.
|
||||
2. Leave validation, expansion order, provenance, and budgets unchanged.
|
||||
Detection is not validation and must still find nested invalid syntax.
|
||||
3. Reuse inheritance tests; add missing sequence-contained or invalid nested
|
||||
cases without assertions about how many nodes were visited.
|
||||
|
||||
Validate: `go test ./internal/engine -run 'Inheritance|Inherited'` and the
|
||||
everyday check. Inputs without inheritance still need a linear scan.
|
||||
|
||||
## Stage 4: Compare path tokens without whole-pointer formatting — Complete
|
||||
|
||||
Roadmap candidate 2. Owners: engine `path.go`, `resolution.go`, `metadata.go`.
|
||||
|
||||
1. Add private token-spelling equality. Key/parsed segments compare literal
|
||||
bytes; two indexes compare integers. For index versus key/parsed, render the
|
||||
index into stack scratch space with `strconv.AppendInt` and compare bytes.
|
||||
Do not parse literal tokens as integers: `01`, `+1`, and `1` differ.
|
||||
2. Implement `pathContains` with existing validity/depth checks and token-prefix
|
||||
equality. Root behavior remains unchanged. Escaping is injective, so raw
|
||||
token spelling comparison preserves canonical pointer equivalence.
|
||||
3. Replace textual comparison in `matchesPathShape` with token equality, then
|
||||
retain its existing explicit/contextual shape checks and candidate-side
|
||||
handling. Apply the same spelling predicate to local textual path deduplication.
|
||||
4. Differentially compare with the String-based baseline over roots, invalid
|
||||
paths, empty/slash/tilde/numeric keys, parsed tokens, nested paths, and integer
|
||||
extrema. Reuse existing effective/historical query-shape coverage.
|
||||
|
||||
Validate: `go test ./internal/engine -run 'Path|Metadata|Resolution|Override|Inheritance'`
|
||||
and the everyday check. Public `Path.String` stays unchanged.
|
||||
|
||||
## Stage 5: Format sorting keys once per item — Complete
|
||||
|
||||
Roadmap candidate 2. Owners: engine resolution, overlay, override application,
|
||||
document comparison, and diagnostics.
|
||||
|
||||
1. Decorate path-bearing items with precomputed canonical pointers, sort with
|
||||
existing comparisons/tie-breakers, then project back. Cover
|
||||
`orderedResolutionRecords`, `displacedOrigins`, `affectedOverrideOrigins`,
|
||||
and the final delta sort in `compareConfigurationValues`.
|
||||
2. In `canonicalDiagnostics`, precompute paths without changing comparator
|
||||
precedence, primary/related location ordering, or standalone comparator use.
|
||||
Do not add permanent pointer fields to every record for sorting.
|
||||
3. Gather recursive displaced definitions into one owned output and sort at
|
||||
the outer owner, avoiding repeated sorting of the same nested subtree.
|
||||
Preserve final lexical order and duplicate behavior.
|
||||
4. Reuse ordering tests; add missing tie/escaped-key cases. Do not promise stable
|
||||
order for items whose full comparison is equal.
|
||||
|
||||
Validate: `go test ./internal/engine -run 'Resolution|Override|Overlay|Diagnostic|CompareDocument|Provenance'`
|
||||
and the everyday check. Expected benefit is reduced formatting, not measured CPU.
|
||||
|
||||
## Stage 6: Add a private canonical-pointer prefix index — Complete
|
||||
|
||||
Scope: one helper plus tests, no caller migration yet.
|
||||
|
||||
1. Add `internal/engine/path_prefix.go`. Nodes have child maps keyed by escaped
|
||||
tokens, terminal integer payload IDs, and the minimum inserted ID within the
|
||||
subtree. IDs are nonnegative owner ordinals; represent absence explicitly.
|
||||
2. Support insertion, ancestor payload visits (root/exact included), descendant
|
||||
payload visits (exact included), and earliest intersecting ID using ancestor
|
||||
IDs and the queried subtree minimum. Preserve multiple IDs at one terminal.
|
||||
No deletion, wildcard search, provider abstraction, or serialization API.
|
||||
3. Tokenize canonical pointers by separator scanning without split slices.
|
||||
Preserve root versus empty keys and `/a` versus `/ab` boundaries. Owners
|
||||
supply canonical pointers from valid paths; no new input validation contract.
|
||||
Retain IDs only, not nodes, credential bytes, or mutable path segments.
|
||||
4. Differentially check results against simple prefix scans for ancestors,
|
||||
descendants, duplicates, insertion minima, escaping, and root/empty tokens.
|
||||
|
||||
Validate: focused helper tests and the everyday check. Storage follows indexed
|
||||
token material/payloads; tests do not fix representation or allocation counts.
|
||||
|
||||
## Stage 7: Index resolution coverage and record reads — Complete
|
||||
|
||||
Roadmap candidate 3, first integration. Owner: `internal/engine/resolution.go`.
|
||||
|
||||
1. Index already ordered records after the callback. Select deepest covering
|
||||
ancestor, earliest owner ID for equal depth, matching the current scan.
|
||||
Use this for delta coverage in `runResolution`; retain all validity/helper
|
||||
checks and diagnostic order.
|
||||
2. Add a small snapshot coverage adapter with the same rules for Stage 8.
|
||||
IDs refer to owner-held slices and cannot outlive or survive their reordering.
|
||||
3. Build operation-local read-only mapping lookups for completed before/after
|
||||
trees, reusing `mappingIndex` mechanics without mutating trees. Add a private
|
||||
indexed document lookup variant with the same validity, shape, bounds, and
|
||||
absence checks; retain the plain helper for callers without reuse.
|
||||
4. Cover parent/exact/more-specific records, equal-depth private fixtures,
|
||||
added/removed values, unrecorded changes, and helper-target mismatch.
|
||||
|
||||
Validate: `go test ./internal/engine -run 'Resolution|Resolve|Callback|Path'`
|
||||
and the everyday check. Snapshot ownership/contents remain unchanged.
|
||||
|
||||
## Stage 8: Replace per-delta whole-metadata scans — Complete
|
||||
|
||||
Roadmap candidate 3, second integration. Owner: engine `load_metadata.go`.
|
||||
|
||||
1. In `applyResolutionMetadata`, build one final pointer/node index and one
|
||||
snapshot coverage index. Reuse final membership for descendant history and
|
||||
field refresh; do not rebuild indexes per delta/field.
|
||||
2. Index delta paths by original delta ID. Enumerate the union of historical
|
||||
and final pointers once, obtain covering delta IDs, and bucket affected pointers
|
||||
by delta. Parse paths only when a retained semantic path is actually needed.
|
||||
Sort buckets lexically and retain original delta processing order.
|
||||
3. Apply exact snapshot events first, descendant history second, effective origin
|
||||
updates third, then final field refresh, as today. Exclude the covering record's
|
||||
own pointer from descendant history and retain the original Previous origins.
|
||||
Handle overlapping delta IDs in reference fixtures without assuming every
|
||||
possible input is disjoint. Normal structural comparison emits disjoint deltas.
|
||||
4. For fields, visit pointers once and process matching delta IDs in original
|
||||
order. For new fields, use snapshot coverage; preserve null/Explicit state,
|
||||
default resolution-origin fallback, and removed-path history.
|
||||
5. Extend existing tests for many disjoint changes, replacement, removed/new
|
||||
descendants, and numeric container-shape changes. A bounded test-only
|
||||
reference metadata update can protect phase/order equivalence.
|
||||
|
||||
Validate: `go test ./internal/engine -run 'Resolution|Metadata|Provenance|History|Explain'`,
|
||||
engine race tests, and the everyday check. Complexity is output-sensitive to
|
||||
affected events, not every delta times every field.
|
||||
|
||||
## Stage 9: Collect sensitivity directly from trees — Complete
|
||||
|
||||
Roadmap candidate 5. Owner: engine `secret.go`. Keep fixed-point propagation
|
||||
unchanged until Stage 11 so collection is independently reviewable.
|
||||
|
||||
1. Replace temporary pointer/node indexes used only to enumerate classification
|
||||
paths/flags with direct visitors into local path and sensitivity accumulators.
|
||||
Remove the redundant initial final-tree scan; final still participates in
|
||||
the late projection phase before schema/predicate classification.
|
||||
2. Preserve phase precedence: applications; history/dependencies; inheritance;
|
||||
projections then final. Tree/pointer contributions currently install parsed
|
||||
segments, while history/contributor contributions can install typed paths.
|
||||
Preserve those representations and subsequent overwrites.
|
||||
3. Skip reparsing only if the current stored path already has that parsed
|
||||
representation. Otherwise perform the existing parsed replacement. Do not
|
||||
simply substitute explicit walker key/index paths or retain the first shape.
|
||||
4. Retain all historical/removed paths, controls, dependency-only paths, default/
|
||||
dynamic tagged projections, and cloned predicate arguments.
|
||||
5. Add only missing numeric shape-transition, historical predicate, dynamic tag,
|
||||
and predicate-detachment cases in existing sensitivity/inheritance test owners.
|
||||
|
||||
Validate: `go test ./internal/engine -run 'Secret|Sensitivity|Inherited|Effective|Explain'`,
|
||||
engine race tests, and the everyday check. No new predicate order/count guarantee.
|
||||
|
||||
## Stage 10: Build and test indexed sensitivity triggers — Complete
|
||||
|
||||
Roadmap candidate 6, isolated algorithm. Add private
|
||||
`internal/engine/secret_propagation.go` and focused tests.
|
||||
|
||||
1. Ordinary edges have dependency/target pointers and activate on classified
|
||||
ancestor or descendant intersection. Inheritance links become two directed
|
||||
edges, each activated only by a classified ancestor/equal dependency path.
|
||||
2. Index dependency pointers with payload IDs. A new classified pointer visits
|
||||
ordinary triggers at ancestors and descendants; inheritance triggers only
|
||||
at descendants. Avoid double-visiting exact terminals. Track fired IDs and
|
||||
enqueue a target only when it becomes newly sensitive.
|
||||
3. Seed from the initial sensitive set. Preserve the ordinary dependency filter:
|
||||
the old `secretIntersectsPath` sees only classified pointers present in the
|
||||
collected paths map. Inheritance's `pathIsSecret` does not require membership.
|
||||
Relations use textual spelling, not document query-shape validation.
|
||||
4. Differentially compare final sets to the old fixed-point algorithm for
|
||||
forward/reverse chains, cycles, duplicate edges, classified ancestors/
|
||||
descendants, classified pointers absent from paths, root, escapes/numeric
|
||||
spellings, and mixed ordinary/inheritance graphs. Use no real credentials.
|
||||
|
||||
Validate: focused propagation tests and the everyday check. Match/output-sized
|
||||
work may still be large for broad dependencies; no blanket linear guarantee.
|
||||
|
||||
## Stage 11: Integrate worklist sensitivity into publication — Complete
|
||||
|
||||
Roadmap candidate 6, integration. Owner: engine `secret.go`.
|
||||
|
||||
1. Feed collected paths, seeds, dependencies, and inheritance links into Stage
|
||||
10's closure. Remove the production fixed-point loop/unused scan helper after
|
||||
migration; retain the useful reference only in tests.
|
||||
2. Verify full-load effective rendering and explanations for mixed derived/
|
||||
inherited sensitivity, replaced/removed paths, tagged containers, consumed
|
||||
controls, and nested copies. All credentials remain outside trees/indexes.
|
||||
3. Verify independent loads and repeated queries retain no mutable propagation
|
||||
state. Queues and trigger indexes are phase-local and discarded on return.
|
||||
|
||||
Validate: `go test ./internal/engine -run 'Secret|Sensitivity|Inheritance|History|Explain|Independent'`,
|
||||
relevant root facade secret/inheritance tests, engine race tests, and the everyday
|
||||
check. Record collection and propagation outcomes separately.
|
||||
|
||||
## Stage 12: Index CLI overlaps and validation reads — Complete
|
||||
|
||||
Roadmap candidate 4. Owner: engine `override.go`.
|
||||
|
||||
1. Retain input order and the duplicate seen map. Replace the prior-accepted
|
||||
overlap scan with earliest intersecting trie lookup using original override
|
||||
indices. Duplicate diagnostics continue preceding overlap checks.
|
||||
2. Insert exactly where `pathAccepted[index] = true` occurs today: after path,
|
||||
overlap, version, and directive checks, before value conversion/budgets.
|
||||
A later bad value still influences subsequent overlap diagnostics. Do not
|
||||
index paths rejected earlier or omit those accepted before value failure.
|
||||
3. Reuse an operation-local mapping index for read-only base traversal during
|
||||
path validation. Preserve missing mapping ancestors, preparation-before-apply,
|
||||
CLI source order/identity, rules, and limits.
|
||||
4. Compare with baseline for disjoint/duplicate/ancestor/descendant paths,
|
||||
multiple conflicts, escapes, rejected paths, and later value failures.
|
||||
Protect diagnostic codes, paths, and first related-source selection.
|
||||
|
||||
Validate: `go test ./internal/engine -run 'Override|CLI|Provenance|Diagnostic'`
|
||||
and the everyday check. No policy changes.
|
||||
|
||||
## Stage 13: Coalesce resolution replay once with indexed ancestors — Complete
|
||||
|
||||
Roadmap candidate 7. Owners: engine `history.go`, `load.go`, `load_result.go`.
|
||||
|
||||
1. Replace group scans with ancestor lookup selecting the earliest inserted
|
||||
matching group, not the deepest. Insert only new groups; merged child paths
|
||||
are not independently indexed as groups.
|
||||
2. Retain original before/after trees and normalization precedence. Use ordered
|
||||
membership sets for reason/dependency unions, retaining first occurrence and
|
||||
canonical pointer equivalence.
|
||||
3. Prepare immutable coalesced descriptors once before publication. Explain
|
||||
reads them without mutating Changes or borrowed snapshots. Retain original
|
||||
snapshots for their other metadata/history-budget uses; node accounting is
|
||||
unchanged. No renderer or public records are cached.
|
||||
4. Differentially check independent/parent/child records, multiple ancestors,
|
||||
root, equal writes, union order, normalization, and repeated detached results.
|
||||
|
||||
Validate: `go test ./internal/engine -run 'Resolution|History|Explain|Ownership'`,
|
||||
engine race tests, and the everyday check. Full replay semantics stay intact.
|
||||
|
||||
## Stage 14: Index relevant inheritance history operations — Complete
|
||||
|
||||
Roadmap candidate 7. Owners: engine metadata, metadata cloning, inheritance history.
|
||||
|
||||
1. During final metadata preparation, index inheritance operation IDs by control
|
||||
path or transfer destination. Collect intersecting IDs for History, deduplicate,
|
||||
restore original operation order, and run existing projection logic.
|
||||
2. Preserve stored-history splitting before resolution and Previous-origin
|
||||
construction. Keep fields/history/path clones independent. A fully frozen
|
||||
ID-only index may be shared in `cloneMetadata`, like immutable evidence;
|
||||
no query builds or updates it. Zero/private unprepared metadata must still
|
||||
produce correct results (use the existing scan if no prepared index exists).
|
||||
3. Leave global `appendInheritanceExplanation` replay intact: non-intersecting
|
||||
operations can affect working state and must not be skipped with this filter.
|
||||
4. Compare indexed history to the scan for multi-scope controls, removed paths,
|
||||
ancestor queries, and query mutations/captured metadata independence.
|
||||
|
||||
Validate: `go test ./internal/engine -run 'Inheritance|Inherited|History|Metadata|Independent'`,
|
||||
engine race tests, and the everyday check.
|
||||
|
||||
## Stage 15: Reuse frozen inspection mapping lookups — Complete
|
||||
|
||||
Roadmap candidate 7. Owners: publication, inspection, document/snapshot/history helpers.
|
||||
|
||||
1. Reuse Stage 7's mapping lookup builder for immutable decoded-input, final,
|
||||
and application trees queried repeatedly by inspection. Populate all retained
|
||||
maps before return, visiting shared node identities once. Retain lookup positions
|
||||
and borrowed immutable nodes, not extra trees or Path copies.
|
||||
2. Thread read-only lookups into applicable document/snapshot and application
|
||||
origin/relevance reads. Preserve validity, shape, bounds, and absence. Read
|
||||
populated maps directly, not lazy entries methods that write on first query.
|
||||
3. Keep plain document lookup on mutable replay trees in this stage. Their
|
||||
existing query-local merge indexes are unchanged; do not reuse a frozen map
|
||||
after cloning, deletion, or entry-position changes. This avoids adding an
|
||||
invalidation protocol just to optimize the selected immutable reads.
|
||||
4. Reuse shape/history/render tests; add only missing repeated/concurrent
|
||||
read-only inspection ownership cases. Do not add a public concurrency promise.
|
||||
Record extra retained-index storage honestly; this is a lookup tradeoff,
|
||||
not a free allocation reduction. Baseline/full-tree replay remains unchanged.
|
||||
|
||||
Validate: `go test ./internal/engine -run 'Path|History|Explain|Effective|Independent|Parallel'`,
|
||||
engine race tests, and the everyday check.
|
||||
|
||||
## Stage 16: Index materialized inheritance-step membership — Complete
|
||||
|
||||
Roadmap candidate 8. Owner: engine `inheritance_origin.go`.
|
||||
|
||||
1. Keep output order and compact linked traces. Use a comparable key with three
|
||||
canonical From/To/Reference strings and one encoded ordered Locations string.
|
||||
Encode the location count and each SourceID byte length, then its bytes and
|
||||
delimited decimal line/column integers. This is collision-free, including
|
||||
empty IDs, delimiter characters, and signed coordinates. Arbitrary unescaped
|
||||
delimiter concatenation is incorrect; no hashing dependency is needed.
|
||||
2. Seed membership from existing Origin.Inheritance but retain its existing
|
||||
duplicates/order. Only newly proposed steps are deduped, as today.
|
||||
3. Format candidate pointers once; clone retained paths/locations only for newly
|
||||
accepted steps. Keep contributor traversal and transfer direction intact.
|
||||
4. Compare chains, shared contributors, different location/order steps, existing
|
||||
duplicates, escaped/numeric paths, and returned-record mutation to baseline.
|
||||
|
||||
Validate: `go test ./internal/engine -run 'Inheritance|Inherited|Metadata|Provenance|History'`
|
||||
and the everyday check. Complex contributor-trace equality stays unchanged.
|
||||
|
||||
## Stage 17: Index simple resolution and origin membership — Complete
|
||||
|
||||
Roadmap candidate 8. Owners: engine `resolution.go` and `load_metadata.go`.
|
||||
|
||||
1. Maintain lazily initialized first-occurrence reason/dependency sets on
|
||||
resolution records. Dependencies use canonical pointer equality; retained
|
||||
slices and callback validation/order/nil behavior remain unchanged.
|
||||
2. Use exact Location membership in `originContainedBy`, retaining Layer/
|
||||
Operation and contributor-empty checks. Use SourceID membership in
|
||||
`originRepresentedByChange`, which has a different source-coverage predicate.
|
||||
Do not conflate these equality rules.
|
||||
3. Leave complex contributor scans and `sameInheritanceTrace` intact. Their
|
||||
concrete segment/location equality cannot be replaced by pointer identity
|
||||
or canonical spelling. Sets are phase-local, not exported record fields.
|
||||
4. Reuse existing provenance tests; cover duplicate unions, equal source IDs
|
||||
with differing locations, and location coverage with contributor mismatch
|
||||
only where protection is missing.
|
||||
|
||||
Validate: `go test ./internal/engine -run 'Resolution|Provenance|Overlay|Override|Inheritance|History'`
|
||||
and the everyday check. Small lists and outputs preserve existing semantics.
|
||||
|
||||
## Stage 18: Review the completed result and record outcomes — Complete
|
||||
|
||||
1. Review the diff from baseline. Map all nine candidates to selected implemented
|
||||
changes and expressly deferred portions. Remove dead helpers/imports/state;
|
||||
keep useful small equivalence tests, no alternate production engines.
|
||||
2. Run `./scripts/check.sh` and `go test -race ./...`. Verify public/import
|
||||
boundaries, examples, independent loads, shapes, presence, history, redaction,
|
||||
and complete-on-return behavior through their existing tests.
|
||||
3. Record algorithmic before/after work, lifetime/storage tradeoffs, revisions,
|
||||
actual commands/results, and unmeasured claims in an evidence subsection here.
|
||||
Optional probes may reuse existing startup/assembly/underlay/inheritance
|
||||
benchmarks and small record/override/dependency/query fixtures. Measurements
|
||||
are not prerequisites; timing claims await a representative host.
|
||||
4. Update stage and roadmap status accurately. Update focused internal docs
|
||||
only where they describe changed implemented mechanics. Keep public contracts
|
||||
and policy language unchanged. Do not retire other historical records or
|
||||
roadmaps as incidental cleanup.
|
||||
5. Review links/whitespace/staging, commit the completed unit, and report outcomes,
|
||||
checks, remaining deferred experiments, and workspace status. No release or
|
||||
downstream bump is included.
|
||||
|
||||
Done: selected changes preserve behavior, remain proportionate/maintainable,
|
||||
pass validation, and distinguish algorithmic expectations from measured results.
|
||||
No public performance guarantee is created.
|
||||
|
||||
## Open Questions
|
||||
|
||||
None required for the selected scope. Algorithm, ownership, relation, ordering,
|
||||
and scope choices above are resolved. Expressly deferred experiments require
|
||||
separate design/evidence and do not block this sequence.
|
||||
|
||||
### Final evidence and outcomes
|
||||
|
||||
- Post-review Stage 9 completion: sensitivity collection now reuses existing
|
||||
valid parsed paths without reparsing; later projections still replace typed
|
||||
history paths. Repeated parsed contributions and both numeric mapping/sequence
|
||||
transitions are covered. Focused sensitivity/inheritance/rendering tests,
|
||||
`./scripts/check.sh`, and `go test -race ./internal/engine` passed.
|
||||
|
||||
- Reviewed planning baseline `0c77deed05417edb32f42c7bf24df2fdc2581a25`,
|
||||
production baseline `48619de024d794cf41e90b171b7fcbe609dcd437`, and the
|
||||
complete implementation diff through `799ebab5a6f15e8252474b8cb7ea164ef503f390`.
|
||||
The selected changes map to the audit candidates as follows: candidate 1 in
|
||||
Stage 2; candidate 2 in Stages 4–5; candidate 3 in Stages 6–8; candidate 4
|
||||
in Stage 12; candidate 5 in Stage 9; candidate 6 in Stages 10–11; candidate 7
|
||||
in Stages 13–15; candidate 8 in Stages 16–17; and candidate 9 in Stage 3.
|
||||
- `GOTOOLCHAIN=local go version` reports `go1.26.8 linux/amd64`.
|
||||
`go test ./internal/engine -run 'Resolution|Provenance|Overlay|Override|Inheritance|History'`,
|
||||
`./scripts/check.sh`, and `go test -race ./...` passed. The development check
|
||||
includes formatting, whitespace, all-package tests, vet, and build. No
|
||||
environmental limitations were encountered.
|
||||
- The selected algorithms replace repeated pairwise/list scans or repeated
|
||||
formatting with token comparisons, local prefix indexes, ordered membership
|
||||
maps, direct visitors, and a worklist. Operation-local indexes and propagation
|
||||
state are discarded after use. Inspection retains immutable node-to-mapping
|
||||
position lookups and prepared replay descriptors for repeated read-only
|
||||
queries; this trades additional retained index storage for reuse. Per-record
|
||||
reason/dependency sets and materialized-step keys likewise use memory in
|
||||
proportion to retained unique values while preserving ordered output slices.
|
||||
- Whole-tree explanation replay, public metadata-copy ownership redesign, global
|
||||
schema caching, and complex contributor-trace membership indexing remain
|
||||
deferred. Existing cloning and linked traces remain. No benchmark campaign
|
||||
was run; expected work reductions are algorithmic, not measured runtime
|
||||
claims, and no public performance guarantee is added.
|
||||
@@ -1,311 +0,0 @@
|
||||
# Performance Audit
|
||||
|
||||
## Scope and assessment
|
||||
|
||||
Code-based review of ConfigKit at
|
||||
`48619de024d794cf41e90b171b7fcbe609dcd437`, following the completed
|
||||
[startup experiment](startup.md) and its [measurement record](startup-benchmark.md).
|
||||
This audit covers the public facade and the loading, acquisition, composition,
|
||||
inheritance, decoding, resolution, metadata, sensitivity, and inspection paths.
|
||||
It identifies optimization candidates, not correctness findings or approved
|
||||
changes to public behavior. No production changes or new benchmark measurements
|
||||
were made for this audit, as requested. The host is currently busy; allocation
|
||||
evidence will be more useful than short timing comparisons in follow-up work.
|
||||
|
||||
The core composition algorithms already use mapping indexes, and schema decoding
|
||||
already uses field-name indexes. The strongest remaining scaling opportunities
|
||||
are auxiliary resolution, overlap, and sensitivity operations rather than the
|
||||
recursive merge itself. Several smaller changes could also remove unnecessary
|
||||
provenance work and repeated pointer formatting.
|
||||
|
||||
Complexity below describes inspected algorithms, not measured runtime. Let
|
||||
`N` be the number of relevant tree paths, `R` the number of resolution records,
|
||||
`D` the number of changed subtrees, `K` the number of overrides, `E` the number
|
||||
of sensitivity dependency edges, `S` the number of classified secret paths,
|
||||
and `L` the path depth/text size. Distinguish input size from output size:
|
||||
expanded trees, historical events, inheritance steps, and rendered YAML can
|
||||
themselves grow substantially. An output-sensitive algorithm cannot avoid
|
||||
producing the output required by the existing contract. Expected map lookup
|
||||
costs and path processing costs also qualify any proposed linear bound.
|
||||
|
||||
## Target state and selected scope
|
||||
|
||||
The implementation target is the same public configuration model with less
|
||||
avoidable auxiliary work: presence-only queries do not materialize provenance;
|
||||
path comparisons and sorts avoid repeated formatting; resolution coverage,
|
||||
override overlaps, and sensitivity propagation use local path indexes;
|
||||
classification collects paths directly; and frozen inspection reuses prepared
|
||||
resolution/inheritance lookup data. Inheritance detection does not construct
|
||||
semantic paths merely to detect a keyword. Ordered provenance collections use
|
||||
membership indexes where their equality rules are straightforward.
|
||||
|
||||
The [implementation plan](implementation.md) selects bounded revisions from
|
||||
all nine candidates below. Whole-tree explanation replay remains in place;
|
||||
branch-local replay, changing the public metadata-copy handoff, global schema
|
||||
caching, and indexing complex contributor trace equality remain later
|
||||
experiments. These have larger ownership, memory, or equivalence risks than the
|
||||
selected changes. No new consumer-facing complexity bound or performance
|
||||
threshold is part of the target. The candidate descriptions retain the broader
|
||||
audit context; the plan owns the selected implementation details and sequence.
|
||||
|
||||
## Candidates
|
||||
|
||||
### 1. Avoid materializing provenance for presence-only queries
|
||||
|
||||
**Priority: high; small, isolated change.**
|
||||
|
||||
Owner: [metadata.go](../../internal/engine/metadata.go), `Metadata.IsExplicit`
|
||||
and `Metadata.Lookup`.
|
||||
|
||||
`IsExplicit` calls `Lookup`, which clones `FieldInfo` and its origin and
|
||||
materializes inheritance steps before returning the boolean. This adds work
|
||||
proportional to provenance size to a query that only needs effective-path
|
||||
validation and the stored `Explicit` field. Inherited values can make that
|
||||
extra work particularly expensive.
|
||||
|
||||
Use the same effective-path/shape validation as `Lookup`, followed by a direct
|
||||
read of the private field's boolean. Keep `Lookup` detached and complete.
|
||||
This removes record/trace construction rather than altering tree composition.
|
||||
Preserve zero metadata, root, invalid/unknown paths, contextual numeric tokens,
|
||||
explicit key/index distinctions, and inherited-input presence. Existing
|
||||
presence and metadata tests should cover most of the risk.
|
||||
|
||||
### 2. Reduce allocation in path comparisons and sorting
|
||||
|
||||
**Priority: high; shared helper changes need careful semantic coverage.**
|
||||
|
||||
Owners: [resolution.go](../../internal/engine/resolution.go), `pathContains`,
|
||||
`appendUniquePath`, and `orderedResolutionRecords`;
|
||||
[metadata.go](../../internal/engine/metadata.go), `matchesPathShape` and `Lookup`;
|
||||
[overlay.go](../../internal/engine/overlay.go), `displacedOrigins`;
|
||||
[override_apply.go](../../internal/engine/override_apply.go),
|
||||
`affectedOverrideOrigins`; [document_compare.go](../../internal/engine/document_compare.go)
|
||||
and [diagnostic.go](../../internal/engine/diagnostic.go), sorting comparators.
|
||||
|
||||
`pathContains` constructs canonical pointer strings multiple times per test.
|
||||
Many nested scans below repeatedly invoke it. Sorting comparators also repeatedly
|
||||
format paths, making formatting/allocation work proportional to comparisons
|
||||
rather than items. `Lookup` formats the same query for multiple lookups.
|
||||
|
||||
Reusing an already formatted pointer within a function and decorating sortable
|
||||
items with a pointer once avoid repeated formatting. Allocation-free containment
|
||||
comparison over token spellings is also in the selected scope. Cost remains proportional
|
||||
to path length, but repeated string allocations can disappear.
|
||||
|
||||
Containment currently compares canonical pointer spellings, while metadata
|
||||
shape matching additionally distinguishes explicit key/index shapes. A raw
|
||||
segment-struct equality replacement would change behavior: parsed tokens and
|
||||
explicit numeric keys/indices can spell the same pointer. Preserve each helper's
|
||||
existing equivalence relation, invalid-path handling, root behavior, slash/tilde
|
||||
escaping, empty keys, and lexical sort order. Precomputation must retain original
|
||||
tie-breaking and must not introduce shared mutable path storage.
|
||||
|
||||
### 3. Index resolution coverage and affected subtrees
|
||||
|
||||
**Priority: high; larger, output-sensitive algorithm change.**
|
||||
|
||||
Owners: [resolution.go](../../internal/engine/resolution.go), `runResolution`
|
||||
and `coveringResolutionRecord`;
|
||||
[load_metadata.go](../../internal/engine/load_metadata.go),
|
||||
`applyResolutionMetadata`, `appendResolutionDescendantHistory`,
|
||||
`coveringResolutionSnapshot`, and `refreshResolutionFields`.
|
||||
|
||||
Each delta scans all records to find its most-specific covering record:
|
||||
approximately `O(D * R * L)`. Metadata updates then scan and parse all field,
|
||||
historical, or final-tree pointers for each delta: approximately
|
||||
`O(D * N * L)`, even for disjoint single-leaf changes with small output.
|
||||
Final-tree indexes are also rebuilt in both descendant-history and field-refresh
|
||||
processing. Newly added fields perform further coverage scans.
|
||||
|
||||
Build an operation-local path-prefix index for covering records and changed
|
||||
subtrees. Walk relevant field/history/final paths once, finding covering deltas
|
||||
and records through that index. Reuse a final-tree index within this operation
|
||||
where its ownership and lifetime permit. Target work proportional to indexed
|
||||
path material plus the actual emitted history, instead of every delta times
|
||||
every field. A segment trie or a suitably ordered prefix traversal is a candidate,
|
||||
not a reason to introduce a general indexing framework.
|
||||
|
||||
Preserve most-specific record selection and equal-depth ties, previous origins,
|
||||
removed descendants, newly introduced fields, null state, and deterministic
|
||||
history ordering. Ancestor operations may legitimately emit many descendant
|
||||
events. Do not collapse those events to claim an unconditional `O(N)` bound.
|
||||
|
||||
### 4. Replace pairwise override overlap checks
|
||||
|
||||
**Priority: medium; clear quadratic case, bounded current input size.**
|
||||
|
||||
Owner: [override.go](../../internal/engine/override.go), `prepareOverrides`,
|
||||
`overlappingOverride`, and `validateOverridePath`.
|
||||
|
||||
For `K` accepted, disjoint overrides, overlap checking scans all prior accepted
|
||||
paths: `O(K² * L)`. Validation also repeatedly linearly searches ordered mapping
|
||||
entries, independently of the index used later when applying overrides.
|
||||
|
||||
An incremental prefix index can detect prior ancestors and descendants in
|
||||
approximately total path-processing work. Retain enough insertion-order
|
||||
information to report the same first conflicting accepted override. Reuse a
|
||||
read-only mapping lookup index during validation if wide mappings make that
|
||||
worthwhile. The existing override/source limit bounds the exposure, so this is
|
||||
less urgent than broadly exercised metadata work; retain the limit unchanged.
|
||||
|
||||
Preserve duplicate-versus-overlap diagnostics, accepted-path bookkeeping,
|
||||
original-record order, related source locations, mapping-only traversal, and
|
||||
schema-version/directive handling. Sorting input records before validation
|
||||
would not automatically preserve these semantics.
|
||||
|
||||
### 5. Remove temporary sensitivity indexes and redundant parsing
|
||||
|
||||
**Priority: high; promising narrowly scoped allocation experiment.**
|
||||
|
||||
Owner: [secret.go](../../internal/engine/secret.go), `classifySecretPaths`
|
||||
and `addSecretPath`; entry point:
|
||||
[load_result.go](../../internal/engine/load_result.go), `publishLoadResult`.
|
||||
|
||||
Classification builds temporary pointer-to-node maps for final input,
|
||||
applications, and projections just to enumerate paths or read `Sensitive`.
|
||||
It indexes the final tree initially and again among projections. Paths then
|
||||
pass through `ParsePath`, including repeated contributions to an existing key.
|
||||
These are linear passes, but allocate avoidable intermediate maps and paths.
|
||||
|
||||
Walk trees directly into the local classification accumulator, formatting a
|
||||
pointer once per visit and recording sensitivity in that visitor. Avoid
|
||||
reparsing already known paths when equivalent behavior can be retained.
|
||||
Evaluate same-tree pass reuse locally rather than unifying metadata indexes.
|
||||
|
||||
Historical paths, removed/replaced secrets, static-default and dynamic-value
|
||||
tags, inheritance contributors, and dependency-only paths remain necessary.
|
||||
Reusing only the effective index would be incomplete. Preserve contextual path
|
||||
handling and detached predicate arguments; the current pointer-keyed accumulator
|
||||
can receive the same spelling through different historical shapes. Check that
|
||||
any typed-path reuse preserves classification for those cases.
|
||||
|
||||
### 6. Use indexed, incremental sensitivity propagation
|
||||
|
||||
**Priority: high scaling opportunity; higher semantic risk than candidate 5.**
|
||||
|
||||
Owner: [secret.go](../../internal/engine/secret.go), the fixed-point loop in
|
||||
`classifySecretPaths` and `secretIntersectsPath`.
|
||||
|
||||
Every pass scans all dependency/inheritance edges. Each ordinary dependency
|
||||
test can scan all classified secrets and compare paths. With `P` propagation
|
||||
passes, the dependency portion can approach `O(P * E * S * L)`; unfavorable
|
||||
edge order can require many passes down a chain. Bidirectional inheritance
|
||||
links add repeated work as well. The finite classified-path set makes the
|
||||
procedure terminate, but does not make these repeated scans cheap.
|
||||
|
||||
Consider a worklist of newly sensitive paths with reverse dependency adjacency
|
||||
and a prefix-aware intersection index. Process newly activated relationships
|
||||
instead of rescanning already satisfied edges. Aim for output-sensitive work
|
||||
near the cost of path indexing, edges, and actual matches; broad ancestor
|
||||
dependencies can inherently intersect many paths, so a blanket `O(N + E)`
|
||||
promise would be unjustified.
|
||||
|
||||
Preserve ordinary dependency direction, ancestor/descendant intersection,
|
||||
inheritance's bidirectional propagation, cycles, historical-only paths, and
|
||||
container redaction. Compare the resulting sensitive set and safe output, not
|
||||
private iteration counts. Keep this separate from the simpler traversal change.
|
||||
|
||||
### 7. Reduce repeated global work in inspection
|
||||
|
||||
**Priority: medium; most valuable when consumers inspect many paths.**
|
||||
|
||||
Owners: [explain.go](../../internal/engine/explain.go), `Inspection.Explain`;
|
||||
[history.go](../../internal/engine/history.go), `coalescedResolutionSnapshots`;
|
||||
[inheritance_history.go](../../internal/engine/inheritance_history.go),
|
||||
`historyWithInheritance`; [document_walk.go](../../internal/engine/document_walk.go),
|
||||
`documentAtPath`; [snapshot.go](../../internal/engine/snapshot.go).
|
||||
|
||||
Each explanation clones whole baseline and decoded-input trees, replays all
|
||||
applications, and coalesces the same resolution snapshots. Coalescing scans
|
||||
previous groups for each snapshot and is `O(R² * L)` for independent records.
|
||||
Each historical query also scans every inheritance operation. Document lookup
|
||||
linearly scans each mapping on its path; querying every key in a wide mapping
|
||||
can therefore add quadratic lookup work. These costs compound across queries,
|
||||
even if each query asks about a small unrelated leaf.
|
||||
|
||||
The selected target prepares immutable coalesced replay descriptors once when an
|
||||
inspection is constructed and indexes relevant inheritance history operations
|
||||
and frozen-tree lookups. Coalescing uses ancestor lookup rather than a group
|
||||
scan. Replaying only the queried branch and its necessary dependencies remains
|
||||
a deferred experiment; whole-tree replay remains in the selected target.
|
||||
|
||||
Preserve the first covering coalesced group, ordered reason/dependency unions,
|
||||
normalization precedence, chronology, intermediate absence/replacement, typed
|
||||
reconciliation, destination-local inheritance, and historical redaction.
|
||||
Simply skipping applications marked irrelevant is not a proven safe algorithm:
|
||||
their contributions can affect later inheritance/replay. Avoid eagerly retaining
|
||||
every rendered explanation or tree-per-field; that exchanges query CPU for
|
||||
potentially substantial startup and retained-memory costs. No new public batch
|
||||
API or cache lifecycle is needed to investigate these internal opportunities.
|
||||
|
||||
### 8. Replace growing-list provenance deduplication where worthwhile
|
||||
|
||||
**Priority: medium; important for deep or contribution-heavy inheritance.**
|
||||
|
||||
Owners: [inheritance_origin.go](../../internal/engine/inheritance_origin.go),
|
||||
`projectInheritanceSteps`; [load_metadata.go](../../internal/engine/load_metadata.go),
|
||||
`originContainedBy` and `originRepresentedByChange`;
|
||||
[resolution.go](../../internal/engine/resolution.go), `appendUniquePath`
|
||||
and `appendUniqueString`.
|
||||
|
||||
Step materialization tests each proposed step against all steps already emitted,
|
||||
reformatting multiple paths in each comparison. A chain of `T` distinct steps
|
||||
can require `O(T²)` comparisons, before path/location costs. Origin containment
|
||||
has pairwise location and contributor scans; repeated insertion into reason
|
||||
and dependency lists also scans the accumulated list.
|
||||
|
||||
Use local membership indexes alongside the ordered output slices when their
|
||||
cardinality justifies the additional machinery. Keys must represent the actual
|
||||
equality relation, including locations and trace structure where applicable;
|
||||
hashes alone are insufficient without collision-safe equality. Preserve first
|
||||
occurrence order, duplicate behavior, and detached returned records. Keep tiny
|
||||
lists straightforward; do not impose global interners or flatten the existing
|
||||
compact shared inheritance traces into eager per-value step lists.
|
||||
|
||||
### 9. Reduce path construction in inheritance detection
|
||||
|
||||
**Priority: medium; small common-path allocation candidate.**
|
||||
|
||||
Owner: [inheritance_load.go](../../internal/engine/inheritance_load.go),
|
||||
`containsInheritanceDirective`.
|
||||
|
||||
The presence check uses a full `walkDocument` traversal that creates paths for
|
||||
all descendants, and keeps traversing after finding `extends`. It only needs
|
||||
to detect a mapping key. A direct boolean tree visitor could inspect keys and
|
||||
sequence children without constructing paths and return immediately on a match.
|
||||
The no-inheritance case still requires a linear scan, but with less allocation;
|
||||
early matches can avoid the remainder of this detection pass.
|
||||
|
||||
Detection remains separate from validation/expansion. It must still find keys
|
||||
inside sequences and nested mappings; invalid inheritance syntax must reach
|
||||
the existing diagnostics rather than being skipped. Do not infer permission
|
||||
to change scope rules, traversal ordering during expansion, or resource budgets.
|
||||
|
||||
## Reviewed areas without a priority algorithm replacement
|
||||
|
||||
| Area | Assessment |
|
||||
| --- | --- |
|
||||
| Strict composition and recursive precedence (`compose.go`, `recursive_merge.go`, `mapping_index.go`, `load_request.go`) | Ordered entries already have lazily built per-node lookup maps, avoiding pairwise wide-map merge scans. Acquisition and tier assembly retain source/history evidence for distinct purposes. No wholesale merge rewrite recommended. |
|
||||
| YAML parsing/writing (`internal/yaml`) and mapping-value construction (`value.go`) | Translation uses a duplicate-key map and presized containers. Adapter trees add linear allocation but protect the parser boundary. No parser replacement or custom serializer justified by this review. |
|
||||
| Sources, imports, globs, filesystems, and resource contexts | Import identity/cycle lookup uses maps; directory enumeration is batched when supported. Glob traversal cost follows visited entries and matches, with deterministic sorting. Repeated directory/canonicalization work may merit future workload evidence, but caching could change acquisition observations and accounting. Preserve handle closure and passive returned contexts. |
|
||||
| Schema, typed decode, projection, and typed cloning | Schema descriptors cache types within a load and index struct fields. Reflection and multiple linear projections/clones have costs, but provide defaults snapshots, codec validation, ownership, and callback auditing. Avoid removing these boundaries speculatively. A global schema cache is not the first recommendation. |
|
||||
| Structural inheritance expansion | Siblings are indexed by name and resolved using dependency state rather than repeatedly searching for parents. Expanded output, destination-local processing, provenance, and retained-history accounting explain substantial work. Deep nested scopes may revisit subtrees, but this audit does not establish a safe general replacement for those passes. |
|
||||
| Secrets resolution and explicit directory bootstrap | Environment lookups are already deduplicated within a load. Directory work is bounded, sorted, and staged before application. The classification candidates above do not call for changing bootstrap preflight or process-environment behavior. |
|
||||
| Public facade and detached records | Root-owned wrappers and explicit conversions preserve the public/private boundary. Some records are copied in the engine and again during conversion; eliminating a copy needs a clear internal ownership handoff, not public aliases or borrowed mutable records. `Result.Metadata()` also clones the complete metadata before queries; consumers repeatedly obtaining it pay repeated full-copy costs. Treat redesign of that private handoff as a separate, lower-priority retained-memory/ownership experiment. |
|
||||
| Effective YAML, diagnostics, examples, and release/development tooling | Deterministic rendering and diagnostics require ordering and output-sized work; path comparator allocation is covered above. Safe snapshots and YAML adaptation allocate separate trees. Maintained consumer examples and check tooling are not production library hot paths; no gate or tooling changes proposed. ReleaseKit internals are outside this repository audit. |
|
||||
|
||||
## Acceptance and boundaries
|
||||
|
||||
The selected revisions preserve the existing [configuration contract](../config.md)
|
||||
and [architecture policy](../policy/architecture.md): complete load results,
|
||||
effective versus historical path shapes, provenance, ownership, presence,
|
||||
secret isolation, and synchronous completion remain unchanged. New caps,
|
||||
input restrictions, composition rules, or performance promises are not part of
|
||||
this work. The plan owns implementation sequencing, validation, and evidence.
|
||||
|
||||
Success is removal of the identified redundant work and broad pairwise scans
|
||||
without disproportionate machinery or semantic regressions. Output-sized work,
|
||||
deterministic ordering, and required ownership copies remain legitimate costs.
|
||||
Benchmarks may inform the tradeoff, especially allocations on the busy host,
|
||||
but are not new release gates or test thresholds. Unmeasured improvements
|
||||
remain algorithmic expectations rather than quantified runtime claims.
|
||||
@@ -1,176 +0,0 @@
|
||||
# PromptKit Request: More Efficient Configuration Construction
|
||||
|
||||
Status: Submitted for ConfigKit's consideration; no upstream design or delivery
|
||||
commitment is assumed.
|
||||
|
||||
Date: 2026-10-08.
|
||||
|
||||
## Consumer Need
|
||||
|
||||
PromptKit uses ConfigKit to assemble configuration for an importable Go LLM
|
||||
prompting library. Catalog and consumer sources supply backend, profile, and
|
||||
versioned prompt definitions. ConfigKit owns source composition, inheritance,
|
||||
typed decoding, resolution, and final configuration publication. PromptKit
|
||||
interprets the final configuration and captures execution-ready definitions.
|
||||
|
||||
We request consideration of lower CPU and allocation costs for this existing
|
||||
loading workflow, particularly with hundreds of small definitions. The goal is
|
||||
to benefit consumers through the existing public API, while preserving the
|
||||
configuration semantics and inspection capabilities they rely on. Acceptance
|
||||
and the implementation approach remain ConfigKit decisions.
|
||||
|
||||
## Why This Matters
|
||||
|
||||
PromptKit ordinarily constructs an engine once and reuses its captured
|
||||
configuration across many calls. Construction is therefore startup work, not a
|
||||
cost repeated for each prompt. Current indexed prompt/profile selection is
|
||||
already fast, and this request does not concern upstream inference latency.
|
||||
|
||||
Nevertheless, cumulative allocation during construction can matter for large
|
||||
catalogs, applications creating multiple engines, and repeated configuration
|
||||
loads. Reducing that work can improve startup and reduce garbage-collection
|
||||
pressure without requiring every downstream library to own a specialized
|
||||
configuration loader.
|
||||
|
||||
PromptKit retains ConfigKit's inspection data for explicit operator APIs.
|
||||
Ordinary prompt execution uses captured definitions rather than analyzing
|
||||
configuration provenance. Inspection remains useful even though it is not on
|
||||
the normal execution path.
|
||||
|
||||
## Observed Workload
|
||||
|
||||
PromptKit's committed `internal/configuration/benchmark_test.go` contains
|
||||
`BenchmarkConfigurationConstruction`. Its `independent` fixture has:
|
||||
|
||||
- 128 independently named profiles, each with an endpoint and model;
|
||||
- 128 independently named prompts, each with version `1`;
|
||||
- one message per prompt, containing the small template `Hello {{.name}}`;
|
||||
- a default-profile reference and a text/no-validation output declaration per
|
||||
prompt; and
|
||||
- one in-memory mapping source, with no underlays, imports, inheritance,
|
||||
external file reads, credentials, or model calls.
|
||||
|
||||
The benchmark reports 256 input and effective entries: 128 profiles plus 128
|
||||
prompt versions. This is a definition count, not a count of all configuration
|
||||
fields or metadata paths.
|
||||
|
||||
Each timed iteration calls PromptKit's real configuration adapter. That adapter
|
||||
uses `ConfigKit.LoadLayered` with resolution and validation callbacks, then
|
||||
captures prompt content, compiled templates, hashes, and execution targets.
|
||||
Input fixture construction occurs outside the timed iteration. Consequently,
|
||||
the measurements include downstream adapter work and are not a standalone
|
||||
ConfigKit benchmark.
|
||||
|
||||
Other committed fixtures cover file-backed messages, shared-base profile
|
||||
inheritance, and four-tier layering. They provide additional downstream cases
|
||||
for assessing proposed improvements; the principal evidence below uses the
|
||||
simple independent fixture.
|
||||
|
||||
## Measurements And Reproduction
|
||||
|
||||
Original PromptKit review baseline:
|
||||
`de6443f3aef4a66169e323e9f9a5f41594fbface`.
|
||||
|
||||
The request preparation repeated the benchmark at PromptKit revision
|
||||
`eb830bb61346672f0f6c5bce6119b76ccc6d61ac`. The benchmark, configuration adapter,
|
||||
and `go.mod` are unchanged between these revisions.
|
||||
|
||||
Reproduction environment:
|
||||
|
||||
- ConfigKit: `gitea.maximumdirect.net/eric/ConfigKit v0.5.0`.
|
||||
- Toolchain: `go1.26.8`, `linux/amd64`.
|
||||
- Processor: Intel Core i5-7260U CPU at 2.20 GHz.
|
||||
- Benchmark process parallelism: 4, as reported by the benchmark suffix.
|
||||
- Workspace mode disabled; installed toolchain and already-acquired module
|
||||
dependencies used without downloads.
|
||||
|
||||
From a PromptKit checkout at either recorded revision:
|
||||
|
||||
```sh
|
||||
GOTOOLCHAIN=local GOWORK=off GOPROXY=off \
|
||||
go test -mod=readonly -run '^$' \
|
||||
-bench '^BenchmarkConfigurationConstruction/independent$' \
|
||||
-benchmem -benchtime=500ms -count=5 ./internal/configuration
|
||||
```
|
||||
|
||||
Repeated results:
|
||||
|
||||
| Run | Time per construction (ms) | Allocated bytes per construction | Allocations per construction |
|
||||
| --- | ---: | ---: | ---: |
|
||||
| 1 | 71.052 | 50,128,254 | 538,582 |
|
||||
| 2 | 67.740 | 50,139,198 | 538,539 |
|
||||
| 3 | 66.353 | 50,174,030 | 538,541 |
|
||||
| 4 | 67.170 | 50,159,959 | 538,542 |
|
||||
| 5 | 68.209 | 50,167,214 | 538,541 |
|
||||
|
||||
These agree with the original review's approximate 69 ms, 50 MB, and 539,000
|
||||
allocations. They measure cumulative allocation per operation, not retained
|
||||
heap or a memory leak. Timing depends on the machine and runtime environment.
|
||||
|
||||
### Allocation And CPU Profile Observations
|
||||
|
||||
A separate two-second profiled run reproduced approximately 68.7 ms,
|
||||
50.1 MB, and 538,553 allocations per construction. To generate comparable
|
||||
profiles from the same PromptKit checkout, placing generated artifacts outside
|
||||
the repository:
|
||||
|
||||
```sh
|
||||
profile_dir=$(mktemp -d)
|
||||
GOTOOLCHAIN=local GOWORK=off GOPROXY=off \
|
||||
go test -mod=readonly -run '^$' \
|
||||
-bench '^BenchmarkConfigurationConstruction/independent$' \
|
||||
-benchtime=2s -cpuprofile="$profile_dir/config.cpu" \
|
||||
-memprofile="$profile_dir/config.mem" -o "$profile_dir/config.test" \
|
||||
./internal/configuration
|
||||
GOTOOLCHAIN=local go tool pprof -top -alloc_space \
|
||||
"$profile_dir/config.test" "$profile_dir/config.mem"
|
||||
GOTOOLCHAIN=local go tool pprof -top \
|
||||
"$profile_dir/config.test" "$profile_dir/config.cpu"
|
||||
```
|
||||
|
||||
The sampled allocation profile prominently attributes work to ConfigKit's
|
||||
path and metadata machinery. Selected flat allocation shares were:
|
||||
|
||||
- `clonePathSegments`: approximately 29.9%.
|
||||
- `ParsePath`: approximately 4.7%.
|
||||
- `collectFieldInfo`: approximately 4.3%.
|
||||
- `collectOrigins`: approximately 3.7%.
|
||||
- `effectivePathIndex.func1`: approximately 3.7%.
|
||||
|
||||
The CPU profile also showed substantial allocation/garbage-collection work and
|
||||
path formatting (`Path.String`, approximately 17.4% cumulative CPU samples).
|
||||
These symbol names are profiling evidence, not recommendations for ConfigKit's
|
||||
internal implementation. Flat allocation and cumulative CPU shares describe
|
||||
different measures and should not be added together.
|
||||
|
||||
The profile supports investigating ConfigKit as a major contributor; it does
|
||||
not attribute every byte or millisecond of this assembled benchmark to
|
||||
ConfigKit. A single fixture size also does not establish an asymptotic scaling
|
||||
defect. Repeated measurements across representative sizes would help assess
|
||||
scaling and any proposed improvement.
|
||||
|
||||
## Requested Outcome
|
||||
|
||||
Consider reducing avoidable construction-time work and temporary allocations
|
||||
for this workload while retaining the capabilities defined by ConfigKit's
|
||||
[configuration contract](../config.md), including:
|
||||
|
||||
- layered sources and sibling inheritance;
|
||||
- typed decoding and recorded application resolution;
|
||||
- effective presence, explicit null, and resource/source context;
|
||||
- provenance, history, and safe operator inspection; and
|
||||
- established result/query ownership and secret-handling behavior.
|
||||
|
||||
An improvement through the existing API would be preferable for PromptKit.
|
||||
The request does not prescribe a cache, data structure, new public API, or
|
||||
alternative metadata mode. Any optional API change can be assessed against
|
||||
the downstream need if ConfigKit determines that one is justified.
|
||||
|
||||
Useful evidence of success would be repeatable reductions in CPU or allocation
|
||||
cost for the recorded fixture and representative related workloads, with
|
||||
existing semantics intact. No machine-specific threshold, fixed allocation
|
||||
budget, mandatory complexity guarantee, or additional release gate is requested.
|
||||
|
||||
This request can be considered independently of PromptKit's local runtime
|
||||
optimizations. PromptKit will continue to use ConfigKit's final configuration
|
||||
handoff and can evaluate adoption of an upstream improvement when available.
|
||||
@@ -1,115 +0,0 @@
|
||||
# Adopt ReleaseKit for source releases
|
||||
|
||||
Status: Complete. The ReleaseKit profile, workflow, and isolated fixture
|
||||
verification are in place. This roadmap uses
|
||||
ReleaseKit v0.2.0 (tagged commit
|
||||
`e030794969fb7787e647a90bbc4eb81bd0610f07`) as the initial adoption
|
||||
baseline. The [ReleaseKit release note](https://gitea.maximumdirect.net/eric/ReleaseKit/src/tag/v0.2.0/docs/releases/v0.2.0.md)
|
||||
summarizes its available capabilities. The staged
|
||||
[implementation plan](implementation.md) owns task order and acceptance checks.
|
||||
|
||||
## Purpose and pre-adoption baseline
|
||||
|
||||
ConfigKit adopted a common release process to replace duplicated local release
|
||||
machinery. Before this migration, [the release procedure](../release.md) used
|
||||
`scripts/release.sh` and `scripts/release-check.sh` to validate a committed
|
||||
candidate, push it to `main`, and publish an annotated tag; dedicated tests
|
||||
exercised those scripts. That workflow selected one offline race-enabled Go
|
||||
suite on a maintained Go 1.25 patch, one full vet pass, and CGO-disabled package
|
||||
builds for Linux amd64 and arm64. The checked-in note and immutable tag were
|
||||
the only declared deliverables. CI and macOS checks did not gate publication;
|
||||
no hosted release page was declared.
|
||||
|
||||
ReleaseKit v0.2.0 offers [profile selections](https://gitea.maximumdirect.net/eric/ReleaseKit/src/tag/v0.2.0/docs/config.md)
|
||||
for that same test, vet, and source-only compilation set. It runs the test
|
||||
suite once, disables test-time vet when a separate full vet pass is selected,
|
||||
and can require an exact effective push URL. Its
|
||||
[publication contract](https://gitea.maximumdirect.net/eric/ReleaseKit/src/tag/v0.2.0/docs/consumers/publish.md)
|
||||
requires the candidate to be on authoritative `main` before validation and
|
||||
pushes only the annotated tag.
|
||||
|
||||
## Target end state
|
||||
|
||||
ConfigKit remains a source-only Go library. Maintainers prepare the version,
|
||||
compatibility assessment, and checked-in note as before, then publish the
|
||||
committed candidate to authoritative `main`. One ReleaseKit `publish`
|
||||
invocation validates that exact candidate and publishes its annotated tag.
|
||||
Confirmed publication of the intended tag completes the release. A failed or
|
||||
uncertain push follows ReleaseKit's bounded, exact-object recovery procedure.
|
||||
If validation fails after the candidate is pushed to `main`, no release tag is
|
||||
published; the corrected candidate requires another commit and validation.
|
||||
No CI wait, consumer installation check, hosted page, or extra human sign-off
|
||||
enters the completion path.
|
||||
|
||||
A small `releasekit.json` profile supplies ConfigKit's product identity,
|
||||
`origin`, the expected authoritative push URL, and the selected shared Go
|
||||
validation: race, full vet, and package compilation for `linux/amd64` and
|
||||
`linux/arm64`. It declares no executable build or delivery. macOS remains
|
||||
best effort. ReleaseKit's other optional checks, including formatting,
|
||||
Markdown, module-distribution, and file-size checks, remain unselected unless
|
||||
a separate decision establishes a distinct need for them.
|
||||
|
||||
Declare `gitea.maximumdirect.net/eric/ReleaseKit/cmd/releasekit` with a
|
||||
`go.mod` `tool` directive and pin its module at v0.2.0, with resolved checksums
|
||||
in `go.sum`. The release-tool build uses that module selection; there is no
|
||||
separate executable-version pin. This adds release-tool dependencies to
|
||||
ConfigKit's module graph without importing ReleaseKit into library code or
|
||||
adding a runtime API. Review the resulting module changes and license
|
||||
treatment under the [OSS policy](../policy/oss.md); tooling use alone does not
|
||||
require an attribution entry.
|
||||
|
||||
At each release preparation, select the newest stable Go series available in
|
||||
the development environment. Record that series' first stable release in
|
||||
ConfigKit's `go.mod` `go` directive (for example, `go 1.26.0` when using Go
|
||||
1.26.8), retaining a higher patch minimum when a dependency or specific
|
||||
correctness fix requires it. This directive is the consumer minimum, not the
|
||||
release toolchain version. Use the latest locally available patch in the
|
||||
selected series to build the pinned ReleaseKit tool and run its validation,
|
||||
provided it meets that minimum and has native race prerequisites available.
|
||||
ReleaseKit sets `GOTOOLCHAIN=local` for its child commands, so the same selected
|
||||
Go executable must remain first on `PATH` for the tool build and validation.
|
||||
Prepare the tool and dependencies before offline candidate validation; any
|
||||
`go.mod` change belongs in the committed candidate. Review a minimum-Go increase
|
||||
as a compatibility change and explain its consumer impact in the release note.
|
||||
Before v1.0.0, an intentional increase requires a minor rather than patch
|
||||
release under ConfigKit's
|
||||
[versioning policy](../policy/release.md#versioning-and-compatibility).
|
||||
|
||||
ReleaseKit v0.2.0 checks the release-note H1 and exact effective push URL; it
|
||||
does not assert the required first-line format or the configured fetch URL.
|
||||
Continue writing notes with the required `# ConfigKit <version>` first line.
|
||||
Per-test skip listings are optional diagnostic output, not a release gate;
|
||||
ReleaseKit's one standard suite still blocks publication on test failure.
|
||||
|
||||
ReleaseKit owns generic candidate guards, test and compile coordination, tag
|
||||
creation, publication, and push recovery. ConfigKit owns version selection,
|
||||
compatibility review, note content, its selected validation and support
|
||||
targets, and a small integration check of its ReleaseKit configuration. The
|
||||
release process adds no ConfigKit public API or runtime use of ReleaseKit.
|
||||
|
||||
## Changes from the baseline
|
||||
|
||||
- Added the minimal profile and declared ReleaseKit v0.2.0 as a tool dependency
|
||||
in `go.mod`, with resolved checksums in `go.sum`. The library has no runtime
|
||||
import of ReleaseKit, and tooling use alone requires no attribution entry.
|
||||
- Updated the [release policy](../policy/release.md) and
|
||||
[maintainer procedure](../release.md) for candidate publication to `main`
|
||||
before ReleaseKit validation, recurring Go version selection and compatibility
|
||||
review, the selected validation set, tag-only publication, ReleaseKit's note
|
||||
and URL guards, and its failure states. Source-only deliverables, supported
|
||||
targets, and the no-CI completion rule remain unchanged.
|
||||
- Updated the [development guide](../development.md) to distinguish everyday
|
||||
development checks from release validation and to document the selected
|
||||
profile test.
|
||||
- Retired the local release scripts and duplicated publication/gate tests.
|
||||
The development-check test and small profile-wiring test remain; generic
|
||||
ReleaseKit publication behavior is not duplicated in ConfigKit's tests.
|
||||
- Completed migration verification exercised the selected profile and release
|
||||
command against isolated candidates and a temporary Git remote, including
|
||||
validation failure, successful tag publication, and wrong-endpoint rejection
|
||||
before validation. It confirmed the selected Go minimum and one invocation
|
||||
of each selected gate, without publishing a real ConfigKit version or
|
||||
requiring a separate CI run.
|
||||
|
||||
Historical tags and release notes remain unchanged. This adoption work does
|
||||
not itself authorize publication of a ConfigKit version.
|
||||
@@ -1,131 +0,0 @@
|
||||
# Performance optimization review findings
|
||||
|
||||
Reviewed `5279330..6c62226838a3658e3802ed2926e9273fbd7c34ea` against the
|
||||
[performance audit roadmap](performance-audit.md) and the
|
||||
[implementation plan](implementation.md). The private prefix indexes, worklist
|
||||
sensitivity closure, frozen inspection lookups, prepared replay, and ordered
|
||||
membership maps broadly match the selected architecture. Whole-tree replay and
|
||||
the expressly deferred ownership/trace changes remain unchanged.
|
||||
|
||||
The review found no reproduced functional regression or unintentionally stronger
|
||||
public contract. All three bounded implementation and test-quality findings
|
||||
below are now addressed. They do not authorize changes to public semantics,
|
||||
dependencies, resource limits, release gates, or credential handling.
|
||||
|
||||
The older, completed ReleaseKit review record remains in Git history at
|
||||
`6c62226:docs/roadmap/remediation.md`.
|
||||
|
||||
## 1. Complete parsed-path reuse in sensitivity collection (P2)
|
||||
|
||||
Status: Complete. Parsed accumulator paths are reused, while typed historical
|
||||
paths still receive the later parsed replacement. Repeated root/escaped/numeric
|
||||
contributions and both numeric mapping/sequence transitions are covered.
|
||||
Focused sensitivity/inheritance/rendering tests, `./scripts/check.sh`, and
|
||||
`go test -race ./internal/engine` passed.
|
||||
|
||||
Affected: [secret.go](../../internal/engine/secret.go), `addSecretPath`
|
||||
(lines 85–89); implementation Stage 9, item 3.
|
||||
|
||||
Temporary node indexes and the redundant initial final-tree walk were removed,
|
||||
but `addSecretPath` still unconditionally calls `ParsePath` and replaces the
|
||||
stored path on every visit. Paths common to multiple source applications,
|
||||
baseline/decoded projections, and the final tree are therefore repeatedly split,
|
||||
unescaped, and allocated even when the accumulator already holds the identical
|
||||
parsed representation. This leaves an expressly selected part of candidate 5
|
||||
unimplemented, despite Stage 9 being marked complete.
|
||||
|
||||
Reuse the existing accumulator path only when it already has the same valid
|
||||
parsed representation. If history or inheritance has installed an explicit
|
||||
key/index path, the later tree/pointer contribution must still replace it with
|
||||
parsed segments as before. Do not skip merely because the pointer key exists,
|
||||
substitute walker-derived typed segments, or change collection phase precedence.
|
||||
|
||||
Add focused behavioral coverage for a repeated parsed contribution and a typed
|
||||
historical contribution subsequently normalized to parsed form, including numeric
|
||||
mapping/sequence transitions. Reuse existing tag/predicate/ownership coverage.
|
||||
No allocation threshold or traversal-count test is needed. Re-run the sensitivity,
|
||||
inheritance, and rendering tests and the everyday development check, then reconcile
|
||||
the completion evidence with the actual implementation.
|
||||
|
||||
## 2. Correct the unindexed-sensitivity regression fixture (P2)
|
||||
|
||||
Status: Complete. The fixture now uses canonical `/untracked` while leaving it
|
||||
unindexed, and explicitly checks that `/copy` is sensitive and `/ordinary` is
|
||||
not. The new assertion failed with the original malformed seed and passed after
|
||||
correction. The closure equivalence test and `./scripts/check.sh` passed.
|
||||
|
||||
Affected: [secret_propagation_test.go](../../internal/engine/secret_propagation_test.go),
|
||||
`TestSecretDependencyClosureMatchesFixedPointScan`, the
|
||||
“unindexed seed only activates inherited relations” case (lines 38–48).
|
||||
|
||||
The seed is `map[string]bool{"untracked": true}`, but the dependency and
|
||||
inheritance paths spell `/untracked`. These are different keys; the seed also
|
||||
violates the canonical-pointer convention used by the new private trie. Both
|
||||
algorithms leave the inherited target unclassified, so their equality does not
|
||||
verify the case's intended behavior. The suite can pass without covering the
|
||||
selected distinction between an unindexed classified path and an invalid seed.
|
||||
|
||||
Use `secretSet(Keys("untracked"))` or the literal `"/untracked"`, while keeping
|
||||
that path absent from the collected paths map. Assert that the inherited target
|
||||
`/copy` becomes sensitive and the ordinary target `/ordinary` does not, in
|
||||
addition to the reference-set comparison. This verifies the existing filtering
|
||||
difference; it does not impose a new public guarantee or require malformed-seed
|
||||
validation in production.
|
||||
|
||||
Review exercised this corrected canonical fixture in a disposable checkout;
|
||||
the current production implementation produces the intended result. The finding
|
||||
is missing regression protection, not an observed sensitivity leak.
|
||||
|
||||
## 3. Finish moving reference-only machinery out of production (P3)
|
||||
|
||||
Status: Complete. Reference containment/deduplication is String-based and
|
||||
test-only. The unused constructor/comparator/path helpers are removed; coverage
|
||||
tests exercise both live record/snapshot adapters, and dependency membership
|
||||
tests exercise the actual resolution record. Required live helpers and immutable
|
||||
inspection evidence are unchanged. Focused tests, `./scripts/check.sh`, and
|
||||
`go test -race ./...` passed; code search found no remaining obsolete symbols.
|
||||
|
||||
Affected: [resolution.go](../../internal/engine/resolution.go),
|
||||
`newResolutionCoverageIndex` (lines 373–382), `appendUniqueString`,
|
||||
`appendUniquePath` (lines 439–453);
|
||||
[path.go](../../internal/engine/path.go), `pathSpellingEqual`;
|
||||
[diagnostic.go](../../internal/engine/diagnostic.go), `compareDiagnostic`.
|
||||
Relevant tests: `resolution_index_test.go` and `path_test.go`.
|
||||
|
||||
After the caller migrations, the slice-based coverage constructor has only a
|
||||
test caller. The old list-deduplication helpers likewise serve only the
|
||||
coalescing reference/test fixtures, and `pathSpellingEqual` exists solely to
|
||||
support that reference-only path helper. The standalone diagnostic comparator
|
||||
wrapper has no callers; production sorting uses `compareDiagnosticWithPaths`.
|
||||
Keeping these in production leaves redundant mechanisms and test scaffolding
|
||||
behind after Stage 18's requested cleanup.
|
||||
|
||||
Move useful reference-only helpers into test files, or make the coverage test
|
||||
exercise a production record/snapshot adapter directly. Keep the old reference
|
||||
deduplication relation independent and String-based where it acts as an oracle.
|
||||
Remove unused production wrappers and helpers once callers are accounted for.
|
||||
Consolidate or retarget the obsolete helper-specific deduplication test toward
|
||||
the actual resolution membership behavior; existing repeated-record coverage
|
||||
may already be sufficient.
|
||||
|
||||
Retain required `clonePaths`, token comparison, both live coverage adapters,
|
||||
immutable inspection evidence, and the legitimate unprepared-metadata history
|
||||
fallback. This is a narrow cleanup, not permission to remove behavior or useful
|
||||
equivalence tests. Re-run affected tests and the everyday development check.
|
||||
|
||||
## Review validation
|
||||
|
||||
- `./scripts/check.sh` passed at the reviewed commit: formatting, whitespace,
|
||||
all-package tests, vet, and build.
|
||||
- `GOTOOLCHAIN=local GOPROXY=off GOWORK=off go test -race ./...` passed.
|
||||
- A fresh, uncached `GOTOOLCHAIN=local GOPROXY=off GOWORK=off go test -count=1 ./...`
|
||||
also passed.
|
||||
- A disposable checkout outside the repository passed
|
||||
`go test -mod=readonly ./internal/engine -run '^TestReview' -count=1` with
|
||||
the corrected canonical unindexed-seed case and 1,500 deterministic generated
|
||||
sensitivity graphs compared with the prior fixed-point procedure.
|
||||
- Reviewed all changed production files and their added/modified test coverage.
|
||||
Root public API, configuration contract, adopted policies, module dependencies,
|
||||
and release profile are unchanged by the implementation commits.
|
||||
- No performance measurements were required or collected. Algorithmic changes
|
||||
and retained-index memory tradeoffs are not quantified runtime guarantees.
|
||||
@@ -1,924 +0,0 @@
|
||||
# Startup construction measurements and focused experiments
|
||||
|
||||
Status: Complete. Both focused optimizations are retained; final consumer
|
||||
verification and handoff are recorded below. Acceptance rests on allocation
|
||||
reductions and behavioral checks; busy-host timing remains inconclusive.
|
||||
|
||||
Measured on 2026-10-09 under the [startup roadmap](startup.md). The
|
||||
[consumer request](promptkit-performance-request.md) owns the submitted workload
|
||||
and measurements. This document records the reproduced baselines, maintained
|
||||
benchmark, profiles, and focused experiment results. It introduces no performance
|
||||
contract or release gate.
|
||||
|
||||
## Environment and revisions
|
||||
|
||||
- PromptKit checkout: `eb830bb61346672f0f6c5bce6119b76ccc6d61ac` for both
|
||||
downstream runs. The adapter and benchmark are unchanged from its earlier
|
||||
review revision, `de6443f3aef4a66169e323e9f9a5f41594fbface`.
|
||||
- Submitted ConfigKit dependency: v0.5.0, source commit
|
||||
`dc09959cca4e105f49ec316c6d4ba585bd2340bd`.
|
||||
- Current ConfigKit baseline: `f1722274c02c08ff28556dd66f29e0da07a9f2c3`.
|
||||
The benchmark added with this report changes no production implementation.
|
||||
- Go: `/snap/bin/go`, `go1.26.8`, `linux/amd64`.
|
||||
- CPU: Intel Core i5-7260U at 2.20 GHz; two physical cores, four logical CPUs.
|
||||
- `GOMAXPROCS=4`, `GOTOOLCHAIN=local`, `GOWORK=off`, `GOPROXY=off`, and
|
||||
readonly module mode. `GOGC` and `GOMEMLIMIT` were unset (runtime defaults).
|
||||
- Dependencies were already cached. No consumer repository's committed
|
||||
dependencies were changed, and no adoption was published. Separate temporary
|
||||
clones excluded consumer worktree edits; the current-code comparison used a
|
||||
temporary module replacement.
|
||||
- Logs, test binaries, and profiles are outside both repositories under
|
||||
`/tmp/configkit-startup.L9vnUo`. These are disposable evidence, not repository
|
||||
dependencies; the commands below regenerate them.
|
||||
|
||||
The processor and toolchain match the submitted environment. The machine was
|
||||
shared, with no exclusive CPU allocation or frequency control. An initial cold
|
||||
downstream pass overlapped discovery work and ranged from 90.95 to 256.19 ms;
|
||||
it was excluded from the warmed comparison below. Subsequent local runs also
|
||||
showed substantial timing variation. All final-run observations, including
|
||||
outliers, are recorded below. Allocation counts were much steadier than elapsed
|
||||
time. These measurements do not establish statistical timing significance.
|
||||
|
||||
## Stage 1: Downstream reproduction
|
||||
|
||||
PromptKit's `BenchmarkConfigurationConstruction/independent` prepares 128
|
||||
profiles and 128 prompt versions outside timing, including the captured
|
||||
`configkit.Mapping` source. Each profile supplies an endpoint and model. Each
|
||||
prompt has version key `1`, one user message, a profile reference, and text output
|
||||
with validation disabled. There are 256 definitions, one tier, and no files,
|
||||
imports, inheritance, credentials, or inference calls.
|
||||
|
||||
Each timed iteration invokes the real adapter's `load` function. Within
|
||||
`LoadLayered`, it captures optional-field presence with `Resolution.InputInfo`,
|
||||
normalizes profile values, performs domain validation, and supplies a path-based
|
||||
secret classifier. The supplied endpoints are already canonical, so this
|
||||
fixture records no normalization changes. After ConfigKit returns, the adapter
|
||||
materializes prompts, compiles templates, computes content hashes, and builds
|
||||
execution profiles. Those operations are included in the downstream timing.
|
||||
|
||||
The comparison used an isolated PromptKit checkout at the revision above. First
|
||||
it retained the declared v0.5.0 dependency. Then its temporary `go.mod` received
|
||||
a replacement pointing at an isolated ConfigKit checkout at the current
|
||||
baseline. Both used:
|
||||
|
||||
```sh
|
||||
GOTOOLCHAIN=local GOWORK=off GOPROXY=off GOMAXPROCS=4 \
|
||||
go test -mod=readonly -run '^$' \
|
||||
-bench '^BenchmarkConfigurationConstruction/independent$' \
|
||||
-benchmem -benchtime=500ms -count=5 ./internal/configuration
|
||||
```
|
||||
|
||||
To reproduce the temporary wiring, clone with `--no-hardlinks --local`, check
|
||||
out the recorded revisions, and run this only in the disposable PromptKit copy:
|
||||
|
||||
```sh
|
||||
go mod edit -replace=gitea.maximumdirect.net/eric/ConfigKit=/absolute/path/to/configkit-copy
|
||||
```
|
||||
|
||||
| ConfigKit | Run | ms/construction | B/construction | allocations/construction |
|
||||
| --- | ---: | ---: | ---: | ---: |
|
||||
| v0.5.0 | 1 | 74.339 | 50,155,869 | 538,590 |
|
||||
| v0.5.0 | 2 | 69.340 | 50,159,666 | 538,541 |
|
||||
| v0.5.0 | 3 | 71.453 | 50,131,114 | 538,542 |
|
||||
| v0.5.0 | 4 | 70.164 | 50,113,244 | 538,541 |
|
||||
| v0.5.0 | 5 | 72.495 | 50,165,238 | 538,540 |
|
||||
| current | 1 | 73.567 | 50,147,688 | 538,593 |
|
||||
| current | 2 | 70.959 | 50,152,860 | 538,540 |
|
||||
| current | 3 | 67.509 | 50,167,353 | 538,541 |
|
||||
| current | 4 | 71.491 | 50,181,043 | 538,543 |
|
||||
| current | 5 | 68.786 | 50,131,684 | 538,541 |
|
||||
|
||||
The per-metric medians are 71.453 ms / 50,155,869 B / 538,541 allocations for
|
||||
v0.5.0 and 70.959 ms / 50,152,860 B / 538,541 allocations for current code.
|
||||
The submitted timing median was approximately 67.740 ms; its approximately
|
||||
50.16 MB and 538,542 allocations are reproduced. The modest warmed timing
|
||||
difference and the cold-run outlier are consistent with machine/runtime
|
||||
variation; they do not establish a code regression. The production loading
|
||||
code is identical between these ConfigKit revisions: their Go-file differences
|
||||
are release/development tests. No startup improvement is claimed.
|
||||
|
||||
## Stage 2: Maintained public-API benchmark
|
||||
|
||||
[startup_bench_test.go](../../startup_bench_test.go) adds
|
||||
`BenchmarkStartupConstruction` in the external `configkit_test` package. It
|
||||
measures `LoadLayered` through the supported root facade at 32, 128, and 512
|
||||
profile/prompt pairs (64, 256, and 1,024 definitions).
|
||||
|
||||
The input has two root collections. Profiles contain endpoint and model scalars;
|
||||
prompts contain a numeric mapping key `1`, a profile reference, a message
|
||||
sequence with role/content, and an output mapping with format/validation mode.
|
||||
The source tree has `13 * pairs + 3` nodes (419, 1,667, and 6,659 respectively),
|
||||
before typed projection adds omitted-field metadata. Typed profiles have twelve
|
||||
optional fields; typed prompts retain optional metadata, inputs, messages, and
|
||||
output fields. This records configuration shape as well as definition count.
|
||||
|
||||
Fixture generation, `Mapping` capture, and one successful lifecycle/metadata
|
||||
check are outside timing. Timed work includes typed decoding, each callback,
|
||||
resolution auditing, validation, final publication, and retention of the latest
|
||||
result. The resolver captures all twelve profile field states and parses their
|
||||
canonical endpoints. Validation visits sorted profile IDs. A simple path-based
|
||||
secret predicate is installed, although the fixture contains no credentials.
|
||||
The preflight checks callback execution, collection sizes, explicit versus
|
||||
omitted profile fields, and metadata for the message reached through mapping
|
||||
key `1` and sequence index 0.
|
||||
|
||||
This is representative rather than an exact copy of the consumer adapter: it
|
||||
omits the unused backend collection and message cache-control type, simplifies
|
||||
domain validation/classification, and does not compile templates, compute
|
||||
hashes, or build runtime registries. Its lower allocation total must not be
|
||||
treated as an exact subtraction of PromptKit overhead. It adds no imports of
|
||||
PromptKit and runs offline without credentials. Its measurements are not
|
||||
pass/fail thresholds.
|
||||
|
||||
Run from ConfigKit:
|
||||
|
||||
```sh
|
||||
GOTOOLCHAIN=local GOWORK=off GOPROXY=off GOMAXPROCS=4 \
|
||||
go test -mod=readonly -run '^$' -bench '^BenchmarkStartupConstruction$' \
|
||||
-benchmem -benchtime=500ms -count=5
|
||||
```
|
||||
|
||||
| Pairs | Run | ms/construction | B/construction | allocations/construction |
|
||||
| ---: | ---: | ---: | ---: | ---: |
|
||||
| 32 | 1 | 18.327 | 11,999,640 | 129,136 |
|
||||
| 32 | 2 | 18.418 | 11,999,423 | 129,136 |
|
||||
| 32 | 3 | 50.366 | 11,999,534 | 129,137 |
|
||||
| 32 | 4 | 55.028 | 11,999,548 | 129,137 |
|
||||
| 32 | 5 | 25.824 | 11,999,468 | 129,136 |
|
||||
| 128 | 1 | 67.610 | 47,408,046 | 513,722 |
|
||||
| 128 | 2 | 65.090 | 47,451,718 | 513,724 |
|
||||
| 128 | 3 | 65.682 | 47,584,876 | 513,727 |
|
||||
| 128 | 4 | 62.574 | 47,324,919 | 513,722 |
|
||||
| 128 | 5 | 123.565 | 47,273,050 | 513,721 |
|
||||
| 512 | 1 | 276.663 | 188,313,648 | 2,051,961 |
|
||||
| 512 | 2 | 284.979 | 189,649,296 | 2,052,000 |
|
||||
| 512 | 3 | 497.075 | 189,234,128 | 2,051,985 |
|
||||
| 512 | 4 | 257.534 | 189,084,032 | 2,051,981 |
|
||||
| 512 | 5 | 265.167 | 188,679,712 | 2,051,973 |
|
||||
|
||||
Per-metric medians:
|
||||
|
||||
| Pairs | ms | B | Allocations |
|
||||
| ---: | ---: | ---: | ---: |
|
||||
| 32 | 25.824 | 11,999,534 | 129,136 |
|
||||
| 128 | 65.682 | 47,408,046 | 513,722 |
|
||||
| 512 | 276.663 | 189,084,032 | 2,051,981 |
|
||||
|
||||
Increasing the catalog fourfold multiplies allocation counts by approximately
|
||||
3.98 and 3.99, and allocated bytes by approximately 3.95 and 3.99. At these
|
||||
fixed-depth shapes, allocations scale approximately with catalog size; the
|
||||
measurements do not demonstrate a superlinear defect or establish a general
|
||||
complexity guarantee. Timing is too variable for a precise scaling claim.
|
||||
The original exploratory fixture lacked the classifier and had lower allocation
|
||||
counts; only the final fixture's results above are the maintained baseline.
|
||||
|
||||
### Complementary existing benchmarks
|
||||
|
||||
The following existing benchmarks were run sequentially, with
|
||||
`GOMAXPROCS=4`, local/offline toolchain settings, readonly modules,
|
||||
`-benchtime=200ms`, and `-count=3`:
|
||||
|
||||
```sh
|
||||
GOTOOLCHAIN=local GOWORK=off GOPROXY=off GOMAXPROCS=4 \
|
||||
go test ./internal/engine -mod=readonly -run '^$' \
|
||||
-bench '^(BenchmarkComposeWideMappings|BenchmarkUnderlayBroadCatalog|BenchmarkUnderlayRenderedHistory|BenchmarkInheritanceInspectedLoad)$' \
|
||||
-benchmem -benchtime=200ms -count=3
|
||||
```
|
||||
|
||||
These per-metric medians provide comparison points for later changes. They have
|
||||
different timing boundaries and must not be compared as equivalent loads.
|
||||
|
||||
| Benchmark / size | ms | B | Allocations |
|
||||
| --- | ---: | ---: | ---: |
|
||||
| ComposeWideMappings / 128 | 1.815 | 848,512 | 9,658 |
|
||||
| ComposeWideMappings / 256 | 3.531 | 1,703,590 | 19,652 |
|
||||
| ComposeWideMappings / 512 | 9.532 | 3,626,298 | 39,643 |
|
||||
| UnderlayBroadCatalog / 128 | 3.299 | 1,265,032 | 15,904 |
|
||||
| UnderlayBroadCatalog / 256 | 3.436 | 2,581,000 | 33,473 |
|
||||
| UnderlayBroadCatalog / 512 | 8.472 | 5,268,937 | 68,581 |
|
||||
| InheritanceInspectedLoad / 16 | 2.571 | 876,236 | 11,050 |
|
||||
| InheritanceInspectedLoad / 32 | 4.066 | 1,545,216 | 19,943 |
|
||||
| InheritanceInspectedLoad / 64 | 8.358 | 2,906,067 | 37,779 |
|
||||
| UnderlayRenderedHistory / 32 | 0.336 | 504,546 | 3,294 |
|
||||
| UnderlayRenderedHistory / 64 | 0.661 | 1,006,053 | 6,529 |
|
||||
| UnderlayRenderedHistory / 128 | 1.709 | 2,031,853 | 12,997 |
|
||||
|
||||
Composition measures document combination; broad underlays measure an engine
|
||||
load; inspected inheritance includes parsing, loading, and an explanation query;
|
||||
rendered history measures explanation rendering after an untimed load. The
|
||||
existing inspection benchmark supplies coverage without adding another suite.
|
||||
|
||||
## Stage 3: Profiles and implementation trace
|
||||
|
||||
CPU and sampled allocation profiles were collected in separate two-second runs
|
||||
at 128 pairs. The current-code downstream run measured 71.877 ms, 50,125,624 B,
|
||||
and 538,552 allocations. The final local fixture's profiled run measured
|
||||
66.153 ms, 47,342,971 B, and 513,721 allocations. Profiled timings are not used
|
||||
in the ordinary timing comparison.
|
||||
|
||||
To regenerate local profiles outside the checkout:
|
||||
|
||||
```sh
|
||||
profile_dir=$(mktemp -d)
|
||||
GOTOOLCHAIN=local GOWORK=off GOPROXY=off GOMAXPROCS=4 \
|
||||
go test -mod=readonly -run '^$' -bench '^BenchmarkStartupConstruction/128$' \
|
||||
-benchtime=2s -cpuprofile="$profile_dir/startup.cpu" \
|
||||
-memprofile="$profile_dir/startup.mem" -o "$profile_dir/startup.test"
|
||||
go tool pprof -top -nodecount=12 -alloc_space "$profile_dir/startup.test" "$profile_dir/startup.mem"
|
||||
go tool pprof -top -nodecount=12 "$profile_dir/startup.test" "$profile_dir/startup.cpu"
|
||||
go tool pprof -list='engine.Path.String' -alloc_space "$profile_dir/startup.test" "$profile_dir/startup.mem"
|
||||
```
|
||||
|
||||
For downstream profiles, use the stage 1 benchmark selector and package, the
|
||||
same two-second settings, and separate output filenames in a temporary directory.
|
||||
|
||||
| Allocation site (flat sampled bytes) | Local | Downstream |
|
||||
| --- | ---: | ---: |
|
||||
| `clonePathSegments` | 31.00% | 30.18% |
|
||||
| `strings.Builder.WriteString` | 10.61% | 11.02% |
|
||||
| `collectOrigins` | 5.04% | 4.86% |
|
||||
| `collectFieldInfo` | 4.16% | 5.17% |
|
||||
| `ParsePath` | 4.11% | 4.37% |
|
||||
| `effectivePathIndex.func1` | 3.69% | 3.13% |
|
||||
| `strings.Builder.WriteByte` | 3.30% | 3.48% |
|
||||
|
||||
`Path.String` accounts cumulatively for 13.90% of sampled allocated bytes and
|
||||
23.03% of CPU samples locally, versus 14.51% and 21.80% downstream. The local
|
||||
line profile attributes 48.50 MB to writing separators and 156.01 MB to writing
|
||||
tokens across the profiled run, exposing repeated builder growth. These are
|
||||
sampled totals across many operations, not bytes per construction. Flat and
|
||||
cumulative shares overlap and must not be added together. GC/scanning,
|
||||
allocation, and map operations also occupy substantial CPU samples.
|
||||
|
||||
The source trace identifies these responsibilities:
|
||||
|
||||
- [Path construction](../../internal/engine/path.go): `Key` and `Index` copy
|
||||
segments to preserve immutable parents and contextual mapping/sequence intent.
|
||||
`Keys` builds through repeated `Key` calls. `Path.String` starts an ungrown
|
||||
builder and formats every segment on every invocation.
|
||||
- [Metadata collection](../../internal/engine/load_metadata.go):
|
||||
`collectFieldInfo` formats each visited path and detaches origin records.
|
||||
Decode's `collectOrigins` likewise traverses and formats paths.
|
||||
- [Path indexes](../../internal/engine/metadata.go): `effectivePathIndex`
|
||||
walks and retains contextual paths; `finalizeMetadataPaths` repeats walks for
|
||||
effective and historical trees. These indexes serve different query roles.
|
||||
- [Sensitivity classification](../../internal/engine/secret.go):
|
||||
`classifySecretPaths` indexes several trees and repeatedly parses textual
|
||||
pointers through `addSecretPath`. It also evaluates the caller's classifier.
|
||||
- [Publication](../../internal/engine/load_result.go): final typed detachment,
|
||||
metadata finalization, and sensitivity classification happen before return.
|
||||
Detachment protects against equal borrowed objects supplied by callbacks.
|
||||
- [Root conversion](../../lifecycle.go): callback adapters delegate to live
|
||||
engine contexts; result publication wraps the detached configuration and
|
||||
inspection. Paths delegate to engine paths. The dominant sampled work is
|
||||
inside the engine, not a second root-owned tree conversion.
|
||||
|
||||
The profiles support reducing redundant allocation and formatting. They do not
|
||||
justify dropping ownership copies, historical trees, indexes, or sensitivity
|
||||
processing. The consumer callback work is part of the measured lifecycle;
|
||||
post-return materialization remains downstream work.
|
||||
|
||||
### Proposed first optimization
|
||||
|
||||
Try pre-sizing the local `strings.Builder` in engine `Path.String` using the
|
||||
encoded pointer length before writing tokens. Keep the existing segment kinds,
|
||||
canonical escaping, index formatting, root behavior, and independent returned
|
||||
strings. A straightforward length calculation plus `Grow` should reduce buffer
|
||||
growth without changing the `Path` representation or introducing shared state.
|
||||
Avoid a cache or buffer pool for this first experiment. This is a hypothesis,
|
||||
not an implemented improvement or a promise of one allocation per path.
|
||||
|
||||
Although segment cloning is the largest allocation site, changing it requires
|
||||
more ownership analysis across branches and captured records. Buffer sizing is
|
||||
a smaller first change supported by both profiles, with a clear measurement
|
||||
signal: fewer allocated bytes/allocations at all catalog sizes and less sampled
|
||||
builder-growth work. CPU benefit must be measured because length calculation
|
||||
also adds work. Escaped tokens and larger indexes require particular care;
|
||||
preserve ordinary handling of invalid paths rather than adding validation here.
|
||||
|
||||
Use existing `TestPathRoundTripsSemanticKeys`,
|
||||
`TestPathPreservesSegmentIntentAndImmutability`, and
|
||||
`TestParsePathRejectsMalformedPointers`, plus schema/metadata tests for numeric
|
||||
keys versus sequence indexes. Verify root-facade integration, history,
|
||||
explanation, and secret tests through the ordinary development suite. Add a
|
||||
focused case only for a consequential boundary not covered already. Keep
|
||||
inspection/rendering measurements alongside construction measurements; run
|
||||
race validation if a later change affects ownership or concurrency. No runtime
|
||||
ownership or concurrency behavior changes are proposed for buffer sizing.
|
||||
|
||||
Then compare repeated matching before/after runs at all three sizes and the
|
||||
downstream fixture; reprofile to confirm the attributed reduction. Keep the
|
||||
change only if repeatable benefit justifies its complexity. If it does not,
|
||||
return to profiling and assess pointer reparsing or repeated traversal rather
|
||||
than assuming the next optimization in advance.
|
||||
|
||||
## Initial validation and checkpoint
|
||||
|
||||
The maintained fixture's preflight and measured runs succeeded at every size.
|
||||
`./scripts/check.sh` passed formatting and whitespace checks, the offline suite
|
||||
(including local documentation links and facade boundaries), full vet, and
|
||||
package builds. At that checkpoint, Stages 4–6 remained unstarted and no
|
||||
production implementation had changed. The evidence supported one focused
|
||||
optimization experiment; wider representation changes needed their own measured
|
||||
justification.
|
||||
|
||||
## Focused experiment: Pre-size `Path.String`
|
||||
|
||||
Status: Retained on 2026-10-09.
|
||||
|
||||
### Change and comparison setup
|
||||
|
||||
The fresh unoptimized baseline is `91ae4fa7eff1571e7c05de01f9e922d75df7453d`,
|
||||
including the maintained benchmark above. An isolated checkout preserved that
|
||||
baseline while the working candidate changed only engine `Path.String` and
|
||||
extended the existing path test. Toolchain, CPU, runtime settings, fixture sizes,
|
||||
and timing boundaries are as recorded above. Fresh measurements used the same
|
||||
benchmark commands: five 500-ms observations for construction/PromptKit, three
|
||||
200-ms observations for complementary cases, and separate two-second profiles.
|
||||
All measurement families ran sequentially. Logs and binaries are outside the
|
||||
repositories at `/tmp/configkit-path-sizing.JpbMpD`.
|
||||
|
||||
The sizing pass counts separators, token bytes, and the extra bytes required for
|
||||
`~` and `/` escaping. A local 20-byte scratch array with `strconv.AppendInt`
|
||||
measures index text without allocating another string. `strings.Builder.Grow`
|
||||
reserves the calculated capacity. The original escaping, index-writing, root,
|
||||
and invalid-path behavior remain intact. There is no cache, pool, shared state,
|
||||
new validation, or change to segment ownership. The sizing pass adds work;
|
||||
benefit is measured rather than assumed.
|
||||
|
||||
The PromptKit checkout remained at
|
||||
`eb830bb61346672f0f6c5bce6119b76ccc6d61ac`. Its temporary replacement first
|
||||
selected the isolated baseline and then the ConfigKit working candidate. No
|
||||
consumer repository or committed dependency changed.
|
||||
|
||||
### Fresh construction results
|
||||
|
||||
Each cell below is the median of that metric across five observations. Timing
|
||||
improvements describe this run, without claiming statistical significance or
|
||||
machine-independent speedups.
|
||||
|
||||
| Workload | Before ms | After ms | Before B | After B | Before allocations | After allocations |
|
||||
| --- | ---: | ---: | ---: | ---: | ---: | ---: |
|
||||
| Local / 32 pairs | 18.251 | 17.495 | 11,999,426 | 11,036,879 | 129,136 | 80,465 |
|
||||
| Local / 128 pairs | 68.096 | 60.859 | 47,320,688 | 43,490,008 | 513,720 | 319,131 |
|
||||
| Local / 512 pairs | 262.823 | 252.779 | 188,764,416 | 173,640,328 | 2,051,979 | 1,273,697 |
|
||||
| PromptKit / independent | 73.386 | 65.899 | 50,127,948 | 46,171,529 | 538,541 | 338,171 |
|
||||
|
||||
Allocated bytes fell by 8.02%, 8.10%, and 8.01% locally; allocation counts fell
|
||||
by 37.69%, 37.88%, and 37.93%. PromptKit showed 7.89% fewer allocated bytes and
|
||||
37.21% fewer allocations. These reductions recur across observations and catalog
|
||||
sizes. The downstream total continues to include its adapter and post-return
|
||||
processing; it is not an isolated ConfigKit cost.
|
||||
|
||||
Timing ranges (ms), keeping the first PromptKit outlier:
|
||||
|
||||
| Workload | Before range | After range |
|
||||
| --- | ---: | ---: |
|
||||
| Local / 32 pairs | 17.468–18.518 | 16.719–19.131 |
|
||||
| Local / 128 pairs | 66.211–79.363 | 58.048–63.724 |
|
||||
| Local / 512 pairs | 256.783–287.161 | 241.361–272.743 |
|
||||
| PromptKit / independent | 68.308–134.999 | 63.664–74.520 |
|
||||
|
||||
The local 128-pair median and downstream median were approximately 10% lower;
|
||||
the other local medians were 4% lower. Timing remains sensitive to shared-host
|
||||
load, so the allocation reduction supplies the stronger evidence for retention.
|
||||
|
||||
### Complementary workloads and possible timing regressions
|
||||
|
||||
The original comparison's per-metric medians follow. These fixtures retain the
|
||||
different boundaries explained above.
|
||||
|
||||
| Benchmark / size | Before ms | After ms | Before B | After B | Before allocations | After allocations |
|
||||
| --- | ---: | ---: | ---: | ---: | ---: | ---: |
|
||||
| ComposeWideMappings / 128 | 0.883 | 0.806 | 848,507 | 797,562 | 9,658 | 5,962 |
|
||||
| ComposeWideMappings / 256 | 1.781 | 1.708 | 1,703,582 | 1,595,293 | 19,652 | 11,860 |
|
||||
| ComposeWideMappings / 512 | 4.460 | 3.963 | 3,626,263 | 3,403,282 | 39,643 | 23,659 |
|
||||
| UnderlayBroadCatalog / 128 | 1.718 | 1.520 | 1,265,027 | 1,229,413 | 15,904 | 12,628 |
|
||||
| UnderlayBroadCatalog / 256 | 3.707 | 3.231 | 2,580,999 | 2,459,362 | 33,473 | 24,821 |
|
||||
| UnderlayBroadCatalog / 512 | 7.094 | 6.676 | 5,268,929 | 4,975,269 | 68,581 | 49,177 |
|
||||
| InheritanceInspectedLoad / 16 | 1.124 | 1.296 | 876,204 | 835,688 | 11,050 | 8,060 |
|
||||
| InheritanceInspectedLoad / 32 | 2.248 | 2.180 | 1,545,178 | 1,466,920 | 19,943 | 14,145 |
|
||||
| InheritanceInspectedLoad / 64 | 4.030 | 3.784 | 2,906,020 | 2,752,258 | 37,778 | 26,360 |
|
||||
| UnderlayRenderedHistory / 32 | 0.289 | 0.305 | 504,547 | 503,459 | 3,294 | 3,221 |
|
||||
| UnderlayRenderedHistory / 64 | 0.624 | 0.735 | 1,006,052 | 1,003,942 | 6,529 | 6,392 |
|
||||
| UnderlayRenderedHistory / 128 | 1.879 | 1.283 | 2,031,854 | 2,027,694 | 12,997 | 12,732 |
|
||||
|
||||
Allocations improved in every case. Rendering benefits are smaller because its
|
||||
load is outside timing. The first batch's 16-entry inheritance and smaller
|
||||
rendering times were higher after the change, so they received longer follow-up
|
||||
runs in reverse order (after, then before).
|
||||
|
||||
Inheritance / 16 used `-benchtime=500ms -count=5`: before times were
|
||||
2.537, 1.153, 1.148, 1.155, and 1.153 ms; after times were 1.308, 1.088, 1.086,
|
||||
1.112, and 1.082 ms. Its median changed from 1.153 to 1.088 ms, reversing the
|
||||
initial apparent regression, with the same allocation reduction.
|
||||
|
||||
Rendering used `-benchtime=500ms -count=3`. Its timing medians were:
|
||||
|
||||
| History size | Before ms | After ms |
|
||||
| ---: | ---: | ---: |
|
||||
| 32 | 0.290 | 0.302 |
|
||||
| 64 | 1.538 | 0.610 |
|
||||
| 128 | 7.213 | 1.329 |
|
||||
|
||||
The before / 64 observations spanned 0.643–3.004 ms and before / 128 spanned
|
||||
2.211–7.909 ms, exposing substantial host variation. Rendering / 32 remained
|
||||
about 4% slower in this second batch, but the ranges overlapped (before
|
||||
0.288–0.317 ms; after 0.299–0.307 ms). A small extra cost from the sizing pass
|
||||
on short inspection paths cannot be ruled out. There is insufficient evidence
|
||||
to claim every operation is faster; the stronger construction allocation benefit
|
||||
and limited implementation complexity justify this tradeoff. These measurements
|
||||
concern cumulative allocation; retained heap was not measured separately. The
|
||||
change introduces no cache or altered data lifetime.
|
||||
|
||||
### Reprofiled attribution and retention decision
|
||||
|
||||
Fresh profiles used the local 128-pair fixture. The before profiled run measured
|
||||
72.765 ms, 47,297,038 B, and 513,721 allocations; after measured 64.367 ms,
|
||||
43,532,356 B, and 319,132 allocations. These remain separate from ordinary
|
||||
timing comparisons.
|
||||
|
||||
`Path.String` accounted for 14.38% of sampled allocation bytes before and 7.43%
|
||||
after. The line profile previously attributed 62.00 MB to separator writes and
|
||||
183.01 MB to token writes across the run; afterward it attributed 123.00 MB to
|
||||
the initial `Grow`, with no sampled allocation at those writes. The profile
|
||||
runs completed different operation counts (36 and 39), so raw MB totals must
|
||||
not be treated as per-load differences. Direct benchmark measurements above
|
||||
establish the per-load reduction. `Path.String`'s cumulative CPU share changed
|
||||
from 21.15% to 18.96%; that share includes the added sizing work.
|
||||
|
||||
Retain the change: a short local sizing pass removes approximately 195,000
|
||||
allocations and 3.83 MB from a representative 128-pair construction, with
|
||||
matching benefits in the consumer and complementary cases. The existing
|
||||
segment ownership and public behavior stay intact. The extra scan and uncertain
|
||||
small-rendering timing cost are recorded rather than converted into a new
|
||||
performance promise.
|
||||
|
||||
### Next candidate and remaining work
|
||||
|
||||
`clonePathSegments` remains the largest flat allocation site: 31.29% before and
|
||||
34.99% after. Its larger share reflects other allocations disappearing; the
|
||||
underlying copies were not changed. `ParsePath` remains at 4.92% after, while
|
||||
`effectivePathIndex.func1` accounts for 3.84%. GC and map operations remain
|
||||
prominent. Pointer formatting still accounts for nearly 19% of sampled CPU,
|
||||
including `strings.Count` work, so further formatting changes would need to
|
||||
justify their own extra complexity.
|
||||
|
||||
At completion of the first experiment, the recommended next experiment was to
|
||||
investigate combining the two walks of the final tree in `finalizeMetadataPaths`:
|
||||
it first called `effectivePathIndex` and then walked the same final tree again
|
||||
for historical paths. One traversal could populate both indexes and format each
|
||||
final path once, while retaining
|
||||
the separate records, contextual segment kinds, ownership copies, and all walks
|
||||
of earlier/application/inheritance trees. This is a concrete redundancy to
|
||||
measure before considering broader path representation or ownership changes.
|
||||
Its benefit had not yet been established; the experiment below now records it.
|
||||
A generic `Keys` constructor rewrite is smaller, but only accounted for 0.78% of
|
||||
sampled allocated bytes in the after profile, so it is a lower-priority candidate.
|
||||
|
||||
The path/metadata/provenance/query/explanation/secret tests passed with the
|
||||
candidate. The existing path test now additionally checks decimal formatting
|
||||
at three-digit and signed integer limits, including negative invalid indexes.
|
||||
`./scripts/check.sh` passed formatting, whitespace, the offline suite (including
|
||||
documentation links and facade boundaries), vet, and builds. This experiment
|
||||
does not complete Stage 5: PromptKit's file-backed, shared-base,
|
||||
and layered fixtures plus its applicable configuration tests still belong to
|
||||
the final consumer verification. No consumer upgrade or release is published.
|
||||
|
||||
## Focused experiment: combine final metadata traversal
|
||||
|
||||
Status: Complete; retain the change. Measured on 2026-10-09. This is the second
|
||||
Stage 4 experiment, not completion of the final consumer verification.
|
||||
|
||||
### Change and ownership boundaries
|
||||
|
||||
`finalizeMetadataPaths` formerly built the effective index with
|
||||
`effectivePathIndex(final)`, then walked the same final tree again to seed the
|
||||
historical index. It now allocates the two maps separately and walks the final
|
||||
tree once. Each visited path is formatted once for insertion into those maps.
|
||||
The effective index stores its own `path.clone()`; the existing historical
|
||||
insertion/deduplication logic stores a separate clone. The maps may share an
|
||||
immutable pointer string, not mutable path-segment storage.
|
||||
|
||||
The historical helper accepts the already-formatted pointer so the final
|
||||
callback can reuse it. Earlier trees, override applications, and inheritance
|
||||
controls/transfers retain their existing walks, order, contextual path kinds,
|
||||
and deduplication. `effectivePathIndex` remains unchanged for resolution input.
|
||||
Nil final input still skips traversal while leaving both maps initialized.
|
||||
There is no public API, configuration-semantic, dependency, cache, pooling,
|
||||
release-gate, or result-lifetime change.
|
||||
|
||||
### Revisions and reproduction
|
||||
|
||||
- Before: `673faea097fb9a178b5743e71cbdcbecd215568f`, including the retained
|
||||
`Path.String` optimization. These are fresh measurements, not comparisons
|
||||
that bundle the two optimizations together.
|
||||
- After: the candidate accompanying this section. Its only production change is
|
||||
in `internal/engine/metadata.go`, Git blob
|
||||
`e22804d659efb203a3829382a21e4e8d059b6c5f`; the accompanying tests and this
|
||||
report are not part of timed construction.
|
||||
- PromptKit: `eb830bb61346672f0f6c5bce6119b76ccc6d61ac`, unchanged for both
|
||||
candidates. Its independent fixture retains the timing boundaries described
|
||||
in Stage 1, including adapter processing after ConfigKit returns.
|
||||
- Toolchain, hardware, runtime settings, fixture sizes, and setup boundaries
|
||||
match the environment and maintained benchmark above. All measurements ran
|
||||
sequentially; profiles and correctness checks did not overlap timed runs.
|
||||
Shared-host scheduling and CPU frequency were not controlled.
|
||||
- Raw benchmark logs, profiles, and binaries are outside both repositories at
|
||||
`/tmp/configkit-final-walk.uDFTyU`. A disposable ConfigKit clone holds the
|
||||
before revision; a disposable PromptKit clone alternately uses a module
|
||||
replacement pointing to that clone or the working candidate. No original
|
||||
consumer files or committed dependencies changed.
|
||||
|
||||
Run from each ConfigKit candidate with matching settings:
|
||||
|
||||
```sh
|
||||
GOTOOLCHAIN=local GOWORK=off GOPROXY=off GOMAXPROCS=4 \
|
||||
go test -mod=readonly -run '^$' -bench '^BenchmarkStartupConstruction$' \
|
||||
-benchmem -benchtime=500ms -count=5
|
||||
|
||||
GOTOOLCHAIN=local GOWORK=off GOPROXY=off GOMAXPROCS=4 \
|
||||
go test -mod=readonly ./internal/engine -run '^$' \
|
||||
-bench '^(BenchmarkComposeWideMappings|BenchmarkUnderlayBroadCatalog|BenchmarkUnderlayRenderedHistory|BenchmarkInheritanceInspectedLoad)$' \
|
||||
-benchmem -benchtime=200ms -count=3
|
||||
```
|
||||
|
||||
For each candidate, rewire only the disposable PromptKit copy with the
|
||||
`go mod edit -replace` command in Stage 1, then run its same independent
|
||||
benchmark with `-benchtime=500ms -count=5`. Initial batches ran before, then
|
||||
after. Longer repeats reversed that order: after, then before. Construction
|
||||
and consumer repeats used `-benchtime=1s -count=5`; complementary repeats
|
||||
used `-benchtime=500ms -count=5`.
|
||||
|
||||
### Fresh construction results
|
||||
|
||||
Every cell is the per-metric median of five observations. No outliers were
|
||||
removed. These timing differences are descriptive, not statistical evidence
|
||||
of a speedup or regression.
|
||||
|
||||
| Workload | Before ms | After ms | Before B | After B | Before allocations | After allocations |
|
||||
| --- | ---: | ---: | ---: | ---: | ---: | ---: |
|
||||
| Local / 32 pairs | 17.363 | 17.175 | 11,036,876 | 10,887,031 | 80,465 | 78,605 |
|
||||
| Local / 128 pairs | 61.612 | 62.541 | 43,513,114 | 42,912,759 | 319,130 | 311,702 |
|
||||
| Local / 512 pairs | 251.336 | 256.408 | 173,804,200 | 170,796,144 | 1,273,712 | 1,244,003 |
|
||||
| PromptKit / independent | 66.861 | 87.909 | 46,174,781 | 45,525,968 | 338,168 | 330,483 |
|
||||
|
||||
Local allocation counts fell by 2.31%, 2.33%, and 2.33%, respectively; allocated
|
||||
bytes fell by 1.36%, 1.38%, and 1.73%. PromptKit showed 2.27% fewer allocations
|
||||
and 1.41% fewer allocated bytes. Its total still includes consumer adapter
|
||||
work, rather than measuring only ConfigKit.
|
||||
|
||||
Initial timing ranges (ms):
|
||||
|
||||
| Workload | Before range | After range |
|
||||
| --- | ---: | ---: |
|
||||
| Local / 32 pairs | 16.547–19.567 | 16.740–20.823 |
|
||||
| Local / 128 pairs | 58.494–69.982 | 58.401–68.902 |
|
||||
| Local / 512 pairs | 243.146–279.740 | 249.116–439.932 |
|
||||
| PromptKit / independent | 64.033–126.770 | 84.062–125.970 |
|
||||
|
||||
The initial PromptKit median was 31% higher after the change; the larger local
|
||||
medians were also slightly higher. Longer reverse-order repeats gave:
|
||||
|
||||
| Workload | Before ms | After ms | Before range | After range |
|
||||
| --- | ---: | ---: | ---: | ---: |
|
||||
| Local / 32 pairs | 18.397 | 17.487 | 18.323–19.124 | 17.322–18.628 |
|
||||
| Local / 128 pairs | 62.551 | 61.602 | 58.959–66.066 | 58.348–63.903 |
|
||||
| Local / 512 pairs | 258.394 | 251.583 | 239.896–263.039 | 247.995–304.867 |
|
||||
| PromptKit / independent | 68.283 | 68.134 | 63.908–71.703 | 64.020–73.127 |
|
||||
|
||||
The consumer's initial slowdown did not recur. All four repeat ranges overlap.
|
||||
Allocation reductions recur: local count medians remain 2.31–2.33% lower and
|
||||
byte medians 1.36–1.43% lower; PromptKit remains 2.27% and 1.38% lower,
|
||||
respectively. At 128 pairs, the longer run changed from 43,510,872 B and
|
||||
319,132 allocations to 42,888,256 B and 311,702 allocations.
|
||||
|
||||
### Complementary workloads and timing investigation
|
||||
|
||||
Initial complementary results are per-metric medians of three observations:
|
||||
|
||||
| Benchmark / size | Before ms | After ms | Before B | After B | Before allocations | After allocations |
|
||||
| --- | ---: | ---: | ---: | ---: | ---: | ---: |
|
||||
| ComposeWideMappings / 128 | 0.824 | 1.166 | 797,564 | 797,564 | 5,962 | 5,962 |
|
||||
| ComposeWideMappings / 256 | 1.676 | 1.755 | 1,595,291 | 1,595,293 | 11,860 | 11,860 |
|
||||
| ComposeWideMappings / 512 | 4.235 | 4.293 | 3,403,281 | 3,403,290 | 23,659 | 23,659 |
|
||||
| InheritanceInspectedLoad / 16 | 1.127 | 1.211 | 835,681 | 832,237 | 8,060 | 7,990 |
|
||||
| InheritanceInspectedLoad / 32 | 2.035 | 1.960 | 1,466,907 | 1,460,263 | 14,145 | 14,011 |
|
||||
| InheritanceInspectedLoad / 64 | 4.093 | 4.150 | 2,752,251 | 2,739,211 | 26,360 | 26,098 |
|
||||
| UnderlayBroadCatalog / 128 | 1.618 | 1.613 | 1,229,411 | 1,218,907 | 12,628 | 12,370 |
|
||||
| UnderlayBroadCatalog / 256 | 3.296 | 3.479 | 2,459,362 | 2,437,599 | 24,821 | 24,307 |
|
||||
| UnderlayBroadCatalog / 512 | 6.703 | 8.523 | 4,975,266 | 4,930,978 | 49,177 | 48,151 |
|
||||
| UnderlayRenderedHistory / 32 | 0.290 | 0.659 | 503,458 | 503,461 | 3,221 | 3,221 |
|
||||
| UnderlayRenderedHistory / 64 | 2.085 | 1.204 | 1,003,942 | 1,003,947 | 6,392 | 6,392 |
|
||||
| UnderlayRenderedHistory / 128 | 3.331 | 2.723 | 2,027,690 | 2,027,695 | 12,732 | 12,732 |
|
||||
|
||||
Inheritance loading and broad-underlay loading allocate less in every case.
|
||||
Pure composition does not run metadata finalization and its allocation counts
|
||||
are unchanged. History rendering also has unchanged allocation counts, as
|
||||
expected: loading is outside its timed section. Small byte differences in
|
||||
those unaffected measurements do not establish an allocation improvement.
|
||||
|
||||
The longer reverse-order complementary batches produced these medians and
|
||||
ranges (ms), retaining all five observations:
|
||||
|
||||
| Benchmark / size | Before ms | After ms | Before range | After range |
|
||||
| --- | ---: | ---: | ---: | ---: |
|
||||
| ComposeWideMappings / 128 | 0.915 | 0.858 | 0.883–1.512 | 0.839–0.922 |
|
||||
| ComposeWideMappings / 256 | 1.762 | 1.733 | 1.732–1.857 | 1.613–1.962 |
|
||||
| ComposeWideMappings / 512 | 4.853 | 4.443 | 4.531–5.321 | 4.197–6.346 |
|
||||
| InheritanceInspectedLoad / 16 | 1.200 | 1.822 | 1.133–1.494 | 1.652–1.951 |
|
||||
| InheritanceInspectedLoad / 32 | 2.141 | 3.177 | 2.130–2.208 | 3.055–3.634 |
|
||||
| InheritanceInspectedLoad / 64 | 4.169 | 5.066 | 4.054–4.460 | 3.883–8.171 |
|
||||
| UnderlayBroadCatalog / 128 | 1.736 | 1.852 | 1.595–1.831 | 1.718–2.510 |
|
||||
| UnderlayBroadCatalog / 256 | 3.712 | 3.743 | 3.413–3.843 | 3.378–7.093 |
|
||||
| UnderlayBroadCatalog / 512 | 7.479 | 12.938 | 7.140–8.168 | 11.075–13.368 |
|
||||
| UnderlayRenderedHistory / 32 | 0.326 | 0.563 | 0.311–1.048 | 0.530–0.583 |
|
||||
| UnderlayRenderedHistory / 64 | 0.719 | 1.083 | 0.651–3.182 | 0.931–1.145 |
|
||||
| UnderlayRenderedHistory / 128 | 1.435 | 2.567 | 1.361–5.152 | 2.289–2.887 |
|
||||
|
||||
Allocation-count medians in these repeats match the initial complementary
|
||||
table. Timing remains mixed: several loading and rendering cases were much
|
||||
slower in the after batch, while the unchanged pure-composition cases were
|
||||
faster. To investigate the substantial apparent regressions, two additional
|
||||
alternating rounds ran after/before, then after/before, with
|
||||
`-benchtime=1s -count=3` and this benchmark selector:
|
||||
|
||||
```sh
|
||||
GOTOOLCHAIN=local GOWORK=off GOPROXY=off GOMAXPROCS=4 \
|
||||
go test -mod=readonly ./internal/engine -run '^$' \
|
||||
-bench '^(BenchmarkInheritanceInspectedLoad|BenchmarkUnderlayBroadCatalog|BenchmarkUnderlayRenderedHistory)$/^(16|32|512)$' \
|
||||
-benchmem -benchtime=1s -count=3
|
||||
```
|
||||
|
||||
| Round / benchmark / size | Before ms | After ms | Before range | After range |
|
||||
| --- | ---: | ---: | ---: | ---: |
|
||||
| 1 / InheritanceInspectedLoad / 16 | 1.126 | 1.166 | 1.116–1.151 | 1.139–1.187 |
|
||||
| 1 / InheritanceInspectedLoad / 32 | 2.064 | 2.070 | 2.023–2.133 | 2.053–2.148 |
|
||||
| 1 / UnderlayBroadCatalog / 512 | 6.949 | 6.822 | 6.904–7.461 | 6.785–7.204 |
|
||||
| 1 / UnderlayRenderedHistory / 32 | 0.312 | 0.311 | 0.304–0.317 | 0.306–0.314 |
|
||||
| 2 / InheritanceInspectedLoad / 16 | 1.196 | 1.174 | 1.135–1.270 | 1.125–1.218 |
|
||||
| 2 / InheritanceInspectedLoad / 32 | 2.196 | 2.047 | 2.108–2.224 | 2.023–2.084 |
|
||||
| 2 / UnderlayBroadCatalog / 512 | 7.514 | 7.286 | 7.291–7.645 | 6.767–7.319 |
|
||||
| 2 / UnderlayRenderedHistory / 32 | 0.332 | 0.324 | 0.327–0.345 | 0.323–0.342 |
|
||||
|
||||
The large regressions did not recur. Inheritance / 16 was about 4% slower in
|
||||
the first alternating round and about 2% faster in the second; the ranges
|
||||
overlap in both. The other targeted cases were similar or faster after the
|
||||
change, with the same allocation counts as the initial table. This supports
|
||||
host/run variation as the explanation for the large batch differences, but
|
||||
does not establish that every operation is faster or exclude small timing
|
||||
costs. The allocation reduction is the more repeatable result.
|
||||
|
||||
### Profile attribution and retention decision
|
||||
|
||||
Separate profiles used the local 128-pair fixture and
|
||||
`-benchtime=2s`, with explicit output paths outside the repositories:
|
||||
|
||||
```sh
|
||||
GOTOOLCHAIN=local GOWORK=off GOPROXY=off GOMAXPROCS=4 \
|
||||
go test -mod=readonly -run '^$' -bench '^BenchmarkStartupConstruction/128$' \
|
||||
-benchmem -benchtime=2s -cpuprofile=/absolute/temp/cpu.pprof \
|
||||
-memprofile=/absolute/temp/mem.pprof -o /absolute/temp/configkit.test
|
||||
|
||||
go tool pprof -list='engine.finalizeMetadataPaths' -alloc_space /absolute/temp/mem.pprof
|
||||
go tool pprof -list='engine.finalizeMetadataPaths$' /absolute/temp/cpu.pprof
|
||||
go tool pprof -top -nodecount=12 -alloc_space /absolute/temp/mem.pprof
|
||||
```
|
||||
|
||||
The before profiled run completed 34 timed operations at 66.339 ms,
|
||||
43,468,634 B, and 319,128 allocations per operation. After completed 37 at
|
||||
62.460 ms, 42,872,869 B, and 311,703 allocations. These are separate from
|
||||
ordinary timing comparisons; raw profile totals include different operation
|
||||
counts and must not be compared as per-load costs.
|
||||
|
||||
The finalizer's cumulative share of sampled allocation bytes changed from
|
||||
12.04% to 9.75%. Its line profile now attributes final-index construction to
|
||||
one walk, rather than the former two call sites. The remaining
|
||||
`effectivePathIndex.func1` belongs to resolution input; its flat allocation
|
||||
share changed from 3.67% to 2.31%. `clonePathSegments` remains the largest flat
|
||||
site (34.64% before, 33.49% after); required ownership copies remain in place.
|
||||
The finalizer's sampled cumulative CPU share was 8.01% before and 7.87% after,
|
||||
while `Path.String` changed from 17.21% to 19.10%. These short profiles support
|
||||
allocation attribution, not a general CPU-speedup claim.
|
||||
|
||||
Retain the change. The longer 128-pair run removes approximately 7,430
|
||||
allocations and 623 KB per construction; the reduction is reproduced at all
|
||||
local sizes and in the consumer. Combining the initial walks adds little
|
||||
implementation complexity while retaining both indexes, independent path
|
||||
storage, and historical shapes. The possible small timing costs and noisy
|
||||
batch results are recorded, not turned into a performance promise. Retained
|
||||
heap was not measured separately; no persistent cache or new result lifetime
|
||||
is introduced.
|
||||
|
||||
### Validation and remaining work
|
||||
|
||||
The two new metadata tests passed against the original production code before
|
||||
the refactor and then against the candidate. They verify independently owned
|
||||
effective/historical segments in both mutation directions, and retention of
|
||||
both numeric-key and sequence-index historical shapes despite repeated tree
|
||||
contributions. Existing tests continue to own removed descendants, root and
|
||||
escaped paths, detached queries, resolution snapshots, independent loads,
|
||||
inheritance, safe explanation, and sensitivity.
|
||||
|
||||
The focused engine tests for paths, metadata, provenance, queries, resolution,
|
||||
inheritance, explanation, sensitivity, secrets, and independence passed.
|
||||
`go test -race ./internal/engine` passed. `./scripts/check.sh` passed
|
||||
formatting, whitespace, the offline default suite, vet, and builds.
|
||||
Documentation changes passed the local-link and whitespace checks.
|
||||
|
||||
At completion of this focused experiment, Stage 4 had two retained changes and
|
||||
Stage 5's complete consumer verification remained outstanding. The final pass
|
||||
below completes those checks. No consumer dependency upgrade or release is
|
||||
published.
|
||||
|
||||
## Final consumer verification and handoff
|
||||
|
||||
Status: Complete; retain both optimizations and conclude this performance
|
||||
effort. Measured on 2026-10-09. The maintainer confirmed that the host was very
|
||||
busy and selected allocations as the better comparison metric. This pass
|
||||
therefore establishes allocation improvements and behavioral compatibility,
|
||||
not an elapsed-time or CPU-speedup claim.
|
||||
|
||||
### Comparison and reproduction
|
||||
|
||||
- Before: `91ae4fa7eff1571e7c05de01f9e922d75df7453d`, the maintained benchmark
|
||||
commit before either optimization. Its production code matches the original
|
||||
current-code baseline `f1722274c02c08ff28556dd66f29e0da07a9f2c3`.
|
||||
- After: `c546124029645b0635a9babe2d97a74ac77f7f8f`, containing
|
||||
`673faea`'s path-buffer sizing and `c546124`'s combined final metadata walk.
|
||||
These remain the only production changes relative to the before revision.
|
||||
- PromptKit: `eb830bb61346672f0f6c5bce6119b76ccc6d61ac` for both comparisons,
|
||||
with its benchmark, adapter, tests, and other dependencies unchanged.
|
||||
- Separate disposable ConfigKit clones hold the exact before/after commits;
|
||||
only a disposable PromptKit clone's module replacement selects between them.
|
||||
Original consumer files and committed dependencies remain unchanged.
|
||||
- Toolchain, hardware, runtime settings, and offline/readonly module settings
|
||||
match the recorded environment above. Timed runs were sequential and did not
|
||||
overlap our correctness checks or profile analysis. Other host work was not
|
||||
controlled.
|
||||
- All logs and disposable checkouts are outside the repositories at
|
||||
`/tmp/configkit-startup-final.jO52hW`. This pass adds no production code,
|
||||
dependencies, benchmark fixtures, or release gates.
|
||||
|
||||
The primary tables use per-metric medians of five longer observations, run
|
||||
after, then before. Initial batches ran before, then after with
|
||||
`-benchtime=500ms -count=5`; their allocation reductions were reproduced in
|
||||
the longer batches. No outliers were removed from the completed paired batches.
|
||||
|
||||
Run from each ConfigKit copy:
|
||||
|
||||
```sh
|
||||
GOTOOLCHAIN=local GOWORK=off GOPROXY=off GOMAXPROCS=4 \
|
||||
go test -mod=readonly -run '^$' -bench '^BenchmarkStartupConstruction$' \
|
||||
-benchmem -benchtime=1s -count=5
|
||||
```
|
||||
|
||||
In the disposable PromptKit copy, select the relevant ConfigKit copy using
|
||||
the Stage 1 `go mod edit -replace` command, then run:
|
||||
|
||||
```sh
|
||||
GOTOOLCHAIN=local GOWORK=off GOPROXY=off GOMAXPROCS=4 \
|
||||
go test -mod=readonly -run '^$' -bench '^BenchmarkConfigurationConstruction$' \
|
||||
-benchmem -benchtime=1s -count=5 ./internal/configuration
|
||||
```
|
||||
|
||||
### Combined local construction results
|
||||
|
||||
The maintained fixture and timing boundaries are unchanged. These are total
|
||||
allocated bytes and allocation counts per construction, not retained heap.
|
||||
|
||||
| Catalog | Before B | After B | Byte reduction | Before allocations | After allocations | Allocation reduction |
|
||||
| --- | ---: | ---: | ---: | ---: | ---: | ---: |
|
||||
| 32 pairs | 11,999,452 | 10,887,026 | 9.27% | 129,136 | 78,605 | 39.13% |
|
||||
| 128 pairs | 47,264,285 | 42,917,578 | 9.20% | 513,719 | 311,704 | 39.32% |
|
||||
| 512 pairs | 189,216,372 | 171,145,796 | 9.55% | 2,051,983 | 1,244,004 | 39.38% |
|
||||
|
||||
The reductions hold across all three selected sizes. This is scaling evidence
|
||||
for these fixtures, not an asymptotic guarantee. At 128 pairs, the combined
|
||||
changes remove approximately 202,000 allocations and 4.35 MB per construction.
|
||||
|
||||
### Broader PromptKit workloads
|
||||
|
||||
All fixture preparation, reusable source capture, and fixture files are outside
|
||||
timing. Every construction still includes the real adapter's ConfigKit load,
|
||||
callbacks, validation, and post-return prompt materialization, compilation,
|
||||
hashing, and execution-target capture.
|
||||
|
||||
- `independent`: 128 profiles and 128 prompt versions; 256 input/effective
|
||||
entries in one tier.
|
||||
- `file_backed`: 64 profiles and 64 prompt versions; 128 input/effective
|
||||
entries in one tier. Message bodies are read through a captured
|
||||
`fstest.MapFS` namespace, not a physical-disk benchmark. Native-file behavior
|
||||
is separately covered by the configuration tests.
|
||||
- `shared_base`: one base profile, 128 sibling-derived profiles, and one prompt
|
||||
version; 130 input/effective entries in one tier. Each child uses `extends`
|
||||
and overrides temperature while inheriting endpoint, model, and nested
|
||||
provider parameters.
|
||||
- `layered`: three underlays and main input; 513 input entries become 129
|
||||
effective entries across four tiers. Profile fields come from different
|
||||
precedence tiers; one prompt selects the final profile.
|
||||
|
||||
| Workload | Before B | After B | Byte reduction | Before allocations | After allocations | Allocation reduction |
|
||||
| --- | ---: | ---: | ---: | ---: | ---: | ---: |
|
||||
| independent | 50,158,875 | 45,512,786 | 9.26% | 538,542 | 330,484 | 38.63% |
|
||||
| file_backed | 27,045,487 | 24,715,761 | 8.61% | 280,679 | 175,854 | 37.35% |
|
||||
| shared_base | 30,860,505 | 28,240,215 | 8.49% | 383,695 | 243,357 | 36.58% |
|
||||
| layered | 28,415,777 | 25,542,623 | 10.11% | 342,923 | 208,686 | 39.14% |
|
||||
|
||||
The broader consumer cases reproduce reductions through the existing public
|
||||
API. These totals include PromptKit work and must not be presented as isolated
|
||||
ConfigKit costs. No consumer source or dependency upgrade was needed.
|
||||
|
||||
### Complementary allocation evidence
|
||||
|
||||
The original complementary comparison used
|
||||
`-benchtime=200ms -count=3` with the Stage 4 selector for composition,
|
||||
broad underlays, inheritance loading, and rendered history. Per-metric medians:
|
||||
|
||||
| Benchmark / size | Before B | After B | Before allocations | After allocations |
|
||||
| --- | ---: | ---: | ---: | ---: |
|
||||
| ComposeWideMappings / 128 | 848,509 | 797,562 | 9,658 | 5,962 |
|
||||
| ComposeWideMappings / 256 | 1,703,587 | 1,595,291 | 19,652 | 11,860 |
|
||||
| ComposeWideMappings / 512 | 3,626,286 | 3,403,288 | 39,643 | 23,659 |
|
||||
| InheritanceInspectedLoad / 16 | 876,217 | 832,232 | 11,050 | 7,990 |
|
||||
| InheritanceInspectedLoad / 32 | 1,545,193 | 1,460,268 | 19,943 | 14,011 |
|
||||
| InheritanceInspectedLoad / 64 | 2,906,050 | 2,739,206 | 37,778 | 26,098 |
|
||||
| UnderlayBroadCatalog / 128 | 1,265,034 | 1,218,918 | 15,904 | 12,370 |
|
||||
| UnderlayBroadCatalog / 256 | 2,581,008 | 2,437,650 | 33,473 | 24,307 |
|
||||
| UnderlayBroadCatalog / 512 | 5,269,009 | 4,931,073 | 68,581 | 48,151 |
|
||||
| UnderlayRenderedHistory / 32 | 504,551 | 503,463 | 3,294 | 3,221 |
|
||||
| UnderlayRenderedHistory / 64 | 1,006,071 | 1,003,952 | 6,529 | 6,392 |
|
||||
| UnderlayRenderedHistory / 128 | 2,031,859 | 2,027,696 | 12,997 | 12,732 |
|
||||
|
||||
Every case allocates less. Composition exercises the retained path-formatting
|
||||
change but not final metadata indexing. History rendering loads outside timing;
|
||||
its small allocation reduction comes from path formatting, not the fused load
|
||||
traversal.
|
||||
|
||||
Apparent timing regressions prompted five-observation reverse-order repeats of
|
||||
broad-underlay / 128 and / 256 and rendered-history / 32, / 64, and / 128 using
|
||||
`-benchtime=500ms -count=5` and selector
|
||||
`^(BenchmarkUnderlayBroadCatalog|BenchmarkUnderlayRenderedHistory)$/^(32|64|128|256)$`.
|
||||
Their allocation-count medians exactly match the corresponding initial rows,
|
||||
and allocated-byte reductions recur. No persistent cache or result-lifetime
|
||||
change was introduced; retained heap was not measured separately.
|
||||
|
||||
### Timing limitation
|
||||
|
||||
Timing varied greatly without source changes. For example, the final
|
||||
candidate's local / 32 median was 45.098 ms in the initial batch and 17.362 ms
|
||||
in the longer batch. Both showed approximately 78,605 allocations. The initial
|
||||
baseline's / 512 observations ranged from 574.500 to 2,027.077 ms, whereas its
|
||||
longer repeat ranged from 261.986 to 269.806 ms.
|
||||
|
||||
Complementary timing repeats also produced large apparent regressions. Before
|
||||
the maintainer's busy-host notice, one further candidate-only diagnostic
|
||||
completed: broad-underlay / 128 returned from a 7.556 ms median to 1.589 ms,
|
||||
and rendered-history / 32 from 2.386 ms to 0.306 ms, with unchanged source and
|
||||
allocation counts. The subsequent baseline diagnostic was intentionally
|
||||
interrupted, and further timing comparisons stopped. Its partial output is not
|
||||
included in matched results; the interrupted diagnostic is not a test failure.
|
||||
|
||||
Raw logs retain elapsed-time observations, but this pass establishes neither
|
||||
a general speedup nor a reproducible timing regression attributable to the
|
||||
changes. Optional quiet-host timing work can refine that assessment. It is not
|
||||
a remaining requirement for this allocation-focused outcome or a release gate.
|
||||
|
||||
### Behavioral verification and final decision
|
||||
|
||||
Both ConfigKit selections passed PromptKit's complete configuration-package
|
||||
tests and 15 selected public contract tests:
|
||||
|
||||
```sh
|
||||
GOTOOLCHAIN=local GOWORK=off GOPROXY=off \
|
||||
go test -mod=readonly -count=1 ./internal/configuration
|
||||
|
||||
GOTOOLCHAIN=local GOWORK=off GOPROXY=off \
|
||||
go test -mod=readonly -count=1 \
|
||||
-run '^Test(ConfigurationInspection|PublicConstruction|ProfileInheritance|EngineCapturesPromptContent|PreparationStillReads)' .
|
||||
```
|
||||
|
||||
Those checks exercise final effective values, explicit zero/null/omission,
|
||||
normalization history, strict conflicts and tier precedence, imported and
|
||||
native file content, source-context ownership and closing, public inheritance,
|
||||
required credentials, error identities, captured content, and redacted
|
||||
inspection without provider calls. Expectations and fixtures were unchanged.
|
||||
|
||||
ConfigKit's applicable semantic, ownership, race, facade, and everyday-check
|
||||
results from Stage 4 apply to the identical production candidate. This pass
|
||||
reuses those successful checks; its documentation updates passed local-link
|
||||
and whitespace validation. Component responsibilities and documented
|
||||
internal flow remain accurate, so no architecture, ADR, public-contract, or
|
||||
internal-document change is needed.
|
||||
|
||||
Retain both changes. They remove redundant buffer growth and final-tree
|
||||
traversal while preserving separately owned metadata indexes and the existing
|
||||
configuration model. Allocation reductions reproduce locally and across the
|
||||
recorded consumer's independent, content-file, inheritance, and layered
|
||||
workloads, with existing behavioral tests passing. No further optimization is
|
||||
selected for this effort.
|
||||
|
||||
Stages 4–6 are complete. No required consumer verification remains for this
|
||||
scope; timing and retained-heap limits are explicit. Release preparation,
|
||||
roadmap retirement, publication, and any consumer dependency update remain
|
||||
separate work.
|
||||
@@ -1,203 +0,0 @@
|
||||
# Startup Loading Performance Roadmap
|
||||
|
||||
Status: Complete. Two optimizations are retained, with final consumer
|
||||
verification and handoff recorded. Findings and reproduction commands are in
|
||||
[startup-benchmark.md](startup-benchmark.md).
|
||||
|
||||
Audience: ConfigKit maintainers implementing and reviewing this work.
|
||||
|
||||
## Objective and scope
|
||||
|
||||
Reduce avoidable CPU work and temporary allocations during configuration
|
||||
construction through the existing public loading API. The
|
||||
[PromptKit request](promptkit-performance-request.md) owns the downstream
|
||||
workload, original measurements, profiling observations, and reproduction
|
||||
instructions. Its measurements include PromptKit adapter work; they are not
|
||||
standalone ConfigKit costs or evidence of retained-memory growth.
|
||||
|
||||
The initial scope is measurement and internal optimization. Preserve the
|
||||
[configuration contract](../config.md), the public facade, and the ownership
|
||||
boundaries described by the [architecture policy](../policy/architecture.md).
|
||||
In particular, improvements must preserve presence and explicit null, source
|
||||
contexts, provenance and history, recorded resolution, safe inspection, secret
|
||||
handling, and independence of callbacks and published results. Those documents
|
||||
remain the owners of the exact requirements.
|
||||
|
||||
Choose implementations from measured evidence. Path copying, pointer
|
||||
formatting/parsing, and repeated metadata/index construction are initial
|
||||
investigation candidates, not prescribed solutions. This roadmap does not select
|
||||
a cache, alternate metadata mode, new public API, fixed speedup, allocation budget,
|
||||
complexity guarantee, or additional release gate. Any need to expand scope must
|
||||
be assessed against the consumer evidence and existing policies before adoption.
|
||||
|
||||
## Execution order
|
||||
|
||||
Execute the stages below in order. Update each stage's status and record its
|
||||
evidence here as work proceeds. Within stage 4, complete and measure one focused
|
||||
change before selecting the next. If measurements invalidate a hypothesis, return
|
||||
to stage 3 rather than carrying an unsupported design forward. Follow the
|
||||
[development guide](../development.md) and its task-specific policy reading guide
|
||||
throughout execution.
|
||||
|
||||
### 1. Establish the downstream and current-code baselines
|
||||
|
||||
Status: Complete. The recorded PromptKit revision was measured with v0.5.0 and
|
||||
current ConfigKit; allocation costs were reproduced. See the
|
||||
[downstream results](startup-benchmark.md#stage-1-downstream-reproduction).
|
||||
|
||||
1. Inspect PromptKit's recorded benchmark and configuration adapter at the
|
||||
revision identified in its request. Identify which work is timed, including
|
||||
source construction, callbacks, and processing after ConfigKit returns.
|
||||
2. Reproduce the independent fixture with the request's ConfigKit dependency,
|
||||
then with the current ConfigKit revision, using the same PromptKit revision,
|
||||
environment, and benchmark settings. Use isolated checkouts or temporary
|
||||
module wiring without changing committed consumer dependencies.
|
||||
3. Record exact revisions, toolchain, platform, processor, process parallelism,
|
||||
fixture size, setup/timing boundaries, commands, and repeated results. Report
|
||||
elapsed time, allocated bytes, and allocations per construction separately.
|
||||
Keep generated profiles and binaries outside the repositories.
|
||||
4. Explain material differences from the submitted measurements before using
|
||||
them to attribute costs. If the recorded consumer checkout or dependencies
|
||||
are unavailable, document the missing evidence and continue with the local
|
||||
benchmark stages; downstream verification remains outstanding.
|
||||
|
||||
Completion evidence: A reproducible baseline distinguishing the submitted
|
||||
version from current code, or an explicit record of unavailable downstream
|
||||
evidence. Do not attribute all adapter costs to ConfigKit.
|
||||
|
||||
### 2. Add a representative ConfigKit construction benchmark
|
||||
|
||||
Status: Complete. A maintained root-API benchmark measures 32, 128, and 512
|
||||
profile/prompt pairs, with complementary existing benchmarks. See the
|
||||
[local results](startup-benchmark.md#stage-2-maintained-public-api-benchmark).
|
||||
|
||||
1. Add a maintained benchmark through the root public API, using a synthetic
|
||||
configuration shape representative of PromptKit's profiles and versioned
|
||||
prompts. Exercise mapping input, typed decoding, resolution, validation, and
|
||||
publication. Match the relevant callback work observed in stage 1 without
|
||||
importing PromptKit or reproducing its runtime machinery.
|
||||
2. Include the reported fixture size and smaller and larger catalogs to measure
|
||||
scaling. Record both definition counts and the configuration shape; definition
|
||||
counts alone do not describe all fields and metadata paths.
|
||||
3. Make fixture setup and timed source-construction boundaries explicit. Keep
|
||||
unrelated fixture generation outside timing and align comparisons on the
|
||||
work they actually include. Confirm the fixture loads successfully and
|
||||
exercises the intended behavior before collecting results.
|
||||
4. Reuse the existing assembly, underlay, inheritance, and inspection benchmarks
|
||||
where they supply complementary evidence. Add related cases only when they
|
||||
cover a material gap, such as the callback workload or the public facade.
|
||||
|
||||
Completion evidence: Repeatable local construction measurements at several
|
||||
sizes, with commands and fixture definitions recorded. The benchmark remains
|
||||
offline and independent of credentials. Follow the
|
||||
[testing policy](../policy/testing.md); timings and allocation counts are
|
||||
measurements, not machine-specific pass/fail assertions.
|
||||
|
||||
### 3. Profile and choose the first optimization
|
||||
|
||||
Status: Complete. CPU/allocation profiles and the source trace identify
|
||||
`Path.String` buffer sizing as the first experiment. See the
|
||||
[profile evidence and proposal](startup-benchmark.md#stage-3-profiles-and-implementation-trace).
|
||||
|
||||
1. Collect CPU and allocation profiles for the representative local fixture.
|
||||
Compare them with the downstream profile where available, distinguishing
|
||||
ConfigKit loading costs from adapter work. Keep profiled runs separate from
|
||||
ordinary timing comparisons.
|
||||
2. Trace the dominant costs through the current implementation. Begin with path
|
||||
construction and cloning, repeated pointer formatting/parsing, and repeated
|
||||
metadata traversal or index construction. Inspect both the engine and root
|
||||
conversion boundary where profiles point to them.
|
||||
3. Distinguish required ownership separation from redundant internal work.
|
||||
Assess simple local changes before introducing shared state or a broader
|
||||
representation change. Preserve contextual mapping-key versus sequence-index
|
||||
semantics when considering path reuse.
|
||||
4. Select the smallest useful change supported by the profile. Record the
|
||||
hypothesis, affected boundaries, expected measurement signal, and existing
|
||||
behavioral tests that protect those boundaries. Assess scaling from the
|
||||
measurements rather than inferring it from one fixture size.
|
||||
|
||||
Completion evidence: A concrete first optimization with a measured motivation
|
||||
and proportionate validation plan. If no worthwhile change emerges, record that
|
||||
finding and recommend whether further investigation is justified.
|
||||
|
||||
### 4. Implement and measure focused changes
|
||||
|
||||
Status: Complete. `Path.String` buffer sizing and combined final metadata
|
||||
traversal are retained after fresh construction, consumer, inspection, and
|
||||
profile comparisons. See the [path-formatting experiment](startup-benchmark.md#focused-experiment-pre-size-pathstring)
|
||||
and [metadata-traversal experiment](startup-benchmark.md#focused-experiment-combine-final-metadata-traversal).
|
||||
No further optimization is selected for this effort.
|
||||
|
||||
1. Implement the selected internal change while preserving the supported API
|
||||
and established behavior. Audit any reduced copying or reused data for
|
||||
aliases across traversal branches, callbacks, queries, and independent loads.
|
||||
2. Run the relevant existing behavioral tests. Add targeted coverage only for
|
||||
a consequential risk not already protected, including contextual path
|
||||
semantics, history, ownership, or secret redaction where affected. Do not
|
||||
rewrite behavioral expectations merely to accommodate an internal refactor.
|
||||
3. Compare repeated before/after measurements under matching conditions at all
|
||||
selected sizes. Reprofile when needed to explain the result. Measure affected
|
||||
inspection operations as well as loading; if reuse or caching changes data
|
||||
lifetime, examine retained memory separately from cumulative allocations.
|
||||
4. Keep changes with repeatable benefit and acceptable complexity. Investigate
|
||||
regressions, discard unsuccessful experiments, and return to stage 3 for any
|
||||
next candidate. Record the evidence for each retained change.
|
||||
5. Complete the applicable development checks before committing each finished
|
||||
unit under the [agent policy](../policy/agents.md). Use the development guide's
|
||||
race validation when ownership or concurrency changes warrant it.
|
||||
|
||||
Completion evidence: Reviewed, validated changes with attributable improvements
|
||||
and recorded tradeoffs. Stop iterating when remaining candidates no longer
|
||||
justify their complexity; a predetermined optimization count is unnecessary.
|
||||
|
||||
### 5. Verify the complete result against consumer workloads
|
||||
|
||||
Status: Complete. The combined candidate was compared with the pre-optimization
|
||||
baseline across all local sizes, complementary benchmarks, and all four recorded
|
||||
PromptKit workloads. Consumer configuration and public contract tests passed.
|
||||
Acceptance rests on reproducible allocation reductions; the maintainer confirmed
|
||||
that busy-host timing is not a reliable comparison here. See the
|
||||
[final verification](startup-benchmark.md#final-consumer-verification-and-handoff).
|
||||
|
||||
1. Compare the final candidate with the unoptimized current-code baseline using
|
||||
the local fixture and relevant existing benchmarks. Confirm that gains hold
|
||||
across the selected sizes and inspect any material regressions.
|
||||
2. Repeat PromptKit's independent fixture with the candidate under the same
|
||||
conditions as stage 1. Also run its relevant file-backed, inheritance, and
|
||||
layered fixtures, and applicable consumer configuration tests. Record the
|
||||
exact candidate wiring and separate downstream gains from isolated ConfigKit
|
||||
gains. Do not update or publish a consumer dependency as part of this work.
|
||||
3. Confirm that final configuration, presence, provenance/history, inspection,
|
||||
and secret behavior remain consistent through the affected paths. Complete
|
||||
the everyday development check and any additional checks justified by the
|
||||
changes, reusing successful results for unchanged content.
|
||||
|
||||
Completion evidence: Repeatable reductions for the representative construction
|
||||
workload, with related-workload results, semantic validation, and any regressions
|
||||
or limitations explained. If downstream verification is unavailable, identify
|
||||
the exact outstanding check and report local results without claiming PromptKit
|
||||
benefit has been verified.
|
||||
|
||||
### 6. Record the outcome and hand off
|
||||
|
||||
Status: Complete. Retained changes, revisions, allocation comparisons,
|
||||
reproduction commands, behavioral checks, and timing/retained-heap limitations
|
||||
are recorded in the [final handoff](startup-benchmark.md#final-consumer-verification-and-handoff).
|
||||
No required verification remains for this scope. Release preparation and
|
||||
publication remain separate work.
|
||||
|
||||
Summarize the retained changes, before/after results, revisions, reproduction
|
||||
commands, validation, tradeoffs, and unresolved questions in this roadmap.
|
||||
Separate completed work from outstanding consumer checks. Recommend adoption
|
||||
based on the measured benefit and maintenance cost, without promising a
|
||||
machine-independent threshold.
|
||||
|
||||
Update implemented internal documentation only where responsibilities or flow
|
||||
changed. Keep benchmark evidence here and behavior in its canonical owners;
|
||||
follow the [documentation policy](../policy/documentation.md). Publication and
|
||||
release preparation remain separate work under the existing release procedure.
|
||||
|
||||
Completion evidence: A reviewable handoff that makes the improvement and its
|
||||
limits clear. Mark the roadmap complete only when the applicable implementation
|
||||
and verification work is finished; retire it during release preparation under
|
||||
the documentation policy.
|
||||
Reference in New Issue
Block a user