496 lines
30 KiB
Markdown
496 lines
30 KiB
Markdown
# Implement the performance audit revisions
|
||
|
||
Status: Complete. Audience: a Luna coding agent implementing one stage at a time,
|
||
in numerical order. The [performance audit roadmap](performance-audit.md) owns
|
||
the target and rationale. This plan replaces the completed ReleaseKit plan,
|
||
which remains available in Git history at `0c77dee`.
|
||
|
||
Planning baseline: `0c77deed05417edb32f42c7bf24df2fdc2581a25`.
|
||
The production baseline is unchanged from the audit's `48619de` revision.
|
||
Keep the earlier Path.String sizing and combined final-metadata traversal
|
||
improvements. This sprint changes private algorithms, not public API,
|
||
configuration semantics, dependencies, Go minimum, or release gates.
|
||
|
||
## Working instructions
|
||
|
||
- Read [development.md](../development.md), the applicable agent, architecture,
|
||
testing, and documentation policies, and owning contract/test files before
|
||
each stage. Use the graph tools for code discovery as instructed by AGENTS.md.
|
||
- Preserve user changes. Implement only the current stage and its regressions;
|
||
update its status/evidence here and commit the validated unit with a short
|
||
plain-English message. Do not publish, push, or modify consumer dependencies.
|
||
- Operation indexes are private phase-local data. Inspection indexes are fully
|
||
built before publication and only read afterward. Returned records remain
|
||
detached. Keep effective and historical indexes separate with independently
|
||
owned mutable path storage.
|
||
- Preserve the existing [configuration contract](../config.md): presence,
|
||
null/empty distinctions, composition, diagnostics, chronology, sensitivity,
|
||
callback lifetime, secret isolation, and handle closure. Do not add rejection
|
||
rules, limits, stricter callbacks, concurrency promises, or complexity guarantees.
|
||
- Run each stage's focused checks and `./scripts/check.sh` before handing off
|
||
production changes. Run `go test -race ./internal/engine` after retained-index
|
||
or ownership changes, and the full race suite in Stage 18. Reuse checks for
|
||
unchanged content. No new CI wait, release gate, or performance threshold.
|
||
- Prefer algorithm reasoning for this sprint. Benchmark runs are not prerequisites.
|
||
Optional measurements use the same prepared local Go, offline dependencies,
|
||
sequential runs, and artifacts outside repositories. Favor B/op and allocs/op
|
||
on the busy host; do not infer CPU improvements from noisy timings.
|
||
- Extend existing behavioral tests for missing risks, not traversal counts.
|
||
Small test-only baseline algorithms may support bounded differential coverage.
|
||
Do not keep alternate production engines or switches. If the selected algorithm
|
||
cannot preserve behavior, record the discrepancy and seek review rather than
|
||
strengthening the contract or silently dropping work.
|
||
|
||
## Resolved implementation choices
|
||
|
||
1. Containment and path deduplication use canonical token spelling, as today.
|
||
Metadata/document queries additionally enforce contextual key/index shape.
|
||
These predicates remain distinct.
|
||
2. Use a small private engine prefix trie with canonical **escaped** pointer
|
||
tokens and integer IDs into owner-held slices. Root is zero tokens; `/`
|
||
is one empty token. Scan separator boundaries without decoding/rejoining
|
||
tokens. Escapes contain no slash. The trie does not validate document shape.
|
||
3. The helper returns IDs, not values/provenance. Owners select deepest ancestor,
|
||
earliest inserted ancestor/intersection, or all relevant IDs, explicitly
|
||
restoring owner order. No general path framework or public hook is introduced.
|
||
4. Sensitivity uses a worklist: ordinary dependencies activate on intersection,
|
||
while directed inheritance links activate on a classified ancestor/equal
|
||
path of their dependency. Preserve both rules.
|
||
5. Inspection retains full replay and on-demand safe rendering. Prepare immutable
|
||
coalesced resolution descriptors and inheritance lookup data once, with no
|
||
per-field tree or rendered-explanation cache.
|
||
6. Defer branch-local replay, public metadata-copy redesign, global schema caching,
|
||
and complex contributor-trace membership indexing. Their ownership/memory/
|
||
equality risks exceed the selected scope. Existing clones and linked traces
|
||
stay in place. No new dependency, pool, or package-global mutable cache is needed.
|
||
|
||
## Stage 1: Confirm the baseline and semantic evidence — Complete
|
||
|
||
Scope: orientation and test-only evidence, with no optimization yet.
|
||
|
||
1. Inspect status/revisions. Read the audit and configuration contract sections
|
||
for metadata, sensitivity, explanation, resolution, and overrides, plus their
|
||
implementation/test owners.
|
||
2. Run `./scripts/check.sh`. Record revision, local `go version`, result, and
|
||
environmental limitations in an evidence subsection here. Do not repeat or
|
||
revise the completed startup timing campaign.
|
||
3. Identify existing coverage for numeric shapes, removed paths, escaped keys,
|
||
inheritance, callback detachment, and independent loads. Add test-only reference
|
||
helpers if useful for later containment, coverage/coalescing, overlap, and
|
||
sensitivity equivalence tests. Keep fixtures bounded and deterministic; do not
|
||
copy the loader. Helpers may instead accompany their first consuming stage.
|
||
|
||
Done: baseline checks recorded and semantic test owners identified. Existing
|
||
contracts/tests are authoritative; reference algorithms add equivalence evidence,
|
||
not new public guarantees. A documentation-only stage needs documentation checks.
|
||
|
||
### Evidence
|
||
|
||
- Current revision: `52793301ba80eb49f91956009e60cd812f92e94c`; planning baseline:
|
||
`0c77deed05417edb32f42c7bf24df2fdc2581a25`. The working tree was clean, and
|
||
production Go files match the audit baseline `48619de024d794cf41e90b171b7fcbe609dcd437`.
|
||
- `GOTOOLCHAIN=local go version`: `go1.26.8 linux/amd64`.
|
||
`GOTOOLCHAIN=local ./scripts/check.sh`: passed (`go test ./...`, `go vet ./...`,
|
||
and `go build ./...`, in addition to formatting and whitespace checks). No
|
||
environmental limitations encountered; no benchmark campaign was repeated.
|
||
- Existing semantic coverage includes numeric/key-index and escaped path shapes
|
||
(`path_test.go`, `provenance_test.go`, `override_test.go`), removed/history
|
||
paths (`metadata_test.go`, `resolution_test.go`, `inheritance_history_test.go`),
|
||
nested and sequence-contained inheritance (`inheritance_test.go`,
|
||
`inheritance_walk_test.go`), detached callback/metadata records
|
||
(`callback_ownership_test.go`, `resolution_test.go`, `metadata_test.go`), and
|
||
independent loads (`provenance_test.go`). Differential reference helpers are
|
||
deferred to the first stage that consumes each algorithm so they can remain
|
||
small and bounded.
|
||
|
||
## Stage 2: Make presence-only queries lightweight — Complete
|
||
|
||
Roadmap candidate 1. Owner: `internal/engine/metadata.go`.
|
||
|
||
1. Make `Metadata.IsExplicit` validate effective path/shape and read the stored
|
||
boolean, without calling `Lookup`, cloning origins, or projecting inheritance.
|
||
A shared private effective-field lookup may serve both methods; `Lookup`
|
||
continues returning complete detached records.
|
||
2. Format each query pointer once per lookup. Preserve root, zero metadata,
|
||
invalid/unknown/removed paths, parsed numeric tokens, and explicit key/index
|
||
distinctions. Leave `Resolution.IsExplicit` entry-state semantics unchanged.
|
||
3. Extend metadata/presence tests only where agreement with `Lookup` lacks
|
||
coverage: direct/inherited input, defaults, and explicit false/zero/null.
|
||
|
||
Validate: `go test ./internal/engine -run 'Metadata|Presence|Explicit|Inherited'`,
|
||
relevant root facade tests, and the everyday check. No allocation assertion.
|
||
|
||
## Stage 3: Detect inheritance without constructing paths — Complete
|
||
|
||
Roadmap candidate 9. Owner: `internal/engine/inheritance_load.go`.
|
||
|
||
1. Replace detection's full path-building walk with a nil-safe boolean visitor:
|
||
inspect mapping keys for `extends`, recurse into values/sequence children,
|
||
and return immediately on a match.
|
||
2. Leave validation, expansion order, provenance, and budgets unchanged.
|
||
Detection is not validation and must still find nested invalid syntax.
|
||
3. Reuse inheritance tests; add missing sequence-contained or invalid nested
|
||
cases without assertions about how many nodes were visited.
|
||
|
||
Validate: `go test ./internal/engine -run 'Inheritance|Inherited'` and the
|
||
everyday check. Inputs without inheritance still need a linear scan.
|
||
|
||
## Stage 4: Compare path tokens without whole-pointer formatting — Complete
|
||
|
||
Roadmap candidate 2. Owners: engine `path.go`, `resolution.go`, `metadata.go`.
|
||
|
||
1. Add private token-spelling equality. Key/parsed segments compare literal
|
||
bytes; two indexes compare integers. For index versus key/parsed, render the
|
||
index into stack scratch space with `strconv.AppendInt` and compare bytes.
|
||
Do not parse literal tokens as integers: `01`, `+1`, and `1` differ.
|
||
2. Implement `pathContains` with existing validity/depth checks and token-prefix
|
||
equality. Root behavior remains unchanged. Escaping is injective, so raw
|
||
token spelling comparison preserves canonical pointer equivalence.
|
||
3. Replace textual comparison in `matchesPathShape` with token equality, then
|
||
retain its existing explicit/contextual shape checks and candidate-side
|
||
handling. Apply the same spelling predicate to local textual path deduplication.
|
||
4. Differentially compare with the String-based baseline over roots, invalid
|
||
paths, empty/slash/tilde/numeric keys, parsed tokens, nested paths, and integer
|
||
extrema. Reuse existing effective/historical query-shape coverage.
|
||
|
||
Validate: `go test ./internal/engine -run 'Path|Metadata|Resolution|Override|Inheritance'`
|
||
and the everyday check. Public `Path.String` stays unchanged.
|
||
|
||
## Stage 5: Format sorting keys once per item — Complete
|
||
|
||
Roadmap candidate 2. Owners: engine resolution, overlay, override application,
|
||
document comparison, and diagnostics.
|
||
|
||
1. Decorate path-bearing items with precomputed canonical pointers, sort with
|
||
existing comparisons/tie-breakers, then project back. Cover
|
||
`orderedResolutionRecords`, `displacedOrigins`, `affectedOverrideOrigins`,
|
||
and the final delta sort in `compareConfigurationValues`.
|
||
2. In `canonicalDiagnostics`, precompute paths without changing comparator
|
||
precedence, primary/related location ordering, or standalone comparator use.
|
||
Do not add permanent pointer fields to every record for sorting.
|
||
3. Gather recursive displaced definitions into one owned output and sort at
|
||
the outer owner, avoiding repeated sorting of the same nested subtree.
|
||
Preserve final lexical order and duplicate behavior.
|
||
4. Reuse ordering tests; add missing tie/escaped-key cases. Do not promise stable
|
||
order for items whose full comparison is equal.
|
||
|
||
Validate: `go test ./internal/engine -run 'Resolution|Override|Overlay|Diagnostic|CompareDocument|Provenance'`
|
||
and the everyday check. Expected benefit is reduced formatting, not measured CPU.
|
||
|
||
## Stage 6: Add a private canonical-pointer prefix index — Complete
|
||
|
||
Scope: one helper plus tests, no caller migration yet.
|
||
|
||
1. Add `internal/engine/path_prefix.go`. Nodes have child maps keyed by escaped
|
||
tokens, terminal integer payload IDs, and the minimum inserted ID within the
|
||
subtree. IDs are nonnegative owner ordinals; represent absence explicitly.
|
||
2. Support insertion, ancestor payload visits (root/exact included), descendant
|
||
payload visits (exact included), and earliest intersecting ID using ancestor
|
||
IDs and the queried subtree minimum. Preserve multiple IDs at one terminal.
|
||
No deletion, wildcard search, provider abstraction, or serialization API.
|
||
3. Tokenize canonical pointers by separator scanning without split slices.
|
||
Preserve root versus empty keys and `/a` versus `/ab` boundaries. Owners
|
||
supply canonical pointers from valid paths; no new input validation contract.
|
||
Retain IDs only, not nodes, credential bytes, or mutable path segments.
|
||
4. Differentially check results against simple prefix scans for ancestors,
|
||
descendants, duplicates, insertion minima, escaping, and root/empty tokens.
|
||
|
||
Validate: focused helper tests and the everyday check. Storage follows indexed
|
||
token material/payloads; tests do not fix representation or allocation counts.
|
||
|
||
## Stage 7: Index resolution coverage and record reads — Complete
|
||
|
||
Roadmap candidate 3, first integration. Owner: `internal/engine/resolution.go`.
|
||
|
||
1. Index already ordered records after the callback. Select deepest covering
|
||
ancestor, earliest owner ID for equal depth, matching the current scan.
|
||
Use this for delta coverage in `runResolution`; retain all validity/helper
|
||
checks and diagnostic order.
|
||
2. Add a small snapshot coverage adapter with the same rules for Stage 8.
|
||
IDs refer to owner-held slices and cannot outlive or survive their reordering.
|
||
3. Build operation-local read-only mapping lookups for completed before/after
|
||
trees, reusing `mappingIndex` mechanics without mutating trees. Add a private
|
||
indexed document lookup variant with the same validity, shape, bounds, and
|
||
absence checks; retain the plain helper for callers without reuse.
|
||
4. Cover parent/exact/more-specific records, equal-depth private fixtures,
|
||
added/removed values, unrecorded changes, and helper-target mismatch.
|
||
|
||
Validate: `go test ./internal/engine -run 'Resolution|Resolve|Callback|Path'`
|
||
and the everyday check. Snapshot ownership/contents remain unchanged.
|
||
|
||
## Stage 8: Replace per-delta whole-metadata scans — Complete
|
||
|
||
Roadmap candidate 3, second integration. Owner: engine `load_metadata.go`.
|
||
|
||
1. In `applyResolutionMetadata`, build one final pointer/node index and one
|
||
snapshot coverage index. Reuse final membership for descendant history and
|
||
field refresh; do not rebuild indexes per delta/field.
|
||
2. Index delta paths by original delta ID. Enumerate the union of historical
|
||
and final pointers once, obtain covering delta IDs, and bucket affected pointers
|
||
by delta. Parse paths only when a retained semantic path is actually needed.
|
||
Sort buckets lexically and retain original delta processing order.
|
||
3. Apply exact snapshot events first, descendant history second, effective origin
|
||
updates third, then final field refresh, as today. Exclude the covering record's
|
||
own pointer from descendant history and retain the original Previous origins.
|
||
Handle overlapping delta IDs in reference fixtures without assuming every
|
||
possible input is disjoint. Normal structural comparison emits disjoint deltas.
|
||
4. For fields, visit pointers once and process matching delta IDs in original
|
||
order. For new fields, use snapshot coverage; preserve null/Explicit state,
|
||
default resolution-origin fallback, and removed-path history.
|
||
5. Extend existing tests for many disjoint changes, replacement, removed/new
|
||
descendants, and numeric container-shape changes. A bounded test-only
|
||
reference metadata update can protect phase/order equivalence.
|
||
|
||
Validate: `go test ./internal/engine -run 'Resolution|Metadata|Provenance|History|Explain'`,
|
||
engine race tests, and the everyday check. Complexity is output-sensitive to
|
||
affected events, not every delta times every field.
|
||
|
||
## Stage 9: Collect sensitivity directly from trees — Complete
|
||
|
||
Roadmap candidate 5. Owner: engine `secret.go`. Keep fixed-point propagation
|
||
unchanged until Stage 11 so collection is independently reviewable.
|
||
|
||
1. Replace temporary pointer/node indexes used only to enumerate classification
|
||
paths/flags with direct visitors into local path and sensitivity accumulators.
|
||
Remove the redundant initial final-tree scan; final still participates in
|
||
the late projection phase before schema/predicate classification.
|
||
2. Preserve phase precedence: applications; history/dependencies; inheritance;
|
||
projections then final. Tree/pointer contributions currently install parsed
|
||
segments, while history/contributor contributions can install typed paths.
|
||
Preserve those representations and subsequent overwrites.
|
||
3. Skip reparsing only if the current stored path already has that parsed
|
||
representation. Otherwise perform the existing parsed replacement. Do not
|
||
simply substitute explicit walker key/index paths or retain the first shape.
|
||
4. Retain all historical/removed paths, controls, dependency-only paths, default/
|
||
dynamic tagged projections, and cloned predicate arguments.
|
||
5. Add only missing numeric shape-transition, historical predicate, dynamic tag,
|
||
and predicate-detachment cases in existing sensitivity/inheritance test owners.
|
||
|
||
Validate: `go test ./internal/engine -run 'Secret|Sensitivity|Inherited|Effective|Explain'`,
|
||
engine race tests, and the everyday check. No new predicate order/count guarantee.
|
||
|
||
## Stage 10: Build and test indexed sensitivity triggers — Complete
|
||
|
||
Roadmap candidate 6, isolated algorithm. Add private
|
||
`internal/engine/secret_propagation.go` and focused tests.
|
||
|
||
1. Ordinary edges have dependency/target pointers and activate on classified
|
||
ancestor or descendant intersection. Inheritance links become two directed
|
||
edges, each activated only by a classified ancestor/equal dependency path.
|
||
2. Index dependency pointers with payload IDs. A new classified pointer visits
|
||
ordinary triggers at ancestors and descendants; inheritance triggers only
|
||
at descendants. Avoid double-visiting exact terminals. Track fired IDs and
|
||
enqueue a target only when it becomes newly sensitive.
|
||
3. Seed from the initial sensitive set. Preserve the ordinary dependency filter:
|
||
the old `secretIntersectsPath` sees only classified pointers present in the
|
||
collected paths map. Inheritance's `pathIsSecret` does not require membership.
|
||
Relations use textual spelling, not document query-shape validation.
|
||
4. Differentially compare final sets to the old fixed-point algorithm for
|
||
forward/reverse chains, cycles, duplicate edges, classified ancestors/
|
||
descendants, classified pointers absent from paths, root, escapes/numeric
|
||
spellings, and mixed ordinary/inheritance graphs. Use no real credentials.
|
||
|
||
Validate: focused propagation tests and the everyday check. Match/output-sized
|
||
work may still be large for broad dependencies; no blanket linear guarantee.
|
||
|
||
## Stage 11: Integrate worklist sensitivity into publication — Complete
|
||
|
||
Roadmap candidate 6, integration. Owner: engine `secret.go`.
|
||
|
||
1. Feed collected paths, seeds, dependencies, and inheritance links into Stage
|
||
10's closure. Remove the production fixed-point loop/unused scan helper after
|
||
migration; retain the useful reference only in tests.
|
||
2. Verify full-load effective rendering and explanations for mixed derived/
|
||
inherited sensitivity, replaced/removed paths, tagged containers, consumed
|
||
controls, and nested copies. All credentials remain outside trees/indexes.
|
||
3. Verify independent loads and repeated queries retain no mutable propagation
|
||
state. Queues and trigger indexes are phase-local and discarded on return.
|
||
|
||
Validate: `go test ./internal/engine -run 'Secret|Sensitivity|Inheritance|History|Explain|Independent'`,
|
||
relevant root facade secret/inheritance tests, engine race tests, and the everyday
|
||
check. Record collection and propagation outcomes separately.
|
||
|
||
## Stage 12: Index CLI overlaps and validation reads — Complete
|
||
|
||
Roadmap candidate 4. Owner: engine `override.go`.
|
||
|
||
1. Retain input order and the duplicate seen map. Replace the prior-accepted
|
||
overlap scan with earliest intersecting trie lookup using original override
|
||
indices. Duplicate diagnostics continue preceding overlap checks.
|
||
2. Insert exactly where `pathAccepted[index] = true` occurs today: after path,
|
||
overlap, version, and directive checks, before value conversion/budgets.
|
||
A later bad value still influences subsequent overlap diagnostics. Do not
|
||
index paths rejected earlier or omit those accepted before value failure.
|
||
3. Reuse an operation-local mapping index for read-only base traversal during
|
||
path validation. Preserve missing mapping ancestors, preparation-before-apply,
|
||
CLI source order/identity, rules, and limits.
|
||
4. Compare with baseline for disjoint/duplicate/ancestor/descendant paths,
|
||
multiple conflicts, escapes, rejected paths, and later value failures.
|
||
Protect diagnostic codes, paths, and first related-source selection.
|
||
|
||
Validate: `go test ./internal/engine -run 'Override|CLI|Provenance|Diagnostic'`
|
||
and the everyday check. No policy changes.
|
||
|
||
## Stage 13: Coalesce resolution replay once with indexed ancestors — Complete
|
||
|
||
Roadmap candidate 7. Owners: engine `history.go`, `load.go`, `load_result.go`.
|
||
|
||
1. Replace group scans with ancestor lookup selecting the earliest inserted
|
||
matching group, not the deepest. Insert only new groups; merged child paths
|
||
are not independently indexed as groups.
|
||
2. Retain original before/after trees and normalization precedence. Use ordered
|
||
membership sets for reason/dependency unions, retaining first occurrence and
|
||
canonical pointer equivalence.
|
||
3. Prepare immutable coalesced descriptors once before publication. Explain
|
||
reads them without mutating Changes or borrowed snapshots. Retain original
|
||
snapshots for their other metadata/history-budget uses; node accounting is
|
||
unchanged. No renderer or public records are cached.
|
||
4. Differentially check independent/parent/child records, multiple ancestors,
|
||
root, equal writes, union order, normalization, and repeated detached results.
|
||
|
||
Validate: `go test ./internal/engine -run 'Resolution|History|Explain|Ownership'`,
|
||
engine race tests, and the everyday check. Full replay semantics stay intact.
|
||
|
||
## Stage 14: Index relevant inheritance history operations — Complete
|
||
|
||
Roadmap candidate 7. Owners: engine metadata, metadata cloning, inheritance history.
|
||
|
||
1. During final metadata preparation, index inheritance operation IDs by control
|
||
path or transfer destination. Collect intersecting IDs for History, deduplicate,
|
||
restore original operation order, and run existing projection logic.
|
||
2. Preserve stored-history splitting before resolution and Previous-origin
|
||
construction. Keep fields/history/path clones independent. A fully frozen
|
||
ID-only index may be shared in `cloneMetadata`, like immutable evidence;
|
||
no query builds or updates it. Zero/private unprepared metadata must still
|
||
produce correct results (use the existing scan if no prepared index exists).
|
||
3. Leave global `appendInheritanceExplanation` replay intact: non-intersecting
|
||
operations can affect working state and must not be skipped with this filter.
|
||
4. Compare indexed history to the scan for multi-scope controls, removed paths,
|
||
ancestor queries, and query mutations/captured metadata independence.
|
||
|
||
Validate: `go test ./internal/engine -run 'Inheritance|Inherited|History|Metadata|Independent'`,
|
||
engine race tests, and the everyday check.
|
||
|
||
## Stage 15: Reuse frozen inspection mapping lookups — Complete
|
||
|
||
Roadmap candidate 7. Owners: publication, inspection, document/snapshot/history helpers.
|
||
|
||
1. Reuse Stage 7's mapping lookup builder for immutable decoded-input, final,
|
||
and application trees queried repeatedly by inspection. Populate all retained
|
||
maps before return, visiting shared node identities once. Retain lookup positions
|
||
and borrowed immutable nodes, not extra trees or Path copies.
|
||
2. Thread read-only lookups into applicable document/snapshot and application
|
||
origin/relevance reads. Preserve validity, shape, bounds, and absence. Read
|
||
populated maps directly, not lazy entries methods that write on first query.
|
||
3. Keep plain document lookup on mutable replay trees in this stage. Their
|
||
existing query-local merge indexes are unchanged; do not reuse a frozen map
|
||
after cloning, deletion, or entry-position changes. This avoids adding an
|
||
invalidation protocol just to optimize the selected immutable reads.
|
||
4. Reuse shape/history/render tests; add only missing repeated/concurrent
|
||
read-only inspection ownership cases. Do not add a public concurrency promise.
|
||
Record extra retained-index storage honestly; this is a lookup tradeoff,
|
||
not a free allocation reduction. Baseline/full-tree replay remains unchanged.
|
||
|
||
Validate: `go test ./internal/engine -run 'Path|History|Explain|Effective|Independent|Parallel'`,
|
||
engine race tests, and the everyday check.
|
||
|
||
## Stage 16: Index materialized inheritance-step membership — Complete
|
||
|
||
Roadmap candidate 8. Owner: engine `inheritance_origin.go`.
|
||
|
||
1. Keep output order and compact linked traces. Use a comparable key with three
|
||
canonical From/To/Reference strings and one encoded ordered Locations string.
|
||
Encode the location count and each SourceID byte length, then its bytes and
|
||
delimited decimal line/column integers. This is collision-free, including
|
||
empty IDs, delimiter characters, and signed coordinates. Arbitrary unescaped
|
||
delimiter concatenation is incorrect; no hashing dependency is needed.
|
||
2. Seed membership from existing Origin.Inheritance but retain its existing
|
||
duplicates/order. Only newly proposed steps are deduped, as today.
|
||
3. Format candidate pointers once; clone retained paths/locations only for newly
|
||
accepted steps. Keep contributor traversal and transfer direction intact.
|
||
4. Compare chains, shared contributors, different location/order steps, existing
|
||
duplicates, escaped/numeric paths, and returned-record mutation to baseline.
|
||
|
||
Validate: `go test ./internal/engine -run 'Inheritance|Inherited|Metadata|Provenance|History'`
|
||
and the everyday check. Complex contributor-trace equality stays unchanged.
|
||
|
||
## Stage 17: Index simple resolution and origin membership — Complete
|
||
|
||
Roadmap candidate 8. Owners: engine `resolution.go` and `load_metadata.go`.
|
||
|
||
1. Maintain lazily initialized first-occurrence reason/dependency sets on
|
||
resolution records. Dependencies use canonical pointer equality; retained
|
||
slices and callback validation/order/nil behavior remain unchanged.
|
||
2. Use exact Location membership in `originContainedBy`, retaining Layer/
|
||
Operation and contributor-empty checks. Use SourceID membership in
|
||
`originRepresentedByChange`, which has a different source-coverage predicate.
|
||
Do not conflate these equality rules.
|
||
3. Leave complex contributor scans and `sameInheritanceTrace` intact. Their
|
||
concrete segment/location equality cannot be replaced by pointer identity
|
||
or canonical spelling. Sets are phase-local, not exported record fields.
|
||
4. Reuse existing provenance tests; cover duplicate unions, equal source IDs
|
||
with differing locations, and location coverage with contributor mismatch
|
||
only where protection is missing.
|
||
|
||
Validate: `go test ./internal/engine -run 'Resolution|Provenance|Overlay|Override|Inheritance|History'`
|
||
and the everyday check. Small lists and outputs preserve existing semantics.
|
||
|
||
## Stage 18: Review the completed result and record outcomes — Complete
|
||
|
||
1. Review the diff from baseline. Map all nine candidates to selected implemented
|
||
changes and expressly deferred portions. Remove dead helpers/imports/state;
|
||
keep useful small equivalence tests, no alternate production engines.
|
||
2. Run `./scripts/check.sh` and `go test -race ./...`. Verify public/import
|
||
boundaries, examples, independent loads, shapes, presence, history, redaction,
|
||
and complete-on-return behavior through their existing tests.
|
||
3. Record algorithmic before/after work, lifetime/storage tradeoffs, revisions,
|
||
actual commands/results, and unmeasured claims in an evidence subsection here.
|
||
Optional probes may reuse existing startup/assembly/underlay/inheritance
|
||
benchmarks and small record/override/dependency/query fixtures. Measurements
|
||
are not prerequisites; timing claims await a representative host.
|
||
4. Update stage and roadmap status accurately. Update focused internal docs
|
||
only where they describe changed implemented mechanics. Keep public contracts
|
||
and policy language unchanged. Do not retire other historical records or
|
||
roadmaps as incidental cleanup.
|
||
5. Review links/whitespace/staging, commit the completed unit, and report outcomes,
|
||
checks, remaining deferred experiments, and workspace status. No release or
|
||
downstream bump is included.
|
||
|
||
Done: selected changes preserve behavior, remain proportionate/maintainable,
|
||
pass validation, and distinguish algorithmic expectations from measured results.
|
||
No public performance guarantee is created.
|
||
|
||
## Open Questions
|
||
|
||
None required for the selected scope. Algorithm, ownership, relation, ordering,
|
||
and scope choices above are resolved. Expressly deferred experiments require
|
||
separate design/evidence and do not block this sequence.
|
||
|
||
### Final evidence and outcomes
|
||
|
||
- Reviewed planning baseline `0c77deed05417edb32f42c7bf24df2fdc2581a25`,
|
||
production baseline `48619de024d794cf41e90b171b7fcbe609dcd437`, and the
|
||
complete implementation diff through `799ebab5a6f15e8252474b8cb7ea164ef503f390`.
|
||
The selected changes map to the audit candidates as follows: candidate 1 in
|
||
Stage 2; candidate 2 in Stages 4–5; candidate 3 in Stages 6–8; candidate 4
|
||
in Stage 12; candidate 5 in Stage 9; candidate 6 in Stages 10–11; candidate 7
|
||
in Stages 13–15; candidate 8 in Stages 16–17; and candidate 9 in Stage 3.
|
||
- `GOTOOLCHAIN=local go version` reports `go1.26.8 linux/amd64`.
|
||
`go test ./internal/engine -run 'Resolution|Provenance|Overlay|Override|Inheritance|History'`,
|
||
`./scripts/check.sh`, and `go test -race ./...` passed. The development check
|
||
includes formatting, whitespace, all-package tests, vet, and build. No
|
||
environmental limitations were encountered.
|
||
- The selected algorithms replace repeated pairwise/list scans or repeated
|
||
formatting with token comparisons, local prefix indexes, ordered membership
|
||
maps, direct visitors, and a worklist. Operation-local indexes and propagation
|
||
state are discarded after use. Inspection retains immutable node-to-mapping
|
||
position lookups and prepared replay descriptors for repeated read-only
|
||
queries; this trades additional retained index storage for reuse. Per-record
|
||
reason/dependency sets and materialized-step keys likewise use memory in
|
||
proportion to retained unique values while preserving ordered output slices.
|
||
- Whole-tree explanation replay, public metadata-copy ownership redesign, global
|
||
schema caching, and complex contributor-trace membership indexing remain
|
||
deferred. Existing cloning and linked traces remain. No benchmark campaign
|
||
was run; expected work reductions are algorithmic, not measured runtime
|
||
claims, and no public performance guarantee is added.
|