From 95495ae7fc8d2eb9f90aa55105bebc2dbb26b1e6 Mon Sep 17 00:00:00 2001 From: Eric Rakestraw Date: Mon, 2 Nov 2026 16:42:14 +0000 Subject: [PATCH] Record sequence resolution efficiency findings --- docs/roadmap/remediation.md | 82 +++++++++++++++++++++++++++++++++++++ 1 file changed, 82 insertions(+) diff --git a/docs/roadmap/remediation.md b/docs/roadmap/remediation.md index aff889f..3ccbcf8 100644 --- a/docs/roadmap/remediation.md +++ b/docs/roadmap/remediation.md @@ -16,6 +16,12 @@ used by public diff. A private entry-to-final correspondence for registered collections feeds auditing, snapshots, metadata refresh, and shared history selection/replay. The main design is in place, but the following issues remain. +A follow-up integration and algorithm audit at `27be6ba`, including the same +pre-existing working-tree changes, added Findings 4 and 5. It preserved the +decided comparison, identity, ordering, and history policies. Findings 1–3 remain +open; Finding 3 already covers the principal collection-size quadratic loop and +is not duplicated below. + ## Finding 1 — Metadata history rejects a valid matched historical field **Priority:** P2. **Status:** Open. @@ -146,6 +152,72 @@ tests of incidental traversal counts. Reverting to a single incomplete projection pass would lose the dependency-history protection the working-tree change was added to provide. +## Finding 4 — Correspondence lookup repeatedly serializes growing path prefixes + +**Priority:** P2. **Status:** Open. + +**Code:** [correspondence translations](../../internal/engine/resolution_correspondence.go), +`entryPath` at lines 96–125 and `finalPath` at lines 129–156; consumers include +[metadata refresh](../../internal/engine/load_metadata.go), resolution record +validation/snapshots, shared history queries, diagnostic locations, and sensitivity. + +Both translation methods search for a containing collection by constructing and +serializing successively longer prefixes of the requested path. For an unmatched +path with `d` constant-width segments, prefix serialization performs work +proportional to `0 + 1 + ... + (d-1)`, or Theta(d²), per lookup. Each nonempty +serialization also allocates a pointer string. The final owned path copy only +needs Theta(d) work. This prefix search is not an algorithm selected by the +roadmap or plan. + +This also affects ordinary configurations with no registered collections: +`entryPath` still formats every prefix even when `collections` is empty. +`applyResolutionMetadata` calls it while seeding every final field and again +while refreshing explicitness. Thus a resolution callback introduces this cost +across ordinary metadata, not only when records actually move. On a chain of +depth `d`, the lookups at all levels contribute Theta(d³) prefix-serialization +work, compared with Theta(d²) for linear lookup at each level. Existing depth +limits bound the magnitude; this is an avoidable extra depth factor within those +limits, not a claim of unbounded supported input depth. + +**Remediation:** return an owned positional path immediately when there is no +eligible correspondence. For registered scopes, use a segment-prefix index, or +serialize the requested pointer once and query the existing `pathPrefixIndex` +for matching ancestors. Retain concrete path-kind verification so numeric keys +and sequence indexes remain distinct, and preserve additions, removals, and +wholly absent historical-position behavior. Because registration selectors do +not pass through sequence indexes, the relevant matching record prefix does +not require an exhaustive scan of every collection. This can make translation +linear in encoded path length while retaining the current compact pair tables. +Also avoid the preparatory correspondence tree walk when the compiled +application policy contains no registrations. Verify behavioral equivalence +through existing translation, shape, history, and ownership tests rather than +adding timing thresholds or exact allocation assertions. + +## Finding 5 — Delta sorting formats pointers inside every comparison + +**Priority:** P3. **Status:** Open; lower-priority efficiency improvement. + +**Code:** [resolution comparison](../../internal/engine/resolution_compare.go), +`compareResolutionValues` at lines 64–69; compare the existing decorated sort in +[shared structural comparison](../../internal/engine/document_compare.go), +`compareConfigurationValues`. + +The new resolution comparator invokes `Path.String()` on both operands during +every sorting comparison. With `k` deltas, comparison sorting can invoke this +Theta(k log k) times. For pointer length `L`, that contributes Theta(k L log k) +serialization work and Theta(k log k) non-root pointer allocations in the worst +case. This is newly avoidable overhead; the existing shared comparator already +formats one pointer per delta before sorting. + +**Remediation:** follow the existing decorate/sort/unwrap pattern: compute each +delta's pointer once, sort by the retained pointer and existing kind tie-break, +then return the original semantic paths. Pointer construction becomes Theta(k L) +with Theta(k) pointer allocations. Sorting still performs O(k log k) comparisons +with normal string-comparison costs; this does not claim a better worst-case +bound for string sorting itself. Keep canonical ordering as expressly selected +in the plan. Reuse ordering tests; no alternate sort policy, persistent cache, +benchmark gate, or API change is needed. + ## Validation performed - The current working tree passed `./scripts/check.sh` with Go 1.27.0 on Linux. @@ -157,3 +229,13 @@ change was added to provide. - The review found no additional public API restrictions or earlier identity validation requirements. Shared atomic diff comparison, callback entry views, registered selector scope, and YAML-only identity lifetime remain preserved. +- The follow-up traced identity-table construction, resolution comparison, + coverage and snapshot handoffs, metadata refresh, history selection/replay, + and sensitivity propagation. Endpoint matching uses tuple lookup tables rather + than pairwise record searches. Existing whole-tree explanation replay and + canonical ordering were retained as planned choices, not reconsidered. +- Findings 4 and 5 follow directly from the loops, `Path.String` implementation, + and existing prefix-index/decorated-sort alternatives. No new tests or + benchmarks were needed; prior successful checks apply to the unchanged + production and test files. The amended document's links and whitespace were + checked separately.