Adopt shared history ownership in the interface roadmap
This commit is contained in:
@@ -1,6 +1,7 @@
|
||||
# Interface Contract Remediation Roadmap
|
||||
|
||||
Status: proposed target; implementation has not begun.
|
||||
Status: target settled, including shared history ownership; implementation has
|
||||
not begun.
|
||||
Planning baseline: `1f64673` (v0.10.2 implementation plus documentation changes).
|
||||
|
||||
This roadmap covers the four findings in the [interface audit](interfaces.md).
|
||||
@@ -16,6 +17,11 @@ interfaces under their existing contracts. The first three findings concern
|
||||
semantic paths that share a JSON Pointer spelling; the fourth concerns a
|
||||
composition claimant order that is lost during diagnostic canonicalization.
|
||||
|
||||
Build one shared internal history model from which metadata history and
|
||||
explanation are derived. It will own the historical evidence and semantic
|
||||
attribution used by both views, replacing their independent reconstruction of
|
||||
that attribution. This is the selected resolution of OD-1.
|
||||
|
||||
The [Path declarations](../../path.go), [metadata contract](../config.md#layered-metadata),
|
||||
[resolution contract](../config.md#recorded-application-resolution), and
|
||||
[strict-composition contract](../config.md#strict-document-composition) remain
|
||||
@@ -31,11 +37,17 @@ as `/shape/0`. `ParsePath` deliberately leaves numeric tokens contextual, wherea
|
||||
shape-aware effective/historical path indexes and a frozen resolution-entry view,
|
||||
but some queries and metadata updates still use pointer spelling alone.
|
||||
|
||||
History currently reconstructs definition and previous-origin records, while
|
||||
explanation replays source contributions, inheritance, decoding checkpoints, CLI
|
||||
assignments, and grouped resolution snapshots. Neither public representation
|
||||
contains everything needed by the other. The target is shared underlying evidence
|
||||
with distinct projections for each interface, preserving their existing behavior.
|
||||
|
||||
| Finding | Current discrepancy | Target |
|
||||
| --- | --- | --- |
|
||||
| 1: resolution presence | `IsExplicit` and `Default` consult string-keyed presence after schema validation, which cannot determine actual shape beneath dynamic values. | Presence decisions use the matching semantic path in the fixed resolution-entry state. |
|
||||
| 2: effective explicitness | Resolution retains old field information when pointer spelling survives a container-shape change. | Input explicitness carries forward for the same semantic path; a newly introduced key or index does not inherit another path's explicitness. |
|
||||
| 3: history attribution | A historical-shape check admits the query, then `History` returns the entire textual-pointer bucket. | Exact definitions and previous origins belong to the queried semantic path, with applicable ancestor operations retained. |
|
||||
| 3: history attribution | A historical-shape check admits the query, then `History` returns the entire textual-pointer bucket; explanation selects evidence separately. | Both interfaces derive attribution from one shared history model, selecting exact definitions, previous origins, and applicable ancestor operations by semantic path. |
|
||||
| 4: claimant order | Composition constructs related locations in assembly order, then canonicalization sorts them by location. | Structured and rendered composition conflicts preserve the selected claimant order. |
|
||||
|
||||
For example, if input contains `shape: {'0': old}` and recorded application
|
||||
@@ -92,44 +104,133 @@ Use the existing before/after projection and entry metadata where possible.
|
||||
This work does not require tracking every transient mutation inside a callback;
|
||||
the existing net-change and coalesced-record model continues to apply.
|
||||
|
||||
### 3. Retain Semantic Attribution In History
|
||||
### 3. Build A Shared Internal History Model
|
||||
|
||||
Repair history event construction and selection across
|
||||
[load_metadata.go](../../internal/engine/load_metadata.go),
|
||||
[metadata.go](../../internal/engine/metadata.go), and their inheritance-history
|
||||
integration. Finding 3's choice of internal evidence representation is discussed
|
||||
under [Open Decisions](#open-decisions).
|
||||
#### Evidence And Ownership
|
||||
|
||||
An explicit query should select definitions belonging to its semantic shape.
|
||||
Ancestor replacement or removal operations can still affect that path and must
|
||||
remain in its history where applicable. An ancestor operation's `Change.Path`
|
||||
need not equal the queried descendant; this is existing behavior.
|
||||
Give historical evidence one private owner in the engine. Composition,
|
||||
inheritance, decoding, CLI assignment, and resolution continue to perform their
|
||||
existing work and supply evidence to that owner. The model does not drive a
|
||||
second configuration load or repeat application callbacks.
|
||||
|
||||
Retain enough evidence to distinguish the affected path from the operation's
|
||||
reported path. Merely filtering current `Change.Path` values is insufficient:
|
||||
some construction paths recreate parsed tokens from strings, and ancestor events
|
||||
intentionally name another path. Review event construction for defaults, input
|
||||
definitions, precedence operations, inheritance, CLI, and resolution to identify
|
||||
where these distinctions can be lost. Change the affected owners without
|
||||
duplicating the entire loading lifecycle.
|
||||
Organize the evidence into ordered groups with the phase boundaries required by
|
||||
the existing inspection contracts:
|
||||
|
||||
Select or reconstruct `Previous` origins using the relevant semantic history.
|
||||
Review `historyWithInheritance` as well as stored events: filtering out a false
|
||||
definition while retaining it as the previous origin would leave the attribution
|
||||
defect unresolved. Preserve chronology and genuine ancestor effects, including
|
||||
history of removed paths. A path that never existed does not acquire history
|
||||
solely because an ancestor changed.
|
||||
| Evidence group | Information to preserve |
|
||||
| --- | --- |
|
||||
| Default baseline | Canonical typed application defaults or Go-zero values and their default attribution. |
|
||||
| Source contributions | Underlay, main-tier, and selected-overlay contributions in their existing order; exact definitions, composite origins, displaced definitions, raw policy-validated values, and sequence-position projections. |
|
||||
| Inheritance | Parent transfers, child contributions including equal-valued replacements, controlling directives and their consumption, identity-field preservation, and contribution origins/dependencies. |
|
||||
| Typed checkpoints | Pre-CLI and post-CLI typed projections and attribution needed to reconcile retained defaults and raw contributions with decoded values. |
|
||||
| CLI contributions | Explicit assignments, including equal-valued assignments, raw supplied representation, source attribution, and affected paths. |
|
||||
| Resolution | Original recorded operations, reasons, dependencies, and before/after evidence, with the grouping needed to explain overlapping records as net subtree changes. |
|
||||
|
||||
Retain the existing contextual behavior of `ParsePath` across historical shapes
|
||||
and the one-entry-per-canonical-pointer enumeration of `HistoricalPaths`. Neither
|
||||
needs a public redesign. `History` and `Explain` should agree on which path a
|
||||
definition belongs to and what affected it; their different record forms do not
|
||||
need to become identical lists.
|
||||
The logical order remains baseline, source contributions, inheritance, pre-CLI
|
||||
decoding reconciliation, CLI contributions, post-CLI typed reconciliation, then
|
||||
recorded resolution. The two typed checkpoints belong on their respective sides
|
||||
of the CLI contribution phase.
|
||||
|
||||
Reuse existing evidence where practical. Any additional retained evidence must
|
||||
respect existing ownership, confidentiality, and applicable accounting; avoid
|
||||
retaining extra document trees merely to identify a path's shape. This is not a
|
||||
new memory quota or a general history-storage redesign.
|
||||
The groups describe logical inspection chronology, not a transcript of every
|
||||
physical mutation. Retain the existing net-change and coalescing rules for
|
||||
resolution. Preserve superseded raw input that need not decode into the final
|
||||
application type; typed checkpoints do not replace that evidence.
|
||||
|
||||
Use a small set of ordinary private structures and existing document/path
|
||||
mechanisms. Exact types, indexes, and storage layout belong in the implementation
|
||||
plan. Complete evidence means enough for the existing history, explanation, and
|
||||
internal attribution consumers; it does not add persistence, a public event API,
|
||||
or recording of transient application effects.
|
||||
|
||||
#### Shared Selection And Attribution
|
||||
|
||||
Retain concrete semantic-path evidence when it is available. A query over the
|
||||
model must distinguish exact definitions from operations affecting an ancestor
|
||||
or descendant, determine the relevant previous origins, and apply sequence
|
||||
contribution coordinates at their assigned positions.
|
||||
|
||||
The affected semantic path and the operation's reported path are distinct facts.
|
||||
For example, a replacement at `/shape` can remove a mapping key and introduce a
|
||||
sequence index with the same pointer spelling. Preserve enough before/after shape
|
||||
evidence to answer each explicit query correctly. Merely filtering `Change.Path`
|
||||
after concrete segment kinds have been discarded is insufficient.
|
||||
|
||||
Make this selection and attribution common to both projections, including
|
||||
inheritance integration and `Previous` reconstruction. A false definition must
|
||||
not survive as a previous origin after it has been excluded from the query's
|
||||
history. Preserve genuine ancestor replacement/removal effects and chronology.
|
||||
A path that never existed does not acquire history solely because an ancestor
|
||||
changed.
|
||||
|
||||
Keep the contextual meaning of parsed numeric tokens across historical states
|
||||
and the one-entry-per-canonical-pointer enumeration of `HistoricalPaths`.
|
||||
Indexes and compact derived records may remain, but their attribution must come
|
||||
from the shared evidence. Metadata queries should not require full configuration
|
||||
replay or YAML rendering to obtain definition and origin records.
|
||||
|
||||
#### Distinct Public Projections
|
||||
|
||||
`Metadata.History` produces detached definition, inheritance, and resolution
|
||||
records from the model. `Explain` reconstructs and safely renders values from
|
||||
the same evidence, retaining its existing decoding and grouping behavior. Neither
|
||||
public API should be implemented by translating the other's public result.
|
||||
|
||||
Preserve these intentional differences:
|
||||
|
||||
| Case | Metadata history | Explanation |
|
||||
| --- | --- | --- |
|
||||
| Several strict contributions to a mapping | Composite definition/origin information. | Each contribution's effect on the mapping. |
|
||||
| Raw input differs from canonical typed values | Definition and attribution records. | The applicable decoding reconciliation transition. |
|
||||
| Overlapping resolution records | Specific recorded attribution under the existing coverage rules. | One net subtree event with combined reasons and dependencies. |
|
||||
| Equal-valued input or inheritance operations | Explicit definition and operation attribution remains visible. | Relevant operations remain visible even when before and after values are equal. |
|
||||
|
||||
Agreement means consistent semantic identity and attribution, not identical
|
||||
record lists. Retain original resolution records as evidence from which net
|
||||
replay groups are derived; a grouped explanation alone loses information needed
|
||||
by metadata.
|
||||
|
||||
#### Replace Existing Attribution Machinery
|
||||
|
||||
The common model must replace independent historical attribution, rather than
|
||||
become another representation alongside it. The implementation plan should map
|
||||
the following owners to their replacements:
|
||||
|
||||
| Current owner | Required change |
|
||||
| --- | --- |
|
||||
| History-building portions of `metadataFromDecoded`, `applyOverlayMetadata`, and `appendPostDecodeCLIHistory` in [load_metadata.go](../../internal/engine/load_metadata.go) and [override_context.go](../../internal/engine/override_context.go) | Produce or project shared evidence instead of independently reconstructing pointer-keyed histories. Retain effective-field construction where it is still needed. |
|
||||
| Previous-origin reconstruction and inheritance insertion in [inheritance_history.go](../../internal/engine/inheritance_history.go) | Use common semantic selection and attribution while preserving the existing inheritance projection rules. |
|
||||
| Relevance and chronology orchestration in [explain.go](../../internal/engine/explain.go) | Consume the shared model; retain useful value reconstruction and redaction helpers. |
|
||||
| Separate published inheritance evidence for metadata and explanation in [load_result.go](../../internal/engine/load_result.go) | Consolidate under the frozen history owner where independent copies are unnecessary; keep required output detachment. |
|
||||
| Original and coalesced resolution evidence in [history.go](../../internal/engine/history.go) | Own original records in the model and derive the replay grouping without losing specific attribution or maintaining a second independently interpreted history. |
|
||||
|
||||
Existing source, sequence-combination, and inheritance evidence producers may
|
||||
remain useful. Their algorithms need not be rewritten merely to contribute to
|
||||
the model. Likewise, final effective fields, the frozen resolution-entry view,
|
||||
source records, and final configuration snapshots retain their responsibilities.
|
||||
Findings 1 and 2 still require explicit presence corrections.
|
||||
|
||||
#### Other Consumers, Lifetime, And Resource Accounting
|
||||
|
||||
Move historical dependency reads in
|
||||
[secret classification](../../internal/engine/secret.go) and historical location
|
||||
fallbacks in [override_context.go](../../internal/engine/override_context.go) to
|
||||
the shared evidence. Do not retain the old history machinery solely for those
|
||||
readers. Preserve existing sensitivity propagation and historical classification;
|
||||
correcting attribution does not authorize narrowing redaction for paths that
|
||||
share a pointer spelling.
|
||||
|
||||
Evidence must be available at the lifecycle stages where these internal readers
|
||||
need it, including diagnostic attribution before publication. Keep the callback's
|
||||
entry metadata fixed while later resolution evidence is added. Finalize the
|
||||
model for published queries without introducing lazy mutation of shared state.
|
||||
Private frozen evidence may be shared by the two views while their returned
|
||||
records remain detached under the existing ownership contract.
|
||||
|
||||
Reuse retained trees and existing lookup machinery where practical. Review
|
||||
retained-history accounting against the resulting ownership graph, including
|
||||
typed checkpoints and inheritance/resolution evidence. The model does not require
|
||||
a full document copy per event, a new quota, or a new concurrency guarantee.
|
||||
Reduced duplication is a design benefit to evaluate, not a promised memory or
|
||||
speed improvement.
|
||||
|
||||
### 4. Preserve Composition Claimant Order Through Diagnostics
|
||||
|
||||
@@ -152,17 +253,22 @@ dependency-ready scheduling unchanged.
|
||||
|
||||
## General Implementation Order
|
||||
|
||||
1. Resolve the history-evidence decision and prepare focused public regression
|
||||
cases from the audit. Establish expected semantic-path behavior using existing
|
||||
contracts and shared path-matching helpers.
|
||||
1. Prepare the concrete shared-evidence design and focused regression cases.
|
||||
Map existing producers and consumers to the new owner, distinguish common
|
||||
attribution from presentation rules, and identify the old paths to remove.
|
||||
Select representations and indexes that preserve existing query and lifetime
|
||||
needs without adding public contracts.
|
||||
2. Repair entry-state presence queries and `Default` (finding 1). This is a local
|
||||
correction that can proceed independently of the larger history work.
|
||||
3. Repair final explicitness preservation (finding 2), retaining the shape evidence
|
||||
needed by metadata updates. Coordinate this work with history construction so
|
||||
the two fixes do not introduce competing definitions of path identity.
|
||||
4. Repair historical definitions and previous-origin attribution (finding 3),
|
||||
then check their interaction with final metadata and explanation. A successful
|
||||
presence fix alone does not establish correct history.
|
||||
4. Build the shared model and migrate metadata history, explanation, sensitivity
|
||||
dependencies, and diagnostic-location readers. Preserve phase evidence and
|
||||
derive both public projections through common semantic attribution. Remove
|
||||
the superseded reconstruction paths and reconcile retained-history accounting.
|
||||
Verify finding 3 and its interaction with final presence metadata; repairing
|
||||
pointer buckets alone does not complete the selected architecture.
|
||||
5. Complete composition diagnostic ordering (finding 4) as an independent change;
|
||||
it may proceed alongside the path work. Check rendering as well as the
|
||||
structured error before considering it complete.
|
||||
@@ -190,6 +296,16 @@ The focused cases should cover:
|
||||
`Previous`, and genuine ancestor replacement/removal. Include contextual
|
||||
enumeration/query coverage and preserve existing inheritance and overlay cases;
|
||||
add another case only where those integrations expose an unprotected risk.
|
||||
- Shared-model projections across composite mapping origins, typed decoding and
|
||||
default reconciliation, equal-valued writes, inheritance controls/transfers,
|
||||
mapped sequence replacements and removed descendants, and overlapping
|
||||
resolution records. Reuse existing cases where they protect those semantics;
|
||||
check the intentional differences between the public projections as well as
|
||||
agreement on attribution.
|
||||
- Historical sensitivity/dependency propagation, diagnostic source-location
|
||||
fallback, detached returned records, and complete retained-history accounting
|
||||
under the new ownership graph. Use small controlled budget cases and existing
|
||||
disclosure fixtures rather than memory or timing thresholds.
|
||||
- A composition conflict whose acquisition and assembly orders differ, using the
|
||||
audit fixture. Check both published diagnostics and rendered related-location
|
||||
order, while retaining coverage for ordinary diagnostic canonicalization.
|
||||
@@ -200,40 +316,31 @@ observable behavior rather than a particular private index or number of copies.
|
||||
No benchmark campaign, broader audit, or new release gate is part of this roadmap.
|
||||
|
||||
Completion requires all four findings to have implementation and regression
|
||||
evidence, agreement at the affected public boundaries, and appropriate development
|
||||
checks. The audit's previous passing suite is not validation of these future
|
||||
changes. Scope here does not promise identical output from different inspection
|
||||
interfaces or a universal guarantee about unrelated path-processing code.
|
||||
evidence, appropriate development checks, and both history interfaces deriving
|
||||
semantic attribution from the shared model. Review the final producer/consumer
|
||||
map to confirm superseded independent reconstruction has been removed and
|
||||
internal attribution readers have migrated. Passing the four defect regressions
|
||||
while retaining the old parallel attribution machinery is not completion.
|
||||
|
||||
## Open Decisions
|
||||
The audit's previous passing suite is not validation of these future changes.
|
||||
Scope here does not promise identical output from different inspection interfaces
|
||||
or a universal guarantee about unrelated path-processing code.
|
||||
|
||||
### OD-1. Where Should Shape-Specific History Evidence Live?
|
||||
## Decision Status
|
||||
|
||||
**Recommendation:** retain semantic-shape evidence with the internal history
|
||||
events as they are built, using existing concrete paths and document evidence.
|
||||
Select applicable events and previous origins for each query before producing
|
||||
detached public `Change` records. The exact representation belongs in the
|
||||
implementation plan.
|
||||
OD-1 is settled: build one shared internal history model with distinct metadata
|
||||
and explanation projections. The design investigation found that both views
|
||||
require evidence the other's public representation does not preserve. Shared
|
||||
ownership can replace independent attribution and redundant retained evidence,
|
||||
while keeping the current lifecycle and public contracts.
|
||||
|
||||
This fits the existing stored-history owner and keeps metadata queries independent
|
||||
of explanation rendering. It also permits a focused correction where event
|
||||
construction currently loses shape. The cost is carrying and preserving evidence
|
||||
across several event producers; the design must keep it consistent with replay
|
||||
and avoid redundant retained trees. The amount of new state should follow what
|
||||
existing evidence cannot supply, not an assumed new event framework.
|
||||
The narrower repair of stored history events would leave the separate replay
|
||||
attribution owner in place. Promoting current explanation output to authoritative
|
||||
history would lose composite-definition and specific resolution information.
|
||||
The selected common underlying model addresses both concerns. This accepts a
|
||||
larger internal refactor than a local shape-filtering fix.
|
||||
|
||||
**Viable alternative:** derive the relevant metadata history from existing
|
||||
shape-aware replay evidence, sharing a private selection operation with
|
||||
explanation where their semantics agree. This can reduce duplicated attribution
|
||||
logic and reuse evidence that already handles the reported transition correctly.
|
||||
However, metadata history includes default/definition and inheritance records
|
||||
whose presentation differs from explanation. This option needs a clear account
|
||||
of those differences, query cost, and evidence ownership; routing `History`
|
||||
through the public explanation representation would introduce unnecessary
|
||||
coupling.
|
||||
|
||||
Choose between these approaches after a focused comparison of the affected event
|
||||
producers and replay coverage, before finalizing the implementation plan. The
|
||||
decision concerns private ownership and implementation cost, not public path or
|
||||
history semantics. No other significant product or architectural decision is
|
||||
open for this scope.
|
||||
No significant open decision remains at roadmap level. Private representation,
|
||||
indexing, and helper choices belong in the implementation plan. This roadmap
|
||||
records future implementation ownership; it does not revise accepted ADRs or
|
||||
declare the model implemented.
|
||||
|
||||
@@ -6,7 +6,8 @@ Status: four open findings. This is an audit record, not an implementation plan.
|
||||
No production code or repository tests were changed.
|
||||
|
||||
The [interface remediation roadmap](interface-remediation.md) describes the
|
||||
proposed target, required changes, dependencies, and remaining design decision.
|
||||
settled target, required changes, and dependencies, including a shared internal
|
||||
history model for metadata history and explanation. Implementation remains pending.
|
||||
This audit retains the findings and reproduction evidence.
|
||||
|
||||
## Scope and evidence
|
||||
|
||||
Reference in New Issue
Block a user