Compare commits
35 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| dc09959cca | |||
| 83c59ea7d3 | |||
| b45f7a0410 | |||
| 2ca1f80094 | |||
| df9a02539c | |||
| 501d59f21b | |||
| f916e7d1f9 | |||
| 462e57df3e | |||
| 4207110e2f | |||
| fdbc306cfb | |||
| 78d4b59bbe | |||
| f564fef84c | |||
| 01e38b5840 | |||
| ac1eec2e30 | |||
| 0a4b774bfe | |||
| b604aa167b | |||
| 15f5083282 | |||
| cbf031aedc | |||
| 34e18c84c1 | |||
| 36362ee6af | |||
| 5a77728a81 | |||
| 1cdf90d831 | |||
| 90c6e8ed7e | |||
| 1ff3c812a7 | |||
| 6111cb23a1 | |||
| de28718941 | |||
| 8d9a3a8dfa | |||
| 27df838513 | |||
| 36172ed20e | |||
| e1ce24aabb | |||
| e4bbedc695 | |||
| 798091b276 | |||
| 5c1aa1814c | |||
| ec26591b27 | |||
| 99338baabb |
50
README.md
50
README.md
@@ -5,18 +5,22 @@ An opinionated Go library for applications that use YAML configuration files.
|
||||
Linux is release-validated; macOS support is best-effort. See the
|
||||
[platform support policy](docs/policy/release.md#platform-support) for details.
|
||||
|
||||
ConfigKit's initial loader accepts one strict YAML
|
||||
source (or explicit defaults-only input), optionally traverses explicit path and glob
|
||||
imports, strictly composes their disjoint definitions, applies typed defaults, and
|
||||
can overlay one selected fixed overlay and check an application schema version. It
|
||||
returns detached metadata and safe diagnostics.
|
||||
ConfigKit's simple `Load` accepts one strict YAML source (or explicit
|
||||
defaults-only input). `LoadLayered` additionally combines explicit lower-priority
|
||||
source groups with an independently selected main input. Sources can be host
|
||||
files, native `fs.FS` files, labeled bytes or readers, or snapshotted mappings.
|
||||
Enabled path and glob imports compose strictly within their group; precedence
|
||||
between groups combines mappings recursively. Both entry points return a complete,
|
||||
validated configuration with detached metadata and safe diagnostics.
|
||||
Applications may perform audited defaulting and normalization through a resolver,
|
||||
then report domain errors and warnings through read-only semantic validation. Secret
|
||||
tags and a path predicate drive deterministic redacted effective YAML. Typed secret
|
||||
references resolve environment names before validation, and explicit
|
||||
secrets-directory bootstrap is available as a separate startup operation. Explicit
|
||||
CLI override records apply above overlays. Value explanation is available through
|
||||
detached structured records and deterministic redacted output.
|
||||
detached structured records and deterministic redacted output. Built-in
|
||||
`extends` relationships let a mapping inherit from a named sibling during the
|
||||
same load, retaining explicit presence and source/derivation history.
|
||||
|
||||
```go
|
||||
type Config struct {
|
||||
@@ -33,11 +37,37 @@ func ReadConfig(path string) (Config, error) {
|
||||
}
|
||||
```
|
||||
|
||||
## Design philosophy
|
||||
|
||||
ConfigKit favors one coherent, predictable configuration model over flexibility
|
||||
for reproducing each consumer's conventions. Its goal is to let applications
|
||||
load once, inspect where final values came from, and distinguish explicit values
|
||||
from omissions with minimal configuration bookkeeping. Applications retain their
|
||||
domain rules and execution responsibilities.
|
||||
|
||||
Loading is synchronous and complete on return. Applications normally load once
|
||||
near startup and then use the final configuration and metadata; ConfigKit stays
|
||||
inactive until explicitly called again. Applications may initiate additional
|
||||
loads as needed. See the [loading lifecycle](docs/config.md#completion-and-resource-lifetime)
|
||||
for completion and ownership details.
|
||||
|
||||
Within a configuration tier, independent contributions compose strictly: shared
|
||||
mappings combine, while overlapping non-mapping definitions are errors. Between
|
||||
precedence tiers, mappings combine recursively, retaining omitted lower-priority
|
||||
children; other values replace lower-priority values. The main document and its
|
||||
imports form one strictly composed tier. Explicit underlay groups sit below it
|
||||
and above typed defaults; a selected overlay and explicit CLI overrides sit
|
||||
above it. The [configuration contract](docs/config.md#strict-document-composition)
|
||||
owns the detailed combination rules and input-specific constraints.
|
||||
|
||||
The [architecture goals](docs/policy/architecture.md#success-criteria) explain
|
||||
these priorities and how genuine new needs can justify extending the shared
|
||||
model. The [deferred scope](docs/roadmap/deferred.md) records deliberate exclusions.
|
||||
|
||||
The [configuration contract](docs/config.md) describes implemented behavior. The
|
||||
[consumer adoption guide](docs/consumers/README.md) provides maintained Narratio-,
|
||||
Notarius-, WeatherReporter-, and Seriatim-shaped examples and migration guidance.
|
||||
The [deferred scope](docs/roadmap/deferred.md) records deliberate exclusions. See the
|
||||
[development guide](docs/development.md) for repository orientation and the
|
||||
[consumer adoption guide](docs/consumers/README.md) provides maintained simple
|
||||
native-filesystem, layered, and sibling-inheritance examples alongside the migration shapes.
|
||||
See the [development guide](docs/development.md) for repository orientation and the
|
||||
[release procedure](docs/release.md) for maintainer-only candidate and publication
|
||||
steps.
|
||||
|
||||
|
||||
@@ -43,16 +43,36 @@ func toEngineLocations(values []Location) []engine.Location {
|
||||
return result
|
||||
}
|
||||
func fromEngineSourceInfo(value engine.SourceInfo) SourceInfo {
|
||||
return SourceInfo{ID: value.ID, Label: value.Label, Path: value.Path, Layer: Layer(value.Layer), ImportedFrom: fromEngineLocationPointer(value.ImportedFrom)}
|
||||
return SourceInfo{ID: value.ID, Label: value.Label, Path: value.Path, Layer: Layer(value.Layer), Kind: SourceKind(value.Kind), UnderlayIndex: value.UnderlayIndex, ImportedFrom: fromEngineLocationPointer(value.ImportedFrom)}
|
||||
}
|
||||
func toEngineSourceInfo(value SourceInfo) engine.SourceInfo {
|
||||
return engine.SourceInfo{ID: value.ID, Label: value.Label, Path: value.Path, Layer: engine.Layer(value.Layer), ImportedFrom: toEngineLocationPointer(value.ImportedFrom)}
|
||||
return engine.SourceInfo{ID: value.ID, Label: value.Label, Path: value.Path, Layer: engine.Layer(value.Layer), Kind: engine.SourceKind(value.Kind), UnderlayIndex: value.UnderlayIndex, ImportedFrom: toEngineLocationPointer(value.ImportedFrom)}
|
||||
}
|
||||
func fromEngineOrigin(value engine.Origin) Origin {
|
||||
return Origin{Layer: Layer(value.Layer), Locations: fromEngineLocations(value.Locations), Operation: ChangeKind(value.Operation)}
|
||||
result := Origin{Layer: Layer(value.Layer), Locations: fromEngineLocations(value.Locations), Operation: ChangeKind(value.Operation)}
|
||||
if value.Inheritance != nil {
|
||||
result.Inheritance = make([]InheritanceStep, len(value.Inheritance))
|
||||
for index, step := range value.Inheritance {
|
||||
result.Inheritance[index] = InheritanceStep{
|
||||
From: fromEnginePath(step.From), To: fromEnginePath(step.To), Reference: fromEnginePath(step.Reference),
|
||||
Locations: fromEngineLocations(step.Locations),
|
||||
}
|
||||
}
|
||||
}
|
||||
return result
|
||||
}
|
||||
func toEngineOrigin(value Origin) engine.Origin {
|
||||
return engine.Origin{Layer: engine.Layer(value.Layer), Locations: toEngineLocations(value.Locations), Operation: engine.ChangeKind(value.Operation)}
|
||||
result := engine.Origin{Layer: engine.Layer(value.Layer), Locations: toEngineLocations(value.Locations), Operation: engine.ChangeKind(value.Operation)}
|
||||
if value.Inheritance != nil {
|
||||
result.Inheritance = make([]engine.InheritanceStep, len(value.Inheritance))
|
||||
for index, step := range value.Inheritance {
|
||||
result.Inheritance[index] = engine.InheritanceStep{
|
||||
From: toEnginePath(step.From), To: toEnginePath(step.To), Reference: toEnginePath(step.Reference),
|
||||
Locations: toEngineLocations(step.Locations),
|
||||
}
|
||||
}
|
||||
}
|
||||
return result
|
||||
}
|
||||
func fromEngineDiagnostic(value engine.Diagnostic) Diagnostic {
|
||||
return Diagnostic{Code: Code(value.Code), Severity: Severity(value.Severity), Path: fromEnginePath(value.Path), Message: value.Message, Primary: fromEngineLocationPointer(value.Primary), Related: fromEngineLocations(value.Related)}
|
||||
|
||||
118
docs/adr/0007-source-backed-underlays.md
Normal file
118
docs/adr/0007-source-backed-underlays.md
Normal file
@@ -0,0 +1,118 @@
|
||||
# ADR 0007: Add Source-Backed Underlays with Consistent Recursive Precedence
|
||||
|
||||
## Status
|
||||
|
||||
Proposed draft for review. Completed planning and remediation records are
|
||||
preserved in Git history at `b604aa1` under `docs/roadmap/`; the
|
||||
[configuration contract](../config.md#layered-loading) owns current behavior.
|
||||
|
||||
This extends [ADR 0001](0001-separate-composition-and-overlay.md) and preserves
|
||||
its distinction between strict composition and intentional overlay.
|
||||
|
||||
## Date
|
||||
|
||||
2026-09-29
|
||||
|
||||
## Context
|
||||
|
||||
Configuration-backed defaults provide lower-priority settings that applications
|
||||
can override. These sources carry imports, locations, and relative-resource
|
||||
contexts. Integrating them into ConfigKit's loading lifecycle gives their values
|
||||
the same presence tracking and inspectable provenance as other configuration
|
||||
inputs.
|
||||
|
||||
The [architecture goals](../policy/architecture.md#success-criteria) prioritize
|
||||
one coherent, predictable configuration model with consistent rules across
|
||||
source representations and loading interfaces. Consumers should load once,
|
||||
receive a complete validated configuration, and inspect its final values' origins.
|
||||
Reducing integration code is important subject to those principles.
|
||||
|
||||
## Decision
|
||||
|
||||
### Add underlays to the existing precedence model
|
||||
|
||||
Introduce explicitly supplied source-backed underlays between typed defaults
|
||||
and main configuration. The proposed precedence, from lowest to highest, is:
|
||||
|
||||
**Typed application defaults < source underlays < strictly composed main
|
||||
configuration < selected overlay < explicit CLI overrides.**
|
||||
|
||||
Typed defaults remain the Go-value baseline. Underlays supply lower-priority
|
||||
configuration with source provenance and, where applicable, imports and resource
|
||||
context. This ordering describes value precedence; typed defaults fill fields
|
||||
absent from the assembled input under the existing defaulting rules.
|
||||
|
||||
Preserve strict composition of the main document and its imports before applying
|
||||
the resulting configuration over underlays. Independent contributions within an
|
||||
underlay tier also compose strictly. Explicitly ordered tiers express intentional
|
||||
precedence. Strict composition reports conflicting contributions within each tier.
|
||||
|
||||
Across precedence tiers, reuse the existing recursive mapping-overlay semantics.
|
||||
Omitted children retain lower-priority input; existing scalar, sequence, null,
|
||||
and cross-kind replacement semantics remain unchanged. Reusing these operations
|
||||
keeps the combination rules consistent across the loading model.
|
||||
|
||||
### Keep source representation separate from composition semantics
|
||||
|
||||
Native `fs.FS` and programmatic inputs join the shared loading model.
|
||||
Equivalent inputs using the same
|
||||
operation have the same presence and precedence semantics regardless of medium.
|
||||
Programmatic values enter directly as labeled configuration sources.
|
||||
|
||||
Retain the contributing source's identity and resource context through layering.
|
||||
A retained underlay value keeps its originating source's resource context.
|
||||
Consumers remain responsible for opening and interpreting their domain resources
|
||||
under the supplied filesystem's lifetime and access semantics. Exact
|
||||
context-access and input APIs remain roadmap decisions.
|
||||
|
||||
### Deliver one final configuration and its inspection metadata
|
||||
|
||||
Assemble effective input, then run the typed loading lifecycle once for the load.
|
||||
Preserve explicit zero,
|
||||
false, empty, and null input versus omission through configuration and metadata
|
||||
together.
|
||||
|
||||
Extend existing provenance and history through underlays, defaults, and recorded
|
||||
application resolution. The returned result includes the completed combination,
|
||||
defaulting, and provenance work. Domain resolution and validation
|
||||
remain application-owned within the recorded lifecycle, consistent with
|
||||
[ADR 0003](0003-preserve-presence-and-resolution-history.md).
|
||||
|
||||
Preserve the existing opt-in schema-version ownership, path protections, and
|
||||
check timing before precedence
|
||||
assembly and typed decoding. The roadmap owns how layered input identifies that
|
||||
owner when version checking is configured.
|
||||
|
||||
Favor near-linear processing in input, retained history, and output size through
|
||||
incremental assembly of load-owned state. This is an engineering objective
|
||||
subordinate to clarity and semantic consistency. Preserve the root-owned facade and
|
||||
private implementation boundary established by [ADR 0006](0006-public-facade.md).
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
- **Leave bundled defaults to consumers.** Avoids extending loading, but makes
|
||||
consumers reproduce merging, presence tracking, and provenance reconciliation.
|
||||
It does not meet the goal of obtaining a complete configuration in one load.
|
||||
- **Offer recursive or atomic combination per layer.** Makes both behaviors
|
||||
explicit and could serve more consumers, but expands the semantic surface and
|
||||
its interactions with presence and provenance without a demonstrated need that
|
||||
outweighs a single consistent rule. A future concrete requirement may justify
|
||||
reconsideration; this feature does not add the option.
|
||||
- **Flatten all documents into last-wins layers.** Simplifies ordering but turns
|
||||
accidental main/import conflicts into overrides, weakening the intentional
|
||||
distinction established by ADR 0001.
|
||||
|
||||
## Consequences
|
||||
|
||||
Consumers gain source-backed defaults within the shared loading lifecycle.
|
||||
Strict contributions and intentional precedence remain distinct, with consistent
|
||||
combination rules across source media. Presence and provenance remain
|
||||
part of the same loading lifecycle as the final values.
|
||||
|
||||
Higher-priority mappings retain omitted lower-priority children, so application
|
||||
validation assesses the final combined configuration.
|
||||
|
||||
Implementation must integrate source context and underlay history into the shared
|
||||
engine with attention to assembly and history-retention costs. Current behavior
|
||||
documentation will change when the feature lands. The roadmap retains ownership
|
||||
of the remaining API decisions.
|
||||
82
docs/adr/0008-complete-on-return-loading.md
Normal file
82
docs/adr/0008-complete-on-return-loading.md
Normal file
@@ -0,0 +1,82 @@
|
||||
# ADR 0008: Synchronous, Complete-on-Return Configuration Loading
|
||||
|
||||
## Status
|
||||
|
||||
Proposed draft for review. The
|
||||
[configuration contract](../config.md#completion-and-resource-lifetime) describes
|
||||
the implemented loading lifecycle and
|
||||
[source resource contexts](../config.md#source-resource-contexts).
|
||||
|
||||
## Date
|
||||
|
||||
2026-09-29
|
||||
|
||||
## Context
|
||||
|
||||
Applications need a clear point at which configuration is ready for use.
|
||||
Completing configuration processing and inspection metadata together gives
|
||||
applications a coherent result whose values and origins agree. A bounded loading
|
||||
lifecycle also makes resource ownership and startup failures straightforward to
|
||||
reason about.
|
||||
|
||||
ConfigKit's [architecture goals](../policy/architecture.md#success-criteria)
|
||||
emphasize complete configuration, inspectable provenance, predictable behavior,
|
||||
and minimal consumer bookkeeping. Applications own their execution lifecycle and
|
||||
the timing of configuration acquisition.
|
||||
|
||||
## Decision
|
||||
|
||||
Make loading a synchronous, complete-on-return operation. Within each load,
|
||||
ConfigKit completes input assembly, defaulting, typed decoding, configured
|
||||
application resolution, secret lookup, validation, and snapshotting before
|
||||
returning the final configuration and metadata. Reported failures return
|
||||
diagnostics and a nil result.
|
||||
|
||||
Release all library-owned loading handles before returning on success or failure.
|
||||
The resulting configuration and metadata are independently usable, caller-owned
|
||||
data with a retained inspection snapshot. Caller-supplied readers and filesystems
|
||||
remain under caller ownership and lifetime control.
|
||||
|
||||
After the call completes, ConfigKit remains inactive until explicitly invoked
|
||||
again. Applications normally load once at or near startup and may initiate
|
||||
additional loads when needed. Each load follows the same lifecycle and source
|
||||
reuse contracts. Inspection, rendering, and separately requested operations run
|
||||
in response to explicit application calls.
|
||||
|
||||
For the planned resource-context capability, retain source context as passive
|
||||
information or a borrowed caller-supplied filesystem. Subsequent resource access
|
||||
acquires its own handles with explicit ownership. This preserves a clear boundary
|
||||
between configuration completion and application resource use.
|
||||
|
||||
These choices give consumers one completion boundary for configuration and
|
||||
provenance, predictable resource lifetimes, and control over when ConfigKit runs.
|
||||
Application callbacks remain responsible for their own effects and resources.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
- **Background watching or refresh.** Can automate responses to source changes,
|
||||
but adds scheduling, shutdown, publication, and resource-lifetime concerns to
|
||||
the library. Explicit calls keep those responsibilities with the application.
|
||||
- **Lazy completion after return.** Can defer work until particular values are
|
||||
used, but moves configuration failures into application execution and makes
|
||||
the readiness of values and metadata harder to establish.
|
||||
- **Transactional atomic loading.** Could imply simultaneous observation of all
|
||||
external inputs and rollback of callback effects. Filesystems, environment
|
||||
variables, and application callbacks do not share a transaction mechanism.
|
||||
Synchronous completion provides the required usable-result boundary without
|
||||
promising those external guarantees.
|
||||
- **Retain loading handles in results.** Can preserve open filesystem access,
|
||||
but gives configuration results an ongoing resource lifecycle and cleanup
|
||||
obligation. Passive source context supports later explicit resource access
|
||||
while keeping loading resources scoped to the load call.
|
||||
|
||||
## Consequences
|
||||
|
||||
Applications can treat successful return as the point at which the final
|
||||
configuration and its metadata are ready for use. Startup loading remains simple,
|
||||
and additional loads have an explicit application-owned trigger.
|
||||
|
||||
Configuration inspection operates on retained data. Resource access observes the
|
||||
backing resource when explicitly requested and has its own failure and ownership
|
||||
boundary. Implementations of new source kinds and layering features preserve the
|
||||
same completion boundary, including cleanup on reported load failures.
|
||||
76
docs/adr/0009-sibling-inheritance.md
Normal file
76
docs/adr/0009-sibling-inheritance.md
Normal file
@@ -0,0 +1,76 @@
|
||||
# ADR 0009: Resolve Sibling Inheritance After Source Assembly
|
||||
|
||||
## Status
|
||||
|
||||
Accepted and implemented. The [configuration contract](../config.md#sibling-inheritance)
|
||||
owns current behavior. The completed implementation plan is preserved in Git
|
||||
history at `83c59ea`.
|
||||
|
||||
## Date
|
||||
|
||||
2026-09-29
|
||||
|
||||
## Context
|
||||
|
||||
Named configuration mappings often share settings while retaining individual
|
||||
overrides. ConfigKit already distinguishes strict composition of independent
|
||||
source contributions from recursive overlay between explicit precedence tiers.
|
||||
Doing another untracked merge in applications would lose omission, source
|
||||
context, and historical explanation information. A shared structural operation
|
||||
must preserve the same presence and provenance model as the rest of loading.
|
||||
|
||||
Inheritance also interacts with source precedence, nested namespaces, version
|
||||
ownership, and secrets. Resolving it before source assembly would validate
|
||||
superseded references and copy values that a later tier should replace. Fully
|
||||
resolving nested relationships inside a reusable parent would bind them to the
|
||||
wrong sibling scope when copied into a child.
|
||||
|
||||
## Decision
|
||||
|
||||
Reserve `extends` as a built-in input key on mappings. A string names one exact
|
||||
sibling mapping; null cancels the relationship. Complete source acquisition,
|
||||
strict composition, selected overlay, and CLI precedence first. Then resolve
|
||||
each effective relationship in the parser-independent document tree, before
|
||||
typed defaults and decoding. A child recursively overlays its own payload on
|
||||
its resolved parent using the existing mapping-combination rules. This creates
|
||||
neither a source tier nor a configurable merge strategy.
|
||||
|
||||
Resolve enclosing sibling chains before nested relationships. Reuse memoized
|
||||
parent templates with unresolved nested instructions; evaluate those instructions
|
||||
at each destination. Preserve original source applications separately from
|
||||
structural transfer/control evidence. Effective origins carry concrete
|
||||
source-to-destination derivation, and inspection projects relevant inheritance
|
||||
events between source replay and typed reconciliation. Inherited input counts as
|
||||
explicit presence, while direct replacement retains its own effective origin.
|
||||
|
||||
The existing main-input version check remains at its original point. Reject
|
||||
structural contributions or directive consumption affecting its protected path.
|
||||
Classify sensitivity at corresponding source and destination paths and retain
|
||||
historical redaction. Secret references remain unresolved input until the
|
||||
existing lookup point. Keep all machinery behind the root-owned public facade.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
- **Application-owned inheritance after `Load`.** Leaves every consumer to
|
||||
recreate ordering, presence, provenance, error reporting, and redaction after
|
||||
ConfigKit has already declared a result complete.
|
||||
- **Resolve each source before precedence assembly.** Makes superseded
|
||||
references active and can copy a parent definition before higher-priority
|
||||
input finishes changing it.
|
||||
- **Resolve nested relationships in a cached parent subtree.** Shares more
|
||||
ready-made output but binds nested sibling lookups to the parent's scope,
|
||||
giving incorrect values at a child destination.
|
||||
- **Registered collections, configurable keywords, or multiple-parent merge
|
||||
modes.** Increase public setup and semantic branches without a requirement
|
||||
that warrants diverging from one structural rule.
|
||||
|
||||
## Consequences
|
||||
|
||||
Consumers can load a configuration once and inspect direct, inherited,
|
||||
defaulted, and application-derived values through the existing result surfaces.
|
||||
`extends` is no longer an ordinary application field, so existing configurations
|
||||
using that key as data must migrate. Expanding many children can materialize
|
||||
substantially more output than the input contains; projection and retained-history
|
||||
budgets bound that work without charging copies as newly acquired sources.
|
||||
Applications still own domain validation, entry selection, and runtime resource
|
||||
use.
|
||||
228
docs/config.md
228
docs/config.md
@@ -12,13 +12,21 @@ ConfigKit exposes opaque sources through these constructors:
|
||||
| Constructor | Current contract |
|
||||
| --- | --- |
|
||||
| `File(path)` | Requires a nonempty path, opens a regular file for each consumption, and closes only the library-owned file handle. Missing, inaccessible, unreadable, directory, and other nonregular inputs fail. |
|
||||
| `NewFilesystem(label, filesystem).File(name)` | Borrows a caller-owned `fs.FS` under a safe nonempty display label. `name` must be a non-root slash-relative `fs.ValidPath`. Each load reopens the selected regular file; copies of the `Filesystem` preserve namespace identity. Separate constructors create separate namespaces even for the same label or filesystem. |
|
||||
| `Bytes(label, data)` | Requires a nonempty, non-secret diagnostic label and snapshots bytes during construction. Later caller mutation cannot affect the source. |
|
||||
| `Reader(label, input)` | Requires a nonempty, non-secret label and non-nil reader. The reader is claimed once across all copies of the source, consumed but never closed, and cannot be reused. |
|
||||
| `Mapping(label, values)` | Requires a safe nonempty label and a non-nil root `map[string]any`. Snapshots supported built-in values into an owned mapping document at construction; `{}` is valid. Constructor conversion failures become safe deferred diagnostics. |
|
||||
| `NoSource()` | Selects defaults-only loading, creates no source record, and marks no paths explicit. |
|
||||
|
||||
The zero `Source` is invalid. Constructor argument failures are retained inside the
|
||||
opaque source and become safe structured diagnostics when the source is consumed.
|
||||
File sources can observe file changes between consumptions; byte sources cannot.
|
||||
File and filesystem sources can observe changes between consumptions; byte and
|
||||
mapping sources cannot. Mapping accepts nil, built-in string, boolean, signed or
|
||||
unsigned integer, finite float, `[]any`, and `map[string]any` values recursively.
|
||||
Typed nil maps and slices represent explicit null; empty collections remain empty.
|
||||
Keys are sorted for stable provenance, integer decimal spelling stays exact, and
|
||||
cycles or unsupported types are rejected without exposing values. Mapping input
|
||||
uses the same directive and version rules as YAML, not CLI override restrictions.
|
||||
|
||||
Each source is limited to 8 MiB. A load using imports or an active overlay is limited
|
||||
to 64 MiB across all accepted sources, 1,024 accepted sources, 64 active import
|
||||
@@ -26,6 +34,8 @@ edges, and 1,000,000 input nodes in aggregate. Glob expansion separately permits
|
||||
1,000,000 visited directory entries, including nonmatches. Reads request at most the
|
||||
remaining allowance plus one byte so excess data is rejected rather than silently
|
||||
truncated. Data returned together with a read error is not accepted as partial input.
|
||||
For mapping sources, key and string bytes use the same accounting as programmatic
|
||||
overrides, and the converted tree uses the same node and depth bounds.
|
||||
|
||||
## Single-source loading
|
||||
|
||||
@@ -44,13 +54,14 @@ The current lifecycle is:
|
||||
```text
|
||||
validate options → parse the selected source → traverse/compose enabled imports
|
||||
→ build schema → check the optional application version → apply a selected overlay
|
||||
→ apply explicit CLI overrides → create and clone defaults → strictly decode
|
||||
→ apply explicit CLI overrides → expand sibling inheritance
|
||||
→ create and clone defaults → strictly decode
|
||||
→ recorded application resolution
|
||||
→ resolve typed secret references → semantic validation → classify secrets
|
||||
→ snapshot configuration and metadata
|
||||
```
|
||||
|
||||
`File`, `Bytes`, and `Reader` each contribute the first base source with deterministic
|
||||
`File`, `Filesystem.File`, `Bytes`, `Reader`, and `Mapping` each contribute the first base source with deterministic
|
||||
ID `source-1`; enabled imports continue in depth-first order as `source-2` onward.
|
||||
`NoSource` skips parsing, supplies an absent version to a configured version checker,
|
||||
contributes no source, and continues through defaults, decode, and projection. A
|
||||
@@ -66,11 +77,166 @@ disabled.
|
||||
|
||||
Every source, options, schema, version, defaults, decode, resolution, or validation
|
||||
failure returns a nil result and a `*LoadError`. Successful results contain caller-owned `Config`, detached
|
||||
`Metadata`, a defensive source list, and defensive nonfatal diagnostics. Mutating
|
||||
`Metadata`, a defensive source list, and defensive nonfatal diagnostics. Each source
|
||||
record's `Kind` identifies file, filesystem, bytes, reader, mapping, or CLI acquisition
|
||||
independently of its precedence `Layer`. Host file paths remain canonical;
|
||||
filesystem paths are slash-relative within their namespace; synthetic inputs have
|
||||
no path. Mutating
|
||||
the returned configuration or inspection records cannot alter ConfigKit's private
|
||||
snapshot. ConfigKit performs no logging. `Result.WriteEffective` exposes only the
|
||||
frozen redacted projection.
|
||||
|
||||
### Completion and resource lifetime
|
||||
|
||||
`Load` is synchronous and complete on return. Success means input assembly,
|
||||
defaulting, typed decoding, configured application resolution, secret lookup,
|
||||
validation, and configuration/metadata snapshotting have completed. A reported
|
||||
load failure returns diagnostics and a nil result, as described above.
|
||||
|
||||
Before returning on success or failure, ConfigKit closes the filesystem handles
|
||||
it opened for loading, including files and directory/root handles. Caller-owned
|
||||
filesystems are borrowed, never closed. The returned
|
||||
configuration and inspection metadata require no close operation and remain
|
||||
usable independently of the source files. A supplied `Reader` remains
|
||||
caller-owned and is never closed by ConfigKit.
|
||||
|
||||
ConfigKit performs no background work after returning: it does not watch inputs,
|
||||
refresh secrets, or mutate published results. Inspection and rendering run only
|
||||
when explicitly called and use the retained snapshot. Applications normally load
|
||||
once near startup, but may explicitly load again using sources that satisfy their
|
||||
documented reuse rules. Additional loads produce independently owned results.
|
||||
|
||||
Completion is not a transaction over external state. Files and environment
|
||||
variables are observed during processing rather than as one simultaneous external
|
||||
snapshot. ConfigKit does not roll back application callback effects or manage work
|
||||
or resources created by application callbacks. The explicit secrets-directory
|
||||
bootstrap remains a separate operation invoked by the application.
|
||||
|
||||
## Layered loading
|
||||
|
||||
`LoadLayered[T](main, options...)` accepts zero or one `LayeredOptions[T]` value.
|
||||
Its `main` is an `Input` containing a source, import opt-in, and optional
|
||||
filesystem context. A zero source is invalid; `Input{Source: NoSource()}` omits
|
||||
the main document. Each `Underlay` contains inputs that compose strictly as
|
||||
one tier, including each input's enabled imports. Empty groups contribute
|
||||
nothing. Underlay groups are listed from lowest to highest priority. Repeating
|
||||
a file within one group remains an import/repetition error; the same reusable
|
||||
source may be selected again in another tier.
|
||||
|
||||
ConfigKit acquires and composes the main input first, constructs the schema,
|
||||
and checks the optional application version from the main tier only. Underlay
|
||||
roots cannot claim that version path (`underlay.version`); imported fragments
|
||||
retain `import.version` protection. It then acquires and composes underlay
|
||||
groups in order, recursively combines the groups, and applies the main tier,
|
||||
one selected overlay, and explicit CLI overrides. Scalar, sequence, and null
|
||||
definitions replace lower values between tiers; mappings combine recursively.
|
||||
Syntax and strict-composition errors inside a tier cannot be masked by a
|
||||
higher-priority tier. Defaults, typed decoding, resolution, secret lookup,
|
||||
validation, and publication run once on the final assembled input.
|
||||
|
||||
An active layered overlay requires `LayeredOptions.OverlayFilesystem` explicitly,
|
||||
independent of the main source's namespace. Its name and root-relative lookup
|
||||
follow the selected-overlay rules below; inactive overlays ignore the context.
|
||||
The same path in a different filesystem namespace is not a repeated file.
|
||||
Ordinary `Load` continues to infer its selected-overlay context from its source.
|
||||
|
||||
Source IDs follow acquisition order—main and its depth-first imports, then each
|
||||
underlay tier's inputs and depth-first imports, then overlay and CLI—even though
|
||||
history follows precedence order. Source `Layer` describes precedence while
|
||||
`Kind` describes acquisition. Underlay records use `LayerUnderlay` and a
|
||||
one-based `UnderlayIndex` giving their original tier position; all other records
|
||||
have index zero. Explicit empty groups and `NoSource` consume no source ID.
|
||||
Byte, source, node, and glob-entry budgets are shared across the load, while
|
||||
the existing per-source and import-depth bounds still apply.
|
||||
|
||||
The maintained [layered consumer example](consumers/README.md#native-filesystem-and-layered-input)
|
||||
shows a main host file, two underlay tiers, embedded defaults, a mapping source,
|
||||
and explicit resource access without a second configuration pass.
|
||||
|
||||
### Sibling inheritance
|
||||
|
||||
Any input mapping may declare `extends: name` to inherit from the mapping named
|
||||
`name` among its immediate siblings. Names are exact keys, including the empty
|
||||
string; lookup does not search ancestor or unrelated scopes. The document root
|
||||
and a sequence-element mapping have no sibling namespace for their own string
|
||||
relationship, though mappings nested inside them can have one. An `extends`
|
||||
value of null cancels a relationship without looking up a parent. Other value
|
||||
types, missing or non-mapping parents, and cycles fail the load with value-free
|
||||
diagnostics.
|
||||
|
||||
`extends` is reserved structural input across ordinary and layered loads, every
|
||||
source kind, imports, selected overlays, and explicit CLI overrides. It is
|
||||
consumed before typed decoding and need not appear in the application's Go
|
||||
types. Applications previously using `extends` as an ordinary YAML field must
|
||||
rename that key and its field. No collection registration or keyword setup is
|
||||
required.
|
||||
|
||||
ConfigKit completes strict composition within tiers and precedence assembly
|
||||
between tiers before resolving relationships. An omitted `extends` retains a
|
||||
lower-priority relationship; `extends: null` cancels it while preserving the
|
||||
child's own settings. Superseded references are historical evidence, not active
|
||||
relationships to validate. Each child recursively overlays its own settings
|
||||
over its resolved parent: explicit zero, false, empty string, sequence, and
|
||||
null values replace inherited values, while mappings combine recursively.
|
||||
The child retains its key. All effective relationships resolve, including
|
||||
entries the application may never select for execution. Partial parent
|
||||
definitions undergo normal typed decoding and application-owned validation,
|
||||
not a special structural exemption.
|
||||
|
||||
Enclosing relationships resolve first. Nested instructions carried by a parent
|
||||
are then resolved separately at each destination, so a child's nested sibling
|
||||
override can change an inherited nested relationship. Typed defaults and
|
||||
application-derived values do not start another inheritance pass. Inherited
|
||||
input counts as explicit presence for `Metadata.IsExplicit`,
|
||||
`Resolution.IsExplicit`, and absence-based resolver defaults, even for false,
|
||||
zero, empty, or null input. `Origin.Inheritance` distinguishes it from a direct
|
||||
declaration with concrete source, destination, and controlling directive paths.
|
||||
A direct child replacement has its own effective origin; displaced ancestry
|
||||
remains inspectable in history. An inherited relative resource keeps the
|
||||
declaring source's passive `SourceContext`, not the child's directory.
|
||||
|
||||
The optional schema-version checker still runs once against the main input at
|
||||
its original point. Inheritance cannot add, replace, or consume the protected
|
||||
version path; an unrelated relationship inside an enclosing mapping does not
|
||||
invalidate an unchanged version. Expansion uses the decoded projection
|
||||
node/depth limits without charging copied values as acquired sources;
|
||||
retained evidence uses the existing history-node limit. See the
|
||||
[consumer example](consumers/README.md#named-sibling-inheritance) for one-load
|
||||
usage.
|
||||
|
||||
### Source resource contexts
|
||||
|
||||
`Result.SourceContext(id)` and callback-scoped `Resolution.SourceContext(id)`
|
||||
return a passive `ResourceContext` for a source with filesystem context, or
|
||||
false for unknown and contextless IDs. The result query is nil-safe; the
|
||||
resolution query follows the callback's lifetime guard. Queries perform no
|
||||
filesystem access. Obtain IDs from exact origin locations rather than guessing
|
||||
one context for a composite origin or an application-derived value.
|
||||
|
||||
`ResourceContext.FS` is rooted at the source's configured root, and `BaseDir`
|
||||
is its slash-relative directory inside that root. This includes imported
|
||||
sources, selected overlays, host files even without enabled imports, native
|
||||
files, and byte, reader, or mapping inputs with an explicit context. CLI and
|
||||
contextless synthetic inputs have no resource context. To open a relative
|
||||
resource, join `BaseDir` with the application resource name using slash-path
|
||||
operations and open that path through `FS`.
|
||||
|
||||
For a resource-valued field, query `result.Metadata().Lookup(fieldPath)`, inspect
|
||||
its origin locations, choose a source ID only when the application's policy can
|
||||
identify the intended contributor, then call `result.SourceContext(id)`. The
|
||||
context belongs to that source even when the field survived from a lower-priority
|
||||
underlay. A composite origin has no automatically selected resource base.
|
||||
|
||||
Native contexts borrow the caller's filesystem within the configured root.
|
||||
Host contexts retain a canonical root path, not an open root handle; each
|
||||
resource `Open` checks containment and opens a fresh handle that the returned
|
||||
file's `Close` releases. Applications own and close the files they open.
|
||||
Resources are application data, not configuration input, and do not consume
|
||||
configuration byte or node budgets. The loaded configuration and metadata stay
|
||||
usable if source files later change or disappear. Later resource operations
|
||||
observe the backing filesystem then and may fail if files move, disappear, or
|
||||
a caller-owned filesystem closes. This is not a snapshot or universal sandbox.
|
||||
|
||||
### Application schema versions
|
||||
|
||||
`VersionRule` declares a semantic path and non-nil checker. Before defaults or typed
|
||||
@@ -108,8 +274,9 @@ successful result. Callback panics remain programming failures.
|
||||
`Result.Metadata` describes every final canonical path. Queries retain explicit
|
||||
key/index distinctions using actual effective and historical container shapes;
|
||||
only parsed numeric tokens are resolved contextually. A descendant that never
|
||||
existed is not historical merely because an atomic ancestor was replaced. Exact surviving input paths
|
||||
are explicit and have a base or overlay origin; omitted paths have a default origin.
|
||||
existed is not historical merely because an atomic ancestor was replaced. Exact
|
||||
surviving direct or inherited input paths are explicit and retain their source
|
||||
layer and derivation; omitted paths have a default origin.
|
||||
An explicit parent mapping does not make omitted children explicit. Null state and
|
||||
available source coordinates are retained. `History` records the Go-zero or application
|
||||
default baseline followed by exact base or overlay definitions, including
|
||||
@@ -124,8 +291,9 @@ domain ownership rules remain application responsibilities. The method's Go
|
||||
documentation in [metadata.go](../metadata.go) defines enumeration and path-ordering
|
||||
details.
|
||||
|
||||
History is chronological: a path can have a synthetic default record, its exact base
|
||||
definition, an overlay replacement, and a recorded resolution operation. A descendant
|
||||
History is chronological: a path can have a synthetic default record, its exact
|
||||
base definition, an overlay replacement, structural inheritance, and a recorded
|
||||
resolution operation. A descendant
|
||||
query uses the ancestor's path for a whole-container or whole-sequence replacement;
|
||||
the descendant can therefore retain history after `Lookup` stops reporting it as
|
||||
effective. Equal-valued input is still an explicit base definition, while a derived
|
||||
@@ -141,6 +309,15 @@ operation. Reasons and dependency paths retain first occurrence order with dupli
|
||||
removed. Source records remain ordered as selected source, depth-first imports, the
|
||||
selected overlay, then synthetic CLI records in supplied order.
|
||||
|
||||
Inheritance adds no acquired source or precedence layer. `Origin.Inheritance`
|
||||
lists concrete transfers from contributor paths through destinations, with
|
||||
each step's controlling `extends` path and location. `History` retains source
|
||||
applications at their original paths and projects parent contributions, child
|
||||
replacements, and consumed controls at destination paths after source assembly
|
||||
and before typed decoding. `HistoricalPaths` includes consumed directives and
|
||||
displaced inherited descendants. A control can have history even when `Lookup`
|
||||
reports it absent. Queries return detached records.
|
||||
|
||||
### Secret classification and effective rendering
|
||||
|
||||
Secret classification combines `configkit:"secret"` field tags with the optional
|
||||
@@ -156,6 +333,11 @@ Propagation reaches a fixed point, so record/map iteration order does not affect
|
||||
result and dependency cycles terminate. A dependency on a classified descendant or
|
||||
on a container that contains one is treated conservatively as sensitive. Applications
|
||||
must classify secret-producing transformations whose dependencies are not recorded.
|
||||
Inherited value and control paths link source and destination classifications
|
||||
through chains and nested copies; unrelated sibling fields are not classified
|
||||
merely because another field is secret. Displaced historical evidence remains
|
||||
redacted. Secret references are copied only as unresolved names; credential
|
||||
lookup stays after decoding and application resolution.
|
||||
|
||||
`Result.WriteEffective` writes the frozen resolved snapshot as deterministic redacted
|
||||
YAML. Mappings sort by decoded key string, sequences retain order, indentation is two
|
||||
@@ -190,7 +372,10 @@ parser values. `Present=false` with empty YAML means absence or deletion; explic
|
||||
null is present and renders as `null`. The history starts with the canonical typed
|
||||
application-default or Go-zero baseline, then includes only relevant base sources in
|
||||
traversal order, the selected overlay, supplied CLI records, and audited net
|
||||
resolution operations. Overlapping parent/child resolution records are rendered
|
||||
resolution operations. Structural inheritance events appear after source replay
|
||||
and before typed reconciliation, including consumed or canceled `extends` input
|
||||
and equal-valued child replacements. Overlapping parent/child resolution records
|
||||
are rendered
|
||||
as one net subtree event with combined reasons and dependencies, rather than
|
||||
pretending that the recorded assignments ran in lexical order. Explicit equal
|
||||
writes remain visible. When input replay
|
||||
@@ -442,8 +627,8 @@ input document.
|
||||
`[a-zA-Z0-9_-]+`; they are not paths. The empty name and exact lowercase name
|
||||
`default` select no overlay, and `overlays/default.yml` is never loaded implicitly.
|
||||
Every other name resolves only to `<configuration-root>/overlays/<name>.yml`, with
|
||||
no extension fallback, case normalization, inheritance, or overlay stack. A selected
|
||||
missing overlay is an error.
|
||||
no extension fallback, case normalization, inheritance between overlay files,
|
||||
or overlay stack. A selected missing overlay is an error.
|
||||
|
||||
The overlay is a separately policy-validated YAML mapping. It is loaded after the
|
||||
base import graph is composed and its schema version is checked, but before defaults
|
||||
@@ -557,18 +742,25 @@ walk failures, matched directories, invalid documents, escapes, and repeated tar
|
||||
remain errors. Final candidates are bounded by the remaining source limit before
|
||||
their contents are accumulated.
|
||||
|
||||
The configuration root defaults to a selected file's canonical directory. A file
|
||||
source may supply `FilesystemContext.Root` to widen that root but may not supply
|
||||
`BaseDir`. Whenever byte and reader sources supply filesystem context, they must
|
||||
provide both `BaseDir` and `Root` as absolute existing directories; their diagnostic
|
||||
labels, including URI-looking labels, never imply filesystem paths. The base and
|
||||
selected file must be inside the canonical root. Nonregular targets are rejected before opening to avoid blocking on FIFOs;
|
||||
For host files, the configuration root defaults to the selected file's canonical
|
||||
directory. A host file may supply `FilesystemContext.Root` to widen that root but
|
||||
may not supply `BaseDir` or a native `FS`. Byte, reader, and mapping inputs with host context
|
||||
must provide both `BaseDir` and `Root` as absolute existing directories.
|
||||
For `Filesystem.File`, the default root is `.` in its namespace and the base is
|
||||
the selected file's directory. An explicit context must use the same `Filesystem`
|
||||
namespace and may set `Root` to contain the file and its base; it may not supply
|
||||
`BaseDir`. Byte, reader, and mapping inputs with native context provide both `BaseDir` and
|
||||
`Root` as slash-relative `fs.ValidPath` directories in `FS` (`.` means its root),
|
||||
with the base inside the root. Context paths are checked even when imports and
|
||||
overlays are inactive. Diagnostic labels, including URI-looking labels, never
|
||||
imply filesystem paths. Nonregular targets are rejected before opening to avoid blocking on FIFOs;
|
||||
post-open regular-file checks remain in place under the stable-input assumption.
|
||||
ConfigKit canonicalizes symlinks,
|
||||
checks containment with relative paths, and opens the canonical root-relative target
|
||||
through a library-owned root handle. Contained relative or absolute symlink targets
|
||||
are accepted; escapes are rejected. Canonical paths define identity, so hard links at
|
||||
distinct paths remain distinct sources.
|
||||
distinct paths remain distinct sources. Native filesystems use slash-path operations
|
||||
and their own symlink behavior; imports stay in their selected namespace and root.
|
||||
|
||||
An active-stack repeat is `import.cycle` and reports the complete cycle's import
|
||||
sites. A previously visited canonical path is `import.repeated` and reports both
|
||||
|
||||
@@ -7,7 +7,7 @@ validation, schema-version policy, CLI parsing, runtime matching and conversion,
|
||||
resource checks, remote acquisition, and service initialization.
|
||||
|
||||
The maintained [`examples/consumers`](../../examples/consumers) package demonstrates
|
||||
four sanitized shapes using only ConfigKit and the Go standard library:
|
||||
sanitized shapes using only ConfigKit and the Go standard library:
|
||||
|
||||
| Shape | Demonstrated fit | Application code deliberately retained |
|
||||
| --- | --- | --- |
|
||||
@@ -15,6 +15,9 @@ four sanitized shapes using only ConfigKit and the Go standard library:
|
||||
| Notarius-like | Canonical object bindings, a string-backed schema version accepting string or integer YAML, strict dynamic options, and `int64` dynamic integers | Catalog selection, allowed option keys, range-checked conversion to runtime `int` |
|
||||
| WeatherReporter-like | Defaults-only loading, report maps, canonical module objects, typed option unions, explicit CLI records, and typed secret startup | Report/module compatibility, rejection of explicitly empty domain lists, CLI parsing, and startup ordering |
|
||||
| Seriatim-like | Independent strict ordered speaker and autocorrect rule files | First-match and ordered replacement algorithms; no central application configuration file |
|
||||
| Bundled file | Simple native `fs.FS` source through ordinary `Load` | Selection and lifetime of the application-owned embedded filesystem |
|
||||
| Layered catalog | Embedded defaults with an import, a mapping underlay, an independent host main file, explicit false/zero input, and source-relative resource access | Domain validation and opening/interpreting the template resource |
|
||||
| Named presets | One-load sibling inheritance, explicit false, nested destination resolution, and source/derivation inspection | Choosing and executing a preset |
|
||||
|
||||
The YAML under [`testdata`](../../examples/consumers/testdata) is loaded by the
|
||||
executable examples and ordinary tests. It contains no private application data,
|
||||
@@ -48,6 +51,43 @@ both without redefining their syntax. ConfigKit overlays are configuration overl
|
||||
they do not select or replace an application's LLM, model, pipeline, report, or
|
||||
execution profile concepts.
|
||||
|
||||
## Named sibling inheritance
|
||||
|
||||
[`LoadPresets`](../../examples/consumers/inheritance.go) loads the maintained
|
||||
[`presets.yml`](../../examples/consumers/testdata/inheritance/presets.yml) in one
|
||||
call. `custom` inherits `enabled: false` from `base`, while its nested
|
||||
`settings.primary` changes the inherited `settings.secondary` result at the
|
||||
custom destination. The application struct has no `extends` field; ConfigKit
|
||||
consumes the relationship before strict typed decoding.
|
||||
|
||||
The [executable example](../../examples/consumers/consumers_test.go) reads
|
||||
`Metadata.IsExplicit` for the inherited false value, then follows
|
||||
`Origin.Inheritance` back to the parent path while `Origin.Locations` identifies
|
||||
the declaring source. Applications still choose which preset to execute and
|
||||
validate domain meaning. The [configuration contract](../config.md#sibling-inheritance)
|
||||
owns exact lookup, precedence, cancellation, and migration rules.
|
||||
|
||||
## Native filesystem and layered input
|
||||
|
||||
[`LoadBundled`](../../examples/consumers/layered.go) keeps the simple `Load` shape:
|
||||
it selects a file in an application-owned embedded filesystem and needs no
|
||||
underlay setup. [`LoadCatalog`](../../examples/consumers/layered.go) uses
|
||||
`LoadLayered` with an imported embedded-defaults group, a separate
|
||||
`Mapping` group, and a host main file supplied by its caller. The groups
|
||||
compose strictly within themselves and recursively across precedence boundaries;
|
||||
the main file's explicit `false` and `0` replace lower values without being
|
||||
mistaken for omissions. Its validator owns allowed modes and resource-name policy.
|
||||
|
||||
[`ReadWelcome`](../../examples/consumers/layered.go) follows the effective field's
|
||||
origin location to a source ID, queries `Result.SourceContext`, and opens the
|
||||
resource relative to that context's `BaseDir`. It does not assume the main file
|
||||
owns a retained underlay value. A composite origin needs an application choice;
|
||||
ConfigKit does not guess a directory. The context query is passive, while later
|
||||
resource reads observe the caller-owned filesystem or a freshly opened host root
|
||||
and may fail independently of the completed configuration load. The
|
||||
[configuration contract](../config.md#source-resource-contexts) owns exact
|
||||
lifetime and precedence rules.
|
||||
|
||||
Known module catalogs should use canonical object entries with a module identifier
|
||||
and typed options. A small union struct is sufficient when the available option
|
||||
fields are known; validation selects which fields apply to each module. Runtime-
|
||||
@@ -126,6 +166,8 @@ focused behavioral owner:
|
||||
| Source/import/selected-overlay pipeline rules and artifact defaults | Narratio-shaped executable example and fixture |
|
||||
| Canonical module objects and dynamic option handling | WeatherReporter- and Notarius-shaped examples and migration tests |
|
||||
| `LoadSecretsDir`, bootstrap reporting, typed lookup, CLI explanation | `Example_weatherStartup` |
|
||||
| Native filesystem, grouped underlays, programmatic mapping, retained resource context | `Example_nativeFilesystem`, `Example_layeredCatalog`, and their focused consumer tests |
|
||||
| Named sibling inheritance, presence, and derivation | `Example_inheritedPresets` and `TestPresetCatalogOneLoadAndDerivation` |
|
||||
| Results, metadata, diagnostics, effective output, and writer failures | Focused root-package load, provenance, rendering, and explanation tests |
|
||||
|
||||
The package-level import-boundary test covers the consumer examples too: supported
|
||||
|
||||
@@ -80,6 +80,14 @@ and current facade ownership belongs to the [component overview](internal/overvi
|
||||
|
||||
## Baseline validation
|
||||
|
||||
The completed filesystem-source and underlay roadmap, original feature request,
|
||||
implementation plan, and resolved remediation record are preserved in Git history
|
||||
at `b604aa1` under `docs/roadmap/`. Current behavior belongs to the
|
||||
[configuration contract](config.md#layered-loading). Built-in sibling
|
||||
inheritance is now part of the [configuration contract](config.md#sibling-inheritance).
|
||||
The completed request, roadmap, plan, and resolved review findings are preserved
|
||||
in Git history at `83c59ea` under `docs/roadmap/`.
|
||||
|
||||
The completed overlay rename plan and release-workflow simplification record are
|
||||
preserved in Git history at `73ce6e9` under `docs/roadmap/`. Current overlay
|
||||
behavior belongs to the [configuration contract](config.md#selected-overlays),
|
||||
@@ -133,12 +141,18 @@ code changes. Use the component overview to find the corresponding implementatio
|
||||
| --- | --- | --- |
|
||||
| YAML syntax, diagnostics, parser limits | [Adapter tests](../internal/yaml/parse_test.go), [source translation](../internal/engine/parse_test.go) | `go test ./internal/yaml` and `go test ./internal/engine -run '^TestParseSource'` |
|
||||
| File/reader access, imports, globs, containment | [Sources](../internal/engine/source_test.go), [FIFO](../internal/engine/source_fifo_test.go), [imports](../internal/engine/imports_test.go), [globs](../internal/engine/glob_test.go) | `go test ./internal/engine -run 'Source\|Import\|Glob\|Filesystem\|Nonregular\|RegularFile'` |
|
||||
| Native filesystem and mapping sources | [Native acquisition](../internal/engine/native_filesystem_test.go), [filesystem facade](../facade_filesystem_test.go), [mapping facade](../facade_mapping_test.go) | `go test ./internal/engine -run 'NativeFilesystem'` and `go test . -run 'TestFacade'` |
|
||||
| Composition, selected overlays, CLI precedence | [Composition](../internal/engine/compose_test.go), [overlay](../internal/engine/overlay_test.go), [selected overlays](../internal/engine/selected_overlay_test.go), [overrides](../internal/engine/override_test.go) | `go test ./internal/engine -run 'Compose\|Overlay\|Override'` |
|
||||
| Sibling inheritance, provenance, history, sensitivity | [Expansion](../internal/engine/inheritance_test.go), [destination scopes](../internal/engine/inheritance_walk_test.go), [origins](../internal/engine/inheritance_origin_test.go), [history](../internal/engine/inheritance_history_test.go), [secrets](../internal/engine/inheritance_secret_test.go), [public loading](../facade_inheritance_test.go) | `go test ./internal/engine -run 'Inheritance\|Inherited'` and `go test . -run '^TestFacadeInheritance\|^TestFacadeInheritedSecrets'` |
|
||||
| Layered loading and source equivalence | [Grouped lifecycle](../internal/engine/load_request_test.go), [layered facade](../facade_layered_test.go), [equivalence](../facade_equivalence_test.go) | `go test ./internal/engine -run '^TestGrouped'`, `go test . -run '^TestFacadeLayered'`, and `go test . -run '^TestEquivalentSourceForms'` |
|
||||
| Schema, defaults, projection, cloning | [Schema](../internal/engine/schema_test.go), [decoding](../internal/engine/decode_test.go), [projection](../internal/engine/projection_test.go) | `go test ./internal/engine -run 'Schema\|Decode\|Projection\|Clone\|Inline\|CompareDocument'` |
|
||||
| Lifecycle, resolution, validation, callback ownership | [Loading](../internal/engine/load_test.go), [resolution](../internal/engine/resolution_test.go), [validation](../internal/engine/validation_test.go), [ownership](../internal/engine/callback_ownership_test.go), [independence](../internal/engine/independence_test.go) | `go test ./internal/engine -run 'Load\|Resolution\|Validation\|Validator\|Hook\|Callback\|Parallel'` |
|
||||
| Paths, metadata, provenance | [Paths](../internal/engine/path_test.go), [metadata](../internal/engine/metadata_test.go), [provenance](../internal/engine/provenance_test.go) | `go test ./internal/engine -run 'Path\|Metadata\|Provenance\|Queries'` |
|
||||
| Passive resource contexts | [Engine contexts](../internal/engine/resource_context_test.go), [public contexts](../facade_resource_context_test.go) | `go test ./internal/engine -run 'Resource'` and `go test . -run 'SourceContext'` |
|
||||
| Secrets, bootstrap, redaction, explanation | [References](../internal/engine/secret_reference_test.go), [bootstrap](../internal/engine/secrets_test.go), [rendering](../internal/engine/render_test.go), [explanation](../internal/engine/explain_test.go) | `go test ./internal/engine -run 'Secret\|Bootstrap\|Effective\|Explain\|Explanation\|History\|Sensitivity'` |
|
||||
| Consumer examples and migration fixtures | [Consumer tests](../examples/consumers/consumers_test.go), [API examples](../example_test.go) | `go test ./examples/consumers` and `go test . -run '^Example'` |
|
||||
| Consumer examples and migration fixtures | [Consumer tests](../examples/consumers/consumers_test.go), [native and layered examples](../examples/consumers/layered_test.go), [API examples](../example_test.go) | `go test ./examples/consumers` and `go test . -run '^Example'` |
|
||||
| Assembly and layered scaling evidence | [Assembly benchmarks](../internal/engine/assembly_bench_test.go), [layered benchmarks](../internal/engine/underlay_bench_test.go) | `go test ./internal/engine -run '^$' -bench '^BenchmarkUnderlay' -benchmem`; repeat with `^BenchmarkCompose` and `^BenchmarkOverlay` |
|
||||
| Inheritance scaling evidence | [Expansion benchmarks](../internal/engine/inheritance_bench_test.go) | `go test ./internal/engine -run '^$' -bench '^BenchmarkInheritance' -benchmem` |
|
||||
| Documentation links and dependency isolation | [Link checks](../documentation_test.go), [import boundary](../import_boundary_test.go), [public boundary](../public_boundary_test.go) | `go test . -run 'TestLocalDocumentationLinks\|TestDependencyImportBoundary\|TestPublicFacadeBoundary'` |
|
||||
| Development and release checker behavior | [Shell regression](../release_check_test.go) | `go test . -run 'TestCheckScripts\|TestDevelopmentCheck'` |
|
||||
| Guarded release publication and recovery | [Publication regression](../release_publish_test.go) | `go test . -run '^TestReleasePublication'` |
|
||||
|
||||
@@ -7,23 +7,28 @@ private state; public signatures and fields do not expose engine types.
|
||||
|
||||
Implemented component owners:
|
||||
|
||||
- `internal/engine` owns source acquisition, composition, schema inspection,
|
||||
decoding, lifecycle coordination, callback auditing, provenance/history,
|
||||
secret resolution and bootstrap, and safe inspection rendering. It never
|
||||
imports the root package.
|
||||
- Root `Source`, `Path`, `Secret`, records, lifecycle callbacks, results, and
|
||||
bootstrap reports are root-defined wrappers or records. Callback wrappers pass
|
||||
only live engine context during a callback; `Result.Config` is detached and its
|
||||
private inspection remains frozen independently.
|
||||
- `internal/engine` owns host and native-filesystem source acquisition,
|
||||
grouped strict composition, recursive precedence assembly, and destination-local
|
||||
sibling inheritance before typed decoding. It also owns schema
|
||||
inspection, lifecycle coordination, callback auditing,
|
||||
provenance/history, passive source-context retention, secret resolution and
|
||||
bootstrap, and safe inspection rendering. It never imports the root package.
|
||||
- Root `Source`, `Filesystem`, `Input`, `Underlay`, `Path`, `Secret`, records,
|
||||
lifecycle callbacks, results, and bootstrap reports are root-defined wrappers
|
||||
or records. Callback wrappers pass only live engine context during a callback;
|
||||
`Result.Config` is detached and its private inspection remains frozen
|
||||
independently.
|
||||
- Root conversion files explicitly cross the path, record, callback, secret, and
|
||||
error boundaries. There are no public aliases, anonymous engine embeddings, or
|
||||
public engine bridge methods.
|
||||
- `internal/document` owns the parser-independent ordered tree and cloning.
|
||||
`internal/yaml` is the sole upstream YAML adapter and parser/dumper boundary.
|
||||
- Engine mechanism tests own source, composition, decoding, callback, provenance,
|
||||
secret, history, rendering, and bootstrap behavior. Root tests own public API
|
||||
compatibility, facade integration, documentation links, and dependency/public
|
||||
boundary checks. Consumer examples remain under `examples/consumers`.
|
||||
- Engine mechanism tests own source, composition, inheritance, decoding,
|
||||
callback, provenance,
|
||||
source-context, secret, history, rendering, and bootstrap behavior. Root tests
|
||||
own public API compatibility, facade integration, documentation links, and
|
||||
dependency/public boundary checks. Consumer examples remain under
|
||||
`examples/consumers`.
|
||||
|
||||
Dependency direction is root to engine to document/YAML support. The root may
|
||||
import only `internal/engine` among project internal production packages; engine
|
||||
@@ -36,4 +41,6 @@ semantics. The [consumer adoption guide](../consumers/README.md) owns applicatio
|
||||
guidance. The [architecture policy](../policy/architecture.md) owns contributor
|
||||
boundaries, and [ADR 0006](../adr/0006-public-facade.md) records the facade
|
||||
decision. See the [development guide](../development.md) for validation commands
|
||||
and [release procedure](../release.md) for source-candidate gates.
|
||||
and [release procedure](../release.md) for source-candidate gates. [ADR
|
||||
0009](../adr/0009-sibling-inheritance.md) records why inheritance is a built-in
|
||||
post-assembly structural operation.
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
|
||||
**Base Policy Version:** 1.3
|
||||
|
||||
**Last Updated**: 2026-09-28
|
||||
**Last Updated**: 2026-09-29
|
||||
|
||||
**Upstream Source:** [Standard Repository Policies](https://gitea.maximumdirect.net/eric/policy/src/commit/d84a38ebab26f048cafbe59a2246700c35d29843/docs/policy/architecture.md)
|
||||
|
||||
@@ -34,6 +34,23 @@ Do not promote incidental implementation choices into architectural invariants.
|
||||
|
||||
## Success Criteria
|
||||
|
||||
ConfigKit prioritizes a coherent configuration model: consistency and
|
||||
predictability take precedence over flexibility for accommodating individual
|
||||
downstream conventions. Consumers should learn one set of rules, not different
|
||||
combination or presence semantics depending on the source format or consuming
|
||||
application. Distinct operations such as strict composition and intentional
|
||||
overlay express distinct purposes within that model.
|
||||
|
||||
Consumers should be able to load once and receive a complete, validated
|
||||
configuration with inspectable provenance for its final values, including
|
||||
defaults and recorded application-derived values. Configuration and metadata
|
||||
together must distinguish explicit zero, false, empty, and null input from
|
||||
omission; ordinary Go values alone need not encode those distinctions.
|
||||
Application-owned resolution belongs within the recorded loading lifecycle, so
|
||||
consumers need no subsequent generic merging, defaulting, or provenance
|
||||
reconstruction. This does not transfer domain execution or runtime resource
|
||||
management into ConfigKit.
|
||||
|
||||
1. **Reduce application infrastructure:** Can applications delete substantial
|
||||
configuration infrastructure while retaining clear ownership of their domain
|
||||
rules?
|
||||
@@ -41,8 +58,11 @@ Do not promote incidental implementation choices into architectural invariants.
|
||||
configuration model, with only the flexibility needed to express genuine
|
||||
application requirements?
|
||||
|
||||
Both criteria must hold. ConfigKit should not preserve divergent consumer
|
||||
conventions merely for compatibility, nor impose restrictions that prevent useful
|
||||
Both criteria must hold. Minimize consumer loading and processing code subject
|
||||
to the consistency, presence, and provenance principles above, rather than
|
||||
trading those principles for consumer-specific shortcuts. ConfigKit should not
|
||||
preserve divergent consumer conventions merely for compatibility, nor impose
|
||||
restrictions that prevent useful
|
||||
domain information from being expressed. Prefer a canonical representation using
|
||||
existing primitives before adding another mechanism. Extend the model when a
|
||||
concrete requirement cannot be accommodated cleanly, not simply because a
|
||||
@@ -88,6 +108,22 @@ must never run implicitly as part of configuration loading.
|
||||
Remote data can enter through application-supplied input without introducing a
|
||||
network provider into the library.
|
||||
|
||||
## Loading Lifecycle
|
||||
|
||||
Configuration loading is a synchronous, complete-on-return operation. A successful
|
||||
load provides the final configuration and inspection metadata after the configured
|
||||
processing and validation phases complete. A failed load reports diagnostics and
|
||||
returns no usable partial result. The normal application pattern is one load near
|
||||
startup; applications may explicitly initiate additional loads.
|
||||
|
||||
ConfigKit releases its library-owned loading handles before returning and remains
|
||||
inactive until the application explicitly invokes it again. It performs no
|
||||
background watching, refresh, or mutation of published results. Caller-owned
|
||||
resources and application callback effects remain the caller's responsibility.
|
||||
The [configuration contract](../config.md#completion-and-resource-lifetime) owns
|
||||
the precise completion and ownership behavior. The proposed decision record is
|
||||
[ADR 0008](../adr/0008-complete-on-return-loading.md).
|
||||
|
||||
## Public API And Dependency Direction
|
||||
|
||||
Keep one small, coherent public surface for the supported lifecycle. Simple
|
||||
@@ -116,18 +152,41 @@ framework on this library.
|
||||
|
||||
## Configuration Integrity
|
||||
|
||||
Composition and intentional overlay are distinct operations. Composition must
|
||||
not silently become last-wins override through file ordering. The decision and
|
||||
alternatives are recorded in [ADR 0001](../adr/0001-separate-composition-and-overlay.md).
|
||||
Within a configuration tier, independent contributions use strict composition:
|
||||
shared mappings combine recursively, while overlapping non-mapping definitions
|
||||
are conflicts, including equal values. Between precedence tiers, intentional
|
||||
overlay combines mappings recursively, retains omitted lower-priority children,
|
||||
and replaces other present values according to the existing overlay rules.
|
||||
Tier precedence is explicit; traversal order within a tier establishes stable
|
||||
reporting, not permission to override. The distinction and its rationale are
|
||||
recorded in [ADR 0001](../adr/0001-separate-composition-and-overlay.md).
|
||||
|
||||
The main document and its imports form a strictly composed tier. Explicit
|
||||
underlay groups compose strictly within each group, then combine recursively
|
||||
from lowest to highest beneath the main tier. Source acquisition kind does not
|
||||
determine its precedence role. Input-specific constraints, including CLI path
|
||||
rules and opt-in version ownership, remain part of the configuration contract. Consistent combination
|
||||
semantics do not require identical acquisition or lifetime behavior from every
|
||||
source kind.
|
||||
|
||||
After source precedence is complete, built-in sibling `extends` relationships
|
||||
resolve structurally in the same parser-independent representation, using the
|
||||
existing recursive overlay rules. This adds neither an acquired source nor a
|
||||
consumer registration mechanism. Enclosing relationships precede nested
|
||||
destination-local resolution; typed defaults, application resolution, and
|
||||
secret lookup retain their existing lifecycle roles. The [configuration
|
||||
contract](../config.md#sibling-inheritance) owns the exact syntax and presence
|
||||
rules, and [ADR 0009](../adr/0009-sibling-inheritance.md) records the rationale.
|
||||
|
||||
ConfigKit must define supported YAML semantics explicitly rather than inherit
|
||||
parser defaults. Processing order and diagnostics must be deterministic for the
|
||||
same inputs and options. Invalid input must not produce a successful usable
|
||||
configuration. Do not run application hooks on invalid typed configuration.
|
||||
|
||||
User-controlled precedence is application defaults, composed base, selected
|
||||
overlay, then explicitly supplied CLI overrides. Applications own flag parsing;
|
||||
ConfigKit owns overlay semantics and provenance. Domain resolution follows these
|
||||
User-controlled precedence is application defaults, ordered underlays, composed
|
||||
main input, selected overlay, then explicitly supplied CLI overrides.
|
||||
Applications own flag parsing; ConfigKit owns overlay semantics and provenance.
|
||||
Domain resolution follows these
|
||||
inputs and must remain recorded, not become a hidden override channel.
|
||||
|
||||
Preserve node-based composition until the effective input is ready for typed
|
||||
|
||||
53
docs/releases/v0.5.0.md
Normal file
53
docs/releases/v0.5.0.md
Normal file
@@ -0,0 +1,53 @@
|
||||
# ConfigKit v0.5.0
|
||||
|
||||
This minor release adds source-backed layered configuration and built-in sibling
|
||||
inheritance, so applications can load one complete configuration while retaining
|
||||
presence, source context, and explainable derivation.
|
||||
|
||||
## Summary
|
||||
|
||||
- `LoadLayered` combines ordered underlay groups beneath the main input, selected
|
||||
overlay, and explicit CLI overrides. Inputs within a group still compose
|
||||
strictly; precedence tiers combine mappings recursively. The main input
|
||||
remains the owner of the optional schema-version check.
|
||||
- `NewFilesystem(...).File(...)` accepts caller-owned `fs.FS` sources, and
|
||||
`Mapping(...)` snapshots supported built-in values as a labeled source.
|
||||
`Result.SourceContext` and `Resolution.SourceContext` expose passive resource
|
||||
contexts for values from file-backed sources.
|
||||
- `extends` now resolves a mapping against one named sibling after source
|
||||
precedence and before typed decoding. Inherited values count as explicit
|
||||
input. Origins, history, explanations, and redacted rendering retain the
|
||||
source and the structural path by which a value reached its destination.
|
||||
Review corrections preserve immutable nested-history evidence and bounded
|
||||
expansion work.
|
||||
|
||||
See the [configuration contract](../config.md#layered-loading),
|
||||
[inheritance rules](../config.md#sibling-inheritance), and
|
||||
[consumer examples](../consumers/README.md) for current behavior.
|
||||
|
||||
## Compatibility and upgrade
|
||||
|
||||
This is an intentional pre-v1 incompatible minor release for configurations
|
||||
that use `extends` as ordinary data. That key is now reserved wherever an input
|
||||
mapping occurs; it is consumed before decoding. Rename any such YAML key and
|
||||
corresponding Go field, then rebuild. No collection registration or opt-in flag
|
||||
is available. Configurations without `extends` retain their existing meaning.
|
||||
|
||||
Applications adopting the new features may move source-backed defaults into
|
||||
explicit `Underlay` groups, use `SourceContext` for relative resources, or
|
||||
replace application-owned sibling merging with one `extends` relationship per
|
||||
mapping. Adoption is optional. The maintained
|
||||
[named-preset example](../consumers/README.md#named-sibling-inheritance)
|
||||
shows inherited `false` and destination-local nested resolution.
|
||||
|
||||
Select the new module version:
|
||||
|
||||
```sh
|
||||
go get gitea.maximumdirect.net/eric/ConfigKit@v0.5.0
|
||||
```
|
||||
|
||||
The minimum Go line remains 1.25. Linux amd64 and arm64 remain the
|
||||
release-validated targets; macOS remains best-effort. The YAML dependency is
|
||||
unchanged at `go.yaml.in/yaml/v4 v4.0.0-rc.6`.
|
||||
|
||||
[COPYRIGHT](https://gitea.maximumdirect.net/eric/ConfigKit/src/tag/v0.5.0/COPYRIGHT) | [LICENSE](https://gitea.maximumdirect.net/eric/ConfigKit/src/tag/v0.5.0/LICENSE) | [ATTRIBUTION](https://gitea.maximumdirect.net/eric/ConfigKit/src/tag/v0.5.0/docs/oss/attribution.md)
|
||||
@@ -11,11 +11,13 @@ General environment-variable configuration overlays are not supported. Typed sec
|
||||
references and explicit directory bootstrap are the environment convention. Literal
|
||||
credential configuration and implicit secret-directory discovery are unsupported.
|
||||
|
||||
The initial release deliberately excludes:
|
||||
The following remain deliberately excluded:
|
||||
|
||||
- automatic `conf.d` discovery;
|
||||
- remote imports, arbitrary providers, and additional configuration formats;
|
||||
- custom merge strategies, sequence merging, reset operators, and overlay inheritance;
|
||||
- custom merge strategies, sequence merging, reset operators, multiple-parent
|
||||
inheritance, and cross-scope references;
|
||||
- named overlay inheritance;
|
||||
- parser-specific or custom structural-decoding extension APIs;
|
||||
- generic dynamic-subtree conversion and module-registry APIs;
|
||||
- automatic schema migrations;
|
||||
@@ -26,3 +28,7 @@ The initial release deliberately excludes:
|
||||
Applications may build their own validate, effective-value, source, bootstrap, and
|
||||
explanation commands around the library. Broadening these boundaries requires
|
||||
concrete consumer evidence, compatibility review, and an explicit design change.
|
||||
|
||||
Built-in sibling inheritance using `extends` is implemented as described in the
|
||||
[configuration contract](../config.md#sibling-inheritance). It is distinct from
|
||||
inheritance between selected overlay files, which remains excluded above.
|
||||
|
||||
@@ -52,6 +52,22 @@ func Example_seriatim() {
|
||||
// Output: visitor true the event occurred <nil> <nil>
|
||||
}
|
||||
|
||||
func Example_inheritedPresets() {
|
||||
result, err := consumers.LoadPresets(configkit.File(fixture("inheritance", "presets.yml")))
|
||||
if err != nil {
|
||||
fmt.Println(err)
|
||||
return
|
||||
}
|
||||
path := configkit.Keys("presets", "custom", "enabled")
|
||||
info, _ := result.Metadata().Lookup(path)
|
||||
custom := result.Config.Presets["custom"]
|
||||
fmt.Println(custom.Enabled, result.Metadata().IsExplicit(path), custom.Settings["secondary"]["width"])
|
||||
fmt.Println(info.Origin.Locations[0].SourceID, info.Origin.Inheritance[0].From)
|
||||
// Output:
|
||||
// false true 20
|
||||
// source-1 /presets/base/enabled
|
||||
}
|
||||
|
||||
func Example_weatherStartup() {
|
||||
const blockedName = "CONFIGKIT_WEATHER_EXAMPLE_BLOCKED"
|
||||
const tokenName = "CONFIGKIT_WEATHER_EXAMPLE_TOKEN"
|
||||
|
||||
20
examples/consumers/inheritance.go
Normal file
20
examples/consumers/inheritance.go
Normal file
@@ -0,0 +1,20 @@
|
||||
package consumers
|
||||
|
||||
import configkit "gitea.maximumdirect.net/eric/ConfigKit"
|
||||
|
||||
// PresetCatalog is ordinary application data; no inheritance fields or
|
||||
// registration hooks are required in the typed configuration.
|
||||
type PresetCatalog struct {
|
||||
Presets map[string]Preset `yaml:"presets"`
|
||||
}
|
||||
|
||||
type Preset struct {
|
||||
Enabled bool `yaml:"enabled"`
|
||||
Settings map[string]map[string]int `yaml:"settings"`
|
||||
}
|
||||
|
||||
// LoadPresets performs one load; callers may inspect inherited presence and
|
||||
// source origins on the returned result before choosing a preset to execute.
|
||||
func LoadPresets(source configkit.Source) (*configkit.Result[PresetCatalog], error) {
|
||||
return configkit.Load[PresetCatalog](source)
|
||||
}
|
||||
35
examples/consumers/inheritance_test.go
Normal file
35
examples/consumers/inheritance_test.go
Normal file
@@ -0,0 +1,35 @@
|
||||
package consumers_test
|
||||
|
||||
import (
|
||||
"testing"
|
||||
|
||||
configkit "gitea.maximumdirect.net/eric/ConfigKit"
|
||||
"gitea.maximumdirect.net/eric/ConfigKit/examples/consumers"
|
||||
)
|
||||
|
||||
func TestPresetCatalogOneLoadAndDerivation(t *testing.T) {
|
||||
result, err := consumers.LoadPresets(configkit.File(fixture("inheritance", "presets.yml")))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if result.Config.Presets["base"].Settings["secondary"]["width"] != 10 ||
|
||||
result.Config.Presets["custom"].Settings["secondary"]["width"] != 20 ||
|
||||
result.Config.Presets["custom"].Enabled {
|
||||
t.Fatalf("expanded presets = %#v", result.Config.Presets)
|
||||
}
|
||||
enabled := configkit.Keys("presets", "custom", "enabled")
|
||||
if !result.Metadata().IsExplicit(enabled) {
|
||||
t.Fatal("inherited false lost explicit presence")
|
||||
}
|
||||
info, found := result.Metadata().Lookup(enabled)
|
||||
if !found || info.Origin.Locations[0].SourceID != "source-1" ||
|
||||
len(info.Origin.Inheritance) != 1 || info.Origin.Inheritance[0].From.String() != "/presets/base/enabled" {
|
||||
t.Fatalf("origin and derivation = %#v, %v", info, found)
|
||||
}
|
||||
nested := configkit.Keys("presets", "custom", "settings", "secondary", "width")
|
||||
info, found = result.Metadata().Lookup(nested)
|
||||
if !found || len(info.Origin.Inheritance) != 1 ||
|
||||
info.Origin.Inheritance[0].From.String() != "/presets/custom/settings/primary/width" {
|
||||
t.Fatalf("destination-local derivation = %#v, %v", info, found)
|
||||
}
|
||||
}
|
||||
95
examples/consumers/layered.go
Normal file
95
examples/consumers/layered.go
Normal file
@@ -0,0 +1,95 @@
|
||||
package consumers
|
||||
|
||||
import (
|
||||
"embed"
|
||||
"errors"
|
||||
"io/fs"
|
||||
"path"
|
||||
|
||||
configkit "gitea.maximumdirect.net/eric/ConfigKit"
|
||||
)
|
||||
|
||||
//go:embed testdata/native-source testdata/layered-bundle
|
||||
var bundledFiles embed.FS
|
||||
|
||||
// BundledConfig is a small configuration loaded directly from an embedded file.
|
||||
type BundledConfig struct {
|
||||
Name string `yaml:"name"`
|
||||
Enabled bool `yaml:"enabled"`
|
||||
}
|
||||
|
||||
// LoadBundled demonstrates a native filesystem source without layered options.
|
||||
func LoadBundled() (*configkit.Result[BundledConfig], error) {
|
||||
files, err := fs.Sub(bundledFiles, "testdata/native-source")
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return configkit.Load[BundledConfig](configkit.NewFilesystem("bundled", files).File("config.yml"))
|
||||
}
|
||||
|
||||
// CatalogConfig keeps resource names and domain rules in the application.
|
||||
type CatalogConfig struct {
|
||||
Mode string `yaml:"mode"`
|
||||
Enabled bool `yaml:"enabled"`
|
||||
Workers int `yaml:"workers"`
|
||||
Templates map[string]string `yaml:"templates"`
|
||||
Labels map[string]string `yaml:"labels"`
|
||||
}
|
||||
|
||||
// LoadCatalog combines two underlay tiers beneath an independently located
|
||||
// host main file. ConfigKit assembles and validates the complete result once.
|
||||
func LoadCatalog(mainPath string) (*configkit.Result[CatalogConfig], error) {
|
||||
files, err := fs.Sub(bundledFiles, "testdata/layered-bundle")
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
bundle := configkit.NewFilesystem("catalog bundle", files)
|
||||
return configkit.LoadLayered(configkit.Input{Source: configkit.File(mainPath)},
|
||||
configkit.LayeredOptions[CatalogConfig]{
|
||||
Underlays: []configkit.Underlay{
|
||||
{Inputs: []configkit.Input{{Source: bundle.File("defaults/config.yml"), Imports: true}}},
|
||||
{Inputs: []configkit.Input{{Source: configkit.Mapping("local policy", map[string]any{
|
||||
"workers": 2,
|
||||
"labels": map[string]any{"tenant": "example"},
|
||||
})}}},
|
||||
},
|
||||
Definition: configkit.Definition[CatalogConfig]{
|
||||
Defaults: func() CatalogConfig { return CatalogConfig{Mode: "demo"} },
|
||||
Validate: validateCatalog,
|
||||
},
|
||||
})
|
||||
}
|
||||
|
||||
func validateCatalog(configuration *CatalogConfig, validation *configkit.Validation) {
|
||||
if configuration.Mode != "demo" && configuration.Mode != "production" {
|
||||
validation.Error(configkit.Keys("mode"), "must be demo or production")
|
||||
}
|
||||
if configuration.Workers < 0 {
|
||||
validation.Error(configkit.Keys("workers"), "must be nonnegative")
|
||||
}
|
||||
if name := configuration.Templates["welcome"]; name == "." || !fs.ValidPath(name) {
|
||||
validation.Error(configkit.Keys("templates", "welcome"), "must be a relative resource name")
|
||||
}
|
||||
}
|
||||
|
||||
// ReadWelcome follows the effective template field's one declaring source to
|
||||
// its retained resource context. Opening and interpreting the resource remain
|
||||
// application work; this does not reopen configuration or change metadata.
|
||||
func ReadWelcome(result *configkit.Result[CatalogConfig]) (string, error) {
|
||||
if result == nil {
|
||||
return "", errors.New("catalog result is nil")
|
||||
}
|
||||
info, found := result.Metadata().Lookup(configkit.Keys("templates", "welcome"))
|
||||
if !found || len(info.Origin.Locations) != 1 {
|
||||
return "", errors.New("welcome template has no single declaring source")
|
||||
}
|
||||
context, found := result.SourceContext(info.Origin.Locations[0].SourceID)
|
||||
if !found {
|
||||
return "", errors.New("welcome template has no resource context")
|
||||
}
|
||||
data, err := fs.ReadFile(context.FS, path.Join(context.BaseDir, result.Config.Templates["welcome"]))
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
return string(data), nil
|
||||
}
|
||||
71
examples/consumers/layered_test.go
Normal file
71
examples/consumers/layered_test.go
Normal file
@@ -0,0 +1,71 @@
|
||||
package consumers_test
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"fmt"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
configkit "gitea.maximumdirect.net/eric/ConfigKit"
|
||||
"gitea.maximumdirect.net/eric/ConfigKit/examples/consumers"
|
||||
)
|
||||
|
||||
func Example_nativeFilesystem() {
|
||||
result, err := consumers.LoadBundled()
|
||||
if err != nil {
|
||||
fmt.Println(err)
|
||||
return
|
||||
}
|
||||
fmt.Println(result.Config.Name, result.Config.Enabled)
|
||||
// Output: embedded false
|
||||
}
|
||||
|
||||
func Example_layeredCatalog() {
|
||||
result, err := consumers.LoadCatalog(fixture("layered-main", "config.yml"))
|
||||
if err != nil {
|
||||
fmt.Println(err)
|
||||
return
|
||||
}
|
||||
welcome, err := consumers.ReadWelcome(result)
|
||||
fmt.Println(result.Config.Mode, result.Config.Enabled, result.Config.Workers,
|
||||
result.Config.Labels["origin"], result.Config.Labels["tenant"], strings.TrimSpace(welcome), err)
|
||||
// Output: production false 0 bundle example Welcome from the bundle. <nil>
|
||||
}
|
||||
|
||||
func TestCatalogRetainsPresenceAndSourceContext(t *testing.T) {
|
||||
result, err := consumers.LoadCatalog(fixture("layered-main", "config.yml"))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
for _, key := range []string{"enabled", "workers"} {
|
||||
info, found := result.Metadata().Lookup(configkit.Keys(key))
|
||||
if !found || !info.Explicit || info.Origin.Layer != configkit.LayerBase {
|
||||
t.Fatalf("%s presence = %#v, %v", key, info, found)
|
||||
}
|
||||
}
|
||||
info, found := result.Metadata().Lookup(configkit.Keys("templates", "welcome"))
|
||||
if !found || info.Origin.Layer != configkit.LayerUnderlay || len(info.Origin.Locations) != 1 {
|
||||
t.Fatalf("welcome origin = %#v, %v", info, found)
|
||||
}
|
||||
context, found := result.SourceContext(info.Origin.Locations[0].SourceID)
|
||||
if !found || context.BaseDir != "defaults" {
|
||||
t.Fatalf("welcome context = %#v, %v", context, found)
|
||||
}
|
||||
welcome, err := consumers.ReadWelcome(result)
|
||||
if err != nil || strings.TrimSpace(welcome) != "Welcome from the bundle." {
|
||||
t.Fatalf("welcome = %q, %v", welcome, err)
|
||||
}
|
||||
if result.Config.Labels["tenant"] != "example" || result.Config.Labels["origin"] != "bundle" {
|
||||
t.Fatalf("recursive underlay labels = %#v", result.Config.Labels)
|
||||
}
|
||||
}
|
||||
|
||||
func TestCatalogValidationRejectsInvalidMode(t *testing.T) {
|
||||
_, err := consumers.LoadCatalog(fixture("layered-main", "invalid.yml"))
|
||||
var loadError *configkit.LoadError
|
||||
if !errors.As(err, &loadError) || len(loadError.Diagnostics()) == 0 ||
|
||||
loadError.Diagnostics()[0].Code != configkit.CodeValidateError ||
|
||||
loadError.Diagnostics()[0].Path.String() != "/mode" {
|
||||
t.Fatalf("invalid mode error = %v", err)
|
||||
}
|
||||
}
|
||||
10
examples/consumers/testdata/inheritance/presets.yml
vendored
Normal file
10
examples/consumers/testdata/inheritance/presets.yml
vendored
Normal file
@@ -0,0 +1,10 @@
|
||||
presets:
|
||||
base:
|
||||
enabled: false
|
||||
settings:
|
||||
primary: {width: 10}
|
||||
secondary: {extends: primary}
|
||||
custom:
|
||||
extends: base
|
||||
settings:
|
||||
primary: {width: 20}
|
||||
7
examples/consumers/testdata/layered-bundle/defaults/config.yml
vendored
Normal file
7
examples/consumers/testdata/layered-bundle/defaults/config.yml
vendored
Normal file
@@ -0,0 +1,7 @@
|
||||
configkit:
|
||||
imports:
|
||||
- path: ../shared.yml
|
||||
enabled: true
|
||||
workers: 4
|
||||
templates:
|
||||
welcome: messages/welcome.txt
|
||||
1
examples/consumers/testdata/layered-bundle/defaults/messages/welcome.txt
vendored
Normal file
1
examples/consumers/testdata/layered-bundle/defaults/messages/welcome.txt
vendored
Normal file
@@ -0,0 +1 @@
|
||||
Welcome from the bundle.
|
||||
2
examples/consumers/testdata/layered-bundle/shared.yml
vendored
Normal file
2
examples/consumers/testdata/layered-bundle/shared.yml
vendored
Normal file
@@ -0,0 +1,2 @@
|
||||
labels:
|
||||
origin: bundle
|
||||
3
examples/consumers/testdata/layered-main/config.yml
vendored
Normal file
3
examples/consumers/testdata/layered-main/config.yml
vendored
Normal file
@@ -0,0 +1,3 @@
|
||||
mode: production
|
||||
enabled: false
|
||||
workers: 0
|
||||
1
examples/consumers/testdata/layered-main/invalid.yml
vendored
Normal file
1
examples/consumers/testdata/layered-main/invalid.yml
vendored
Normal file
@@ -0,0 +1 @@
|
||||
mode: unsupported
|
||||
2
examples/consumers/testdata/native-source/config.yml
vendored
Normal file
2
examples/consumers/testdata/native-source/config.yml
vendored
Normal file
@@ -0,0 +1,2 @@
|
||||
name: embedded
|
||||
enabled: false
|
||||
@@ -31,6 +31,9 @@ func TestFacadeValuesRemainRootOwned(t *testing.T) {
|
||||
reflect.TypeFor[configkit.Resolution](), reflect.TypeFor[configkit.Validation](),
|
||||
reflect.TypeFor[configkit.Version](), reflect.TypeFor[configkit.VersionRule](),
|
||||
reflect.TypeFor[configkit.FilesystemContext](), reflect.TypeFor[configkit.Override](),
|
||||
reflect.TypeFor[configkit.Filesystem](), reflect.TypeFor[configkit.ResourceContext](),
|
||||
reflect.TypeFor[configkit.Input](), reflect.TypeFor[configkit.Underlay](),
|
||||
reflect.TypeFor[configkit.LayeredOptions[struct{}]](),
|
||||
reflect.TypeFor[configkit.Definition[struct{}]](), reflect.TypeFor[configkit.Options[struct{}]](),
|
||||
reflect.TypeFor[configkit.Result[struct{}]](),
|
||||
reflect.TypeFor[configkit.ValueSnapshot](), reflect.TypeFor[configkit.ValueChange](),
|
||||
|
||||
@@ -26,6 +26,7 @@ func TestFacadeInactiveContextsPreserveClosedFailure(t *testing.T) {
|
||||
{"IsExplicit", func() { state.resolution.IsExplicit(path) }},
|
||||
{"InputInfo", func() { state.resolution.InputInfo(path) }},
|
||||
{"Source", func() { state.resolution.Source("source") }},
|
||||
{"SourceContext", func() { state.resolution.SourceContext("source") }},
|
||||
{"Record", func() { state.resolution.Record(path, configkit.ChangeNormalization, "reason") }},
|
||||
{"Resolution.Error", func() { state.resolution.Error(path, "message") }},
|
||||
{"Resolution.Warning", func() { state.resolution.Warning(path, "message") }},
|
||||
|
||||
164
facade_equivalence_test.go
Normal file
164
facade_equivalence_test.go
Normal file
@@ -0,0 +1,164 @@
|
||||
package configkit_test
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"reflect"
|
||||
"strings"
|
||||
"testing"
|
||||
"testing/fstest"
|
||||
|
||||
configkit "gitea.maximumdirect.net/eric/ConfigKit"
|
||||
)
|
||||
|
||||
type equivalentConfig struct {
|
||||
Version uint64 `yaml:"version"`
|
||||
Name string `yaml:"name"`
|
||||
Enabled bool `yaml:"enabled"`
|
||||
Count int `yaml:"count"`
|
||||
Items []string `yaml:"items"`
|
||||
Optional *string `yaml:"optional"`
|
||||
}
|
||||
|
||||
func TestEquivalentSourceFormsPreserveInputSemantics(t *testing.T) {
|
||||
const yaml = "version: 1\nname: retained\nenabled: false\ncount: 0\nitems: []\noptional: null\n"
|
||||
hostPath := filepath.Join(t.TempDir(), "main.yml")
|
||||
if err := os.WriteFile(hostPath, []byte(yaml), 0o600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
namespace := configkit.NewFilesystem("bundle", fstest.MapFS{"main.yml": &fstest.MapFile{Data: []byte(yaml)}})
|
||||
context := &configkit.FilesystemContext{Root: ".", BaseDir: ".", FS: &namespace}
|
||||
for _, test := range []struct {
|
||||
name string
|
||||
source configkit.Source
|
||||
context *configkit.FilesystemContext
|
||||
kind configkit.SourceKind
|
||||
}{
|
||||
{"host", configkit.File(hostPath), nil, configkit.SourceKindFile},
|
||||
{"native", namespace.File("main.yml"), nil, configkit.SourceKindFilesystem},
|
||||
{"bytes", configkit.Bytes("memory", []byte(yaml)), context, configkit.SourceKindBytes},
|
||||
{"reader", configkit.Reader("stream", strings.NewReader(yaml)), context, configkit.SourceKindReader},
|
||||
{"mapping", configkit.Mapping("programmatic", map[string]any{
|
||||
"version": uint64(1), "name": "retained", "enabled": false, "count": 0,
|
||||
"items": []any{}, "optional": nil,
|
||||
}), context, configkit.SourceKindMapping},
|
||||
} {
|
||||
t.Run(test.name, func(t *testing.T) {
|
||||
checks := 0
|
||||
result, err := configkit.LoadLayered(configkit.Input{Source: test.source, Filesystem: test.context}, configkit.LayeredOptions[equivalentConfig]{
|
||||
Definition: configkit.Definition[equivalentConfig]{Version: &configkit.VersionRule{
|
||||
Path: configkit.Keys("version"), Check: func(version configkit.Version, validation *configkit.Validation) error {
|
||||
checks++
|
||||
if !version.Present || version.Text != "1" {
|
||||
t.Errorf("version = %#v", version)
|
||||
}
|
||||
return nil
|
||||
},
|
||||
}},
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
want := equivalentConfig{Version: 1, Name: "retained", Items: []string{}}
|
||||
if !reflect.DeepEqual(result.Config, want) || checks != 1 {
|
||||
t.Fatalf("config = %#v, checks %d", result.Config, checks)
|
||||
}
|
||||
for _, field := range []string{"version", "name", "enabled", "count", "items", "optional"} {
|
||||
info, ok := result.Metadata().Lookup(configkit.Keys(field))
|
||||
if !ok || !info.Explicit || info.Origin.Layer != configkit.LayerBase || info.Null != (field == "optional") {
|
||||
t.Errorf("%s input info = %#v, %v", field, info, ok)
|
||||
}
|
||||
}
|
||||
sources := result.Sources()
|
||||
if len(sources) != 1 || sources[0].Kind != test.kind || sources[0].ID != "source-1" || sources[0].UnderlayIndex != 0 {
|
||||
t.Fatalf("sources = %#v", sources)
|
||||
}
|
||||
resource, ok := result.SourceContext("source-1")
|
||||
if !ok || resource.BaseDir != "." || resource.FS == nil {
|
||||
t.Fatalf("source context = %#v, %v", resource, ok)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestRepeatedImportsAcrossTiersAndReaderConsumption(t *testing.T) {
|
||||
namespace := configkit.NewFilesystem("bundle", fstest.MapFS{
|
||||
"left.yml": &fstest.MapFile{Data: []byte("configkit: {imports: [{path: shared.yml}]}\n")},
|
||||
"right.yml": &fstest.MapFile{Data: []byte("configkit: {imports: [{path: shared.yml}]}\n")},
|
||||
"shared.yml": &fstest.MapFile{Data: []byte("name: retained\n")},
|
||||
})
|
||||
type config struct {
|
||||
Name string `yaml:"name"`
|
||||
}
|
||||
left := configkit.Input{Source: namespace.File("left.yml"), Imports: true}
|
||||
right := configkit.Input{Source: namespace.File("right.yml"), Imports: true}
|
||||
_, err := configkit.LoadLayered(configkit.Input{Source: configkit.NoSource()}, configkit.LayeredOptions[config]{
|
||||
Underlays: []configkit.Underlay{{Inputs: []configkit.Input{left, right}}},
|
||||
})
|
||||
if got := layeredErrorCode(t, err); got != configkit.CodeImportRepeated {
|
||||
t.Fatalf("same-tier repeated import = %s", got)
|
||||
}
|
||||
result, err := configkit.LoadLayered(configkit.Input{Source: configkit.NoSource()}, configkit.LayeredOptions[config]{
|
||||
Underlays: []configkit.Underlay{{Inputs: []configkit.Input{left}}, {Inputs: []configkit.Input{right}}},
|
||||
})
|
||||
if err != nil || result.Config.Name != "retained" || len(result.Sources()) != 4 {
|
||||
t.Fatalf("cross-tier repeated import = %#v, %v", result, err)
|
||||
}
|
||||
reader := configkit.Reader("single-use", strings.NewReader("name: retained\n"))
|
||||
_, err = configkit.LoadLayered(configkit.Input{Source: configkit.NoSource()}, configkit.LayeredOptions[config]{
|
||||
Underlays: []configkit.Underlay{{Inputs: []configkit.Input{{Source: reader}}}, {Inputs: []configkit.Input{{Source: reader}}}},
|
||||
})
|
||||
if got := layeredErrorCode(t, err); got != configkit.CodeSourceReaderUsed {
|
||||
t.Fatalf("reader re-consumption = %s", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestReplacedSecretRemainsAbsentFromSafeOutput(t *testing.T) {
|
||||
type config struct {
|
||||
Secret string `yaml:"secret"`
|
||||
}
|
||||
result, err := configkit.LoadLayered(configkit.Input{Source: configkit.Bytes("main", []byte("secret: new-secret-canary\n"))}, configkit.LayeredOptions[config]{
|
||||
Underlays: []configkit.Underlay{{Inputs: []configkit.Input{{Source: configkit.Mapping("lower", map[string]any{"secret": "old-secret-canary"})}}}},
|
||||
Definition: configkit.Definition[config]{Secret: func(path configkit.Path) bool { return path.String() == "/secret" }},
|
||||
})
|
||||
if err != nil || result.Config.Secret != "new-secret-canary" {
|
||||
t.Fatalf("secret load = %#v, %v", result, err)
|
||||
}
|
||||
var effective, history bytes.Buffer
|
||||
if err := result.WriteEffective(&effective); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := result.WriteExplanation(&history, configkit.Keys("secret")); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if strings.Contains(effective.String()+history.String(), "old-secret-canary") || strings.Contains(effective.String()+history.String(), "new-secret-canary") {
|
||||
t.Fatalf("secret output leaked: %q %q", effective.String(), history.String())
|
||||
}
|
||||
_, err = configkit.LoadLayered(configkit.Input{Source: configkit.Bytes("main", []byte("secret: new-secret-canary\n"))}, configkit.LayeredOptions[config]{
|
||||
Underlays: []configkit.Underlay{{Inputs: []configkit.Input{{Source: configkit.Mapping("lower", map[string]any{"secret": "old-secret-canary"})}}}},
|
||||
Definition: configkit.Definition[config]{Validate: func(config *config, validation *configkit.Validation) {
|
||||
validation.Error(configkit.Keys("secret"), "invalid secret")
|
||||
}},
|
||||
})
|
||||
loadError, ok := err.(*configkit.LoadError)
|
||||
if !ok {
|
||||
t.Fatalf("validation error = %#v", err)
|
||||
}
|
||||
var diagnostics bytes.Buffer
|
||||
if err := configkit.WriteDiagnostics(&diagnostics, loadError.Diagnostics()); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if strings.Contains(diagnostics.String(), "secret-canary") {
|
||||
t.Fatalf("diagnostic leaked secret: %q", diagnostics.String())
|
||||
}
|
||||
}
|
||||
|
||||
func layeredErrorCode(t *testing.T, err error) configkit.Code {
|
||||
t.Helper()
|
||||
loadError, ok := err.(*configkit.LoadError)
|
||||
if !ok || len(loadError.Diagnostics()) == 0 {
|
||||
t.Fatalf("load error = %#v", err)
|
||||
}
|
||||
return loadError.Diagnostics()[0].Code
|
||||
}
|
||||
142
facade_filesystem_test.go
Normal file
142
facade_filesystem_test.go
Normal file
@@ -0,0 +1,142 @@
|
||||
package configkit_test
|
||||
|
||||
import (
|
||||
"embed"
|
||||
"io/fs"
|
||||
"reflect"
|
||||
"strconv"
|
||||
"strings"
|
||||
"testing"
|
||||
"testing/fstest"
|
||||
|
||||
configkit "gitea.maximumdirect.net/eric/ConfigKit"
|
||||
)
|
||||
|
||||
//go:embed testdata/filesystem
|
||||
var filesystemFixtures embed.FS
|
||||
|
||||
type filesystemFacadeConfig struct {
|
||||
Items map[string]string `yaml:"items"`
|
||||
}
|
||||
|
||||
func TestFacadeFilesystemSourceImportsOverlayAndReuse(t *testing.T) {
|
||||
fixture, err := fs.Sub(filesystemFixtures, "testdata/filesystem")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
namespace := configkit.NewFilesystem("embedded", fixture)
|
||||
copyOfNamespace := namespace
|
||||
source := namespace.File("config/main.yml")
|
||||
for range 2 {
|
||||
result, err := configkit.Load(source, configkit.Options[filesystemFacadeConfig]{
|
||||
Imports: true, Overlay: "testing", Filesystem: &configkit.FilesystemContext{FS: ©OfNamespace},
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if want := (map[string]string{"main": "selected", "shared": "default"}); !reflect.DeepEqual(result.Config.Items, want) {
|
||||
t.Fatalf("items = %#v, want %#v", result.Config.Items, want)
|
||||
}
|
||||
sources := result.Sources()
|
||||
if len(sources) != 3 {
|
||||
t.Fatalf("sources = %#v", sources)
|
||||
}
|
||||
for index, want := range []string{"config/main.yml", "shared.yml", "overlays/testing.yml"} {
|
||||
if sources[index].ID != "source-"+strconv.Itoa(index+1) || sources[index].Path != want || sources[index].Kind != configkit.SourceKindFilesystem {
|
||||
t.Errorf("source %d = %#v", index, sources[index])
|
||||
}
|
||||
if sources[index].Label != "embedded/"+want {
|
||||
t.Errorf("source label = %q", sources[index].Label)
|
||||
}
|
||||
}
|
||||
if sources[1].ImportedFrom == nil || sources[1].ImportedFrom.SourceID != sources[0].ID || sources[1].ImportedFrom.Line == 0 {
|
||||
t.Fatalf("import location = %#v", sources[1].ImportedFrom)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestFacadeFilesystemContextsAndIdentity(t *testing.T) {
|
||||
first := configkit.NewFilesystem("same", fstest.MapFS{
|
||||
"root/config/main.yml": &fstest.MapFile{Data: []byte("configkit: {imports: [{path: ../shared.yml}]}\n")},
|
||||
"root/shared.yml": &fstest.MapFile{Data: []byte("items: {value: first}\n")},
|
||||
"root/overlays/testing.yml": &fstest.MapFile{Data: []byte("items: {overlay: first}\n")},
|
||||
})
|
||||
second := configkit.NewFilesystem("same", fstest.MapFS{
|
||||
"root/config/main.yml": &fstest.MapFile{Data: []byte("items: {value: second}\n")},
|
||||
})
|
||||
context := &configkit.FilesystemContext{Root: "root", FS: &first}
|
||||
result, err := configkit.Load(first.File("root/config/main.yml"), configkit.Options[filesystemFacadeConfig]{Imports: true, Overlay: "testing", Filesystem: context})
|
||||
if err != nil || result.Config.Items["value"] != "first" || result.Config.Items["overlay"] != "first" {
|
||||
t.Fatalf("contained load = %#v, %v", result, err)
|
||||
}
|
||||
result, err = configkit.Load[filesystemFacadeConfig](second.File("root/config/main.yml"))
|
||||
if err != nil || result.Config.Items["value"] != "second" {
|
||||
t.Fatalf("same-label independent namespace = %#v, %v", result, err)
|
||||
}
|
||||
checkCode(t, second.File("root/config/main.yml"), configkit.Options[filesystemFacadeConfig]{Filesystem: context}, configkit.CodeOptionsInvalid)
|
||||
checkCode(t, first.File("root/config/main.yml"), configkit.Options[filesystemFacadeConfig]{Filesystem: &configkit.FilesystemContext{Root: "root/config", FS: &first}, Imports: true}, configkit.CodeImportOutsideRoot)
|
||||
checkCode(t, configkit.File("testdata/filesystem/config/main.yml"), configkit.Options[filesystemFacadeConfig]{Filesystem: context}, configkit.CodeOptionsInvalid)
|
||||
|
||||
for _, input := range []struct {
|
||||
name string
|
||||
source configkit.Source
|
||||
kind configkit.SourceKind
|
||||
}{
|
||||
{"bytes", configkit.Bytes("memory", []byte("configkit: {imports: [{path: shared.yml}]}\n")), configkit.SourceKindBytes},
|
||||
{"reader", configkit.Reader("stream", strings.NewReader("configkit: {imports: [{path: shared.yml}]}\n")), configkit.SourceKindReader},
|
||||
} {
|
||||
t.Run(input.name, func(t *testing.T) {
|
||||
result, err := configkit.Load(input.source, configkit.Options[filesystemFacadeConfig]{Imports: true, Filesystem: &configkit.FilesystemContext{Root: "root", BaseDir: "root", FS: &first}})
|
||||
if err != nil || result.Config.Items["value"] != "first" {
|
||||
t.Fatalf("synthetic import = %#v, %v", result, err)
|
||||
}
|
||||
if got := result.Sources(); len(got) != 2 || got[0].Kind != input.kind || got[0].Path != "" || got[1].Kind != configkit.SourceKindFilesystem {
|
||||
t.Fatalf("sources = %#v", got)
|
||||
}
|
||||
})
|
||||
}
|
||||
checkCode(t, configkit.Bytes("memory", []byte("items: {}\n")), configkit.Options[filesystemFacadeConfig]{Filesystem: &configkit.FilesystemContext{Root: "root", BaseDir: ".", FS: &first}}, configkit.CodeImportOutsideRoot)
|
||||
result, err = configkit.Load(configkit.Bytes("memory", []byte("items: {value: main}\n")), configkit.Options[filesystemFacadeConfig]{
|
||||
Overlay: "testing", Filesystem: &configkit.FilesystemContext{Root: "root", BaseDir: "root", FS: &first},
|
||||
})
|
||||
if err != nil || result.Config.Items["value"] != "main" || result.Config.Items["overlay"] != "first" {
|
||||
t.Fatalf("synthetic selected overlay = %#v, %v", result, err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestFacadeSourceKindsIncludeHostAndCLI(t *testing.T) {
|
||||
result, err := configkit.Load(configkit.File("testdata/filesystem/shared.yml"), configkit.Options[filesystemFacadeConfig]{
|
||||
Overrides: []configkit.Override{{Path: configkit.Keys("items", "shared"), Value: "changed", Flag: "--shared"}},
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
sources := result.Sources()
|
||||
if len(sources) != 2 || sources[0].Kind != configkit.SourceKindFile || sources[0].Path == "" || sources[1].Kind != configkit.SourceKindCLI || sources[1].Path != "" {
|
||||
t.Fatalf("host and CLI sources = %#v", sources)
|
||||
}
|
||||
}
|
||||
|
||||
func TestFacadeFilesystemConstructorDiagnostics(t *testing.T) {
|
||||
var nilFilesystem fs.FS
|
||||
var zero configkit.Filesystem
|
||||
for _, source := range []configkit.Source{
|
||||
zero.File("main.yml"),
|
||||
configkit.NewFilesystem("", fstest.MapFS{}).File("main.yml"),
|
||||
configkit.NewFilesystem("bad\nlabel", fstest.MapFS{}).File("main.yml"),
|
||||
configkit.NewFilesystem("good", nilFilesystem).File("main.yml"),
|
||||
configkit.NewFilesystem("good", fstest.MapFS{}).File("."),
|
||||
configkit.NewFilesystem("good", fstest.MapFS{}).File("../outside.yml"),
|
||||
} {
|
||||
checkCode(t, source, configkit.Options[filesystemFacadeConfig]{}, configkit.CodeSourceInvalid)
|
||||
}
|
||||
}
|
||||
|
||||
func checkCode(t *testing.T, source configkit.Source, options configkit.Options[filesystemFacadeConfig], want configkit.Code) {
|
||||
t.Helper()
|
||||
_, err := configkit.Load(source, options)
|
||||
loadError, ok := err.(*configkit.LoadError)
|
||||
if !ok || len(loadError.Diagnostics()) == 0 || loadError.Diagnostics()[0].Code != want {
|
||||
t.Fatalf("error = %#v, want %s", err, want)
|
||||
}
|
||||
}
|
||||
284
facade_inheritance_test.go
Normal file
284
facade_inheritance_test.go
Normal file
@@ -0,0 +1,284 @@
|
||||
package configkit_test
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"io/fs"
|
||||
"reflect"
|
||||
"strings"
|
||||
"testing"
|
||||
"testing/fstest"
|
||||
|
||||
configkit "gitea.maximumdirect.net/eric/ConfigKit"
|
||||
)
|
||||
|
||||
type inheritedEntry struct {
|
||||
Enabled bool `yaml:"enabled"`
|
||||
Count int `yaml:"count"`
|
||||
Resource string `yaml:"resource"`
|
||||
Settings map[string]map[string]string `yaml:"settings"`
|
||||
}
|
||||
|
||||
type inheritedConfiguration struct {
|
||||
Version string `yaml:"version"`
|
||||
Entries map[string]inheritedEntry `yaml:"entries"`
|
||||
}
|
||||
|
||||
func TestFacadeInheritancePreservesPresenceOriginsAndHistory(t *testing.T) {
|
||||
input := []byte(`entries:
|
||||
parent:
|
||||
enabled: false
|
||||
count: 0
|
||||
settings:
|
||||
primary: {value: original}
|
||||
secondary: {extends: primary}
|
||||
child:
|
||||
extends: parent
|
||||
settings:
|
||||
primary: {value: custom}
|
||||
`)
|
||||
defaultRan := false
|
||||
result, err := configkit.Load(configkit.Bytes("config.yml", input), configkit.Options[inheritedConfiguration]{Definition: configkit.Definition[inheritedConfiguration]{
|
||||
Resolve: func(config *inheritedConfiguration, resolution *configkit.Resolution) error {
|
||||
if !resolution.IsExplicit(configkit.Keys("entries", "child", "enabled")) {
|
||||
t.Error("inherited false is not explicit")
|
||||
}
|
||||
info, found := resolution.InputInfo(configkit.Keys("entries", "child", "enabled"))
|
||||
if !found || len(info.Origin.Inheritance) != 1 {
|
||||
t.Errorf("resolver inherited origin = %#v, %v", info, found)
|
||||
}
|
||||
child := config.Entries["child"]
|
||||
configkit.Default(resolution, configkit.Keys("entries", "child", "enabled"), &child.Enabled, true, "use fallback")
|
||||
defaultRan = child.Enabled
|
||||
config.Entries["child"] = child
|
||||
return nil
|
||||
},
|
||||
}})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
child := result.Config.Entries["child"]
|
||||
if defaultRan || child.Enabled || child.Count != 0 || child.Settings["secondary"]["value"] != "custom" {
|
||||
t.Fatalf("inherited values = %#v, default ran %v", child, defaultRan)
|
||||
}
|
||||
path := configkit.Keys("entries", "child", "settings", "secondary", "value")
|
||||
info, found := result.Metadata().Lookup(path)
|
||||
if !found || !info.Explicit || len(info.Origin.Inheritance) != 1 ||
|
||||
info.Origin.Inheritance[0].From.String() != "/entries/child/settings/primary/value" ||
|
||||
info.Origin.Inheritance[0].To.String() != path.String() {
|
||||
t.Fatalf("nested destination origin = %#v, %v", info, found)
|
||||
}
|
||||
info.Origin.Inheritance[0].Locations[0].SourceID = "mutated"
|
||||
info, _ = result.Metadata().Lookup(path)
|
||||
if info.Origin.Inheritance[0].Locations[0].SourceID == "mutated" {
|
||||
t.Fatal("public inheritance record aliases inspection state")
|
||||
}
|
||||
if _, found := result.Metadata().Lookup(configkit.Keys("entries", "child", "extends")); found {
|
||||
t.Fatal("consumed directive remained effective")
|
||||
}
|
||||
control, found := result.Explain(configkit.Keys("entries", "child", "extends"))
|
||||
if !found || control.Effective.Present || len(control.History) < 2 ||
|
||||
control.History[len(control.History)-1].Origin.Operation != configkit.ChangeInheritance {
|
||||
t.Fatalf("consumed directive = %#v, %v", control, found)
|
||||
}
|
||||
explanation, found := result.Explain(path)
|
||||
if !found || explanation.Effective.YAML != "custom\n" {
|
||||
t.Fatalf("nested explanation = %#v, %v", explanation, found)
|
||||
}
|
||||
for _, change := range explanation.History {
|
||||
if change.Origin.Operation == configkit.ChangeDecoding {
|
||||
t.Fatalf("inherited value attributed to decoding: %#v", explanation.History)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestFacadeInheritanceLayeredPrecedenceAndSourceContext(t *testing.T) {
|
||||
filesystem := configkit.NewFilesystem("bundle", fstest.MapFS{
|
||||
"config/main.yml": &fstest.MapFile{Data: []byte("entries: {child: {extends: parent}}\n")},
|
||||
"shared/parent.yml": &fstest.MapFile{Data: []byte("entries: {parent: {resource: icon.txt, count: 1}}\n")},
|
||||
"shared/flags.yml": &fstest.MapFile{Data: []byte("entries: {parent: {enabled: false}}\n")},
|
||||
"shared/icon.txt": &fstest.MapFile{Data: []byte("parent-resource")},
|
||||
"config/overlays/test.yml": &fstest.MapFile{Data: []byte("entries: {parent: {count: 2}}\n")},
|
||||
})
|
||||
result, err := configkit.LoadLayered(configkit.Input{Source: filesystem.File("config/main.yml")}, configkit.LayeredOptions[inheritedConfiguration]{
|
||||
Underlays: []configkit.Underlay{
|
||||
{Inputs: []configkit.Input{{Source: filesystem.File("shared/parent.yml")}}},
|
||||
{Inputs: []configkit.Input{{Source: filesystem.File("shared/flags.yml")}}},
|
||||
},
|
||||
Overlay: "test", OverlayFilesystem: &configkit.FilesystemContext{Root: "config", BaseDir: "config", FS: &filesystem},
|
||||
Overrides: []configkit.Override{{Path: configkit.Keys("entries", "child", "count"), Value: 3, Flag: "--count"}},
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
child := result.Config.Entries["child"]
|
||||
if child.Count != 3 || child.Resource != "icon.txt" {
|
||||
t.Fatalf("layered inheritance = %#v", child)
|
||||
}
|
||||
info, found := result.Metadata().Lookup(configkit.Keys("entries", "child", "resource"))
|
||||
if !found || len(info.Origin.Inheritance) != 1 || len(info.Origin.Locations) != 1 {
|
||||
t.Fatalf("resource origin = %#v, %v", info, found)
|
||||
}
|
||||
context, found := result.SourceContext(info.Origin.Locations[0].SourceID)
|
||||
if !found || context.BaseDir != "shared" || context.FS == nil {
|
||||
t.Fatalf("source context = %#v, %v", context, found)
|
||||
}
|
||||
data, err := fs.ReadFile(context.FS, context.BaseDir+"/"+child.Resource)
|
||||
if err != nil || string(data) != "parent-resource" {
|
||||
t.Fatalf("source-relative resource = %q, %v", data, err)
|
||||
}
|
||||
count, found := result.Metadata().Lookup(configkit.Keys("entries", "child", "count"))
|
||||
if !found || count.Origin.Layer != configkit.LayerCLI || len(count.Origin.Inheritance) != 0 {
|
||||
t.Fatalf("CLI replacement origin = %#v, %v", count, found)
|
||||
}
|
||||
var rendered bytes.Buffer
|
||||
if err := result.WriteEffective(&rendered); err != nil || strings.Contains(rendered.String(), "extends:") {
|
||||
t.Fatalf("effective rendering = %q, %v", rendered.String(), err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestFacadeInheritanceMappingMatchesBytes(t *testing.T) {
|
||||
values := map[string]any{"entries": map[string]any{
|
||||
"parent": map[string]any{"enabled": false, "count": 0},
|
||||
"child": map[string]any{"extends": "parent"},
|
||||
}}
|
||||
mapped, err := configkit.Load[inheritedConfiguration](configkit.Mapping("mapping", values))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
fromYAML, err := configkit.Load[inheritedConfiguration](configkit.Bytes("config.yml", []byte("entries: {parent: {enabled: false, count: 0}, child: {extends: parent}}\n")))
|
||||
if err != nil || !reflect.DeepEqual(mapped.Config, fromYAML.Config) {
|
||||
t.Fatalf("mapping and bytes differ: %#v, %#v, %v", mapped, fromYAML, err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestFacadeInheritanceProtectsVersionOwnership(t *testing.T) {
|
||||
for _, test := range []struct {
|
||||
name, input string
|
||||
path configkit.Path
|
||||
wantFailure bool
|
||||
}{
|
||||
{"inherited version", "entries: {parent: {count: 1}, child: {extends: parent}}\n", configkit.Keys("entries", "child", "count"), true},
|
||||
{"overridden version", "entries: {parent: {count: 1}, child: {extends: parent, count: 1}}\n", configkit.Keys("entries", "child", "count"), true},
|
||||
{"direct unchanged version", "entries: {parent: {enabled: false}, child: {extends: parent, count: 1}}\n", configkit.Keys("entries", "child", "count"), false},
|
||||
{"consumed checked directive", "entries: {parent: {count: 1}, child: {extends: parent}}\n", configkit.Keys("entries", "child", "extends"), true},
|
||||
{"independent top-level version", "version: v1\nentries: {parent: {count: 1}, child: {extends: parent}}\n", configkit.Keys("version"), false},
|
||||
} {
|
||||
t.Run(test.name, func(t *testing.T) {
|
||||
checks := struct{ version, defaults, resolution, validation int }{}
|
||||
result, err := configkit.Load(configkit.Bytes("version.yml", []byte(test.input)), configkit.Options[inheritedConfiguration]{Definition: configkit.Definition[inheritedConfiguration]{
|
||||
Version: &configkit.VersionRule{Path: test.path, Check: func(configkit.Version, *configkit.Validation) error {
|
||||
checks.version++
|
||||
return nil
|
||||
}},
|
||||
Defaults: func() inheritedConfiguration { checks.defaults++; return inheritedConfiguration{} },
|
||||
Resolve: func(*inheritedConfiguration, *configkit.Resolution) error { checks.resolution++; return nil },
|
||||
Validate: func(*inheritedConfiguration, *configkit.Validation) { checks.validation++ },
|
||||
}})
|
||||
if test.wantFailure {
|
||||
if result != nil || err == nil {
|
||||
t.Fatalf("protected version = %#v, %v", result, err)
|
||||
}
|
||||
loadError, ok := err.(*configkit.LoadError)
|
||||
if !ok || len(loadError.Diagnostics()) == 0 || loadError.Diagnostics()[0].Code != configkit.CodeInheritanceVersion ||
|
||||
checks != (struct{ version, defaults, resolution, validation int }{version: 1}) {
|
||||
t.Fatalf("version failure = %v, callbacks %#v", err, checks)
|
||||
}
|
||||
} else if err != nil || result == nil || checks != (struct{ version, defaults, resolution, validation int }{1, 1, 1, 1}) {
|
||||
t.Fatalf("unchanged version = %#v, %v, callbacks %#v", result, err, checks)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestFacadeInheritanceFailureStopsLaterCallbacks(t *testing.T) {
|
||||
checks := struct{ defaults, resolution, validation int }{}
|
||||
result, err := configkit.Load(configkit.Bytes("invalid.yml", []byte("entries: {child: {extends: PRIVATE-REFERENCE-CANARY}}\n")), configkit.Options[inheritedConfiguration]{Definition: configkit.Definition[inheritedConfiguration]{
|
||||
Defaults: func() inheritedConfiguration { checks.defaults++; return inheritedConfiguration{} },
|
||||
Resolve: func(*inheritedConfiguration, *configkit.Resolution) error { checks.resolution++; return nil },
|
||||
Validate: func(*inheritedConfiguration, *configkit.Validation) { checks.validation++ },
|
||||
}})
|
||||
if result != nil || err == nil || checks != (struct{ defaults, resolution, validation int }{}) || strings.Contains(err.Error(), "PRIVATE-REFERENCE-CANARY") {
|
||||
t.Fatalf("invalid inheritance = %#v, %v, callbacks %#v", result, err, checks)
|
||||
}
|
||||
loadError, ok := err.(*configkit.LoadError)
|
||||
if !ok || len(loadError.Diagnostics()) == 0 || loadError.Diagnostics()[0].Code != configkit.CodeInheritanceMissing {
|
||||
t.Fatalf("missing reference diagnostics = %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestFacadeInheritanceResolvesAfterImportsAndCLIReferenceChange(t *testing.T) {
|
||||
filesystem := configkit.NewFilesystem("bundle", fstest.MapFS{
|
||||
"main.yml": &fstest.MapFile{Data: []byte("configkit: {imports: [{path: parent.yml}]}\nentries: {child: {extends: first}}\n")},
|
||||
"parent.yml": &fstest.MapFile{Data: []byte("entries: {first: {count: 1}, second: {count: 2}}\n")},
|
||||
})
|
||||
result, err := configkit.Load(filesystem.File("main.yml"), configkit.Options[inheritedConfiguration]{
|
||||
Imports: true, Overrides: []configkit.Override{{Path: configkit.Keys("entries", "child", "extends"), Value: "second", Flag: "--extends"}},
|
||||
})
|
||||
if err != nil || result.Config.Entries["child"].Count != 2 {
|
||||
t.Fatalf("import and CLI reference = %#v, %v", result, err)
|
||||
}
|
||||
history := result.Metadata().History(configkit.Keys("entries", "child", "extends"))
|
||||
if len(history) < 3 || history[len(history)-1].Origin.Operation != configkit.ChangeInheritance {
|
||||
t.Fatalf("superseded reference history = %#v", history)
|
||||
}
|
||||
conflicting := configkit.NewFilesystem("bundle", fstest.MapFS{
|
||||
"main.yml": &fstest.MapFile{Data: []byte("configkit: {imports: [{path: parent.yml}]}\nentries: {child: {extends: first}, first: {count: 3}}\n")},
|
||||
"parent.yml": &fstest.MapFile{Data: []byte("entries: {first: {count: 1}}\n")},
|
||||
})
|
||||
_, err = configkit.Load[inheritedConfiguration](conflicting.File("main.yml"), configkit.Options[inheritedConfiguration]{Imports: true})
|
||||
loadError, ok := err.(*configkit.LoadError)
|
||||
if !ok || len(loadError.Diagnostics()) == 0 || loadError.Diagnostics()[0].Code != configkit.CodeComposeConflict {
|
||||
t.Fatalf("strict conflict before expansion = %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestFacadeInheritanceSelectedOverlayCancelsRelationship(t *testing.T) {
|
||||
filesystem := configkit.NewFilesystem("bundle", fstest.MapFS{
|
||||
"main.yml": &fstest.MapFile{Data: []byte("entries: {parent: {count: 7}, child: {extends: parent, enabled: false}}\n")},
|
||||
"overlays/test.yml": &fstest.MapFile{Data: []byte("entries: {child: {extends: null}}\n")},
|
||||
})
|
||||
result, err := configkit.Load(filesystem.File("main.yml"), configkit.Options[inheritedConfiguration]{
|
||||
Overlay: "test", Filesystem: &configkit.FilesystemContext{Root: ".", FS: &filesystem},
|
||||
})
|
||||
if err != nil || result.Config.Entries["child"].Count != 0 || result.Config.Entries["child"].Enabled {
|
||||
if loadError, ok := err.(*configkit.LoadError); ok {
|
||||
t.Fatalf("cancelled relationship = %#v, %v, diagnostics %#v", result, err, loadError.Diagnostics())
|
||||
}
|
||||
t.Fatalf("cancelled relationship = %#v, %v", result, err)
|
||||
}
|
||||
history := result.Metadata().History(configkit.Keys("entries", "child", "extends"))
|
||||
if len(history) < 3 || history[len(history)-1].Origin.Operation != configkit.ChangeInheritance {
|
||||
t.Fatalf("cancellation history = %#v", history)
|
||||
}
|
||||
}
|
||||
|
||||
func TestFacadeInheritedSecretsStayRedactedAfterLookup(t *testing.T) {
|
||||
const canary = "INHERITED-RELEASE-TEST-CREDENTIAL"
|
||||
t.Setenv("CONFIGKIT_INHERITED_TEST_SECRET", canary)
|
||||
type entry struct {
|
||||
Credential configkit.Secret `yaml:"credential"`
|
||||
Plain string `yaml:"plain"`
|
||||
}
|
||||
type configuration struct {
|
||||
Entries map[string]entry `yaml:"entries"`
|
||||
}
|
||||
result, err := configkit.Load[configuration](configkit.Bytes("secret.yml", []byte("entries: {parent: {credential: CONFIGKIT_INHERITED_TEST_SECRET, plain: visible}, child: {extends: parent}}\n")), configkit.Options[configuration]{Definition: configkit.Definition[configuration]{
|
||||
Secret: func(path configkit.Path) bool { return path.String() == "/entries/parent/credential" },
|
||||
}})
|
||||
if err != nil || result.Config.Entries["child"].Credential.Value() != canary {
|
||||
t.Fatalf("inherited credential = %#v, %v", result, err)
|
||||
}
|
||||
path := configkit.Keys("entries", "child", "credential")
|
||||
explanation, found := result.Explain(path)
|
||||
if !found || !explanation.Effective.Redacted || strings.Contains(explanation.Effective.YAML, canary) {
|
||||
t.Fatalf("credential explanation = %#v, %v", explanation, found)
|
||||
}
|
||||
var rendered bytes.Buffer
|
||||
if err := result.WriteEffective(&rendered); err != nil || strings.Contains(rendered.String(), canary) || !strings.Contains(rendered.String(), "visible") {
|
||||
t.Fatalf("effective secret rendering = %q, %v", rendered.String(), err)
|
||||
}
|
||||
rendered.Reset()
|
||||
if err := result.WriteExplanation(&rendered, path); err != nil || strings.Contains(rendered.String(), canary) {
|
||||
t.Fatalf("historical secret rendering = %q, %v", rendered.String(), err)
|
||||
}
|
||||
}
|
||||
243
facade_layered_test.go
Normal file
243
facade_layered_test.go
Normal file
@@ -0,0 +1,243 @@
|
||||
package configkit_test
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"reflect"
|
||||
"strconv"
|
||||
"strings"
|
||||
"testing"
|
||||
"testing/fstest"
|
||||
|
||||
configkit "gitea.maximumdirect.net/eric/ConfigKit"
|
||||
)
|
||||
|
||||
type layeredFacadeConfig struct {
|
||||
Settings map[string]string `yaml:"settings"`
|
||||
Count int `yaml:"count"`
|
||||
Enabled bool `yaml:"enabled"`
|
||||
List []string `yaml:"list"`
|
||||
Optional *string `yaml:"optional"`
|
||||
Derived string `yaml:"derived"`
|
||||
}
|
||||
|
||||
func TestFacadeLayeredMixedSourcesAndInspection(t *testing.T) {
|
||||
mainPath := filepath.Join(t.TempDir(), "main.yml")
|
||||
if err := os.WriteFile(mainPath, []byte("settings: {shared: main, secret: new-canary}\ncount: 0\nenabled: false\nlist: []\noptional: null\n"), 0o600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
lower := configkit.NewFilesystem("bundle", fstest.MapFS{
|
||||
"main.yml": &fstest.MapFile{Data: []byte("configkit: {imports: [{path: defaults.yml}]}\nsettings: {shared: lower, native: retained, secret: old-canary}\nlist: [lower]\n")},
|
||||
"defaults.yml": &fstest.MapFile{Data: []byte("settings: {imported: retained}\n")},
|
||||
"overlays/testing.yml": &fstest.MapFile{Data: []byte("settings: {from_lower_namespace: retained}\n")},
|
||||
})
|
||||
selected := configkit.NewFilesystem("bundle", fstest.MapFS{
|
||||
"overlays/testing.yml": &fstest.MapFile{Data: []byte("settings: {shared: overlay}\n")},
|
||||
})
|
||||
checks := struct{ defaults, resolution, validation int }{}
|
||||
result, err := configkit.LoadLayered(configkit.Input{Source: configkit.File(mainPath)}, configkit.LayeredOptions[layeredFacadeConfig]{
|
||||
Underlays: []configkit.Underlay{
|
||||
{Inputs: []configkit.Input{
|
||||
{Source: lower.File("main.yml"), Imports: true},
|
||||
{Source: configkit.Mapping("programmatic-low", map[string]any{"settings": map[string]any{"mapping": "retained"}, "count": 7, "enabled": true})},
|
||||
{Source: lower.File("overlays/testing.yml")},
|
||||
}},
|
||||
{Inputs: []configkit.Input{{Source: configkit.Mapping("programmatic-high", map[string]any{
|
||||
"settings": map[string]any{"shared": "middle", "second": "retained"}, "list": []any{"middle"},
|
||||
})}}},
|
||||
},
|
||||
Overlay: "testing", OverlayFilesystem: &configkit.FilesystemContext{Root: ".", BaseDir: ".", FS: &selected},
|
||||
Overrides: []configkit.Override{{Path: configkit.Keys("settings", "shared"), Value: "cli", Flag: "--shared"}},
|
||||
Definition: configkit.Definition[layeredFacadeConfig]{
|
||||
Defaults: func() layeredFacadeConfig { checks.defaults++; return layeredFacadeConfig{} },
|
||||
Resolve: func(config *layeredFacadeConfig, resolution *configkit.Resolution) error {
|
||||
checks.resolution++
|
||||
if source, ok := resolution.Source("source-2"); !ok || source.Layer != configkit.LayerUnderlay || source.UnderlayIndex != 1 || source.Kind != configkit.SourceKindFilesystem {
|
||||
t.Errorf("resolver source = %#v, %v", source, ok)
|
||||
}
|
||||
configkit.Default(resolution, configkit.Keys("derived"), &config.Derived, config.Settings["native"], "derive from native", configkit.Keys("settings", "native"))
|
||||
return nil
|
||||
},
|
||||
Validate: func(config *layeredFacadeConfig, validation *configkit.Validation) { checks.validation++ },
|
||||
Secret: func(path configkit.Path) bool { return path.String() == "/settings/secret" },
|
||||
},
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("layered load: %v, diagnostics %#v", err, err.(*configkit.LoadError).Diagnostics())
|
||||
}
|
||||
wantSettings := map[string]string{
|
||||
"shared": "cli", "secret": "new-canary", "native": "retained", "imported": "retained",
|
||||
"mapping": "retained", "from_lower_namespace": "retained", "second": "retained",
|
||||
}
|
||||
if !reflect.DeepEqual(result.Config.Settings, wantSettings) || result.Config.Count != 0 || result.Config.Enabled || result.Config.List == nil || len(result.Config.List) != 0 || result.Config.Optional != nil || result.Config.Derived != "retained" {
|
||||
t.Fatalf("configuration = %#v", result.Config)
|
||||
}
|
||||
if checks != (struct{ defaults, resolution, validation int }{1, 1, 1}) {
|
||||
t.Fatalf("lifecycle calls = %#v", checks)
|
||||
}
|
||||
sources := result.Sources()
|
||||
if len(sources) != 8 {
|
||||
t.Fatalf("sources = %#v", sources)
|
||||
}
|
||||
wantLayers := []configkit.Layer{configkit.LayerBase, configkit.LayerUnderlay, configkit.LayerUnderlay, configkit.LayerUnderlay, configkit.LayerUnderlay, configkit.LayerUnderlay, configkit.LayerOverlay, configkit.LayerCLI}
|
||||
wantIndexes := []int{0, 1, 1, 1, 1, 2, 0, 0}
|
||||
wantKinds := []configkit.SourceKind{configkit.SourceKindFile, configkit.SourceKindFilesystem, configkit.SourceKindFilesystem, configkit.SourceKindMapping, configkit.SourceKindFilesystem, configkit.SourceKindMapping, configkit.SourceKindFilesystem, configkit.SourceKindCLI}
|
||||
for index, source := range sources {
|
||||
if source.ID != "source-"+strconv.Itoa(index+1) || source.Layer != wantLayers[index] || source.UnderlayIndex != wantIndexes[index] || source.Kind != wantKinds[index] {
|
||||
t.Errorf("source %d = %#v", index, source)
|
||||
}
|
||||
}
|
||||
if sources[4].Path != sources[6].Path || sources[4].Label != sources[6].Label {
|
||||
t.Fatalf("namespace-collision fixture changed: %#v, %#v", sources[4], sources[6])
|
||||
}
|
||||
sources[1].Label = "changed"
|
||||
if result.Sources()[1].Label != "bundle/main.yml" {
|
||||
t.Fatal("source records alias retained metadata")
|
||||
}
|
||||
for _, test := range []struct {
|
||||
path string
|
||||
want configkit.Layer
|
||||
}{
|
||||
{"/settings/native", configkit.LayerUnderlay},
|
||||
{"/settings/shared", configkit.LayerCLI},
|
||||
{"/derived", configkit.LayerResolution},
|
||||
} {
|
||||
path, err := configkit.ParsePath(test.path)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
info, ok := result.Metadata().Lookup(path)
|
||||
if !ok || info.Origin.Layer != test.want {
|
||||
t.Errorf("%s metadata = %#v, %v", test.path, info, ok)
|
||||
}
|
||||
}
|
||||
container, ok := result.Metadata().Lookup(configkit.Keys("settings"))
|
||||
if !ok || len(container.Origin.Locations) < 4 {
|
||||
t.Fatalf("composite mapping origin = %#v, %v", container, ok)
|
||||
}
|
||||
explanation, ok := result.Explain(configkit.Keys("settings", "shared"))
|
||||
if !ok || len(explanation.History) < 5 || explanation.Origin.Layer != configkit.LayerCLI {
|
||||
t.Fatalf("shared history = %#v, %v", explanation, ok)
|
||||
}
|
||||
var effective, rendered bytes.Buffer
|
||||
if err := result.WriteEffective(&effective); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := result.WriteExplanation(&rendered, configkit.Keys("settings", "secret")); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if strings.Contains(effective.String()+rendered.String(), "old-canary") || strings.Contains(effective.String()+rendered.String(), "new-canary") || !strings.Contains(rendered.String(), "redacted") {
|
||||
t.Fatalf("secret rendering leaked: effective %q, explanation %q", effective.String(), rendered.String())
|
||||
}
|
||||
}
|
||||
|
||||
func TestFacadeLayeredDisplacedDescendantRemainsInspectable(t *testing.T) {
|
||||
type config struct {
|
||||
Choice string `yaml:"choice"`
|
||||
}
|
||||
result, err := configkit.LoadLayered(configkit.Input{Source: configkit.Mapping("main", map[string]any{"choice": "flat"})}, configkit.LayeredOptions[config]{
|
||||
Underlays: []configkit.Underlay{{Inputs: []configkit.Input{{Source: configkit.Mapping("lower", map[string]any{
|
||||
"choice": map[string]any{"nested": "historical"},
|
||||
})}}}},
|
||||
})
|
||||
if err != nil || result.Config.Choice != "flat" {
|
||||
t.Fatalf("replacement load = %#v, %v", result, err)
|
||||
}
|
||||
path := configkit.Keys("choice", "nested")
|
||||
explanation, ok := result.Explain(path)
|
||||
if !ok || explanation.Effective.Present || len(explanation.History) == 0 || explanation.History[0].Origin.Layer != configkit.LayerUnderlay {
|
||||
t.Fatalf("deleted descendant = %#v, %v", explanation, ok)
|
||||
}
|
||||
}
|
||||
|
||||
func TestFacadeLayeredZeroUnderlaysMatchesOrdinaryLoad(t *testing.T) {
|
||||
filesystem := configkit.NewFilesystem("bundle", fstest.MapFS{
|
||||
"overlays/testing.yml": &fstest.MapFile{Data: []byte("value: selected\n")},
|
||||
})
|
||||
context := &configkit.FilesystemContext{Root: ".", BaseDir: ".", FS: &filesystem}
|
||||
source := configkit.Bytes("main", []byte("value: base\n"))
|
||||
type config struct {
|
||||
Value string `yaml:"value"`
|
||||
}
|
||||
ordinary, err := configkit.Load(source, configkit.Options[config]{Overlay: "testing", Filesystem: context})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
layered, err := configkit.LoadLayered(configkit.Input{Source: source}, configkit.LayeredOptions[config]{Overlay: "testing", OverlayFilesystem: context})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if ordinary.Config != layered.Config || !reflect.DeepEqual(ordinary.Sources(), layered.Sources()) {
|
||||
t.Fatalf("ordinary = %#v, %#v; layered = %#v, %#v", ordinary.Config, ordinary.Sources(), layered.Config, layered.Sources())
|
||||
}
|
||||
var first, second bytes.Buffer
|
||||
if err := ordinary.WriteEffective(&first); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := layered.WriteEffective(&second); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if first.String() != second.String() {
|
||||
t.Fatalf("effective ordinary %q, layered %q", first.String(), second.String())
|
||||
}
|
||||
if ordinaryExplanation, ok := ordinary.Explain(configkit.Keys("value")); !ok {
|
||||
t.Fatal("ordinary explanation missing")
|
||||
} else if layeredExplanation, ok := layered.Explain(configkit.Keys("value")); !ok || !reflect.DeepEqual(ordinaryExplanation, layeredExplanation) {
|
||||
t.Fatalf("explanations differ: %#v, %#v", ordinaryExplanation, layeredExplanation)
|
||||
}
|
||||
}
|
||||
|
||||
func TestFacadeLayeredOptionAndVersionFailures(t *testing.T) {
|
||||
type config struct {
|
||||
Version string `yaml:"version"`
|
||||
}
|
||||
namespace := configkit.NewFilesystem("bundle", fstest.MapFS{
|
||||
"overlays/testing.yml": &fstest.MapFile{Data: []byte("{}\n")},
|
||||
})
|
||||
for _, test := range []struct {
|
||||
name string
|
||||
main configkit.Input
|
||||
options []configkit.LayeredOptions[config]
|
||||
want configkit.Code
|
||||
}{
|
||||
{"zero source", configkit.Input{}, nil, configkit.CodeSourceInvalid},
|
||||
{"multiple options", configkit.Input{Source: configkit.NoSource()}, []configkit.LayeredOptions[config]{{}, {}}, configkit.CodeOptionsInvalid},
|
||||
{"missing overlay context", configkit.Input{Source: configkit.NoSource()}, []configkit.LayeredOptions[config]{{Overlay: "testing"}}, configkit.CodeOptionsInvalid},
|
||||
{"selected file already acquired", configkit.Input{Source: configkit.NoSource()}, []configkit.LayeredOptions[config]{{
|
||||
Overlay: "testing", OverlayFilesystem: &configkit.FilesystemContext{Root: ".", BaseDir: ".", FS: &namespace},
|
||||
Underlays: []configkit.Underlay{{Inputs: []configkit.Input{{Source: namespace.File("overlays/testing.yml")}}}},
|
||||
}}, configkit.CodeOverlayRepeated},
|
||||
{"underlay version", configkit.Input{Source: configkit.NoSource()}, []configkit.LayeredOptions[config]{{
|
||||
Definition: configkit.Definition[config]{Version: &configkit.VersionRule{Path: configkit.Keys("version"), Check: func(configkit.Version, *configkit.Validation) error { return nil }}},
|
||||
Underlays: []configkit.Underlay{{Inputs: []configkit.Input{{Source: configkit.Mapping("lower", map[string]any{"version": "forbidden"})}}}},
|
||||
}}, configkit.CodeUnderlayVersion},
|
||||
} {
|
||||
t.Run(test.name, func(t *testing.T) {
|
||||
_, err := configkit.LoadLayered(test.main, test.options...)
|
||||
loadError, ok := err.(*configkit.LoadError)
|
||||
if !ok || len(loadError.Diagnostics()) == 0 || loadError.Diagnostics()[0].Code != test.want {
|
||||
t.Fatalf("error = %#v, want %s", err, test.want)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestFacadeLayeredExplicitHostOverlayWithoutMain(t *testing.T) {
|
||||
root := t.TempDir()
|
||||
if err := os.Mkdir(filepath.Join(root, "overlays"), 0o700); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := os.WriteFile(filepath.Join(root, "overlays", "testing.yml"), []byte("value: selected\n"), 0o600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
type config struct {
|
||||
Value string `yaml:"value"`
|
||||
}
|
||||
result, err := configkit.LoadLayered(configkit.Input{Source: configkit.NoSource()}, configkit.LayeredOptions[config]{
|
||||
Overlay: "testing", OverlayFilesystem: &configkit.FilesystemContext{Root: root, BaseDir: root},
|
||||
})
|
||||
if err != nil || result.Config.Value != "selected" || len(result.Sources()) != 1 || result.Sources()[0].Kind != configkit.SourceKindFile {
|
||||
t.Fatalf("explicit host overlay = %#v, %v", result, err)
|
||||
}
|
||||
}
|
||||
133
facade_mapping_test.go
Normal file
133
facade_mapping_test.go
Normal file
@@ -0,0 +1,133 @@
|
||||
package configkit_test
|
||||
|
||||
import (
|
||||
"math"
|
||||
"reflect"
|
||||
"strings"
|
||||
"testing"
|
||||
"testing/fstest"
|
||||
|
||||
configkit "gitea.maximumdirect.net/eric/ConfigKit"
|
||||
)
|
||||
|
||||
type mappingFacadeConfig struct {
|
||||
Enabled bool `yaml:"enabled"`
|
||||
Count uint64 `yaml:"count"`
|
||||
Optional *string `yaml:"optional"`
|
||||
Items []string `yaml:"items"`
|
||||
Name string `yaml:"name"`
|
||||
}
|
||||
|
||||
func TestFacadeMappingSnapshotsAndMatchesYAML(t *testing.T) {
|
||||
items := []any{"one", "two"}
|
||||
values := map[string]any{
|
||||
"name": "original", "enabled": false, "count": ^uint64(0),
|
||||
"optional": nil, "items": items,
|
||||
}
|
||||
source := configkit.Mapping("programmatic", values)
|
||||
values["name"] = "mutated"
|
||||
items[0] = "mutated"
|
||||
for range 2 {
|
||||
result, err := configkit.Load[mappingFacadeConfig](source)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
want := mappingFacadeConfig{Count: ^uint64(0), Items: []string{"one", "two"}, Name: "original"}
|
||||
if !reflect.DeepEqual(result.Config, want) {
|
||||
t.Fatalf("mapping config = %#v, want %#v", result.Config, want)
|
||||
}
|
||||
if sources := result.Sources(); len(sources) != 1 || sources[0].ID != "source-1" || sources[0].Kind != configkit.SourceKindMapping || sources[0].Layer != configkit.LayerBase || sources[0].Path != "" {
|
||||
t.Fatalf("mapping sources = %#v", sources)
|
||||
}
|
||||
info, ok := result.Metadata().Lookup(configkit.Keys("name"))
|
||||
if !ok || len(info.Origin.Locations) != 1 || info.Origin.Locations[0].SourceID != "source-1" || info.Origin.Locations[0].Line != 0 {
|
||||
t.Fatalf("mapping name origin = %#v, %v", info, ok)
|
||||
}
|
||||
fromYAML, err := configkit.Load[mappingFacadeConfig](configkit.Bytes("equivalent", []byte("enabled: false\ncount: 18446744073709551615\noptional: null\nitems: [one, two]\nname: original\n")))
|
||||
if err != nil || !reflect.DeepEqual(result.Config, fromYAML.Config) {
|
||||
t.Fatalf("equivalent YAML = %#v, %v", fromYAML, err)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestFacadeMappingNullEmptyAndVersion(t *testing.T) {
|
||||
result, err := configkit.Load[mappingFacadeConfig](configkit.Mapping("empty", map[string]any{}))
|
||||
if err != nil || len(result.Sources()) != 1 {
|
||||
t.Fatalf("empty mapping = %#v, %v", result, err)
|
||||
}
|
||||
var nilSequence []any
|
||||
var nilMap map[string]any
|
||||
var checked string
|
||||
result, err = configkit.Load(configkit.Mapping("nullable", map[string]any{
|
||||
"items": nilSequence, "optional": nilMap, "enabled": false, "count": uint64(0),
|
||||
}), configkit.Options[mappingFacadeConfig]{Definition: configkit.Definition[mappingFacadeConfig]{
|
||||
Version: &configkit.VersionRule{Path: configkit.Keys("count"), Check: func(version configkit.Version, validation *configkit.Validation) error {
|
||||
checked = version.Text
|
||||
return nil
|
||||
}},
|
||||
}})
|
||||
if err != nil || checked != "0" || result.Config.Items != nil || result.Config.Optional != nil {
|
||||
t.Fatalf("null and zero mapping = %#v, checked %q, %v", result, checked, err)
|
||||
}
|
||||
info, ok := result.Metadata().Lookup(configkit.Keys("items"))
|
||||
if !ok || !info.Null || !info.Explicit {
|
||||
t.Fatalf("null metadata = %#v, %v", info, ok)
|
||||
}
|
||||
}
|
||||
|
||||
func TestFacadeMappingImportsUseExplicitContext(t *testing.T) {
|
||||
namespace := configkit.NewFilesystem("bundle", fstest.MapFS{
|
||||
"root/shared.yml": &fstest.MapFile{Data: []byte("name: imported\n")},
|
||||
})
|
||||
source := configkit.Mapping("programmatic", map[string]any{
|
||||
"configkit": map[string]any{"imports": []any{map[string]any{"path": "shared.yml"}}},
|
||||
"enabled": true,
|
||||
})
|
||||
result, err := configkit.Load(source, configkit.Options[mappingFacadeConfig]{
|
||||
Imports: true, Filesystem: &configkit.FilesystemContext{Root: "root", BaseDir: "root", FS: &namespace},
|
||||
})
|
||||
if err != nil || result.Config.Name != "imported" || !result.Config.Enabled {
|
||||
t.Fatalf("mapping import = %#v, %v", result, err)
|
||||
}
|
||||
if sources := result.Sources(); len(sources) != 2 || sources[0].Kind != configkit.SourceKindMapping || sources[1].Kind != configkit.SourceKindFilesystem || sources[1].Layer != configkit.LayerBase {
|
||||
t.Fatalf("mapping import sources = %#v", sources)
|
||||
}
|
||||
checkMappingCode(t, source, configkit.Options[mappingFacadeConfig]{Imports: true}, configkit.CodeOptionsInvalid)
|
||||
checkMappingCode(t, source, configkit.Options[mappingFacadeConfig]{}, configkit.CodeImportDisabled)
|
||||
}
|
||||
|
||||
func TestFacadeMappingDeferredValueDiagnostics(t *testing.T) {
|
||||
cycle := map[string]any{}
|
||||
cycle["secret"] = cycle
|
||||
for _, test := range []struct {
|
||||
name string
|
||||
source configkit.Source
|
||||
want configkit.Code
|
||||
path string
|
||||
}{
|
||||
{"nil root", configkit.Mapping("nil", nil), configkit.CodeSourceInvalid, ""},
|
||||
{"bad label", configkit.Mapping("bad\nlabel", map[string]any{}), configkit.CodeSourceInvalid, ""},
|
||||
{"unsupported", configkit.Mapping("bad", map[string]any{"secret": struct{ Value string }{"canary-value"}}), configkit.CodeSourceInvalid, "/secret"},
|
||||
{"nonfinite", configkit.Mapping("bad", map[string]any{"count": math.NaN()}), configkit.CodeSourceInvalid, "/count"},
|
||||
{"cycle", configkit.Mapping("bad", cycle), configkit.CodeSourceInvalid, "/secret"},
|
||||
{"too many bytes", configkit.Mapping("bad", map[string]any{"name": strings.Repeat("x", 8<<20+1)}), configkit.CodeSourceLimitExceeded, "/name"},
|
||||
} {
|
||||
t.Run(test.name, func(t *testing.T) {
|
||||
loadError := checkMappingCode(t, test.source, configkit.Options[mappingFacadeConfig]{}, test.want)
|
||||
diagnostic := loadError.Diagnostics()[0]
|
||||
if diagnostic.Path.String() != test.path || diagnostic.Primary == nil || diagnostic.Primary.SourceID != "source-1" || strings.Contains(diagnostic.Message, "canary-value") {
|
||||
t.Fatalf("diagnostic = %#v", diagnostic)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func checkMappingCode(t *testing.T, source configkit.Source, options configkit.Options[mappingFacadeConfig], want configkit.Code) *configkit.LoadError {
|
||||
t.Helper()
|
||||
_, err := configkit.Load(source, options)
|
||||
loadError, ok := err.(*configkit.LoadError)
|
||||
if !ok || len(loadError.Diagnostics()) == 0 || loadError.Diagnostics()[0].Code != want {
|
||||
t.Fatalf("error = %#v, want %s", err, want)
|
||||
}
|
||||
return loadError
|
||||
}
|
||||
221
facade_resource_context_test.go
Normal file
221
facade_resource_context_test.go
Normal file
@@ -0,0 +1,221 @@
|
||||
package configkit_test
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"io/fs"
|
||||
"os"
|
||||
"path"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
"testing/fstest"
|
||||
|
||||
configkit "gitea.maximumdirect.net/eric/ConfigKit"
|
||||
)
|
||||
|
||||
type observedResourceFS struct {
|
||||
files fstest.MapFS
|
||||
opens int
|
||||
closed bool
|
||||
}
|
||||
|
||||
func (f *observedResourceFS) Open(name string) (fs.File, error) {
|
||||
f.opens++
|
||||
if f.closed {
|
||||
return nil, fs.ErrClosed
|
||||
}
|
||||
return f.files.Open(name)
|
||||
}
|
||||
|
||||
type resourceFacadeConfig struct {
|
||||
Native string `yaml:"native"`
|
||||
Shared string `yaml:"shared"`
|
||||
Host string `yaml:"host"`
|
||||
Selected string `yaml:"selected"`
|
||||
}
|
||||
|
||||
func TestFacadeSourceContextsFollowOriginsWithoutQueryIO(t *testing.T) {
|
||||
host := t.TempDir()
|
||||
if err := os.WriteFile(filepath.Join(host, "main.yml"), []byte("host: host.txt\n"), 0o600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := os.WriteFile(filepath.Join(host, "host.txt"), []byte("host-v1"), 0o600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
borrowed := &observedResourceFS{files: fstest.MapFS{
|
||||
"bundle/config/defaults.yml": &fstest.MapFile{Data: []byte("configkit: {imports: [{path: ../shared.yml}]}\nnative: icon.txt\n")},
|
||||
"bundle/shared.yml": &fstest.MapFile{Data: []byte("shared: shared.txt\n")},
|
||||
"bundle/overlays/test.yml": &fstest.MapFile{Data: []byte("selected: overlay.txt\n")},
|
||||
"bundle/config/icon.txt": &fstest.MapFile{Data: []byte("icon-v1")},
|
||||
"bundle/shared.txt": &fstest.MapFile{Data: []byte("shared-v1")},
|
||||
"bundle/overlays/overlay.txt": &fstest.MapFile{Data: []byte("overlay-v1")},
|
||||
}}
|
||||
namespace := configkit.NewFilesystem("bundle", borrowed)
|
||||
var resolverContext configkit.ResourceContext
|
||||
result, err := configkit.LoadLayered(configkit.Input{Source: configkit.File(filepath.Join(host, "main.yml"))}, configkit.LayeredOptions[resourceFacadeConfig]{
|
||||
Underlays: []configkit.Underlay{{Inputs: []configkit.Input{{
|
||||
Source: namespace.File("bundle/config/defaults.yml"), Imports: true,
|
||||
Filesystem: &configkit.FilesystemContext{Root: "bundle", FS: &namespace},
|
||||
}}}},
|
||||
Overlay: "test", OverlayFilesystem: &configkit.FilesystemContext{Root: "bundle", BaseDir: "bundle", FS: &namespace},
|
||||
Definition: configkit.Definition[resourceFacadeConfig]{Resolve: func(config *resourceFacadeConfig, resolution *configkit.Resolution) error {
|
||||
before := borrowed.opens
|
||||
var ok bool
|
||||
resolverContext, ok = resolution.SourceContext("source-2")
|
||||
if !ok || resolverContext.BaseDir != "config" || resolverContext.FS == nil {
|
||||
t.Errorf("resolver context = %#v, %v", resolverContext, ok)
|
||||
}
|
||||
if _, ok := resolution.SourceContext("missing"); ok || borrowed.opens != before {
|
||||
t.Error("resolution context query performed I/O or guessed an unknown source")
|
||||
}
|
||||
return nil
|
||||
}},
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("load: %v, diagnostics %#v", err, err.(*configkit.LoadError).Diagnostics())
|
||||
}
|
||||
queriesBefore := borrowed.opens
|
||||
for _, test := range []struct {
|
||||
field, wantBase, wantData string
|
||||
}{
|
||||
{"native", "config", "icon-v1"},
|
||||
{"shared", ".", "shared-v1"},
|
||||
{"selected", "overlays", "overlay-v1"},
|
||||
{"host", ".", "host-v1"},
|
||||
} {
|
||||
origin, ok := result.Metadata().Lookup(configkit.Keys(test.field))
|
||||
if !ok || len(origin.Origin.Locations) != 1 {
|
||||
t.Fatalf("%s origin = %#v, %v", test.field, origin, ok)
|
||||
}
|
||||
context, ok := result.SourceContext(origin.Origin.Locations[0].SourceID)
|
||||
if !ok || context.FS == nil || context.BaseDir != test.wantBase {
|
||||
t.Fatalf("%s context = %#v, %v", test.field, context, ok)
|
||||
}
|
||||
if borrowed.opens != queriesBefore {
|
||||
t.Fatal("result SourceContext query performed filesystem I/O")
|
||||
}
|
||||
resourceName := path.Join(context.BaseDir, map[string]string{
|
||||
"native": result.Config.Native, "shared": result.Config.Shared,
|
||||
"selected": result.Config.Selected, "host": result.Config.Host,
|
||||
}[test.field])
|
||||
data, err := fs.ReadFile(context.FS, resourceName)
|
||||
if err != nil || string(data) != test.wantData {
|
||||
t.Fatalf("%s resource = %q, %v", test.field, data, err)
|
||||
}
|
||||
queriesBefore = borrowed.opens
|
||||
}
|
||||
if _, ok := result.SourceContext(""); ok {
|
||||
t.Fatal("empty source ID returned a context")
|
||||
}
|
||||
var nilResult *configkit.Result[resourceFacadeConfig]
|
||||
if _, ok := nilResult.SourceContext("source-2"); ok {
|
||||
t.Fatal("nil result returned a context")
|
||||
}
|
||||
if _, err := fs.ReadFile(resolverContext.FS, "../shared.txt"); !errors.Is(err, fs.ErrInvalid) {
|
||||
t.Fatalf("native root escape = %v", err)
|
||||
}
|
||||
borrowed.files["bundle/config/icon.txt"].Data = []byte("icon-v2")
|
||||
data, err := fs.ReadFile(resolverContext.FS, "config/icon.txt")
|
||||
if err != nil || string(data) != "icon-v2" {
|
||||
t.Fatalf("updated native resource = %q, %v", data, err)
|
||||
}
|
||||
borrowed.closed = true
|
||||
if result.Config.Native != "icon.txt" {
|
||||
t.Fatal("closing caller filesystem altered loaded configuration")
|
||||
}
|
||||
if _, err := fs.ReadFile(resolverContext.FS, "config/icon.txt"); !errors.Is(err, fs.ErrClosed) {
|
||||
t.Fatalf("closed caller filesystem error = %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestFacadeHostContextSurvivesConfigurationDeletion(t *testing.T) {
|
||||
root := t.TempDir()
|
||||
main := filepath.Join(root, "main.yml")
|
||||
resource := filepath.Join(root, "asset.txt")
|
||||
if err := os.WriteFile(main, []byte("native: asset.txt\n"), 0o600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := os.WriteFile(resource, []byte("first"), 0o600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
result, err := configkit.Load[resourceFacadeConfig](configkit.File(main))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
context, ok := result.SourceContext("source-1")
|
||||
if !ok || context.BaseDir != "." {
|
||||
t.Fatalf("host context = %#v, %v", context, ok)
|
||||
}
|
||||
if err := os.Remove(main); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := os.WriteFile(resource, []byte("second"), 0o600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
data, err := fs.ReadFile(context.FS, path.Join(context.BaseDir, result.Config.Native))
|
||||
if err != nil || string(data) != "second" || result.Config.Native != "asset.txt" {
|
||||
t.Fatalf("resource after source deletion = %q, %v", data, err)
|
||||
}
|
||||
if _, err := fs.ReadFile(context.FS, "../asset.txt"); !errors.Is(err, fs.ErrInvalid) {
|
||||
t.Fatalf("invalid host path = %v", err)
|
||||
}
|
||||
if err := os.Remove(resource); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err := fs.ReadFile(context.FS, "asset.txt"); !errors.Is(err, fs.ErrNotExist) {
|
||||
t.Fatalf("missing later resource = %v", err)
|
||||
}
|
||||
outside := filepath.Join(t.TempDir(), "outside.txt")
|
||||
if err := os.WriteFile(outside, []byte("outside"), 0o600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := os.Symlink(outside, filepath.Join(root, "escape")); err == nil {
|
||||
if _, err := fs.ReadFile(context.FS, "escape"); !errors.Is(err, fs.ErrPermission) {
|
||||
t.Fatalf("host symlink escape = %v", err)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestFacadeSyntheticSourceContextsAndContextlessSources(t *testing.T) {
|
||||
namespace := configkit.NewFilesystem("bundle", fstest.MapFS{
|
||||
"bundle/config/icon.txt": &fstest.MapFile{Data: []byte("icon")},
|
||||
})
|
||||
context := &configkit.FilesystemContext{Root: "bundle", BaseDir: "bundle/config", FS: &namespace}
|
||||
for _, test := range []struct {
|
||||
name string
|
||||
source configkit.Source
|
||||
}{
|
||||
{"bytes", configkit.Bytes("memory", []byte("native: icon.txt\n"))},
|
||||
{"reader", configkit.Reader("stream", strings.NewReader("native: icon.txt\n"))},
|
||||
{"mapping", configkit.Mapping("values", map[string]any{"native": "icon.txt"})},
|
||||
} {
|
||||
t.Run(test.name, func(t *testing.T) {
|
||||
result, err := configkit.Load(test.source, configkit.Options[resourceFacadeConfig]{Filesystem: context})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
resource, ok := result.SourceContext("source-1")
|
||||
if !ok || resource.BaseDir != "config" {
|
||||
t.Fatalf("synthetic context = %#v, %v", resource, ok)
|
||||
}
|
||||
data, err := fs.ReadFile(resource.FS, path.Join(resource.BaseDir, result.Config.Native))
|
||||
if err != nil || string(data) != "icon" {
|
||||
t.Fatalf("synthetic resource = %q, %v", data, err)
|
||||
}
|
||||
})
|
||||
}
|
||||
result, err := configkit.Load[resourceFacadeConfig](configkit.Bytes("plain", []byte("native: none\n")))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, ok := result.SourceContext("source-1"); ok {
|
||||
t.Fatal("contextless byte source returned a context")
|
||||
}
|
||||
result, err = configkit.Load(configkit.NoSource(), configkit.Options[resourceFacadeConfig]{Overrides: []configkit.Override{{Path: configkit.Keys("native"), Value: "from-cli", Flag: "--native"}}})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, ok := result.SourceContext("source-1"); ok {
|
||||
t.Fatal("CLI source returned a context")
|
||||
}
|
||||
}
|
||||
107
facade_underlay_origins_test.go
Normal file
107
facade_underlay_origins_test.go
Normal file
@@ -0,0 +1,107 @@
|
||||
package configkit_test
|
||||
|
||||
import (
|
||||
"reflect"
|
||||
"testing"
|
||||
"testing/fstest"
|
||||
|
||||
configkit "gitea.maximumdirect.net/eric/ConfigKit"
|
||||
)
|
||||
|
||||
func TestUnderlaySequenceDescendantOrigins(t *testing.T) {
|
||||
for _, later := range []bool{false, true} {
|
||||
name := "first tier"
|
||||
if later {
|
||||
name = "later tier"
|
||||
}
|
||||
t.Run(name, func(t *testing.T) {
|
||||
bundle := configkit.NewFilesystem("bundle", fstest.MapFS{
|
||||
"main.yml": &fstest.MapFile{Data: []byte("configkit: {imports: [{path: items.yml}]}\n")},
|
||||
"items.yml": &fstest.MapFile{Data: []byte("items: [{name: first, nested: [false]}]\n")},
|
||||
})
|
||||
tiers := []configkit.Underlay{}
|
||||
if later {
|
||||
tiers = append(tiers, configkit.Underlay{Inputs: []configkit.Input{{Source: configkit.Mapping("lower", map[string]any{"other": true})}}})
|
||||
}
|
||||
tiers = append(tiers, configkit.Underlay{Inputs: []configkit.Input{{Source: bundle.File("main.yml"), Imports: true}}})
|
||||
paths := []string{"/items", "/items/0", "/items/0/name", "/items/0/nested", "/items/0/nested/0"}
|
||||
entry := map[string]configkit.Origin{}
|
||||
result, err := configkit.LoadLayered[map[string]any](configkit.Input{Source: configkit.Mapping("main", map[string]any{"main": true})}, configkit.LayeredOptions[map[string]any]{
|
||||
Underlays: tiers,
|
||||
Definition: configkit.Definition[map[string]any]{Resolve: func(_ *map[string]any, resolution *configkit.Resolution) error {
|
||||
for _, pointer := range paths {
|
||||
path, _ := configkit.ParsePath(pointer)
|
||||
info, ok := resolution.InputInfo(path)
|
||||
if !ok {
|
||||
t.Errorf("missing resolver input %s", pointer)
|
||||
}
|
||||
entry[pointer] = info.Origin
|
||||
}
|
||||
return nil
|
||||
}},
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var sourceID string
|
||||
for _, source := range result.Sources() {
|
||||
if source.Path == "items.yml" {
|
||||
sourceID = source.ID
|
||||
}
|
||||
}
|
||||
if sourceID == "" {
|
||||
t.Fatal("missing imported source")
|
||||
}
|
||||
for _, pointer := range paths {
|
||||
path, _ := configkit.ParsePath(pointer)
|
||||
info, ok := result.Metadata().Lookup(path)
|
||||
if !ok || info.Origin.Layer != configkit.LayerUnderlay || len(info.Origin.Locations) != 1 || info.Origin.Locations[0].SourceID != sourceID {
|
||||
t.Errorf("%s origin = %+v", pointer, info.Origin)
|
||||
}
|
||||
if !reflect.DeepEqual(entry[pointer], info.Origin) {
|
||||
t.Errorf("%s resolver origin differs", pointer)
|
||||
}
|
||||
explanation, ok := result.Explain(path)
|
||||
if !ok || !reflect.DeepEqual(explanation.Origin, info.Origin) {
|
||||
t.Errorf("%s explanation origin differs", pointer)
|
||||
}
|
||||
history := result.Metadata().History(path)
|
||||
if len(history) == 0 || !reflect.DeepEqual(history[len(history)-1].Origin, info.Origin) {
|
||||
t.Errorf("%s history = %+v", pointer, history)
|
||||
}
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestUnderlayHistoryRetainsLaterMappingContributors(t *testing.T) {
|
||||
result, err := configkit.LoadLayered[map[string]any](configkit.Input{Source: configkit.NoSource()}, configkit.LayeredOptions[map[string]any]{Underlays: []configkit.Underlay{
|
||||
{Inputs: []configkit.Input{{Source: configkit.Mapping("one", map[string]any{"x": 1})}}},
|
||||
{Inputs: []configkit.Input{{Source: configkit.Mapping("two", map[string]any{"x": map[string]any{}})}}},
|
||||
{Inputs: []configkit.Input{{Source: configkit.Mapping("three", map[string]any{"x": map[string]any{"y": 2}})}}},
|
||||
}})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
path := configkit.Keys("x")
|
||||
info, _ := result.Metadata().Lookup(path)
|
||||
wantOrigin := configkit.Origin{Layer: configkit.LayerUnderlay, Locations: []configkit.Location{{SourceID: "source-2"}, {SourceID: "source-3"}}}
|
||||
if !reflect.DeepEqual(info.Origin, wantOrigin) {
|
||||
t.Fatalf("effective origin = %+v, want %+v", info.Origin, wantOrigin)
|
||||
}
|
||||
history := result.Metadata().History(path)
|
||||
if len(history) != 3 || !reflect.DeepEqual(history[len(history)-1].Origin, info.Origin) {
|
||||
t.Fatalf("history = %+v, effective origin = %+v", history, info.Origin)
|
||||
}
|
||||
explanation, ok := result.Explain(path)
|
||||
if !ok || !reflect.DeepEqual(explanation.Origin, info.Origin) {
|
||||
t.Fatal("explanation and metadata disagree")
|
||||
}
|
||||
if len(explanation.History) != 3 {
|
||||
t.Fatalf("explanation history = %+v", explanation.History)
|
||||
}
|
||||
last := explanation.History[len(explanation.History)-1].Origin
|
||||
if last.Layer != configkit.LayerUnderlay || len(last.Locations) != 1 || last.Locations[0].SourceID != "source-3" {
|
||||
t.Fatalf("last explanation origin = %+v", last)
|
||||
}
|
||||
}
|
||||
@@ -39,7 +39,8 @@ type Explanation struct {
|
||||
}
|
||||
|
||||
// Explain returns safe effective and chronological values for a known final or
|
||||
// historical path. Unknown and malformed paths return false.
|
||||
// historical path. Structural inheritance appears between source applications
|
||||
// and typed reconciliation. Unknown and malformed paths return false.
|
||||
func (r *Result[T]) Explain(path Path) (Explanation, bool) {
|
||||
if r == nil {
|
||||
return Explanation{}, false
|
||||
|
||||
119
internal/engine/assembly_bench_test.go
Normal file
119
internal/engine/assembly_bench_test.go
Normal file
@@ -0,0 +1,119 @@
|
||||
package engine
|
||||
|
||||
import (
|
||||
"strconv"
|
||||
"testing"
|
||||
|
||||
"gitea.maximumdirect.net/eric/ConfigKit/internal/document"
|
||||
)
|
||||
|
||||
func BenchmarkComposeWideMappings(b *testing.B) {
|
||||
for _, width := range []int{128, 256, 512} {
|
||||
b.Run(strconv.Itoa(width), func(b *testing.B) {
|
||||
left := benchmarkCatalog("source-1", 0, width)
|
||||
right := benchmarkCatalog("source-2", width, width)
|
||||
b.ReportAllocs()
|
||||
b.ResetTimer()
|
||||
for range b.N {
|
||||
if _, diagnostics := composeDocuments(left, right); len(diagnostics) != 0 {
|
||||
b.Fatal(diagnostics)
|
||||
}
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func BenchmarkOverlayRecursiveMappings(b *testing.B) {
|
||||
for _, width := range []int{128, 256, 512} {
|
||||
b.Run(strconv.Itoa(width), func(b *testing.B) {
|
||||
base, diagnostics := composeDocuments(benchmarkCatalog("source-1", 0, width))
|
||||
if len(diagnostics) != 0 {
|
||||
b.Fatal(diagnostics)
|
||||
}
|
||||
overlay := benchmarkCatalog("source-2", width/2, width)
|
||||
b.ReportAllocs()
|
||||
b.ResetTimer()
|
||||
for range b.N {
|
||||
if _, diagnostics := overlayDocuments(base, overlay); len(diagnostics) != 0 {
|
||||
b.Fatal(diagnostics)
|
||||
}
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func BenchmarkOverlayManyApplications(b *testing.B) {
|
||||
for _, count := range []int{32, 64, 128} {
|
||||
inputs := make([]*document.Node, count)
|
||||
for index := range inputs {
|
||||
inputs[index] = benchmarkCatalog("source-2", index, 1)
|
||||
}
|
||||
benchmarkOverlayApplications(b, count, inputs)
|
||||
}
|
||||
}
|
||||
|
||||
func BenchmarkOverlayRepeatedReplacement(b *testing.B) {
|
||||
for _, count := range []int{32, 64, 128} {
|
||||
inputs := make([]*document.Node, count)
|
||||
for index := range inputs {
|
||||
location := document.Location{SourceID: "source-2"}
|
||||
value := &document.Node{Kind: document.String, Text: "replacement", Location: location}
|
||||
if index%2 == 0 {
|
||||
value = &document.Node{Kind: document.Mapping, Location: location, Mapping: []document.Entry{{
|
||||
Key: "enabled", Value: &document.Node{Kind: document.Boolean, Text: "false", Location: location},
|
||||
}}}
|
||||
}
|
||||
inputs[index] = &document.Node{Kind: document.Mapping, Location: location, Mapping: []document.Entry{{
|
||||
Key: "catalog", Value: &document.Node{Kind: document.Mapping, Location: location, Mapping: []document.Entry{{
|
||||
Key: "item-0", Value: value,
|
||||
}}},
|
||||
}}}
|
||||
}
|
||||
benchmarkOverlayApplications(b, count, inputs)
|
||||
}
|
||||
}
|
||||
|
||||
func benchmarkOverlayApplications(b *testing.B, count int, inputs []*document.Node) {
|
||||
base, diagnostics := composeDocuments(benchmarkCatalog("source-1", 0, 1))
|
||||
if len(diagnostics) != 0 {
|
||||
b.Fatal(diagnostics)
|
||||
}
|
||||
for _, mode := range []string{"owned", "copy-preserving"} {
|
||||
b.Run(mode+"/"+strconv.Itoa(count), func(b *testing.B) {
|
||||
b.ReportAllocs()
|
||||
b.ResetTimer()
|
||||
for range b.N {
|
||||
if mode == "owned" {
|
||||
assembly := newAssemblyAccumulator(cloneComposedDocument(base))
|
||||
for _, input := range inputs {
|
||||
if diagnostics := assembly.applyOverlay(input, nil, nil, LayerOverlay); len(diagnostics) != 0 {
|
||||
b.Fatal(diagnostics)
|
||||
}
|
||||
}
|
||||
continue
|
||||
}
|
||||
current := base
|
||||
for _, input := range inputs {
|
||||
current, diagnostics = overlayDocuments(current, input)
|
||||
if len(diagnostics) != 0 {
|
||||
b.Fatal(diagnostics)
|
||||
}
|
||||
}
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func benchmarkCatalog(source string, first, width int) *document.Node {
|
||||
location := document.Location{SourceID: source}
|
||||
entries := make([]document.Entry, width)
|
||||
for index := range entries {
|
||||
entries[index] = document.Entry{
|
||||
Key: "item-" + strconv.Itoa(first+index),
|
||||
Value: &document.Node{Kind: document.Mapping, Location: location, Mapping: []document.Entry{{Key: "enabled", Value: &document.Node{Kind: document.Boolean, Text: "true", Location: location}}}},
|
||||
}
|
||||
}
|
||||
return &document.Node{Kind: document.Mapping, Location: location, Mapping: []document.Entry{{
|
||||
Key: "catalog", Value: &document.Node{Kind: document.Mapping, Location: location, Mapping: entries},
|
||||
}}}
|
||||
}
|
||||
@@ -15,6 +15,17 @@ type composedDocument struct {
|
||||
inputNodes int
|
||||
}
|
||||
|
||||
// assemblyAccumulator owns one mutable working document. Acquired trees held by
|
||||
// history applications are separate from its root and remain unchanged.
|
||||
type assemblyAccumulator struct {
|
||||
current *composedDocument
|
||||
index mappingIndex
|
||||
}
|
||||
|
||||
func newAssemblyAccumulator(owned *composedDocument) *assemblyAccumulator {
|
||||
return &assemblyAccumulator{current: owned}
|
||||
}
|
||||
|
||||
// cloneComposedDocument detaches mutable composition and provenance state;
|
||||
// history application trees are immutable and retain their existing sharing.
|
||||
func cloneComposedDocument(base *composedDocument) *composedDocument {
|
||||
@@ -58,8 +69,14 @@ func composeDocuments(inputs ...*document.Node) (*composedDocument, []Diagnostic
|
||||
collected.append(input, Path{})
|
||||
}
|
||||
|
||||
checked := make(map[string]struct{}, len(collected.byPath))
|
||||
for _, claim := range collected.ordered {
|
||||
claims := collected.byPath[claim.path.String()]
|
||||
pointer := claim.path.String()
|
||||
if _, seen := checked[pointer]; seen {
|
||||
continue
|
||||
}
|
||||
checked[pointer] = struct{}{}
|
||||
claims := collected.byPath[pointer]
|
||||
if len(claims) < 2 || allMappingClaims(claims) {
|
||||
continue
|
||||
}
|
||||
@@ -67,21 +84,43 @@ func composeDocuments(inputs ...*document.Node) (*composedDocument, []Diagnostic
|
||||
}
|
||||
|
||||
root := document.Clone(inputs[0])
|
||||
index := new(mappingIndex)
|
||||
for _, input := range inputs[1:] {
|
||||
mergeDisjointMappings(root, input)
|
||||
mergeDisjointMappings(root, input, index)
|
||||
}
|
||||
applications := make([]historyApplication, 0, len(inputs))
|
||||
for _, input := range inputs {
|
||||
applications = append(applications, historyApplication{layer: LayerBase, value: input})
|
||||
}
|
||||
origins := compositionOrigins(collected.ordered)
|
||||
completeCompositionOrigins(root, Path{}, origins)
|
||||
return &composedDocument{
|
||||
root: root,
|
||||
claims: cloneCompositionClaims(collected.ordered),
|
||||
origins: compositionOrigins(collected.ordered),
|
||||
origins: origins,
|
||||
applications: applications,
|
||||
}, nil
|
||||
}
|
||||
|
||||
// Conflict claims treat sequences atomically, but inspection needs origins for
|
||||
// every descendant. Preserve composite mapping claims while filling those gaps.
|
||||
func completeCompositionOrigins(node *document.Node, path Path, origins map[string]Origin) {
|
||||
pointer := path.String()
|
||||
if _, present := origins[pointer]; !present {
|
||||
origins[pointer] = Origin{Layer: LayerBase, Locations: []Location{publicLocation(node.Location)}}
|
||||
}
|
||||
switch node.Kind {
|
||||
case document.Mapping:
|
||||
for _, entry := range node.Mapping {
|
||||
completeCompositionOrigins(entry.Value, path.Key(entry.Key), origins)
|
||||
}
|
||||
case document.Sequence:
|
||||
for index, child := range node.Sequence {
|
||||
completeCompositionOrigins(child, path.Index(index), origins)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func (c *compositionClaims) append(node *document.Node, path Path) {
|
||||
claim := compositionClaim{
|
||||
path: path.clone(),
|
||||
@@ -109,18 +148,18 @@ func allMappingClaims(claims []compositionClaim) bool {
|
||||
return true
|
||||
}
|
||||
|
||||
func mergeDisjointMappings(target, incoming *document.Node) {
|
||||
func mergeDisjointMappings(target, incoming *document.Node, index *mappingIndex) {
|
||||
for _, incomingEntry := range incoming.Mapping {
|
||||
index := documentEntryIndex(target.Mapping, incomingEntry.Key)
|
||||
if index < 0 {
|
||||
position := index.find(target, incomingEntry.Key)
|
||||
if position < 0 {
|
||||
cloned := incomingEntry
|
||||
cloned.Value = document.Clone(incomingEntry.Value)
|
||||
target.Mapping = append(target.Mapping, cloned)
|
||||
index.append(target, cloned)
|
||||
continue
|
||||
}
|
||||
// Claim validation guarantees that the only remaining overlap is a
|
||||
// mapping pair. Empty mappings naturally contribute no child entries.
|
||||
mergeDisjointMappings(target.Mapping[index].Value, incomingEntry.Value)
|
||||
mergeDisjointMappings(target.Mapping[position].Value, incomingEntry.Value, index)
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -116,6 +116,31 @@ func TestComposeDocumentsReportsAllClaimantsAtExactLocations(t *testing.T) {
|
||||
assertCompositionConflict(t, diagnostics, "/group", []string{"source-1", "source-2", "source-3"})
|
||||
}
|
||||
|
||||
func TestComposeDocumentsKeepsFirstConflictAndAppendedEntryOrder(t *testing.T) {
|
||||
first := parseCompositionDocument(t, "source-1", "catalog:\n first: 1\n shared: one\n later: one\n")
|
||||
second := parseCompositionDocument(t, "source-2", "catalog:\n second: 2\n shared: two\n later: two\n")
|
||||
third := parseCompositionDocument(t, "source-3", "catalog:\n third: 3\n shared: three\n")
|
||||
_, diagnostics := composeDocuments(first, second, third)
|
||||
assertCompositionConflict(t, diagnostics, "/catalog/shared", []string{"source-1", "source-2", "source-3"})
|
||||
|
||||
composed, diagnostics := composeDocuments(
|
||||
parseCompositionDocument(t, "source-1", "catalog:\n first: 1\n"),
|
||||
parseCompositionDocument(t, "source-2", "catalog:\n second: 2\n"),
|
||||
parseCompositionDocument(t, "source-3", "catalog:\n third: 3\n"),
|
||||
)
|
||||
if len(diagnostics) != 0 {
|
||||
t.Fatalf("composeDocuments() diagnostics = %#v", diagnostics)
|
||||
}
|
||||
entries := composed.root.Mapping[0].Value.Mapping
|
||||
got := make([]string, len(entries))
|
||||
for index, entry := range entries {
|
||||
got[index] = entry.Key
|
||||
}
|
||||
if want := []string{"first", "second", "third"}; !reflect.DeepEqual(got, want) {
|
||||
t.Fatalf("catalog keys = %#v, want %#v", got, want)
|
||||
}
|
||||
}
|
||||
|
||||
func TestComposeDocumentsTreatsEmptyMappingsAsChildlessContributions(t *testing.T) {
|
||||
empty, diagnostics := composeDocuments(
|
||||
parseCompositionDocument(t, "source-1", "{}\n"),
|
||||
|
||||
@@ -51,13 +51,14 @@ func (r *Inspection) Explain(path Path) (Explanation, bool) {
|
||||
After: valueSnapshot(working, path, r.secrets), Origin: Origin{Layer: LayerDefault},
|
||||
})
|
||||
}
|
||||
replayIndex := new(mappingIndex)
|
||||
for _, application := range r.applications {
|
||||
operationPath, relevant := applicationOperationPath(application, path)
|
||||
var before ValueSnapshot
|
||||
if relevant {
|
||||
before = valueSnapshot(working, path, r.secrets)
|
||||
}
|
||||
applyHistoryApplication(&working, application)
|
||||
applyHistoryApplication(&working, application, replayIndex)
|
||||
if !relevant {
|
||||
continue
|
||||
}
|
||||
@@ -71,6 +72,7 @@ func (r *Inspection) Explain(path Path) (Explanation, bool) {
|
||||
Origin: historyApplicationOrigin(application, operationPath),
|
||||
})
|
||||
}
|
||||
appendInheritanceExplanation(&explanation, &working, path, r.inheritance, r.secrets)
|
||||
|
||||
// Intermediate input replacements do not permanently suppress defaults:
|
||||
// decoding applies them to the final input tree. Show that reconciliation
|
||||
|
||||
@@ -23,10 +23,14 @@ func (r *Inspection) WriteExplanation(output io.Writer, path Path) error {
|
||||
return errExplanationUnknownPath
|
||||
}
|
||||
var rendered strings.Builder
|
||||
labels := make(map[string]string, len(r.sources))
|
||||
for _, source := range r.sources {
|
||||
labels[source.ID] = source.Label
|
||||
}
|
||||
rendered.WriteString("path: ")
|
||||
rendered.WriteString(strconv.Quote(explanation.Path.String()))
|
||||
rendered.WriteByte('\n')
|
||||
writeExplanationOrigin(&rendered, "effective source", explanation.Origin, r.sources)
|
||||
writeExplanationOrigin(&rendered, "effective source", explanation.Origin, labels)
|
||||
writeExplanationSnapshot(&rendered, "effective (canonical typed)", explanation.Effective, 0)
|
||||
if explanation.SecretReference != nil {
|
||||
rendered.WriteString("secret reference: ")
|
||||
@@ -38,12 +42,14 @@ func (r *Inspection) WriteExplanation(output io.Writer, path Path) error {
|
||||
rendered.WriteString(" - representation: ")
|
||||
if change.Origin.Operation == ChangeDecoding {
|
||||
rendered.WriteString("input to canonical typed\n")
|
||||
} else if change.Origin.Layer == LayerBase || change.Origin.Layer == LayerOverlay || change.Origin.Layer == LayerCLI {
|
||||
} else if change.Origin.Operation == ChangeInheritance {
|
||||
rendered.WriteString("structural inheritance\n")
|
||||
} else if change.Origin.Layer == LayerBase || change.Origin.Layer == LayerUnderlay || change.Origin.Layer == LayerOverlay || change.Origin.Layer == LayerCLI {
|
||||
rendered.WriteString("policy-validated input\n")
|
||||
} else {
|
||||
rendered.WriteString("canonical typed\n")
|
||||
}
|
||||
writeExplanationOrigin(&rendered, " source", change.Origin, r.sources)
|
||||
writeExplanationOrigin(&rendered, " source", change.Origin, labels)
|
||||
writeExplanationSnapshot(&rendered, " before", change.Before, 4)
|
||||
writeExplanationSnapshot(&rendered, " after", change.After, 4)
|
||||
for _, reason := range change.Reasons {
|
||||
@@ -65,7 +71,7 @@ func (r *Inspection) WriteExplanation(output io.Writer, path Path) error {
|
||||
return err
|
||||
}
|
||||
|
||||
func writeExplanationOrigin(output *strings.Builder, label string, origin Origin, sources []SourceInfo) {
|
||||
func writeExplanationOrigin(output *strings.Builder, label string, origin Origin, labels map[string]string) {
|
||||
output.WriteString(label)
|
||||
output.WriteString(": layer=")
|
||||
output.WriteString(strconv.Quote(string(origin.Layer)))
|
||||
@@ -77,7 +83,7 @@ func writeExplanationOrigin(output *strings.Builder, label string, origin Origin
|
||||
for _, location := range origin.Locations {
|
||||
output.WriteString(" location: source=")
|
||||
output.WriteString(strconv.Quote(location.SourceID))
|
||||
if label := sourceLabel(sources, location.SourceID); label != "" {
|
||||
if label := labels[location.SourceID]; label != "" {
|
||||
output.WriteString(" label=")
|
||||
output.WriteString(strconv.Quote(label))
|
||||
}
|
||||
@@ -87,15 +93,15 @@ func writeExplanationOrigin(output *strings.Builder, label string, origin Origin
|
||||
output.WriteString(strconv.Itoa(location.Column))
|
||||
output.WriteByte('\n')
|
||||
}
|
||||
}
|
||||
|
||||
func sourceLabel(sources []SourceInfo, sourceID string) string {
|
||||
for _, source := range sources {
|
||||
if source.ID == sourceID {
|
||||
return source.Label
|
||||
}
|
||||
for _, step := range origin.Inheritance {
|
||||
output.WriteString(" inherited: from=")
|
||||
output.WriteString(strconv.Quote(step.From.String()))
|
||||
output.WriteString(" to=")
|
||||
output.WriteString(strconv.Quote(step.To.String()))
|
||||
output.WriteString(" reference=")
|
||||
output.WriteString(strconv.Quote(step.Reference.String()))
|
||||
output.WriteByte('\n')
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
func writeExplanationSnapshot(output *strings.Builder, label string, snapshot ValueSnapshot, indent int) {
|
||||
|
||||
@@ -1,12 +1,21 @@
|
||||
package engine
|
||||
|
||||
import (
|
||||
"io/fs"
|
||||
"os"
|
||||
"path"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
)
|
||||
|
||||
func validateFilesystemContext(source Source, context *FilesystemContext, main parsedSource) *Diagnostic {
|
||||
if nativeContext(source, context) {
|
||||
access, basePath, mainPath, diagnostic := nativeFilesystemAccess(source, context)
|
||||
if diagnostic != nil {
|
||||
return diagnostic
|
||||
}
|
||||
return validateNativeDirectories(access, basePath, mainPath)
|
||||
}
|
||||
rootPath, _, _, diagnostic := importFilesystem(source, context, main)
|
||||
if diagnostic != nil {
|
||||
return diagnostic
|
||||
@@ -29,20 +38,87 @@ func filesystemOptionDiagnostics(source Source, context *FilesystemContext) []Di
|
||||
case sourceNone:
|
||||
return []Diagnostic{optionDiagnostic(CodeOptionsInvalid, "NoSource cannot use filesystem context")}
|
||||
case sourceFile:
|
||||
if context.FS != nil {
|
||||
return []Diagnostic{optionDiagnostic(CodeOptionsInvalid, "host file sources cannot use a native filesystem context")}
|
||||
}
|
||||
if context.BaseDir != "" {
|
||||
return []Diagnostic{optionDiagnostic(CodeOptionsInvalid, "file sources determine their own import base directory")}
|
||||
}
|
||||
case sourceBytes, sourceReader:
|
||||
case sourceFilesystem:
|
||||
if context.FS == nil || context.FS.namespace == nil || source.namespace == nil || context.FS.namespace.token != source.namespace.token {
|
||||
return []Diagnostic{optionDiagnostic(CodeOptionsInvalid, "filesystem file context must use the source namespace")}
|
||||
}
|
||||
if context.BaseDir != "" {
|
||||
return []Diagnostic{optionDiagnostic(CodeOptionsInvalid, "file sources determine their own import base directory")}
|
||||
}
|
||||
if context.Root != "" && !fs.ValidPath(context.Root) {
|
||||
return []Diagnostic{optionDiagnostic(CodeOptionsInvalid, "filesystem root must be a slash-relative valid path")}
|
||||
}
|
||||
case sourceBytes, sourceReader, sourceMapping:
|
||||
if context.BaseDir == "" || context.Root == "" {
|
||||
return []Diagnostic{optionDiagnostic(CodeOptionsInvalid, "byte and reader filesystem context requires base and root directories")}
|
||||
}
|
||||
if !filepath.IsAbs(context.BaseDir) || !filepath.IsAbs(context.Root) {
|
||||
if context.FS != nil {
|
||||
if context.FS.namespace == nil || !fs.ValidPath(context.BaseDir) || !fs.ValidPath(context.Root) {
|
||||
return []Diagnostic{optionDiagnostic(CodeOptionsInvalid, "filesystem base and root must be slash-relative valid paths")}
|
||||
}
|
||||
} else if !filepath.IsAbs(context.BaseDir) || !filepath.IsAbs(context.Root) {
|
||||
return []Diagnostic{optionDiagnostic(CodeOptionsInvalid, "byte and reader filesystem base and root directories must be absolute")}
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func nativeContext(source Source, context *FilesystemContext) bool {
|
||||
return source.kind == sourceFilesystem || context != nil && context.FS != nil
|
||||
}
|
||||
|
||||
// nativeFilesystemAccess resolves namespace context without opening caller
|
||||
// resources. File inputs always derive their base from the selected file.
|
||||
func nativeFilesystemAccess(source Source, context *FilesystemContext) (*filesystemAccess, string, string, *Diagnostic) {
|
||||
var namespace *nativeNamespace
|
||||
rootPath := "."
|
||||
basePath := "."
|
||||
mainPath := ""
|
||||
if source.kind == sourceFilesystem {
|
||||
namespace = source.namespace
|
||||
mainPath = source.path
|
||||
basePath = path.Dir(source.path)
|
||||
if context != nil && context.Root != "" {
|
||||
rootPath = context.Root
|
||||
}
|
||||
} else if context != nil && context.FS != nil {
|
||||
namespace = context.FS.namespace
|
||||
rootPath, basePath = context.Root, context.BaseDir
|
||||
}
|
||||
if namespace == nil || nilFilesystem(namespace.filesystem) || !fs.ValidPath(rootPath) || !fs.ValidPath(basePath) {
|
||||
value := optionDiagnostic(CodeOptionsInvalid, "filesystem context is invalid")
|
||||
return nil, "", "", &value
|
||||
}
|
||||
access := newNativeAccess(namespace.filesystem, rootPath, namespace.token)
|
||||
access.label = namespace.label
|
||||
if !access.contains(basePath) || mainPath != "" && !access.contains(mainPath) {
|
||||
value := importDiagnostic(CodeImportOutsideRoot, "selected source and import base must stay inside the configuration root", nil)
|
||||
return nil, "", "", &value
|
||||
}
|
||||
return access, basePath, mainPath, nil
|
||||
}
|
||||
|
||||
func validateNativeDirectories(access *filesystemAccess, basePath, mainPath string) *Diagnostic {
|
||||
if !access.contains(basePath) || mainPath != "" && !access.contains(mainPath) {
|
||||
value := importDiagnostic(CodeImportOutsideRoot, "selected source and import base must stay inside the configuration root", nil)
|
||||
return &value
|
||||
}
|
||||
for _, directory := range []string{access.rootPath, basePath} {
|
||||
info, err := access.stat(directory)
|
||||
if err != nil || !info.IsDir() {
|
||||
value := importDiagnostic(CodeImportInvalid, "configuration root and import base must be directories", nil)
|
||||
return &value
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func importFilesystem(source Source, context *FilesystemContext, main parsedSource) (rootPath, basePath, mainPath string, diagnostic *Diagnostic) {
|
||||
if main.source == nil || main.document == nil {
|
||||
value := importDiagnostic(CodeImportInvalid, "imports require a selected configuration source", nil)
|
||||
|
||||
185
internal/engine/filesystem_access.go
Normal file
185
internal/engine/filesystem_access.go
Normal file
@@ -0,0 +1,185 @@
|
||||
package engine
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"io"
|
||||
"io/fs"
|
||||
"os"
|
||||
"path"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// namespaceToken identifies one caller-selected filesystem without comparing
|
||||
// arbitrary fs.FS values. A token can be shared by copies of a source wrapper.
|
||||
type namespaceToken struct{ marker byte }
|
||||
|
||||
type nativeNamespace struct {
|
||||
filesystem fs.FS
|
||||
label string
|
||||
token *namespaceToken
|
||||
}
|
||||
|
||||
func newNativeNamespace(label string, filesystem fs.FS) *nativeNamespace {
|
||||
return &nativeNamespace{filesystem: filesystem, label: label, token: &namespaceToken{}}
|
||||
}
|
||||
|
||||
type resourceIdentity struct {
|
||||
namespace *namespaceToken
|
||||
path string
|
||||
occurrence *namespaceToken
|
||||
}
|
||||
|
||||
// filesystemAccess is a load-scoped view of either a contained host root or a
|
||||
// borrowed native filesystem. Names are canonical host paths or fs.FS paths.
|
||||
type filesystemAccess struct {
|
||||
rootPath string
|
||||
root *os.Root
|
||||
native fs.FS
|
||||
token *namespaceToken
|
||||
label string
|
||||
}
|
||||
|
||||
func openHostAccess(rootPath string) (*filesystemAccess, error) {
|
||||
root, err := os.OpenRoot(rootPath)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return &filesystemAccess{rootPath: rootPath, root: root, token: &namespaceToken{}}, nil
|
||||
}
|
||||
|
||||
func newNativeAccess(filesystem fs.FS, root string, token *namespaceToken) *filesystemAccess {
|
||||
return &filesystemAccess{rootPath: root, native: filesystem, token: token}
|
||||
}
|
||||
|
||||
func (a *filesystemAccess) close() error {
|
||||
if a.root != nil {
|
||||
return a.root.Close()
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (a *filesystemAccess) identity(name string) resourceIdentity {
|
||||
return resourceIdentity{namespace: a.token, path: name}
|
||||
}
|
||||
|
||||
func (a *filesystemAccess) syntheticIdentity() resourceIdentity {
|
||||
return resourceIdentity{namespace: a.token, occurrence: &namespaceToken{}}
|
||||
}
|
||||
|
||||
func (a *filesystemAccess) join(base, name string) string {
|
||||
if a.native != nil {
|
||||
return path.Join(base, name)
|
||||
}
|
||||
return filepath.Join(base, filepath.FromSlash(path.Clean(name)))
|
||||
}
|
||||
|
||||
func (a *filesystemAccess) dir(name string) string {
|
||||
if a.native != nil {
|
||||
return path.Dir(name)
|
||||
}
|
||||
return filepath.Dir(name)
|
||||
}
|
||||
|
||||
func (a *filesystemAccess) contains(name string) bool {
|
||||
if a.native == nil {
|
||||
return containedPath(a.rootPath, filepath.Clean(name))
|
||||
}
|
||||
if !fs.ValidPath(name) || !fs.ValidPath(a.rootPath) {
|
||||
return false
|
||||
}
|
||||
return a.rootPath == "." || name == a.rootPath || strings.HasPrefix(name, a.rootPath+"/")
|
||||
}
|
||||
|
||||
func (a *filesystemAccess) canonical(name string) (string, error) {
|
||||
if a.native == nil {
|
||||
return canonicalExistingPath(name)
|
||||
}
|
||||
if !fs.ValidPath(name) {
|
||||
return "", fs.ErrInvalid
|
||||
}
|
||||
_, err := fs.Stat(a.native, name)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
return name, nil
|
||||
}
|
||||
|
||||
func (a *filesystemAccess) relative(name string) (string, error) {
|
||||
if a.native != nil {
|
||||
if !a.contains(name) {
|
||||
return "", fs.ErrInvalid
|
||||
}
|
||||
return name, nil
|
||||
}
|
||||
relative, err := filepath.Rel(a.rootPath, name)
|
||||
if err != nil || filepath.IsAbs(relative) || relative == ".." || strings.HasPrefix(relative, ".."+string(filepath.Separator)) {
|
||||
return "", fs.ErrInvalid
|
||||
}
|
||||
return relative, nil
|
||||
}
|
||||
|
||||
func (a *filesystemAccess) stat(name string) (fs.FileInfo, error) {
|
||||
if a.native != nil {
|
||||
return fs.Stat(a.native, name)
|
||||
}
|
||||
relative, err := a.relative(name)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return a.root.Stat(relative)
|
||||
}
|
||||
|
||||
func (a *filesystemAccess) open(name string) (fs.File, error) {
|
||||
if a.native != nil {
|
||||
return a.native.Open(name)
|
||||
}
|
||||
relative, err := a.relative(name)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return a.root.Open(relative)
|
||||
}
|
||||
|
||||
func (a *filesystemAccess) readDirectory(name string, visit func(fs.DirEntry) bool) error {
|
||||
file, err := a.open(name)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if reader, ok := file.(fs.ReadDirFile); ok {
|
||||
for {
|
||||
entries, readErr := reader.ReadDir(globReadBatch)
|
||||
for _, entry := range entries {
|
||||
if !visit(entry) {
|
||||
return file.Close()
|
||||
}
|
||||
}
|
||||
if errors.Is(readErr, io.EOF) {
|
||||
return file.Close()
|
||||
}
|
||||
if readErr != nil {
|
||||
_ = file.Close()
|
||||
return readErr
|
||||
}
|
||||
if len(entries) == 0 {
|
||||
_ = file.Close()
|
||||
return io.ErrNoProgress
|
||||
}
|
||||
}
|
||||
}
|
||||
if err := file.Close(); err != nil {
|
||||
return err
|
||||
}
|
||||
// fs.ReadDir provides the standard fallback for implementations whose opened
|
||||
// files do not implement fs.ReadDirFile. It may allocate all entries itself.
|
||||
entries, err := fs.ReadDir(a.native, name)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
for _, entry := range entries {
|
||||
if !visit(entry) {
|
||||
break
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
@@ -2,11 +2,8 @@ package engine
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"io"
|
||||
"io/fs"
|
||||
"os"
|
||||
"path"
|
||||
"path/filepath"
|
||||
"slices"
|
||||
"strings"
|
||||
)
|
||||
@@ -42,29 +39,29 @@ func (s *importTraversal) expandImportGlob(basePath string, declaration importDe
|
||||
names = []string{literal}
|
||||
}
|
||||
for _, name := range names {
|
||||
absolute := filepath.Join(candidate.absolute, filepath.FromSlash(name))
|
||||
absolute := s.access.join(candidate.absolute, name)
|
||||
relative := path.Join(candidate.relative, name)
|
||||
if !containedPath(s.rootPath, filepath.Clean(absolute)) {
|
||||
if !s.access.contains(absolute) {
|
||||
return nil, []Diagnostic{importDiagnostic(CodeImportOutsideRoot, "glob traversal is outside the configuration root", &location)}
|
||||
}
|
||||
canonical, err := canonicalExistingPath(absolute)
|
||||
canonical, err := s.access.canonical(absolute)
|
||||
if err != nil {
|
||||
if errors.Is(err, fs.ErrNotExist) && !patterned {
|
||||
continue
|
||||
}
|
||||
return nil, []Diagnostic{globFilesystemDiagnostic(err, "glob match could not be canonicalized", location)}
|
||||
}
|
||||
if !containedPath(s.rootPath, canonical) {
|
||||
if !s.access.contains(canonical) {
|
||||
return nil, []Diagnostic{importDiagnostic(CodeImportOutsideRoot, "glob traversal resolves outside the configuration root", &location)}
|
||||
}
|
||||
if last {
|
||||
next = append(next, globCandidate{absolute: canonical, relative: relative})
|
||||
if len(s.sources)+len(next) > s.limits.sources {
|
||||
if s.accounting.sources+len(next) > s.limits.sources {
|
||||
return nil, []Diagnostic{importDiagnostic(CodeImportLimitExceeded, "glob matches exceed the remaining source-count limit", &location)}
|
||||
}
|
||||
continue
|
||||
}
|
||||
info, err := os.Stat(canonical)
|
||||
info, err := s.access.stat(canonical)
|
||||
if err != nil {
|
||||
return nil, []Diagnostic{globFilesystemDiagnostic(err, "glob traversal target could not be inspected", location)}
|
||||
}
|
||||
@@ -88,56 +85,47 @@ func (s *importTraversal) expandImportGlob(basePath string, declaration importDe
|
||||
}
|
||||
|
||||
func (s *importTraversal) readGlobMatches(directory, pattern string, location Location) ([]string, []Diagnostic) {
|
||||
canonical, err := canonicalExistingPath(directory)
|
||||
canonical, err := s.access.canonical(directory)
|
||||
if err != nil {
|
||||
if errors.Is(err, fs.ErrNotExist) {
|
||||
return nil, nil
|
||||
}
|
||||
return nil, []Diagnostic{globFilesystemDiagnostic(err, "glob directory could not be canonicalized", location)}
|
||||
}
|
||||
if !containedPath(s.rootPath, canonical) {
|
||||
if !s.access.contains(canonical) {
|
||||
return nil, []Diagnostic{importDiagnostic(CodeImportOutsideRoot, "glob directory resolves outside the configuration root", &location)}
|
||||
}
|
||||
relative, err := filepath.Rel(s.rootPath, canonical)
|
||||
if err != nil || filepath.IsAbs(relative) || relative == ".." || strings.HasPrefix(relative, ".."+string(filepath.Separator)) {
|
||||
if _, err := s.access.relative(canonical); err != nil {
|
||||
return nil, []Diagnostic{importDiagnostic(CodeImportOutsideRoot, "glob directory could not be addressed inside the configuration root", &location)}
|
||||
}
|
||||
directoryFile, err := s.root.Open(relative)
|
||||
if err != nil {
|
||||
if errors.Is(err, fs.ErrNotExist) {
|
||||
return nil, nil
|
||||
}
|
||||
return nil, []Diagnostic{globFilesystemDiagnostic(err, "glob directory could not be opened", location)}
|
||||
}
|
||||
|
||||
var matches []string
|
||||
for {
|
||||
entries, readErr := directoryFile.ReadDir(globReadBatch)
|
||||
for _, entry := range entries {
|
||||
s.usedEntries++
|
||||
if s.usedEntries > s.limits.entries {
|
||||
_ = directoryFile.Close()
|
||||
return nil, []Diagnostic{importDiagnostic(CodeImportLimitExceeded, "glob traversal exceeds the directory-entry visit limit", &location)}
|
||||
}
|
||||
matched, matchErr := path.Match(pattern, entry.Name())
|
||||
if matchErr != nil {
|
||||
_ = directoryFile.Close()
|
||||
return nil, []Diagnostic{importDiagnostic(CodeImportInvalid, "import glob pattern is invalid", &location)}
|
||||
}
|
||||
if matched {
|
||||
matches = append(matches, entry.Name())
|
||||
}
|
||||
var problem *Diagnostic
|
||||
err = s.access.readDirectory(canonical, func(entry fs.DirEntry) bool {
|
||||
s.accounting.entries++
|
||||
if s.accounting.entries > s.limits.entries {
|
||||
value := importDiagnostic(CodeImportLimitExceeded, "glob traversal exceeds the directory-entry visit limit", &location)
|
||||
problem = &value
|
||||
return false
|
||||
}
|
||||
if errors.Is(readErr, io.EOF) {
|
||||
break
|
||||
matched, matchErr := path.Match(pattern, entry.Name())
|
||||
if matchErr != nil {
|
||||
value := importDiagnostic(CodeImportInvalid, "import glob pattern is invalid", &location)
|
||||
problem = &value
|
||||
return false
|
||||
}
|
||||
if readErr != nil {
|
||||
_ = directoryFile.Close()
|
||||
return nil, []Diagnostic{globFilesystemDiagnostic(readErr, "glob directory could not be read", location)}
|
||||
if matched {
|
||||
matches = append(matches, entry.Name())
|
||||
}
|
||||
return true
|
||||
})
|
||||
if problem != nil {
|
||||
return nil, []Diagnostic{*problem}
|
||||
}
|
||||
if err := directoryFile.Close(); err != nil {
|
||||
return nil, []Diagnostic{globFilesystemDiagnostic(err, "glob directory could not be closed", location)}
|
||||
if errors.Is(err, fs.ErrNotExist) {
|
||||
return nil, nil
|
||||
}
|
||||
if err != nil {
|
||||
return nil, []Diagnostic{globFilesystemDiagnostic(err, "glob directory could not be traversed", location)}
|
||||
}
|
||||
slices.Sort(matches)
|
||||
return matches, nil
|
||||
|
||||
@@ -6,6 +6,40 @@ import (
|
||||
|
||||
const historyNodeLimit = 4_000_000
|
||||
|
||||
type retainedNodeCounter struct {
|
||||
seen map[*document.Node]bool
|
||||
count int
|
||||
limit int
|
||||
}
|
||||
|
||||
func newRetainedNodeCounter(limit int) *retainedNodeCounter {
|
||||
return &retainedNodeCounter{seen: make(map[*document.Node]bool), limit: limit}
|
||||
}
|
||||
|
||||
func (c *retainedNodeCounter) add(root *document.Node) bool {
|
||||
if c == nil {
|
||||
return true
|
||||
}
|
||||
stack := []*document.Node{root}
|
||||
for len(stack) != 0 {
|
||||
node := stack[len(stack)-1]
|
||||
stack = stack[:len(stack)-1]
|
||||
if node == nil || c.seen[node] {
|
||||
continue
|
||||
}
|
||||
c.seen[node] = true
|
||||
c.count++
|
||||
if c.limit >= 0 && c.count > c.limit {
|
||||
return false
|
||||
}
|
||||
for _, entry := range node.Mapping {
|
||||
stack = append(stack, entry.Value)
|
||||
}
|
||||
stack = append(stack, node.Sequence...)
|
||||
}
|
||||
return true
|
||||
}
|
||||
|
||||
// History applications borrow immutable policy-validated input trees. Metadata
|
||||
// records answer presence and origin queries; replay reconstructs value changes.
|
||||
// Replay reconciles with canonical decoding before domain resolution so both
|
||||
@@ -83,15 +117,15 @@ func pathsIntersect(left, right Path) bool {
|
||||
return pathContains(left, right) || pathContains(right, left)
|
||||
}
|
||||
|
||||
func applyHistoryApplication(root **document.Node, application historyApplication) {
|
||||
func applyHistoryApplication(root **document.Node, application historyApplication, index *mappingIndex) {
|
||||
if application.path.String() == "" {
|
||||
mergeHistoryDocument(root, application.value)
|
||||
mergeHistoryDocument(root, application.value, index)
|
||||
return
|
||||
}
|
||||
mergeHistoryAtPath(root, application.path, application.value)
|
||||
mergeHistoryAtPath(root, application.path, application.value, index)
|
||||
}
|
||||
|
||||
func mergeHistoryDocument(target **document.Node, incoming *document.Node) {
|
||||
func mergeHistoryDocument(target **document.Node, incoming *document.Node, index *mappingIndex) {
|
||||
if incoming == nil {
|
||||
return
|
||||
}
|
||||
@@ -100,39 +134,39 @@ func mergeHistoryDocument(target **document.Node, incoming *document.Node) {
|
||||
return
|
||||
}
|
||||
for _, entry := range incoming.Mapping {
|
||||
position := documentEntryIndex((*target).Mapping, entry.Key)
|
||||
position := index.find(*target, entry.Key)
|
||||
if position < 0 {
|
||||
cloned := entry
|
||||
cloned.Value = document.Clone(entry.Value)
|
||||
(*target).Mapping = append((*target).Mapping, cloned)
|
||||
index.append(*target, cloned)
|
||||
continue
|
||||
}
|
||||
mergeHistoryDocument(&(*target).Mapping[position].Value, entry.Value)
|
||||
mergeHistoryDocument(&(*target).Mapping[position].Value, entry.Value, index)
|
||||
}
|
||||
}
|
||||
|
||||
func mergeHistoryAtPath(root **document.Node, path Path, incoming *document.Node) {
|
||||
func mergeHistoryAtPath(root **document.Node, path Path, incoming *document.Node, index *mappingIndex) {
|
||||
if root == nil || *root == nil || len(path.segments) == 0 {
|
||||
mergeHistoryDocument(root, incoming)
|
||||
mergeHistoryDocument(root, incoming, index)
|
||||
return
|
||||
}
|
||||
current := *root
|
||||
for index, segment := range path.segments {
|
||||
for segmentIndex, segment := range path.segments {
|
||||
if current.Kind != document.Mapping || segment.kind == pathSegmentIndex {
|
||||
return
|
||||
}
|
||||
position := documentEntryIndex(current.Mapping, segment.key)
|
||||
if index == len(path.segments)-1 {
|
||||
position := index.find(current, segment.key)
|
||||
if segmentIndex == len(path.segments)-1 {
|
||||
if position < 0 {
|
||||
current.Mapping = append(current.Mapping, document.Entry{Key: segment.key, Value: document.Clone(incoming)})
|
||||
index.append(current, document.Entry{Key: segment.key, Value: document.Clone(incoming)})
|
||||
} else {
|
||||
mergeHistoryDocument(¤t.Mapping[position].Value, incoming)
|
||||
mergeHistoryDocument(¤t.Mapping[position].Value, incoming, index)
|
||||
}
|
||||
return
|
||||
}
|
||||
if position < 0 {
|
||||
child := &document.Node{Kind: document.Mapping}
|
||||
current.Mapping = append(current.Mapping, document.Entry{Key: segment.key, Value: child})
|
||||
index.append(current, document.Entry{Key: segment.key, Value: child})
|
||||
current = child
|
||||
} else {
|
||||
current = current.Mapping[position].Value
|
||||
@@ -230,11 +264,11 @@ func replaceHistoryPath(root **document.Node, path Path, replacement *document.N
|
||||
}
|
||||
}
|
||||
|
||||
func historyNodesWithinLimit(baseline, input, final *document.Node, applications []historyApplication, resolutions []resolutionSnapshot) bool {
|
||||
return historyNodesWithin(historyNodeLimit, baseline, input, final, applications, resolutions)
|
||||
func historyNodesWithinLimit(baseline, input, final *document.Node, applications []historyApplication, resolutions []resolutionSnapshot, inherited ...*inheritanceScope) bool {
|
||||
return historyNodesWithin(historyNodeLimit, baseline, input, final, applications, resolutions, inherited...)
|
||||
}
|
||||
|
||||
func historyNodesWithin(limit int, baseline, input, final *document.Node, applications []historyApplication, resolutions []resolutionSnapshot) bool {
|
||||
func historyNodesWithin(limit int, baseline, input, final *document.Node, applications []historyApplication, resolutions []resolutionSnapshot, inherited ...*inheritanceScope) bool {
|
||||
seen := make(map[*document.Node]bool)
|
||||
count := 0
|
||||
var visit func(*document.Node) bool
|
||||
@@ -272,5 +306,23 @@ func historyNodesWithin(limit int, baseline, input, final *document.Node, applic
|
||||
return false
|
||||
}
|
||||
}
|
||||
for _, expansion := range inherited {
|
||||
if expansion == nil {
|
||||
continue
|
||||
}
|
||||
if !visit(expansion.root) {
|
||||
return false
|
||||
}
|
||||
for _, operation := range expansion.operations {
|
||||
if !visit(operation.parent) || !visit(operation.effective) {
|
||||
return false
|
||||
}
|
||||
}
|
||||
for _, replacement := range expansion.replacements {
|
||||
if !visit(replacement.before) || !visit(replacement.after) {
|
||||
return false
|
||||
}
|
||||
}
|
||||
}
|
||||
return true
|
||||
}
|
||||
|
||||
@@ -4,9 +4,6 @@ import (
|
||||
"bytes"
|
||||
"errors"
|
||||
"io/fs"
|
||||
"os"
|
||||
"path"
|
||||
"path/filepath"
|
||||
"strconv"
|
||||
"strings"
|
||||
|
||||
@@ -40,47 +37,96 @@ type importRoute struct {
|
||||
locations []Location
|
||||
}
|
||||
|
||||
type importTraversal struct {
|
||||
rootPath string
|
||||
root *os.Root
|
||||
limits importLimits
|
||||
rule *VersionRule
|
||||
type importAccounting struct {
|
||||
sources int
|
||||
bytes int64
|
||||
nodes int
|
||||
entries int
|
||||
}
|
||||
|
||||
documents []*document.Node
|
||||
sources []SourceInfo
|
||||
active map[string]int
|
||||
visited map[string]importRoute
|
||||
stack []importRoute
|
||||
usedBytes int64
|
||||
usedNodes int
|
||||
usedEntries int
|
||||
type importTraversal struct {
|
||||
access *filesystemAccess
|
||||
accounting *importAccounting
|
||||
limits importLimits
|
||||
rule *VersionRule
|
||||
|
||||
documents []*document.Node
|
||||
sources []SourceInfo
|
||||
active map[resourceIdentity]int
|
||||
visited map[resourceIdentity]importRoute
|
||||
stack []importRoute
|
||||
}
|
||||
|
||||
func loadImportedDocuments(source Source, context *FilesystemContext, main parsedSource, rule *VersionRule, limits importLimits) (*composedDocument, []SourceInfo, []Diagnostic) {
|
||||
return loadImportedDocumentsTracked(source, context, main, rule, limits, &importAccounting{}, nil, nil)
|
||||
}
|
||||
|
||||
func loadImportedDocumentsTracked(source Source, context *FilesystemContext, main parsedSource, rule *VersionRule, limits importLimits, accounting *importAccounting, visited map[resourceIdentity]importRoute, hostToken *namespaceToken) (*composedDocument, []SourceInfo, []Diagnostic) {
|
||||
if nativeContext(source, context) {
|
||||
access, basePath, mainPath, diagnostic := nativeFilesystemAccess(source, context)
|
||||
if diagnostic != nil {
|
||||
return nil, sourceInfos(main.source), []Diagnostic{*diagnostic}
|
||||
}
|
||||
return loadImportedWithAccessTracked(access, basePath, mainPath, main, rule, limits, accounting, visited)
|
||||
}
|
||||
rootPath, basePath, mainPath, diagnostic := importFilesystem(source, context, main)
|
||||
if diagnostic != nil {
|
||||
return nil, sourceInfos(main.source), []Diagnostic{*diagnostic}
|
||||
}
|
||||
root, err := os.OpenRoot(rootPath)
|
||||
access, err := openHostAccess(rootPath)
|
||||
if err != nil {
|
||||
diagnostic := globFilesystemDiagnostic(err, "configuration root could not be opened", Location{})
|
||||
diagnostic.Primary = nil
|
||||
return nil, sourceInfos(main.source), []Diagnostic{diagnostic}
|
||||
}
|
||||
if hostToken != nil {
|
||||
access.token = hostToken
|
||||
}
|
||||
return loadImportedWithAccessTracked(access, basePath, mainPath, main, rule, limits, accounting, visited)
|
||||
}
|
||||
|
||||
func loadImportedWithAccess(access *filesystemAccess, basePath, mainPath string, main parsedSource, rule *VersionRule, limits importLimits, accounting *importAccounting) (*composedDocument, []SourceInfo, []Diagnostic) {
|
||||
return loadImportedWithAccessTracked(access, basePath, mainPath, main, rule, limits, accounting, nil)
|
||||
}
|
||||
|
||||
func loadImportedWithAccessTracked(access *filesystemAccess, basePath, mainPath string, main parsedSource, rule *VersionRule, limits importLimits, accounting *importAccounting, visited map[resourceIdentity]importRoute) (*composedDocument, []SourceInfo, []Diagnostic) {
|
||||
if !access.contains(basePath) || (mainPath != "" && !access.contains(mainPath)) {
|
||||
_ = access.close()
|
||||
return nil, sourceInfos(main.source), []Diagnostic{importDiagnostic(CodeImportOutsideRoot, "selected source and import base must stay inside the configuration root", nil)}
|
||||
}
|
||||
for _, directory := range []string{access.rootPath, basePath} {
|
||||
info, err := access.stat(directory)
|
||||
if err != nil || !info.IsDir() {
|
||||
_ = access.close()
|
||||
return nil, sourceInfos(main.source), []Diagnostic{importDiagnostic(CodeImportInvalid, "configuration root and import base must be directories", nil)}
|
||||
}
|
||||
}
|
||||
mainInfo := cloneSourceInfo(*main.source)
|
||||
if mainPath != "" {
|
||||
mainInfo.Path = mainPath
|
||||
}
|
||||
mainKey := access.identity(mainPath)
|
||||
if mainPath == "" {
|
||||
mainKey = access.syntheticIdentity()
|
||||
}
|
||||
if previous, repeated := visited[mainKey]; repeated {
|
||||
_ = access.close()
|
||||
return nil, sourceInfos(main.source), []Diagnostic{importRepeatedDiagnostic(Location{SourceID: main.source.ID}, previous, importRoute{})}
|
||||
}
|
||||
accounting.sources++
|
||||
accounting.bytes += main.bytes
|
||||
accounting.nodes += countInputNodes(main.document)
|
||||
state := importTraversal{
|
||||
rootPath: rootPath, root: root, limits: limits, rule: rule,
|
||||
access: access, accounting: accounting, limits: limits, rule: rule,
|
||||
documents: make([]*document.Node, 0, 1),
|
||||
sources: []SourceInfo{mainInfo},
|
||||
active: make(map[string]int), visited: make(map[string]importRoute),
|
||||
usedBytes: main.bytes, usedNodes: countInputNodes(main.document),
|
||||
active: make(map[resourceIdentity]int), visited: visited,
|
||||
}
|
||||
if state.visited == nil {
|
||||
state.visited = make(map[resourceIdentity]importRoute)
|
||||
}
|
||||
if diagnostic := state.checkInitialLimits(); diagnostic != nil {
|
||||
_ = root.Close()
|
||||
_ = access.close()
|
||||
return nil, cloneSourceInfos(state.sources), []Diagnostic{*diagnostic}
|
||||
}
|
||||
|
||||
@@ -89,8 +135,8 @@ func loadImportedDocuments(source Source, context *FilesystemContext, main parse
|
||||
mainIdentity = "<selected-source>"
|
||||
}
|
||||
mainRoute := importRoute{canonical: mainIdentity}
|
||||
state.active[mainIdentity] = 0
|
||||
state.visited[mainIdentity] = mainRoute
|
||||
state.active[mainKey] = 0
|
||||
state.visited[mainKey] = mainRoute
|
||||
state.stack = append(state.stack, mainRoute)
|
||||
body, declarations, diagnostics := splitImportDirectives(main.document)
|
||||
if len(diagnostics) == 0 {
|
||||
@@ -98,7 +144,7 @@ func loadImportedDocuments(source Source, context *FilesystemContext, main parse
|
||||
diagnostics = state.walkDeclarations(basePath, declarations, 0)
|
||||
}
|
||||
|
||||
closeErr := root.Close()
|
||||
closeErr := access.close()
|
||||
if len(diagnostics) != 0 {
|
||||
return nil, cloneSourceInfos(state.sources), diagnostics
|
||||
}
|
||||
@@ -107,22 +153,22 @@ func loadImportedDocuments(source Source, context *FilesystemContext, main parse
|
||||
}
|
||||
composed, diagnostics := composeDocuments(state.documents...)
|
||||
if composed != nil {
|
||||
composed.inputBytes = state.usedBytes
|
||||
composed.inputNodes = state.usedNodes
|
||||
composed.inputBytes = accounting.bytes
|
||||
composed.inputNodes = accounting.nodes
|
||||
}
|
||||
return composed, cloneSourceInfos(state.sources), diagnostics
|
||||
}
|
||||
|
||||
func (s *importTraversal) checkInitialLimits() *Diagnostic {
|
||||
if s.limits.sources < 1 {
|
||||
if s.accounting.sources > s.limits.sources {
|
||||
value := importDiagnostic(CodeImportLimitExceeded, "configuration exceeds the source-count limit", nil)
|
||||
return &value
|
||||
}
|
||||
if s.usedBytes > s.limits.bytes {
|
||||
if s.accounting.bytes > s.limits.bytes {
|
||||
value := importDiagnostic(CodeImportLimitExceeded, "configuration exceeds the aggregate byte limit", nil)
|
||||
return &value
|
||||
}
|
||||
if s.usedNodes > s.limits.nodes {
|
||||
if s.accounting.nodes > s.limits.nodes {
|
||||
value := importDiagnostic(CodeImportLimitExceeded, "configuration exceeds the aggregate node limit", nil)
|
||||
return &value
|
||||
}
|
||||
@@ -169,11 +215,11 @@ func (s *importTraversal) visitPath(basePath string, declaration importDeclarati
|
||||
return []Diagnostic{importDiagnostic(CodeImportInvalid, "import path must be a nonempty slash-relative path without a network scheme", &location)}
|
||||
}
|
||||
|
||||
candidate := filepath.Join(basePath, filepath.FromSlash(path.Clean(declaration.path)))
|
||||
if !containedPath(s.rootPath, filepath.Clean(candidate)) {
|
||||
candidate := s.access.join(basePath, declaration.path)
|
||||
if !s.access.contains(candidate) {
|
||||
return []Diagnostic{importDiagnostic(CodeImportOutsideRoot, "import target is outside the configuration root", &location)}
|
||||
}
|
||||
canonical, err := canonicalExistingPath(candidate)
|
||||
canonical, err := s.access.canonical(candidate)
|
||||
if err != nil {
|
||||
if declaration.optional && errors.Is(err, fs.ErrNotExist) {
|
||||
return nil
|
||||
@@ -187,30 +233,39 @@ func (s *importTraversal) visitPath(basePath string, declaration importDeclarati
|
||||
}
|
||||
return []Diagnostic{importDiagnostic(code, message, &location)}
|
||||
}
|
||||
if !containedPath(s.rootPath, canonical) {
|
||||
if !s.access.contains(canonical) {
|
||||
return []Diagnostic{importDiagnostic(CodeImportOutsideRoot, "import target resolves outside the configuration root", &location)}
|
||||
}
|
||||
|
||||
currentRoute := s.currentRoute(location)
|
||||
if activeIndex, ok := s.active[canonical]; ok {
|
||||
identity := s.access.identity(canonical)
|
||||
if activeIndex, ok := s.active[identity]; ok {
|
||||
cycle := append([]importRoute(nil), s.stack[activeIndex:]...)
|
||||
return []Diagnostic{importCycleDiagnostic(canonical, location, cycle)}
|
||||
}
|
||||
if previous, ok := s.visited[canonical]; ok {
|
||||
if previous, ok := s.visited[identity]; ok {
|
||||
return []Diagnostic{importRepeatedDiagnostic(location, previous, currentRoute)}
|
||||
}
|
||||
if len(s.sources)+1 > s.limits.sources {
|
||||
if s.accounting.sources+1 > s.limits.sources {
|
||||
return []Diagnostic{importDiagnostic(CodeImportLimitExceeded, "configuration exceeds the source-count limit", &location)}
|
||||
}
|
||||
|
||||
relative, err := filepath.Rel(s.rootPath, canonical)
|
||||
if err != nil || filepath.IsAbs(relative) || relative == ".." || strings.HasPrefix(relative, ".."+string(filepath.Separator)) {
|
||||
if _, err := s.access.relative(canonical); err != nil {
|
||||
return []Diagnostic{importDiagnostic(CodeImportOutsideRoot, "import target could not be addressed inside the configuration root", &location)}
|
||||
}
|
||||
sourceID := "source-" + strconv.Itoa(len(s.sources)+1)
|
||||
info := SourceInfo{ID: sourceID, Label: declaration.path, Path: canonical, Layer: LayerBase, ImportedFrom: &location}
|
||||
sourceID := "source-" + strconv.Itoa(s.accounting.sources+1)
|
||||
kind := SourceKindFile
|
||||
label := declaration.path
|
||||
if s.access.native != nil {
|
||||
kind = SourceKindFilesystem
|
||||
label = s.access.label + "/" + canonical
|
||||
}
|
||||
info := SourceInfo{ID: sourceID, Label: label, Path: canonical, Layer: LayerBase, Kind: kind, ImportedFrom: &location}
|
||||
if kind == SourceKindFilesystem {
|
||||
info.namespace = s.access.token
|
||||
}
|
||||
s.sources = append(s.sources, info)
|
||||
data, skipped, diagnostic := s.readImport(relative, sourceID, declaration.optional)
|
||||
data, skipped, diagnostic := s.readImport(canonical, sourceID, declaration.optional)
|
||||
if diagnostic != nil {
|
||||
return []Diagnostic{*diagnostic}
|
||||
}
|
||||
@@ -218,6 +273,7 @@ func (s *importTraversal) visitPath(basePath string, declaration importDeclarati
|
||||
s.sources = s.sources[:len(s.sources)-1]
|
||||
return nil
|
||||
}
|
||||
s.accounting.sources++
|
||||
|
||||
root, issues := yamladapter.Parse(bytes.NewReader(data), sourceID, yamladapter.Limits{
|
||||
Depth: maxDocumentDepth, Nodes: maxDocumentNodes,
|
||||
@@ -226,10 +282,10 @@ func (s *importTraversal) visitPath(basePath string, declaration importDeclarati
|
||||
return parserDiagnostics(sourceID, issues)
|
||||
}
|
||||
nodeCount := countInputNodes(root)
|
||||
if s.usedNodes+nodeCount > s.limits.nodes {
|
||||
if s.accounting.nodes+nodeCount > s.limits.nodes {
|
||||
return []Diagnostic{importDiagnostic(CodeImportLimitExceeded, "configuration exceeds the aggregate node limit", &location)}
|
||||
}
|
||||
s.usedNodes += nodeCount
|
||||
s.accounting.nodes += nodeCount
|
||||
|
||||
body, declarations, diagnostics := splitImportDirectives(root)
|
||||
if len(diagnostics) != 0 {
|
||||
@@ -245,22 +301,22 @@ func (s *importTraversal) visitPath(basePath string, declaration importDeclarati
|
||||
|
||||
route := importRoute{canonical: canonical, locations: currentRoute.locations}
|
||||
s.documents = append(s.documents, body)
|
||||
s.active[canonical] = len(s.stack)
|
||||
s.visited[canonical] = route
|
||||
s.active[identity] = len(s.stack)
|
||||
s.visited[identity] = route
|
||||
s.stack = append(s.stack, route)
|
||||
diagnostics = s.walkDeclarations(filepath.Dir(canonical), declarations, depth)
|
||||
delete(s.active, canonical)
|
||||
diagnostics = s.walkDeclarations(s.access.dir(canonical), declarations, depth)
|
||||
delete(s.active, identity)
|
||||
s.stack = s.stack[:len(s.stack)-1]
|
||||
return diagnostics
|
||||
}
|
||||
|
||||
func (s *importTraversal) readImport(relative, sourceID string, optional bool) ([]byte, bool, *Diagnostic) {
|
||||
remaining := s.limits.bytes - s.usedBytes
|
||||
func (s *importTraversal) readImport(name, sourceID string, optional bool) ([]byte, bool, *Diagnostic) {
|
||||
remaining := s.limits.bytes - s.accounting.bytes
|
||||
limit := min(int64(maxSourceBytes), remaining)
|
||||
if limit < 0 {
|
||||
limit = 0
|
||||
}
|
||||
data, failure := readRegularFile(relative, s.root.Stat, s.root.Open, limit)
|
||||
data, failure := readRegularFile(name, s.access.stat, s.access.open, limit)
|
||||
if failure != nil {
|
||||
if optional && failure.missingTarget {
|
||||
return nil, true, nil
|
||||
@@ -268,7 +324,7 @@ func (s *importTraversal) readImport(relative, sourceID string, optional bool) (
|
||||
value := sourceReadDiagnostic(failure, sourceID, CodeImportLimitExceeded, "configuration exceeds a source or aggregate byte limit")
|
||||
return nil, false, &value
|
||||
}
|
||||
s.usedBytes += int64(len(data))
|
||||
s.accounting.bytes += int64(len(data))
|
||||
return data, false, nil
|
||||
}
|
||||
|
||||
|
||||
313
internal/engine/inheritance.go
Normal file
313
internal/engine/inheritance.go
Normal file
@@ -0,0 +1,313 @@
|
||||
package engine
|
||||
|
||||
import (
|
||||
"slices"
|
||||
|
||||
"gitea.maximumdirect.net/eric/ConfigKit/internal/document"
|
||||
)
|
||||
|
||||
const inheritanceKey = "extends"
|
||||
|
||||
// These private records keep structural evidence separate from acquired source
|
||||
// records. Later inspection can project a transfer onto individual descendants.
|
||||
type inheritanceTransfer struct {
|
||||
from Path
|
||||
to Path
|
||||
reference Path
|
||||
locations []Location
|
||||
}
|
||||
|
||||
type inheritanceControl struct {
|
||||
path Path
|
||||
locations []Location
|
||||
cancelled bool
|
||||
}
|
||||
|
||||
type inheritanceScope struct {
|
||||
root *document.Node
|
||||
origins map[string]Origin
|
||||
transfers []inheritanceTransfer
|
||||
controls []inheritanceControl
|
||||
replacements []inheritanceReplacement
|
||||
operations []inheritanceOperation
|
||||
}
|
||||
|
||||
type inheritanceOperation struct {
|
||||
control *inheritanceControl
|
||||
transfer *inheritanceTransfer
|
||||
controlOrigin Origin
|
||||
parent *document.Node
|
||||
effective *document.Node
|
||||
parentOrigins map[string]Origin
|
||||
effectiveOrigins map[string]Origin
|
||||
}
|
||||
|
||||
// A replacement retains the displaced source/derivation record while the
|
||||
// effective origin belongs to the child. Value trees are borrowed immutable
|
||||
// inputs or detached templates and are budgeted with retained history later.
|
||||
type inheritanceReplacement struct {
|
||||
path Path
|
||||
before *document.Node
|
||||
after *document.Node
|
||||
previous []displacedDefinition
|
||||
effective Origin
|
||||
}
|
||||
|
||||
// A link represents one shared relationship edge. Each contributor retains
|
||||
// its original semantic path independently of source line/column coordinates.
|
||||
type inheritanceTrace struct {
|
||||
previous *inheritanceTrace
|
||||
transfer *inheritanceTransfer
|
||||
}
|
||||
|
||||
type inheritanceContributor struct {
|
||||
source Path
|
||||
location Location
|
||||
trace *inheritanceTrace
|
||||
}
|
||||
|
||||
type siblingFrame struct {
|
||||
name string
|
||||
parent string
|
||||
hasParent bool
|
||||
entered bool
|
||||
}
|
||||
|
||||
// expandSiblingDefinitions resolves only the mappings directly in scope.
|
||||
// Nested instructions remain in the returned templates for destination-local
|
||||
// resolution. This private entry point does not participate in public loading.
|
||||
func expandSiblingDefinitions(scope *document.Node, scopePath Path) (*inheritanceScope, []Diagnostic) {
|
||||
return expandSiblingDefinitionsWithLimits(scope, scopePath, defaultProjectionLimits())
|
||||
}
|
||||
|
||||
func expandSiblingDefinitionsWithLimits(scope *document.Node, scopePath Path, limits projectionLimits) (*inheritanceScope, []Diagnostic) {
|
||||
return expandSiblingDefinitionsWithOrigins(scope, scopePath, limits, nil)
|
||||
}
|
||||
|
||||
func expandSiblingDefinitionsWithOrigins(scope *document.Node, scopePath Path, limits projectionLimits, inputOrigins map[string]Origin, budget ...*retainedNodeCounter) (*inheritanceScope, []Diagnostic) {
|
||||
if scope == nil || scope.Kind != document.Mapping {
|
||||
return nil, []Diagnostic{{
|
||||
Code: CodeInheritanceInvalid, Severity: SeverityError, Path: scopePath,
|
||||
Message: "inheritance scope must be a mapping",
|
||||
}}
|
||||
}
|
||||
|
||||
byName := make(map[string]document.Entry, len(scope.Mapping))
|
||||
names := make([]string, 0, len(scope.Mapping))
|
||||
for _, entry := range scope.Mapping {
|
||||
byName[entry.Key] = entry
|
||||
names = append(names, entry.Key)
|
||||
}
|
||||
slices.Sort(names)
|
||||
|
||||
resolved := make(map[string]*document.Node, len(byName))
|
||||
resolvedOrigins := make(map[string]map[string]Origin, len(byName))
|
||||
state := make(map[string]uint8, len(byName))
|
||||
result := &inheritanceScope{}
|
||||
counter := newRetainedNodeCounter(historyNodeLimit)
|
||||
if len(budget) != 0 {
|
||||
counter = budget[0]
|
||||
}
|
||||
retained := 1 // scope mapping node
|
||||
for _, entry := range scope.Mapping {
|
||||
if entry.Value.Kind == document.Mapping {
|
||||
continue
|
||||
}
|
||||
remaining := limits.nodes - retained - 1
|
||||
if limits.nodes >= 0 && remaining < 0 {
|
||||
return nil, []Diagnostic{inheritanceBudgetDiagnostic(scopePath.Key(entry.Key), entry.Value.Location)}
|
||||
}
|
||||
count, within := inheritanceLowerBoundCount(entry.Value, remaining)
|
||||
if !within {
|
||||
return nil, []Diagnostic{inheritanceBudgetDiagnostic(scopePath.Key(entry.Key), entry.Value.Location)}
|
||||
}
|
||||
retained += 1 + count // key and value subtree
|
||||
}
|
||||
for _, name := range names {
|
||||
if byName[name].Value.Kind != document.Mapping || state[name] == 2 {
|
||||
continue
|
||||
}
|
||||
stack := []siblingFrame{{name: name}}
|
||||
for len(stack) != 0 {
|
||||
frame := &stack[len(stack)-1]
|
||||
entry := byName[frame.name]
|
||||
path := scopePath.Key(frame.name)
|
||||
if !frame.entered {
|
||||
state[frame.name] = 1
|
||||
frame.entered = true
|
||||
reference, found := inheritanceDirective(entry.Value)
|
||||
if found {
|
||||
referencePath := path.Key(inheritanceKey)
|
||||
location := publicLocation(reference.Value.Location)
|
||||
result.controls = append(result.controls, inheritanceControl{
|
||||
path: referencePath, locations: []Location{location},
|
||||
cancelled: reference.Value.Kind == document.Null,
|
||||
})
|
||||
switch reference.Value.Kind {
|
||||
case document.Null:
|
||||
case document.String:
|
||||
frame.parent, frame.hasParent = reference.Value.Text, true
|
||||
parent, exists := byName[frame.parent]
|
||||
if !exists {
|
||||
return nil, []Diagnostic{inheritanceDiagnostic(CodeInheritanceMissing, referencePath, location, nil,
|
||||
"referenced sibling does not exist")}
|
||||
}
|
||||
if parent.Value.Kind != document.Mapping {
|
||||
related := []Location{publicLocation(parent.Value.Location)}
|
||||
return nil, []Diagnostic{inheritanceDiagnostic(CodeInheritanceInvalid, referencePath, location, related,
|
||||
"referenced sibling is not a mapping")}
|
||||
}
|
||||
if state[frame.parent] == 1 {
|
||||
related := []Location{publicLocation(parent.Value.Location)}
|
||||
return nil, []Diagnostic{inheritanceDiagnostic(CodeInheritanceCycle, referencePath, location, related,
|
||||
"sibling inheritance contains a cycle")}
|
||||
}
|
||||
if state[frame.parent] == 0 {
|
||||
stack = append(stack, siblingFrame{name: frame.parent})
|
||||
continue
|
||||
}
|
||||
default:
|
||||
return nil, []Diagnostic{inheritanceDiagnostic(CodeInheritanceInvalid, referencePath, location, nil,
|
||||
"inheritance reference must be a sibling name or null")}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
payload := withoutInheritanceDirective(entry.Value)
|
||||
operationStart, replacementStart := len(result.operations), len(result.replacements)
|
||||
childOrigins := directInheritanceOrigins(payload, path, inputOrigins)
|
||||
if reference, found := inheritanceDirective(entry.Value); found {
|
||||
control := inheritanceControl{
|
||||
path: path.Key(inheritanceKey),
|
||||
locations: []Location{publicLocation(reference.Value.Location)},
|
||||
cancelled: reference.Value.Kind == document.Null,
|
||||
}
|
||||
controlOrigins := directInheritanceOrigins(reference.Value, control.path, inputOrigins)
|
||||
result.operations = append(result.operations, inheritanceOperation{
|
||||
control: &control, controlOrigin: controlOrigins[control.path.String()],
|
||||
})
|
||||
}
|
||||
if limits.nodes >= 0 {
|
||||
remaining := limits.nodes - retained - 1 // the sibling key
|
||||
within := false
|
||||
if frame.hasParent {
|
||||
within = inheritanceOverlayLowerBoundWithin(resolved[frame.parent], payload, remaining)
|
||||
} else {
|
||||
within = inheritanceLowerBoundWithin(payload, remaining)
|
||||
}
|
||||
if !within {
|
||||
return nil, []Diagnostic{inheritanceBudgetDiagnostic(path, entry.Value.Location)}
|
||||
}
|
||||
}
|
||||
if frame.hasParent {
|
||||
parent := resolved[frame.parent]
|
||||
merged := document.Clone(parent)
|
||||
control := inheritanceDirectiveLocation(entry.Value)
|
||||
transfer := inheritanceTransfer{
|
||||
from: scopePath.Key(frame.parent), to: path,
|
||||
reference: path.Key(inheritanceKey), locations: []Location{control},
|
||||
}
|
||||
mergedOrigins := transferredInheritanceOrigins(resolvedOrigins[frame.parent], transfer.from, path, transfer)
|
||||
mergeInheritanceOrigin(mergedOrigins, path, childOrigins[path.String()])
|
||||
mergeRecursiveMappings(merged, payload, path, new(mappingIndex), recursiveMergeHooks{
|
||||
added: func(childPath Path, incoming *document.Node) {
|
||||
copyInheritanceSubtree(mergedOrigins, childOrigins, incoming, childPath)
|
||||
},
|
||||
combined: func(childPath Path, _ *document.Node) {
|
||||
mergeInheritanceOrigin(mergedOrigins, childPath, childOrigins[childPath.String()])
|
||||
},
|
||||
replaced: func(childPath Path, existing, incoming *document.Node) {
|
||||
previous := displacedOrigins(existing, mergedOrigins, childPath)
|
||||
result.replacements = append(result.replacements, inheritanceReplacement{
|
||||
path: childPath.clone(), before: existing, after: incoming,
|
||||
previous: previous, effective: cloneOrigin(childOrigins[childPath.String()]),
|
||||
})
|
||||
deleteOriginSubtree(mergedOrigins, childPath, existing)
|
||||
copyInheritanceSubtree(mergedOrigins, childOrigins, incoming, childPath)
|
||||
},
|
||||
})
|
||||
merged.Location = entry.Value.Location
|
||||
resolved[frame.name] = merged
|
||||
resolvedOrigins[frame.name] = mergedOrigins
|
||||
result.transfers = append(result.transfers, transfer)
|
||||
result.operations = append(result.operations, inheritanceOperation{
|
||||
transfer: &transfer, parent: parent, effective: merged,
|
||||
parentOrigins: resolvedOrigins[frame.parent], effectiveOrigins: mergedOrigins,
|
||||
})
|
||||
} else {
|
||||
resolved[frame.name] = document.Clone(payload)
|
||||
resolvedOrigins[frame.name] = childOrigins
|
||||
}
|
||||
count, _ := inheritanceLowerBoundCount(resolved[frame.name], limits.nodes)
|
||||
retained += 1 + count
|
||||
for _, operation := range result.operations[operationStart:] {
|
||||
if !counter.add(operation.parent) || !counter.add(operation.effective) {
|
||||
return nil, []Diagnostic{inheritanceHistoryBudgetDiagnostic(path, entry.Value.Location)}
|
||||
}
|
||||
}
|
||||
for _, replacement := range result.replacements[replacementStart:] {
|
||||
if !counter.add(replacement.before) || !counter.add(replacement.after) {
|
||||
return nil, []Diagnostic{inheritanceHistoryBudgetDiagnostic(path, entry.Value.Location)}
|
||||
}
|
||||
}
|
||||
state[frame.name] = 2
|
||||
stack = stack[:len(stack)-1]
|
||||
}
|
||||
}
|
||||
|
||||
root := *scope
|
||||
root.Mapping = make([]document.Entry, len(scope.Mapping))
|
||||
rootOrigin := directInheritanceOrigins(&document.Node{Kind: document.Mapping, Location: scope.Location}, scopePath, inputOrigins)
|
||||
result.origins = rootOrigin
|
||||
for index, entry := range scope.Mapping {
|
||||
root.Mapping[index] = entry
|
||||
if resolved[entry.Key] != nil {
|
||||
root.Mapping[index].Value = resolved[entry.Key]
|
||||
for pointer, origin := range resolvedOrigins[entry.Key] {
|
||||
result.origins[pointer] = origin
|
||||
}
|
||||
} else {
|
||||
root.Mapping[index].Value = document.Clone(entry.Value)
|
||||
for pointer, origin := range directInheritanceOrigins(entry.Value, scopePath.Key(entry.Key), inputOrigins) {
|
||||
result.origins[pointer] = origin
|
||||
}
|
||||
}
|
||||
}
|
||||
result.root = &root
|
||||
if !inheritanceLowerBoundWithin(result.root, limits.nodes) {
|
||||
return nil, []Diagnostic{inheritanceBudgetDiagnostic(scopePath, scope.Location)}
|
||||
}
|
||||
return result, nil
|
||||
}
|
||||
|
||||
func inheritanceDirective(node *document.Node) (document.Entry, bool) {
|
||||
for _, entry := range node.Mapping {
|
||||
if entry.Key == inheritanceKey {
|
||||
return entry, true
|
||||
}
|
||||
}
|
||||
return document.Entry{}, false
|
||||
}
|
||||
|
||||
func inheritanceDirectiveLocation(node *document.Node) Location {
|
||||
entry, _ := inheritanceDirective(node)
|
||||
return publicLocation(entry.Value.Location)
|
||||
}
|
||||
|
||||
func withoutInheritanceDirective(node *document.Node) *document.Node {
|
||||
payload := *node
|
||||
payload.Mapping = make([]document.Entry, 0, len(node.Mapping))
|
||||
for _, entry := range node.Mapping {
|
||||
if entry.Key != inheritanceKey {
|
||||
payload.Mapping = append(payload.Mapping, entry)
|
||||
}
|
||||
}
|
||||
return &payload
|
||||
}
|
||||
|
||||
func inheritanceDiagnostic(code Code, path Path, primary Location, related []Location, message string) Diagnostic {
|
||||
return Diagnostic{
|
||||
Code: code, Severity: SeverityError, Path: path, Message: message,
|
||||
Primary: &primary, Related: related,
|
||||
}
|
||||
}
|
||||
110
internal/engine/inheritance_bench_test.go
Normal file
110
internal/engine/inheritance_bench_test.go
Normal file
@@ -0,0 +1,110 @@
|
||||
package engine
|
||||
|
||||
import (
|
||||
"strconv"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"gitea.maximumdirect.net/eric/ConfigKit/internal/document"
|
||||
)
|
||||
|
||||
func BenchmarkInheritanceExpansion(b *testing.B) {
|
||||
for _, shape := range []string{"chain", "wide", "nested", "replacement"} {
|
||||
for _, size := range []int{16, 32, 64} {
|
||||
b.Run(shape+"/"+strconv.Itoa(size), func(b *testing.B) {
|
||||
input := inheritanceBenchmarkInput(shape, size)
|
||||
b.ReportAllocs()
|
||||
b.ResetTimer()
|
||||
for range b.N {
|
||||
if _, diagnostics := expandInheritanceWithOrigins(input, nil, defaultProjectionLimits()); len(diagnostics) != 0 {
|
||||
b.Fatal(diagnostics)
|
||||
}
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func BenchmarkInheritanceInspectedLoad(b *testing.B) {
|
||||
type entry struct {
|
||||
Value string `yaml:"value"`
|
||||
}
|
||||
type configuration struct {
|
||||
Entries map[string]entry `yaml:"entries"`
|
||||
}
|
||||
for _, size := range []int{16, 32, 64} {
|
||||
b.Run(strconv.Itoa(size), func(b *testing.B) {
|
||||
var input strings.Builder
|
||||
input.WriteString("entries:\n item-0: {value: base}\n")
|
||||
for index := 1; index <= size; index++ {
|
||||
input.WriteString(" item-")
|
||||
input.WriteString(strconv.Itoa(index))
|
||||
input.WriteString(": {extends: item-0}\n")
|
||||
}
|
||||
source := Bytes("inheritance-benchmark", []byte(input.String()))
|
||||
query := Keys("entries", "item-"+strconv.Itoa(size), "value")
|
||||
b.ReportAllocs()
|
||||
b.ResetTimer()
|
||||
for range b.N {
|
||||
loaded, err := Load[configuration](source, testSecretBinding())
|
||||
if err != nil {
|
||||
b.Fatal(err)
|
||||
}
|
||||
if _, found := loaded.Inspection.Explain(query); !found {
|
||||
b.Fatal("inherited value is not explainable")
|
||||
}
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// Each scope has constant-sized output and evidence. This distinguishes budget
|
||||
// accounting overhead from legitimate growth in expanded parent payloads.
|
||||
func BenchmarkInheritanceIndependentScopes(b *testing.B) {
|
||||
for _, size := range []int{256, 512, 1024} {
|
||||
b.Run(strconv.Itoa(size), func(b *testing.B) {
|
||||
input := &document.Node{Kind: document.Mapping}
|
||||
for index := 0; index < size; index++ {
|
||||
input.Mapping = append(input.Mapping, document.Entry{
|
||||
Key: strconv.Itoa(index), Value: inheritanceBenchmarkInput("wide", 1),
|
||||
})
|
||||
}
|
||||
b.ReportAllocs()
|
||||
b.ResetTimer()
|
||||
for range b.N {
|
||||
if _, diagnostics := expandInheritanceTree(input, defaultProjectionLimits()); len(diagnostics) != 0 {
|
||||
b.Fatal(diagnostics)
|
||||
}
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func inheritanceBenchmarkInput(shape string, size int) *document.Node {
|
||||
value := func(text string) *document.Node { return &document.Node{Kind: document.String, Text: text} }
|
||||
mapping := func(entries ...document.Entry) *document.Node {
|
||||
return &document.Node{Kind: document.Mapping, Mapping: entries}
|
||||
}
|
||||
entry := func(key string, node *document.Node) document.Entry { return document.Entry{Key: key, Value: node} }
|
||||
base := mapping(entry("value", value("base")), entry("settings", mapping(entry("enabled", &document.Node{Kind: document.Boolean, Text: "false"}))))
|
||||
if shape == "nested" {
|
||||
base = mapping(entry("settings", mapping(entry("first", mapping(entry("value", value("base")))), entry("second", mapping(entry("extends", value("first")))))))
|
||||
}
|
||||
definitions := make([]document.Entry, 0, size+1)
|
||||
definitions = append(definitions, entry("item-0", base))
|
||||
for index := 1; index <= size; index++ {
|
||||
parent := "item-0"
|
||||
if shape == "chain" {
|
||||
parent = "item-" + strconv.Itoa(index-1)
|
||||
}
|
||||
payload := []document.Entry{entry("extends", value(parent))}
|
||||
if shape == "nested" {
|
||||
payload = append(payload, entry("settings", mapping(entry("first", mapping(entry("value", value("custom")))))))
|
||||
}
|
||||
if shape == "replacement" {
|
||||
payload = append(payload, entry("value", value("replacement")))
|
||||
}
|
||||
definitions = append(definitions, entry("item-"+strconv.Itoa(index), mapping(payload...)))
|
||||
}
|
||||
return mapping(entry("entries", mapping(definitions...)))
|
||||
}
|
||||
60
internal/engine/inheritance_explain.go
Normal file
60
internal/engine/inheritance_explain.go
Normal file
@@ -0,0 +1,60 @@
|
||||
package engine
|
||||
|
||||
import "gitea.maximumdirect.net/eric/ConfigKit/internal/document"
|
||||
|
||||
func appendInheritanceExplanation(explanation *Explanation, working **document.Node, query Path, operations []inheritanceOperation, secrets map[string]bool) {
|
||||
for _, operation := range operations {
|
||||
changes := projectedInheritanceChanges(query, operation)
|
||||
if operation.control != nil {
|
||||
if !pathsIntersect(operation.control.path, query) {
|
||||
continue
|
||||
}
|
||||
before := valueSnapshot(*working, query, secrets)
|
||||
replaceHistoryPath(working, operation.control.path, nil)
|
||||
after := valueSnapshot(*working, query, secrets)
|
||||
if len(changes) != 0 {
|
||||
explanation.History = append(explanation.History, ValueChange{
|
||||
Before: before, After: after, Origin: cloneOrigin(changes[0].Origin),
|
||||
Dependencies: clonePaths(changes[0].Dependencies),
|
||||
})
|
||||
}
|
||||
continue
|
||||
}
|
||||
if operation.transfer == nil {
|
||||
continue
|
||||
}
|
||||
before := valueSnapshot(*working, query, secrets)
|
||||
replaceHistoryPath(working, operation.transfer.to, operation.parent)
|
||||
afterParent := valueSnapshot(*working, query, secrets)
|
||||
parent, parentPresent := transferredOperationOrigin(operation, operationPathForQuery(operation.transfer.to, query))
|
||||
if pathsIntersect(operation.transfer.to, query) && parentPresent {
|
||||
parent.Operation = ChangeInheritance
|
||||
parent.Inheritance = projectInheritanceSteps(parent)
|
||||
explanation.History = append(explanation.History, ValueChange{
|
||||
Before: before, After: afterParent, Origin: parent,
|
||||
Dependencies: []Path{operation.transfer.from.clone(), operation.transfer.reference.clone()},
|
||||
})
|
||||
}
|
||||
replaceHistoryPath(working, operation.transfer.to, operation.effective)
|
||||
if !pathsIntersect(operation.transfer.to, query) {
|
||||
continue
|
||||
}
|
||||
afterChild := valueSnapshot(*working, query, secrets)
|
||||
if len(changes) != 0 {
|
||||
last := changes[len(changes)-1]
|
||||
if !parentPresent || len(changes) > 1 {
|
||||
explanation.History = append(explanation.History, ValueChange{
|
||||
Before: afterParent, After: afterChild, Origin: cloneOrigin(last.Origin),
|
||||
Dependencies: clonePaths(last.Dependencies),
|
||||
})
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func operationPathForQuery(root, query Path) Path {
|
||||
if pathContains(query, root) {
|
||||
return root
|
||||
}
|
||||
return query
|
||||
}
|
||||
119
internal/engine/inheritance_history.go
Normal file
119
internal/engine/inheritance_history.go
Normal file
@@ -0,0 +1,119 @@
|
||||
package engine
|
||||
|
||||
import "gitea.maximumdirect.net/eric/ConfigKit/internal/document"
|
||||
|
||||
// Retain source history at original paths. Structural operations are held once
|
||||
// per relationship and projected onto a requested destination path only when
|
||||
// History or Explain is queried.
|
||||
func applyInheritanceMetadata(metadata *Metadata, raw *document.Node, rawOrigins map[string]Origin, overlays []overlayRecord, expansion *inheritanceScope) {
|
||||
if metadata == nil || expansion == nil {
|
||||
return
|
||||
}
|
||||
sourceView := Metadata{
|
||||
fields: make(map[string]FieldInfo),
|
||||
history: metadata.history,
|
||||
}
|
||||
walkDocument(raw, Path{}, func(path Path, node *document.Node) {
|
||||
origin, found := rawOrigins[path.String()]
|
||||
if !found {
|
||||
origin = Origin{Layer: LayerBase, Locations: []Location{publicLocation(node.Location)}}
|
||||
}
|
||||
sourceView.fields[path.String()] = FieldInfo{Explicit: true, Null: node.Kind == document.Null, Origin: cloneOrigin(origin)}
|
||||
})
|
||||
applyOverlayMetadata(&sourceView, overlays)
|
||||
metadata.history = sourceView.history
|
||||
metadata.inheritance = append([]inheritanceOperation(nil), expansion.operations...)
|
||||
}
|
||||
|
||||
func historyWithInheritance(path Path, stored []Change, operations []inheritanceOperation) []Change {
|
||||
split := len(stored)
|
||||
for index, change := range stored {
|
||||
if change.Origin.Layer == LayerResolution {
|
||||
split = index
|
||||
break
|
||||
}
|
||||
}
|
||||
history := cloneChanges(stored[:split])
|
||||
for _, operation := range operations {
|
||||
for _, change := range projectedInheritanceChanges(path, operation) {
|
||||
if len(history) != 0 {
|
||||
change.Previous = []Origin{cloneOrigin(history[len(history)-1].Origin)}
|
||||
}
|
||||
history = append(history, change)
|
||||
}
|
||||
}
|
||||
for _, change := range stored[split:] {
|
||||
cloned := cloneChange(change)
|
||||
if len(history) != 0 {
|
||||
cloned.Previous = []Origin{cloneOrigin(history[len(history)-1].Origin)}
|
||||
}
|
||||
history = append(history, cloned)
|
||||
}
|
||||
return history
|
||||
}
|
||||
|
||||
func projectedInheritanceChanges(query Path, operation inheritanceOperation) []Change {
|
||||
if operation.control != nil {
|
||||
if !pathsIntersect(operation.control.path, query) {
|
||||
return nil
|
||||
}
|
||||
origin := cloneOrigin(operation.controlOrigin)
|
||||
origin.Operation = ChangeInheritance
|
||||
origin.Inheritance = projectInheritanceSteps(origin)
|
||||
return []Change{{Path: operation.control.path.clone(), Origin: origin,
|
||||
Dependencies: []Path{operation.control.path.clone()}}}
|
||||
}
|
||||
if operation.transfer == nil || !pathsIntersect(operation.transfer.to, query) {
|
||||
return nil
|
||||
}
|
||||
path := query
|
||||
if pathContains(query, operation.transfer.to) {
|
||||
path = operation.transfer.to
|
||||
}
|
||||
var changes []Change
|
||||
parent, parentFound := transferredOperationOrigin(operation, path)
|
||||
if parentFound {
|
||||
parent.Operation = ChangeInheritance
|
||||
parent.Inheritance = projectInheritanceSteps(parent)
|
||||
changes = append(changes, Change{
|
||||
Path: path.clone(), Origin: parent,
|
||||
Dependencies: []Path{operation.transfer.from.clone(), operation.transfer.reference.clone()},
|
||||
})
|
||||
}
|
||||
effective, effectiveFound := operation.effectiveOrigins[path.String()]
|
||||
if effectiveFound {
|
||||
effective = cloneOrigin(effective)
|
||||
effective.Operation = ChangeInheritance
|
||||
}
|
||||
if effectiveFound && (!parentFound || !originContainedBy(effective, parent)) {
|
||||
effective.Inheritance = projectInheritanceSteps(effective)
|
||||
changes = append(changes, Change{
|
||||
Path: path.clone(), Origin: effective,
|
||||
Dependencies: []Path{operation.transfer.reference.clone()},
|
||||
})
|
||||
} else if parentFound && !effectiveFound {
|
||||
removed := cloneOrigin(operation.effectiveOrigins[operation.transfer.to.String()])
|
||||
removed.Operation = ChangeInheritance
|
||||
removed.Inheritance = projectInheritanceSteps(removed)
|
||||
changes = append(changes, Change{
|
||||
Path: operation.transfer.to.clone(), Origin: removed,
|
||||
Dependencies: []Path{operation.transfer.reference.clone()},
|
||||
})
|
||||
}
|
||||
return changes
|
||||
}
|
||||
|
||||
func transferredOperationOrigin(operation inheritanceOperation, destination Path) (Origin, bool) {
|
||||
relative, ok := relativeHistoryPath(operation.transfer.to, destination)
|
||||
if !ok {
|
||||
return Origin{}, false
|
||||
}
|
||||
source := appendInheritancePath(operation.transfer.from, relative)
|
||||
origin, found := operation.parentOrigins[source.String()]
|
||||
if !found {
|
||||
return Origin{}, false
|
||||
}
|
||||
transferred := transferredInheritanceOrigins(map[string]Origin{source.String(): origin},
|
||||
operation.transfer.from, operation.transfer.to, *operation.transfer)
|
||||
return transferred[destination.String()], true
|
||||
}
|
||||
363
internal/engine/inheritance_history_test.go
Normal file
363
internal/engine/inheritance_history_test.go
Normal file
@@ -0,0 +1,363 @@
|
||||
package engine
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"gitea.maximumdirect.net/eric/ConfigKit/internal/document"
|
||||
)
|
||||
|
||||
func TestNestedInheritanceHistoryRetainsUnresolvedTemplates(t *testing.T) {
|
||||
input := `presets:
|
||||
base:
|
||||
settings:
|
||||
primary: {width: 10}
|
||||
secondary: {extends: primary}
|
||||
custom:
|
||||
extends: base
|
||||
settings:
|
||||
primary: {width: 20}
|
||||
grandchild:
|
||||
extends: custom
|
||||
settings:
|
||||
primary: {width: 30}
|
||||
`
|
||||
loaded, err := Load[map[string]any](Bytes("nested", []byte(input)), testSecretBinding())
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
for name, width := range map[string]string{"custom": "20", "grandchild": "30"} {
|
||||
t.Run(name, func(t *testing.T) {
|
||||
root := Keys("presets", name)
|
||||
value, _ := loaded.Inspection.Explain(root.Key("settings").Key("secondary").Key("width"))
|
||||
introduced := false
|
||||
for _, change := range value.History {
|
||||
if change.Origin.Operation == ChangeInheritance && !change.Before.Present && change.After.YAML == width+"\n" {
|
||||
introduced = true
|
||||
}
|
||||
}
|
||||
if !introduced || value.Effective.YAML != width+"\n" {
|
||||
t.Fatalf("missing nested introduction: %#v", value)
|
||||
}
|
||||
controlPath := root.Key("settings").Key("secondary").Key("extends")
|
||||
control, _ := loaded.Inspection.Explain(controlPath)
|
||||
arrived, consumed := false, false
|
||||
for _, change := range control.History {
|
||||
arrived = arrived || change.After.YAML == "primary\n"
|
||||
consumed = consumed || (change.Before.YAML == "primary\n" && !change.After.Present)
|
||||
}
|
||||
if !arrived || !consumed || control.Effective.Present {
|
||||
t.Fatalf("missing control history: %#v", control)
|
||||
}
|
||||
container, _ := loaded.Inspection.Explain(root)
|
||||
unresolved := false
|
||||
for _, change := range container.History {
|
||||
if change.Origin.Operation == ChangeInheritance && strings.Contains(change.After.YAML, "extends: primary") {
|
||||
unresolved = true
|
||||
}
|
||||
}
|
||||
if !unresolved || strings.Contains(container.Effective.YAML, "extends:") {
|
||||
t.Fatalf("invalid enclosing snapshot: %#v", container)
|
||||
}
|
||||
if _, present := loaded.Inspection.Metadata().Lookup(controlPath); present {
|
||||
t.Fatal("control is effective")
|
||||
}
|
||||
if len(loaded.Inspection.Metadata().History(controlPath)) != len(control.History) {
|
||||
t.Fatal("metadata and explanation disagree")
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestNestedInheritanceHistoryInsideSequence(t *testing.T) {
|
||||
input := `base:
|
||||
items:
|
||||
- first: {value: inherited}
|
||||
second: {extends: first}
|
||||
child: {extends: base}
|
||||
`
|
||||
loaded, err := Load[map[string]any](Bytes("sequence", []byte(input)), testSecretBinding())
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
path := Keys("child", "items").Index(0).Key("second")
|
||||
control, _ := loaded.Inspection.Explain(path.Key("extends"))
|
||||
consumed := false
|
||||
for _, change := range control.History {
|
||||
consumed = consumed || (change.Before.YAML == "first\n" && !change.After.Present)
|
||||
}
|
||||
if !consumed || control.Effective.Present {
|
||||
t.Fatalf("sequence control history = %#v", control)
|
||||
}
|
||||
value, _ := loaded.Inspection.Explain(path.Key("value"))
|
||||
introduced := false
|
||||
for _, change := range value.History {
|
||||
introduced = introduced || (!change.Before.Present && change.After.YAML == "inherited\n")
|
||||
}
|
||||
if !introduced || value.Effective.YAML != "inherited\n" {
|
||||
t.Fatalf("sequence value history = %#v", value)
|
||||
}
|
||||
}
|
||||
|
||||
func inheritanceInspectionForTest(t *testing.T, sources ...*document.Node) (*Inspection, *inheritanceScope) {
|
||||
t.Helper()
|
||||
composed, diagnostics := composeDocuments(sources...)
|
||||
if len(diagnostics) != 0 {
|
||||
t.Fatal(diagnostics)
|
||||
}
|
||||
return inheritanceInspectionFromComposition(t, composed)
|
||||
}
|
||||
|
||||
func inheritanceInspectionFromComposition(t *testing.T, composed *composedDocument) (*Inspection, *inheritanceScope) {
|
||||
t.Helper()
|
||||
expanded, diagnostics := expandInheritanceWithOrigins(composed.root, composed.origins, defaultProjectionLimits())
|
||||
if len(diagnostics) != 0 {
|
||||
t.Fatal(diagnostics)
|
||||
}
|
||||
baseline := &document.Node{Kind: document.Mapping}
|
||||
presence := make(map[string]inputPresence)
|
||||
collectPresence(expanded.root, Path{}, presence)
|
||||
decoded := decodedConfiguration{
|
||||
baseline: baseline, final: expanded.root, presence: presence, origins: expanded.origins,
|
||||
}
|
||||
metadata := metadataFromDecoded(decoded)
|
||||
applyInheritanceMetadata(&metadata, composed.root, composed.origins, composed.overlays, expanded)
|
||||
finalizeMetadataPaths(&metadata, expanded.root, composed.applications, baseline, composed.root)
|
||||
return &Inspection{
|
||||
metadata: metadata, baseline: baseline, input: expanded.root,
|
||||
inputOrigins: expanded.origins, final: expanded.root,
|
||||
applications: composed.applications, inheritance: expanded.operations,
|
||||
}, expanded
|
||||
}
|
||||
|
||||
func TestInheritanceHistoryFollowsFinalParentAfterSourceOverlay(t *testing.T) {
|
||||
base, diagnostics := composeDocuments(parseCompositionDocument(t, "source-1", "parent: {value: original}\nchild: {extends: parent}\n"))
|
||||
if len(diagnostics) != 0 {
|
||||
t.Fatal(diagnostics)
|
||||
}
|
||||
assembly := newAssemblyAccumulator(base)
|
||||
if diagnostics := assembly.applyOverlay(parseCompositionDocument(t, "source-2", "parent: {value: latest}\n"), nil, nil, LayerOverlay); len(diagnostics) != 0 {
|
||||
t.Fatal(diagnostics)
|
||||
}
|
||||
inspection, _ := inheritanceInspectionFromComposition(t, assembly.current)
|
||||
path := Keys("child", "value")
|
||||
explanation, found := inspection.Explain(path)
|
||||
if !found || explanation.Effective.YAML != "latest\n" ||
|
||||
len(explanation.History) == 0 || explanation.History[len(explanation.History)-1].Origin.Locations[0].SourceID != "source-2" {
|
||||
t.Fatalf("overlaid parent contribution = %#v, %v", explanation, found)
|
||||
}
|
||||
parentHistory := inspection.Metadata().History(Keys("parent", "value"))
|
||||
if len(parentHistory) < 2 || parentHistory[0].Origin.Layer != LayerBase ||
|
||||
parentHistory[len(parentHistory)-1].Origin.Layer != LayerOverlay {
|
||||
t.Fatalf("parent source history = %#v", parentHistory)
|
||||
}
|
||||
}
|
||||
|
||||
func TestInheritanceHistoryPreservesCancellationAndSupersededDirective(t *testing.T) {
|
||||
base, diagnostics := composeDocuments(parseCompositionDocument(t, "source-1", "parent: {value: one}\nchild: {extends: missing, value: own}\n"))
|
||||
if len(diagnostics) != 0 {
|
||||
t.Fatal(diagnostics)
|
||||
}
|
||||
assembly := newAssemblyAccumulator(base)
|
||||
if diagnostics := assembly.applyOverlay(parseCompositionDocument(t, "source-2", "child: {extends: null}\n"), nil, nil, LayerOverlay); len(diagnostics) != 0 {
|
||||
t.Fatal(diagnostics)
|
||||
}
|
||||
inspection, _ := inheritanceInspectionFromComposition(t, assembly.current)
|
||||
control := Keys("child", inheritanceKey)
|
||||
history := inspection.Metadata().History(control)
|
||||
if len(history) < 3 || history[0].Origin.Locations[0].SourceID != "source-1" ||
|
||||
history[1].Origin.Locations[0].SourceID != "source-2" ||
|
||||
history[len(history)-1].Origin.Operation != ChangeInheritance {
|
||||
t.Fatalf("superseded and cancelled directive history = %#v", history)
|
||||
}
|
||||
explanation, found := inspection.Explain(control)
|
||||
if !found || explanation.Effective.Present || len(explanation.History) < 3 ||
|
||||
explanation.History[len(explanation.History)-1].After.Present {
|
||||
t.Fatalf("cancelled directive explanation = %#v, %v", explanation, found)
|
||||
}
|
||||
value, found := inspection.Explain(Keys("child", "value"))
|
||||
if !found || value.Effective.YAML != "own\n" {
|
||||
t.Fatalf("cancelled relationship changed own value = %#v, %v", value, found)
|
||||
}
|
||||
}
|
||||
|
||||
func TestInheritanceHistoryProjectsNestedDestinationValues(t *testing.T) {
|
||||
inspection, _ := inheritanceInspectionForTest(t, parseCompositionDocument(t, "source-1", `base:
|
||||
settings:
|
||||
primary: {value: old}
|
||||
secondary: {extends: primary}
|
||||
custom:
|
||||
extends: base
|
||||
settings:
|
||||
primary: {value: new}
|
||||
`))
|
||||
path := Keys("custom", "settings", "secondary", "value")
|
||||
history := inspection.Metadata().History(path)
|
||||
structural := 0
|
||||
for _, change := range history {
|
||||
if change.Origin.Operation == ChangeInheritance {
|
||||
structural++
|
||||
}
|
||||
}
|
||||
if structural == 0 {
|
||||
t.Fatalf("nested destination history = %#v", history)
|
||||
}
|
||||
explanation, found := inspection.Explain(path)
|
||||
if !found || explanation.Effective.YAML != "new\n" ||
|
||||
len(explanation.Origin.Inheritance) == 0 ||
|
||||
explanation.Origin.Inheritance[len(explanation.Origin.Inheritance)-1].To.String() != path.String() {
|
||||
t.Fatalf("nested destination explanation = %#v, %v", explanation, found)
|
||||
}
|
||||
for _, change := range explanation.History {
|
||||
if change.Origin.Operation == ChangeDecoding {
|
||||
t.Fatalf("nested inheritance was attributed to decoding: %#v", explanation.History)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestInheritanceHistoryShowsParentAndEqualChildContribution(t *testing.T) {
|
||||
inspection, _ := inheritanceInspectionForTest(t,
|
||||
parseCompositionDocument(t, "source-1", "parent: {value: same}\n"),
|
||||
parseCompositionDocument(t, "source-2", "child: {extends: parent, value: same}\n"))
|
||||
path := Keys("child", "value")
|
||||
history := inspection.Metadata().History(path)
|
||||
var structural []Change
|
||||
for _, change := range history {
|
||||
if change.Origin.Operation == ChangeInheritance {
|
||||
structural = append(structural, change)
|
||||
}
|
||||
}
|
||||
if len(structural) != 2 || structural[0].Origin.Locations[0].SourceID != "source-1" ||
|
||||
structural[1].Origin.Locations[0].SourceID != "source-2" ||
|
||||
len(structural[0].Dependencies) != 2 || structural[0].Dependencies[0].String() != "/parent" ||
|
||||
len(structural[0].Origin.Inheritance) != 1 {
|
||||
t.Fatalf("equal-valued structural history = %#v", history)
|
||||
}
|
||||
explanation, found := inspection.Explain(path)
|
||||
if !found || explanation.Effective.YAML != "same\n" {
|
||||
t.Fatalf("explanation = %#v, %v", explanation, found)
|
||||
}
|
||||
var equalEvents int
|
||||
for _, change := range explanation.History {
|
||||
if change.Origin.Operation == ChangeDecoding {
|
||||
t.Fatalf("inheritance was attributed to decoding: %#v", explanation.History)
|
||||
}
|
||||
if change.Origin.Operation == ChangeInheritance && change.Before.YAML == change.After.YAML {
|
||||
equalEvents++
|
||||
}
|
||||
}
|
||||
if equalEvents != 2 {
|
||||
t.Fatalf("equal parent and child operations = %#v", explanation.History)
|
||||
}
|
||||
parentHistory := inspection.Metadata().History(Keys("parent", "value"))
|
||||
for _, change := range parentHistory {
|
||||
if change.Origin.Operation == ChangeInheritance {
|
||||
t.Fatalf("source definition acquired a destination operation: %#v", parentHistory)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestInheritanceHistoryRetainsControlsAndDisplacedPaths(t *testing.T) {
|
||||
inspection, _ := inheritanceInspectionForTest(t,
|
||||
parseCompositionDocument(t, "source-1", "parent: {branch: {detail: inherited}}\n"),
|
||||
parseCompositionDocument(t, "source-2", "child: {extends: parent, branch: replaced}\n"))
|
||||
control := Keys("child", inheritanceKey)
|
||||
controlHistory := inspection.Metadata().History(control)
|
||||
if len(controlHistory) < 2 || controlHistory[len(controlHistory)-1].Origin.Operation != ChangeInheritance {
|
||||
t.Fatalf("control history = %#v", controlHistory)
|
||||
}
|
||||
controlExplanation, found := inspection.Explain(control)
|
||||
if !found || controlExplanation.Effective.Present ||
|
||||
controlExplanation.History[len(controlExplanation.History)-1].After.Present {
|
||||
t.Fatalf("consumed control = %#v, %v", controlExplanation, found)
|
||||
}
|
||||
path := Keys("child", "branch", "detail")
|
||||
if _, found := inspection.Metadata().Lookup(path); found {
|
||||
t.Fatal("displaced descendant remained effective")
|
||||
}
|
||||
listed := false
|
||||
for _, historical := range inspection.Metadata().HistoricalPaths() {
|
||||
if historical.String() == path.String() {
|
||||
listed = true
|
||||
}
|
||||
}
|
||||
if !listed {
|
||||
t.Fatal("displaced inherited descendant was not enumerable")
|
||||
}
|
||||
history := inspection.Metadata().History(path)
|
||||
if len(history) < 2 || history[len(history)-1].Origin.Operation != ChangeInheritance {
|
||||
t.Fatalf("displaced descendant history = %#v", history)
|
||||
}
|
||||
explanation, found := inspection.Explain(path)
|
||||
if !found || explanation.Effective.Present {
|
||||
t.Fatalf("displaced descendant explanation = %#v, %v", explanation, found)
|
||||
}
|
||||
var sawInherited, sawRemoval bool
|
||||
for _, change := range explanation.History {
|
||||
if change.Origin.Operation != ChangeInheritance {
|
||||
continue
|
||||
}
|
||||
sawInherited = sawInherited || change.After.Present
|
||||
sawRemoval = sawRemoval || (change.Before.Present && !change.After.Present)
|
||||
}
|
||||
if !sawInherited || !sawRemoval {
|
||||
t.Fatalf("displaced value sequence = %#v", explanation.History)
|
||||
}
|
||||
var rendered bytes.Buffer
|
||||
if err := inspection.WriteExplanation(&rendered, path); err != nil ||
|
||||
!strings.Contains(rendered.String(), "structural inheritance") {
|
||||
t.Fatalf("rendered structural explanation = %q, %v", rendered.String(), err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestInheritanceHistoryPrecedesApplicationResolution(t *testing.T) {
|
||||
inspection, _ := inheritanceInspectionForTest(t,
|
||||
parseCompositionDocument(t, "source-1", "parent: {value: original}\nchild: {extends: parent}\n"))
|
||||
path := Keys("child", "value")
|
||||
replacement := &document.Node{Kind: document.String, Text: "derived"}
|
||||
snapshot := resolutionSnapshot{
|
||||
change: Change{Path: path, Origin: Origin{Layer: LayerResolution, Operation: ChangeNormalization}},
|
||||
after: replacement,
|
||||
}
|
||||
final := document.Clone(inspection.final)
|
||||
replaceHistoryPath(&final, path, replacement)
|
||||
inspection.final = final
|
||||
inspection.resolution = []resolutionSnapshot{snapshot}
|
||||
inspection.metadata.history[path.String()] = append(inspection.metadata.history[path.String()], snapshot.change)
|
||||
inspection.metadata.fields[path.String()] = FieldInfo{Explicit: true, Origin: snapshot.change.Origin}
|
||||
history := inspection.Metadata().History(path)
|
||||
if len(history) < 2 || history[len(history)-2].Origin.Operation != ChangeInheritance ||
|
||||
history[len(history)-1].Origin.Operation != ChangeNormalization {
|
||||
t.Fatalf("resolution order = %#v", history)
|
||||
}
|
||||
explanation, found := inspection.Explain(path)
|
||||
if !found || explanation.Effective.YAML != "derived\n" ||
|
||||
explanation.History[len(explanation.History)-1].Origin.Operation != ChangeNormalization {
|
||||
t.Fatalf("resolved explanation = %#v, %v", explanation, found)
|
||||
}
|
||||
}
|
||||
|
||||
func TestHistoryNodeBudgetCountsSharedInheritanceEvidenceOnce(t *testing.T) {
|
||||
shared := &document.Node{Kind: document.String, Text: "shared"}
|
||||
extra := &document.Node{Kind: document.String, Text: "extra"}
|
||||
expansion := &inheritanceScope{
|
||||
root: shared,
|
||||
operations: []inheritanceOperation{{parent: shared, effective: shared}},
|
||||
replacements: []inheritanceReplacement{{before: shared, after: extra}},
|
||||
}
|
||||
if !historyNodesWithin(2, nil, nil, nil, nil, nil, expansion) ||
|
||||
historyNodesWithin(1, nil, nil, nil, nil, nil, expansion) {
|
||||
t.Fatal("shared evidence node accounting is incorrect")
|
||||
}
|
||||
}
|
||||
|
||||
func TestInheritanceExpansionEnforcesRetainedHistoryBudget(t *testing.T) {
|
||||
input := parseCompositionDocument(t, "source-1", "parent: {value: original}\nchild: {extends: parent, value: replacement}\n")
|
||||
if _, diagnostics := expandInheritanceWithBudgets(input, nil, defaultProjectionLimits(), 2); len(diagnostics) != 1 || diagnostics[0].Code != CodeExplainLimitExceeded {
|
||||
t.Fatalf("retained history limit = %#v", diagnostics)
|
||||
}
|
||||
if _, diagnostics := expandInheritanceWithBudgets(input, nil, defaultProjectionLimits(), 100); len(diagnostics) != 0 {
|
||||
t.Fatalf("sufficient retained history budget = %#v", diagnostics)
|
||||
}
|
||||
}
|
||||
53
internal/engine/inheritance_load.go
Normal file
53
internal/engine/inheritance_load.go
Normal file
@@ -0,0 +1,53 @@
|
||||
package engine
|
||||
|
||||
import "gitea.maximumdirect.net/eric/ConfigKit/internal/document"
|
||||
|
||||
func containsInheritanceDirective(root *document.Node) bool {
|
||||
found := false
|
||||
walkDocument(root, Path{}, func(path Path, _ *document.Node) {
|
||||
if len(path.segments) != 0 && path.segments[len(path.segments)-1].kind == pathSegmentKey &&
|
||||
path.segments[len(path.segments)-1].key == inheritanceKey {
|
||||
found = true
|
||||
}
|
||||
})
|
||||
return found
|
||||
}
|
||||
|
||||
func inheritanceVersionDiagnostic(rule *VersionRule, expanded *inheritanceScope) *Diagnostic {
|
||||
if rule == nil || expanded == nil {
|
||||
return nil
|
||||
}
|
||||
for _, operation := range expanded.operations {
|
||||
var location Location
|
||||
var affected bool
|
||||
if operation.control != nil {
|
||||
affected = pathsIntersect(operation.control.path, rule.Path)
|
||||
if len(operation.control.locations) != 0 {
|
||||
location = operation.control.locations[0]
|
||||
}
|
||||
} else if operation.transfer != nil {
|
||||
if relative, within := relativeHistoryPath(operation.transfer.to, rule.Path); within {
|
||||
_, affected = documentAtPath(operation.parent, relative)
|
||||
} else if pathContains(rule.Path, operation.transfer.to) {
|
||||
affected = true
|
||||
}
|
||||
if len(operation.transfer.locations) != 0 {
|
||||
location = operation.transfer.locations[0]
|
||||
}
|
||||
}
|
||||
if affected {
|
||||
return &Diagnostic{
|
||||
Code: CodeInheritanceVersion, Severity: SeverityError, Path: rule.Path.clone(),
|
||||
Message: "inheritance cannot change the schema-version path", Primary: &location,
|
||||
}
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func inheritanceOperations(expanded *inheritanceScope) []inheritanceOperation {
|
||||
if expanded == nil {
|
||||
return nil
|
||||
}
|
||||
return expanded.operations
|
||||
}
|
||||
127
internal/engine/inheritance_origin.go
Normal file
127
internal/engine/inheritance_origin.go
Normal file
@@ -0,0 +1,127 @@
|
||||
package engine
|
||||
|
||||
import (
|
||||
"slices"
|
||||
|
||||
"gitea.maximumdirect.net/eric/ConfigKit/internal/document"
|
||||
)
|
||||
|
||||
func directInheritanceOrigins(node *document.Node, root Path, input map[string]Origin) map[string]Origin {
|
||||
result := make(map[string]Origin)
|
||||
walkDocument(node, root, func(path Path, value *document.Node) {
|
||||
origin, found := input[path.String()]
|
||||
if !found {
|
||||
origin = Origin{Layer: LayerBase, Locations: []Location{publicLocation(value.Location)}}
|
||||
}
|
||||
origin = cloneOrigin(origin)
|
||||
if len(origin.contributors) == 0 {
|
||||
origin.contributors = make([]inheritanceContributor, len(origin.Locations))
|
||||
for index, location := range origin.Locations {
|
||||
origin.contributors[index] = inheritanceContributor{
|
||||
source: path.clone(), location: location,
|
||||
}
|
||||
}
|
||||
}
|
||||
result[path.String()] = origin
|
||||
})
|
||||
return result
|
||||
}
|
||||
|
||||
func transferredInheritanceOrigins(parent map[string]Origin, from, to Path, control inheritanceTransfer) map[string]Origin {
|
||||
result := make(map[string]Origin, len(parent))
|
||||
transfer := control // one immutable edge shared by every matching contributor
|
||||
links := make(map[*inheritanceTrace]*inheritanceTrace)
|
||||
for pointer, origin := range parent {
|
||||
path, err := ParsePath(pointer)
|
||||
if err != nil {
|
||||
continue
|
||||
}
|
||||
relative, ok := relativeHistoryPath(from, path)
|
||||
if !ok {
|
||||
continue
|
||||
}
|
||||
destination := appendInheritancePath(to, relative)
|
||||
origin = cloneOrigin(origin)
|
||||
for index, contributor := range origin.contributors {
|
||||
link, found := links[contributor.trace]
|
||||
if !found {
|
||||
link = &inheritanceTrace{previous: contributor.trace, transfer: &transfer}
|
||||
links[contributor.trace] = link
|
||||
}
|
||||
origin.contributors[index].trace = link
|
||||
}
|
||||
result[destination.String()] = origin
|
||||
}
|
||||
return result
|
||||
}
|
||||
|
||||
func appendInheritancePath(root, relative Path) Path {
|
||||
return Path{segments: append(clonePathSegments(root.segments, len(relative.segments)), relative.segments...)}
|
||||
}
|
||||
|
||||
func mergeInheritanceOrigin(origins map[string]Origin, path Path, incoming Origin) {
|
||||
pointer := path.String()
|
||||
current := origins[pointer]
|
||||
current.Layer = incoming.Layer
|
||||
current.Locations = append(current.Locations, incoming.Locations...)
|
||||
current.contributors = append(current.contributors, incoming.contributors...)
|
||||
origins[pointer] = current
|
||||
}
|
||||
|
||||
func copyInheritanceSubtree(target, incoming map[string]Origin, node *document.Node, root Path) {
|
||||
walkDocument(node, root, func(path Path, _ *document.Node) {
|
||||
target[path.String()] = cloneOrigin(incoming[path.String()])
|
||||
})
|
||||
}
|
||||
|
||||
// Materialize concrete steps only for an inspection record. The linked private
|
||||
// traces remain compact when one parent supplies many descendant values.
|
||||
func projectInheritanceSteps(origin Origin) []InheritanceStep {
|
||||
steps := cloneInheritanceSteps(origin.Inheritance)
|
||||
for _, contributor := range origin.contributors {
|
||||
var reverse []*inheritanceTrace
|
||||
for link := contributor.trace; link != nil; link = link.previous {
|
||||
reverse = append(reverse, link)
|
||||
}
|
||||
current := contributor.source
|
||||
for index := len(reverse) - 1; index >= 0; index-- {
|
||||
transfer := reverse[index].transfer
|
||||
relative, ok := relativeHistoryPath(transfer.from, current)
|
||||
if !ok {
|
||||
continue
|
||||
}
|
||||
next := appendInheritancePath(transfer.to, relative)
|
||||
step := InheritanceStep{
|
||||
From: current.clone(), To: next.clone(), Reference: transfer.reference.clone(),
|
||||
Locations: cloneLocations(transfer.locations),
|
||||
}
|
||||
if !slices.ContainsFunc(steps, func(existing InheritanceStep) bool {
|
||||
return existing.From.String() == step.From.String() &&
|
||||
existing.To.String() == step.To.String() &&
|
||||
existing.Reference.String() == step.Reference.String() &&
|
||||
slices.Equal(existing.Locations, step.Locations)
|
||||
}) {
|
||||
steps = append(steps, step)
|
||||
}
|
||||
current = next
|
||||
}
|
||||
}
|
||||
return steps
|
||||
}
|
||||
|
||||
func sameInheritanceTrace(left, right *inheritanceTrace) bool {
|
||||
for left != nil && right != nil {
|
||||
if left == right {
|
||||
return true
|
||||
}
|
||||
l, r := left.transfer, right.transfer
|
||||
if !slices.Equal(l.from.segments, r.from.segments) ||
|
||||
!slices.Equal(l.to.segments, r.to.segments) ||
|
||||
!slices.Equal(l.reference.segments, r.reference.segments) ||
|
||||
!slices.Equal(l.locations, r.locations) {
|
||||
return false
|
||||
}
|
||||
left, right = left.previous, right.previous
|
||||
}
|
||||
return left == nil && right == nil
|
||||
}
|
||||
141
internal/engine/inheritance_origin_test.go
Normal file
141
internal/engine/inheritance_origin_test.go
Normal file
@@ -0,0 +1,141 @@
|
||||
package engine
|
||||
|
||||
import (
|
||||
"io/fs"
|
||||
"testing"
|
||||
"testing/fstest"
|
||||
|
||||
"gitea.maximumdirect.net/eric/ConfigKit/internal/document"
|
||||
)
|
||||
|
||||
func TestInheritedOriginsKeepSourcePathsPresenceAndIndependentReplacement(t *testing.T) {
|
||||
parent := parseCompositionDocument(t, "source-1", `parent:
|
||||
enabled: false
|
||||
count: 0
|
||||
optional: null
|
||||
items: [{file: message.txt}]
|
||||
settings: {shared: parent}
|
||||
`)
|
||||
child := parseCompositionDocument(t, "source-2", `child:
|
||||
extends: parent
|
||||
count: 3
|
||||
settings: {own: child}
|
||||
`)
|
||||
composed, diagnostics := composeDocuments(parent, child)
|
||||
if len(diagnostics) != 0 {
|
||||
t.Fatal(diagnostics)
|
||||
}
|
||||
result, diagnostics := expandInheritanceWithOrigins(composed.root, composed.origins, defaultProjectionLimits())
|
||||
if len(diagnostics) != 0 {
|
||||
t.Fatal(diagnostics)
|
||||
}
|
||||
for _, key := range []string{"enabled", "optional"} {
|
||||
path := Keys("child", key)
|
||||
value, found := documentAtPath(result.root, path)
|
||||
origin, hasOrigin := result.origins[path.String()]
|
||||
if !found || !hasOrigin || value == nil || len(origin.Locations) != 1 || origin.Locations[0].SourceID != "source-1" ||
|
||||
len(projectInheritanceSteps(origin)) != 1 {
|
||||
t.Fatalf("inherited %s = %#v, %#v", key, value, origin)
|
||||
}
|
||||
}
|
||||
count := result.origins["/child/count"]
|
||||
if len(count.Locations) != 1 || count.Locations[0].SourceID != "source-2" || len(projectInheritanceSteps(count)) != 0 {
|
||||
t.Fatalf("direct replacement origin = %#v", count)
|
||||
}
|
||||
if len(result.replacements) != 1 || result.replacements[0].path.String() != "/child/count" ||
|
||||
result.replacements[0].before.Text != "0" || result.replacements[0].after.Text != "3" ||
|
||||
len(result.replacements[0].previous) != 1 ||
|
||||
len(projectInheritanceSteps(result.replacements[0].previous[0].origin)) != 1 {
|
||||
t.Fatalf("displaced inheritance = %#v", result.replacements)
|
||||
}
|
||||
settings := result.origins["/child/settings"]
|
||||
if len(settings.Locations) != 2 || settings.Locations[0].SourceID != "source-1" ||
|
||||
settings.Locations[1].SourceID != "source-2" || len(projectInheritanceSteps(settings)) != 1 {
|
||||
t.Fatalf("composite mapping origin = %#v", settings)
|
||||
}
|
||||
if originContainedBy(result.origins["/child/enabled"], result.origins["/parent/enabled"]) ||
|
||||
originContainedBy(result.origins["/parent/enabled"], result.origins["/child/enabled"]) {
|
||||
t.Fatal("direct and inherited evidence collapsed despite matching source coordinates")
|
||||
}
|
||||
original := result.origins["/child/enabled"]
|
||||
detached := cloneOrigin(original)
|
||||
detached.Locations[0].SourceID = "changed"
|
||||
detached.contributors[0].source.segments[0].key = "changed"
|
||||
stepsCopy := projectInheritanceSteps(original)
|
||||
stepsCopy[0].Locations[0].SourceID = "changed"
|
||||
if original.Locations[0].SourceID != "source-1" ||
|
||||
original.contributors[0].source.String() != "/parent/enabled" ||
|
||||
projectInheritanceSteps(original)[0].Locations[0].SourceID != "source-2" {
|
||||
t.Fatal("cloned origin or projected steps changed retained evidence")
|
||||
}
|
||||
resource := result.origins["/child/items/0/file"]
|
||||
steps := projectInheritanceSteps(resource)
|
||||
if len(steps) != 1 || steps[0].From.String() != "/parent/items/0/file" ||
|
||||
steps[0].To.String() != "/child/items/0/file" || steps[0].Reference.String() != "/child/extends" ||
|
||||
len(resource.Locations) != 1 || resource.Locations[0].SourceID != "source-1" {
|
||||
t.Fatalf("sequence resource origin = %#v, steps = %#v", resource, steps)
|
||||
}
|
||||
contexts := map[string]fs.FS{
|
||||
"source-1": fstest.MapFS{"message.txt": &fstest.MapFile{Data: []byte("from parent")}},
|
||||
"source-2": fstest.MapFS{"message.txt": &fstest.MapFile{Data: []byte("from child")}},
|
||||
}
|
||||
data, err := fs.ReadFile(contexts[resource.Locations[0].SourceID], "message.txt")
|
||||
if err != nil || string(data) != "from parent" {
|
||||
t.Fatalf("inherited resource = %q, %v", data, err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestInheritedOriginsRetainChainsAndNestedDestinations(t *testing.T) {
|
||||
first := parseCompositionDocument(t, "source-1", `first:
|
||||
enabled: false
|
||||
settings:
|
||||
primary: {value: original}
|
||||
secondary: {extends: primary}
|
||||
`)
|
||||
second := parseCompositionDocument(t, "source-2", `first: {other: present}
|
||||
middle: {extends: first}
|
||||
last:
|
||||
extends: middle
|
||||
settings:
|
||||
primary: {value: replaced}
|
||||
`)
|
||||
composed, diagnostics := composeDocuments(first, second)
|
||||
if len(diagnostics) != 0 {
|
||||
t.Fatal(diagnostics)
|
||||
}
|
||||
result, diagnostics := expandInheritanceWithOrigins(composed.root, composed.origins, defaultProjectionLimits())
|
||||
if len(diagnostics) != 0 {
|
||||
t.Fatal(diagnostics)
|
||||
}
|
||||
chain := projectInheritanceSteps(result.origins["/last/enabled"])
|
||||
if len(chain) != 2 || chain[0].From.String() != "/first/enabled" ||
|
||||
chain[0].To.String() != "/middle/enabled" || chain[1].From.String() != "/middle/enabled" ||
|
||||
chain[1].To.String() != "/last/enabled" {
|
||||
t.Fatalf("chain = %#v", chain)
|
||||
}
|
||||
composite := result.origins["/last"]
|
||||
if len(composite.Locations) < 2 || len(composite.contributors) != len(composite.Locations) {
|
||||
t.Fatalf("composite contributors = %#v", composite)
|
||||
}
|
||||
seenSources := make(map[string]bool)
|
||||
for _, contributor := range composite.contributors {
|
||||
seenSources[contributor.location.SourceID] = true
|
||||
}
|
||||
if !seenSources["source-1"] || !seenSources["source-2"] {
|
||||
t.Fatalf("same-layer source contributors = %#v", composite.contributors)
|
||||
}
|
||||
nested := projectInheritanceSteps(result.origins["/last/settings/secondary/value"])
|
||||
if len(nested) != 1 || nested[0].From.String() != "/last/settings/primary/value" ||
|
||||
nested[0].To.String() != "/last/settings/secondary/value" {
|
||||
t.Fatalf("destination-specific nested transfer = %#v", nested)
|
||||
}
|
||||
if got := result.origins["/last/settings/secondary/value"].Locations[0].SourceID; got != "source-2" {
|
||||
t.Fatalf("nested replacement source = %q", got)
|
||||
}
|
||||
if original, found := documentAtPath(first, Keys("first", "settings", "primary", "value")); !found || original.Text != "original" {
|
||||
t.Fatal("expansion changed its input")
|
||||
}
|
||||
if inherited, found := documentAtPath(result.root, Keys("last", "settings", "secondary", "value")); !found || inherited.Kind != document.String || inherited.Text != "replaced" {
|
||||
t.Fatalf("nested inherited value = %#v, %v", inherited, found)
|
||||
}
|
||||
}
|
||||
136
internal/engine/inheritance_secret_test.go
Normal file
136
internal/engine/inheritance_secret_test.go
Normal file
@@ -0,0 +1,136 @@
|
||||
package engine
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"reflect"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestInheritanceSensitivityFollowsCopiesWithoutMaskingNeighbors(t *testing.T) {
|
||||
const canary = "INHERITED-CANARY-CREDENTIAL"
|
||||
input := parseCompositionDocument(t, "source-1", "parent: {token: "+canary+", plain: visible}\nchild: {extends: parent}\nnext: {extends: child}\nother: {extends: parent}\n")
|
||||
for _, classified := range []Path{Keys("parent", "token"), Keys("next", "token")} {
|
||||
t.Run(classified.String(), func(t *testing.T) {
|
||||
inspection, _ := inheritanceInspectionForTest(t, input)
|
||||
inspection.secrets = classifySecretPaths(nil, func(path Path) bool {
|
||||
return path.String() == classified.String()
|
||||
}, inspection.metadata, inspection.final, inspection.applications, inspection.input)
|
||||
for _, path := range []Path{Keys("parent", "token"), Keys("child", "token"), Keys("next", "token"), Keys("other", "token")} {
|
||||
explanation, found := inspection.Explain(path)
|
||||
if !found || !explanation.Effective.Redacted || strings.Contains(explanation.Effective.YAML, canary) {
|
||||
t.Fatalf("sensitive copy %s = %#v, %v", path, explanation, found)
|
||||
}
|
||||
for _, change := range explanation.History {
|
||||
if strings.Contains(change.Before.YAML, canary) || strings.Contains(change.After.YAML, canary) {
|
||||
t.Fatalf("historical canary at %s: %#v", path, change)
|
||||
}
|
||||
}
|
||||
}
|
||||
plain, found := inspection.Explain(Keys("other", "plain"))
|
||||
if !found || plain.Effective.Redacted || plain.Effective.YAML != "visible\n" {
|
||||
t.Fatalf("unrelated field = %#v, %v", plain, found)
|
||||
}
|
||||
var output bytes.Buffer
|
||||
if err := inspection.WriteEffective(&output); err != nil || strings.Contains(output.String(), canary) || !strings.Contains(output.String(), "visible") {
|
||||
t.Fatalf("effective rendering = %q, %v", output.String(), err)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestInheritanceSensitivityPreservesDisplacedHistoryAndControl(t *testing.T) {
|
||||
const canary = "DISPLACED-CANARY-CREDENTIAL"
|
||||
inspection, _ := inheritanceInspectionForTest(t, parseCompositionDocument(t, "source-1", "parent: {branch: {token: "+canary+"}}\nchild: {extends: parent, branch: public}\n"))
|
||||
inspection.secrets = classifySecretPaths(nil, func(path Path) bool {
|
||||
return path.String() == Keys("parent", "branch", "token").String() || path.String() == Keys("child", inheritanceKey).String()
|
||||
}, inspection.metadata, inspection.final, inspection.applications, inspection.input)
|
||||
path := Keys("child", "branch", "token")
|
||||
explanation, found := inspection.Explain(path)
|
||||
if !found || explanation.Effective.Present {
|
||||
t.Fatalf("displaced sensitive path = %#v, %v", explanation, found)
|
||||
}
|
||||
for _, change := range explanation.History {
|
||||
if strings.Contains(change.Before.YAML, canary) || strings.Contains(change.After.YAML, canary) ||
|
||||
(change.After.Present && !change.After.Redacted) {
|
||||
t.Fatalf("displaced secret leaked: %#v", explanation.History)
|
||||
}
|
||||
}
|
||||
var rendered bytes.Buffer
|
||||
if err := inspection.WriteExplanation(&rendered, path); err != nil || strings.Contains(rendered.String(), canary) {
|
||||
t.Fatalf("displaced explanation rendering = %q, %v", rendered.String(), err)
|
||||
}
|
||||
control, found := inspection.Explain(Keys("child", inheritanceKey))
|
||||
if !found || control.Effective.Present || len(control.History) == 0 || !control.History[0].After.Redacted {
|
||||
t.Fatalf("sensitive control history = %#v, %v", control, found)
|
||||
}
|
||||
info, found := inspection.Metadata().Lookup(Keys("child", "branch"))
|
||||
if !found || len(info.Origin.Inheritance) != 0 {
|
||||
t.Fatalf("independent replacement provenance = %#v, %v", info, found)
|
||||
}
|
||||
}
|
||||
|
||||
func TestInheritanceSensitivityIncludesSchemaTaggedSource(t *testing.T) {
|
||||
type entry struct {
|
||||
Token string `yaml:"token"`
|
||||
}
|
||||
type configuration struct {
|
||||
Parent struct {
|
||||
Token string `yaml:"token" configkit:"secret"`
|
||||
} `yaml:"parent"`
|
||||
Child entry `yaml:"child"`
|
||||
}
|
||||
schema, issue := buildSchema(reflect.TypeFor[configuration](), testSecretBinding())
|
||||
if issue != nil {
|
||||
t.Fatal(issue)
|
||||
}
|
||||
inspection, _ := inheritanceInspectionForTest(t, parseCompositionDocument(t, "source-1", "parent: {token: TAGGED-CANARY}\nchild: {extends: parent}\n"))
|
||||
inspection.secrets = classifySecretPaths(schema, nil, inspection.metadata, inspection.final, inspection.applications, inspection.input)
|
||||
child, found := inspection.Explain(Keys("child", "token"))
|
||||
if !found || !child.Effective.Redacted || strings.Contains(child.Effective.YAML, "TAGGED-CANARY") {
|
||||
t.Fatalf("tagged source copy = %#v, %v", child, found)
|
||||
}
|
||||
}
|
||||
|
||||
func TestInheritedSecretReferencesResolveOnlyAfterExpansion(t *testing.T) {
|
||||
type entry struct {
|
||||
Credential Secret `yaml:"credential"`
|
||||
}
|
||||
type configuration struct {
|
||||
Entries map[string]entry `yaml:"entries"`
|
||||
}
|
||||
schema, issue := buildSchema(reflect.TypeFor[configuration](), testSecretBinding())
|
||||
if issue != nil {
|
||||
t.Fatal(issue)
|
||||
}
|
||||
composed, diagnostics := composeDocuments(parseCompositionDocument(t, "source-1", "entries:\n parent: {credential: INHERITED_SECRET_NAME}\n child: {extends: parent}\n"))
|
||||
if len(diagnostics) != 0 {
|
||||
t.Fatal(diagnostics)
|
||||
}
|
||||
expanded, diagnostics := expandInheritanceWithOrigins(composed.root, composed.origins, defaultProjectionLimits())
|
||||
if len(diagnostics) != 0 {
|
||||
t.Fatal(diagnostics)
|
||||
}
|
||||
lookupCount := 0
|
||||
decoded, issues := decodeConfiguration(schema, reflect.Zero(reflect.TypeFor[configuration]()), expanded.root)
|
||||
if len(issues) != 0 || lookupCount != 0 {
|
||||
t.Fatalf("decoding and lookup order = %#v, %d", issues, lookupCount)
|
||||
}
|
||||
resolved := resolveSecretReferences(decoded.value, schema, expanded.root, func(name string) (string, bool) {
|
||||
lookupCount++
|
||||
if name != "INHERITED_SECRET_NAME" {
|
||||
t.Fatalf("inherited reference name = %q", name)
|
||||
}
|
||||
return "RESOLVED-CREDENTIAL-CANARY", true
|
||||
})
|
||||
if len(resolved.diagnostics) != 0 || lookupCount != 1 ||
|
||||
decoded.value.Interface().(configuration).Entries["child"].Credential.Value() != "RESOLVED-CREDENTIAL-CANARY" {
|
||||
t.Fatalf("inherited reference resolution = %#v, %d", resolved.diagnostics, lookupCount)
|
||||
}
|
||||
inspection, _ := inheritanceInspectionFromComposition(t, composed)
|
||||
inspection.secrets = classifySecretPaths(schema, nil, inspection.metadata, inspection.final, inspection.applications, inspection.input)
|
||||
var rendered bytes.Buffer
|
||||
if err := inspection.WriteEffective(&rendered); err != nil || strings.Contains(rendered.String(), "RESOLVED-CREDENTIAL-CANARY") {
|
||||
t.Fatalf("reference rendering = %q, %v", rendered.String(), err)
|
||||
}
|
||||
}
|
||||
141
internal/engine/inheritance_test.go
Normal file
141
internal/engine/inheritance_test.go
Normal file
@@ -0,0 +1,141 @@
|
||||
package engine
|
||||
|
||||
import (
|
||||
"reflect"
|
||||
"strconv"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"gitea.maximumdirect.net/eric/ConfigKit/internal/document"
|
||||
)
|
||||
|
||||
func TestSiblingInheritanceResolvesReferencesAndKeepsInputs(t *testing.T) {
|
||||
tests := []struct {
|
||||
name, input string
|
||||
path Path
|
||||
text string
|
||||
transfers int
|
||||
controls int
|
||||
}{
|
||||
{
|
||||
name: "forward reference", input: "child: {extends: parent, value: own}\nparent: {value: parent, kept: preserved}\n",
|
||||
path: Keys("child", "kept"), text: "preserved", transfers: 1, controls: 1,
|
||||
},
|
||||
{
|
||||
name: "backward reference", input: "parent: {value: parent, kept: preserved}\nchild: {extends: parent, value: own}\n",
|
||||
path: Keys("child", "value"), text: "own", transfers: 1, controls: 1,
|
||||
},
|
||||
{
|
||||
name: "chain and shared parent", input: "last: {extends: middle, enabled: false}\nfirst: {value: original}\nmiddle: {extends: first, extra: middle}\nother: {extends: first}\n",
|
||||
path: Keys("last", "value"), text: "original", transfers: 3, controls: 3,
|
||||
},
|
||||
{
|
||||
name: "exact empty sibling name", input: "child: {extends: \"\"}\n\"\": {value: empty}\n",
|
||||
path: Keys("child", "value"), text: "empty", transfers: 1, controls: 1,
|
||||
},
|
||||
{
|
||||
name: "exact unusual sibling name", input: "child: {extends: \"a/b~c\"}\n\"a/b~c\": {value: unusual}\n",
|
||||
path: Keys("child", "value"), text: "unusual", transfers: 1, controls: 1,
|
||||
},
|
||||
{
|
||||
name: "null cancellation", input: "child: {extends: null, value: own}\nparent: {value: parent}\n",
|
||||
path: Keys("child", "value"), text: "own", controls: 1,
|
||||
},
|
||||
}
|
||||
for _, test := range tests {
|
||||
t.Run(test.name, func(t *testing.T) {
|
||||
input := parseCompositionDocument(t, "source-1", test.input)
|
||||
before := document.Clone(input)
|
||||
result, diagnostics := expandSiblingDefinitions(input, Path{})
|
||||
if len(diagnostics) != 0 {
|
||||
t.Fatalf("diagnostics = %#v", diagnostics)
|
||||
}
|
||||
if len(result.transfers) != test.transfers || len(result.controls) != test.controls {
|
||||
t.Fatalf("evidence = %#v, %#v", result.transfers, result.controls)
|
||||
}
|
||||
node, found := documentAtPath(result.root, test.path)
|
||||
if !found || node.Text != test.text {
|
||||
t.Fatalf("%s = %#v, %v", test.path.String(), node, found)
|
||||
}
|
||||
if _, found := documentAtPath(result.root, Keys("child", inheritanceKey)); found {
|
||||
t.Fatal("child directive remained in expanded payload")
|
||||
}
|
||||
if !reflect.DeepEqual(input, before) {
|
||||
t.Fatal("resolver changed input")
|
||||
}
|
||||
node.Text = "changed"
|
||||
if !reflect.DeepEqual(input, before) {
|
||||
t.Fatal("expanded payload aliases input")
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestSiblingInheritanceTracksTransferAndIndependentChildren(t *testing.T) {
|
||||
input := parseCompositionDocument(t, "source-1", "first: {value: parent}\nsecond: {extends: first}\nthird: {extends: first}\n")
|
||||
result, diagnostics := expandSiblingDefinitions(input, Keys("catalog"))
|
||||
if len(diagnostics) != 0 {
|
||||
t.Fatal(diagnostics)
|
||||
}
|
||||
if len(result.transfers) != 2 || result.transfers[0].from.String() != "/catalog/first" ||
|
||||
result.transfers[0].to.String() != "/catalog/second" ||
|
||||
result.transfers[0].reference.String() != "/catalog/second/extends" ||
|
||||
len(result.transfers[0].locations) != 1 || result.transfers[0].locations[0].SourceID != "source-1" {
|
||||
t.Fatalf("transfers = %#v", result.transfers)
|
||||
}
|
||||
second, _ := documentAtPath(result.root, Keys("second", "value"))
|
||||
third, _ := documentAtPath(result.root, Keys("third", "value"))
|
||||
second.Text = "changed"
|
||||
if third.Text != "parent" {
|
||||
t.Fatal("siblings share mutable expanded values")
|
||||
}
|
||||
}
|
||||
|
||||
func TestSiblingInheritanceRejectsInvalidRelationships(t *testing.T) {
|
||||
tests := []struct {
|
||||
name, input, path string
|
||||
code Code
|
||||
related bool
|
||||
}{
|
||||
{"missing", "child: {extends: private-reference-canary}\n", "/child/extends", CodeInheritanceMissing, false},
|
||||
{"nonmapping parent", "child: {extends: parent}\nparent: null\n", "/child/extends", CodeInheritanceInvalid, true},
|
||||
{"boolean reference", "child: {extends: true}\n", "/child/extends", CodeInheritanceInvalid, false},
|
||||
{"sequence reference", "child: {extends: [parent]}\n", "/child/extends", CodeInheritanceInvalid, false},
|
||||
{"self reference", "child: {extends: child}\n", "/child/extends", CodeInheritanceCycle, true},
|
||||
{"longer cycle", "a: {extends: b}\nb: {extends: c}\nc: {extends: a}\n", "/c/extends", CodeInheritanceCycle, true},
|
||||
}
|
||||
for _, test := range tests {
|
||||
t.Run(test.name, func(t *testing.T) {
|
||||
input := parseCompositionDocument(t, "source-1", test.input)
|
||||
result, diagnostics := expandSiblingDefinitions(input, Path{})
|
||||
if result != nil || len(diagnostics) != 1 {
|
||||
t.Fatalf("result and diagnostics = %#v, %#v", result, diagnostics)
|
||||
}
|
||||
problem := diagnostics[0]
|
||||
if problem.Code != test.code || problem.Path.String() != test.path || problem.Primary == nil ||
|
||||
problem.Primary.SourceID != "source-1" || problem.Primary.Line < 1 ||
|
||||
(len(problem.Related) != 0) != test.related || strings.Contains(problem.Message, "private-reference-canary") {
|
||||
t.Fatalf("diagnostic = %#v", problem)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestSiblingInheritanceDetectsDeepCycleWithoutRecursiveDependencyWalk(t *testing.T) {
|
||||
const count = 2048
|
||||
entries := make([]document.Entry, count)
|
||||
for index := range entries {
|
||||
location := document.Location{SourceID: "source-1", Line: index + 1, Column: 1}
|
||||
parent := strconv.Itoa((index + 1) % count)
|
||||
entries[index] = document.Entry{
|
||||
Key: strconv.Itoa(index), Value: &document.Node{Kind: document.Mapping, Location: location,
|
||||
Mapping: []document.Entry{{Key: inheritanceKey, Value: &document.Node{
|
||||
Kind: document.String, Text: parent, Location: location,
|
||||
}}}},
|
||||
}
|
||||
}
|
||||
result, diagnostics := expandSiblingDefinitions(&document.Node{Kind: document.Mapping, Mapping: entries}, Path{})
|
||||
if result != nil || len(diagnostics) != 1 || diagnostics[0].Code != CodeInheritanceCycle {
|
||||
t.Fatalf("deep cycle result = %#v, diagnostics = %#v", result, diagnostics)
|
||||
}
|
||||
}
|
||||
309
internal/engine/inheritance_walk.go
Normal file
309
internal/engine/inheritance_walk.go
Normal file
@@ -0,0 +1,309 @@
|
||||
package engine
|
||||
|
||||
import (
|
||||
"cmp"
|
||||
"slices"
|
||||
|
||||
"gitea.maximumdirect.net/eric/ConfigKit/internal/document"
|
||||
)
|
||||
|
||||
type inheritanceWalker struct {
|
||||
nodes int // logical non-directive nodes in the entire working tree
|
||||
origins map[string]Origin
|
||||
limits projectionLimits
|
||||
retained *retainedNodeCounter
|
||||
transfers []inheritanceTransfer
|
||||
controls []inheritanceControl
|
||||
replacements []inheritanceReplacement
|
||||
operations []inheritanceOperation
|
||||
}
|
||||
|
||||
// expandInheritanceTree resolves each destination scope after its enclosing
|
||||
// relationships. The loading lifecycle uses the origin-aware entry point.
|
||||
func expandInheritanceTree(input *document.Node, limits projectionLimits) (*inheritanceScope, []Diagnostic) {
|
||||
return expandInheritanceWithOrigins(input, nil, limits)
|
||||
}
|
||||
|
||||
func expandInheritanceWithOrigins(input *document.Node, inputOrigins map[string]Origin, limits projectionLimits) (*inheritanceScope, []Diagnostic) {
|
||||
return expandInheritanceWithBudgets(input, inputOrigins, limits, historyNodeLimit)
|
||||
}
|
||||
|
||||
func expandInheritanceWithBudgets(input *document.Node, inputOrigins map[string]Origin, limits projectionLimits, historyLimit int) (*inheritanceScope, []Diagnostic) {
|
||||
if input == nil || input.Kind != document.Mapping {
|
||||
return nil, []Diagnostic{{Code: CodeInheritanceInvalid, Severity: SeverityError,
|
||||
Message: "inheritance requires a mapping document root"}}
|
||||
}
|
||||
root := document.Clone(input)
|
||||
origins := cloneOriginIndex(inputOrigins)
|
||||
if inputOrigins == nil {
|
||||
origins = directInheritanceOrigins(input, Path{}, nil)
|
||||
}
|
||||
nodes, within := inheritanceLowerBoundCount(root, limits.nodes)
|
||||
if !within {
|
||||
return nil, []Diagnostic{inheritanceBudgetDiagnostic(Path{}, root.Location)}
|
||||
}
|
||||
walker := inheritanceWalker{
|
||||
nodes: nodes, limits: limits, origins: origins,
|
||||
retained: newRetainedNodeCounter(historyLimit),
|
||||
}
|
||||
if problem := walker.visit(&root, Path{}, true); problem != nil {
|
||||
return nil, []Diagnostic{*problem}
|
||||
}
|
||||
if problem := inheritanceFinalBudget(root, limits); problem != nil {
|
||||
return nil, []Diagnostic{*problem}
|
||||
}
|
||||
return &inheritanceScope{
|
||||
root: root, origins: walker.origins, transfers: walker.transfers,
|
||||
controls: walker.controls, replacements: walker.replacements, operations: walker.operations,
|
||||
}, nil
|
||||
}
|
||||
|
||||
func (w *inheritanceWalker) visit(node **document.Node, path Path, unscoped bool) *Diagnostic {
|
||||
switch (*node).Kind {
|
||||
case document.Mapping:
|
||||
original := *node
|
||||
if unscoped {
|
||||
reference, found := inheritanceDirective(*node)
|
||||
if found {
|
||||
referencePath := path.Key(inheritanceKey)
|
||||
location := publicLocation(reference.Value.Location)
|
||||
if reference.Value.Kind != document.Null {
|
||||
problem := inheritanceDiagnostic(CodeInheritanceInvalid, referencePath, location, nil,
|
||||
"inheritance reference has no enclosing sibling scope")
|
||||
return &problem
|
||||
}
|
||||
w.controls = append(w.controls, inheritanceControl{
|
||||
path: referencePath, locations: []Location{location}, cancelled: true,
|
||||
})
|
||||
origin := directInheritanceOrigins(reference.Value, referencePath, w.origins)[referencePath.String()]
|
||||
control := inheritanceControl{path: referencePath, locations: []Location{location}, cancelled: true}
|
||||
w.operations = append(w.operations, inheritanceOperation{control: &control, controlOrigin: origin})
|
||||
deleteOriginSubtree(w.origins, referencePath, reference.Value)
|
||||
*node = withoutInheritanceDirective(*node)
|
||||
}
|
||||
}
|
||||
if scopeHasInheritanceDefinitions(*node) {
|
||||
// Only this scope can change the logical total. Reserve the nodes
|
||||
// outside it so expansion checks the remaining budget before cloning.
|
||||
before, _ := inheritanceLowerBoundCount(*node, -1)
|
||||
limits := w.limits
|
||||
if limits.nodes >= 0 {
|
||||
limits.nodes -= w.nodes - before
|
||||
}
|
||||
resolved, diagnostics := expandSiblingDefinitionsWithOrigins(*node, path, limits, w.origins, w.retained)
|
||||
if len(diagnostics) != 0 {
|
||||
return &diagnostics[0]
|
||||
}
|
||||
*node = resolved.root
|
||||
deleteOriginSubtree(w.origins, path, original)
|
||||
for pointer, origin := range resolved.origins {
|
||||
w.origins[pointer] = origin
|
||||
}
|
||||
w.transfers = append(w.transfers, resolved.transfers...)
|
||||
w.controls = append(w.controls, resolved.controls...)
|
||||
w.replacements = append(w.replacements, resolved.replacements...)
|
||||
w.operations = append(w.operations, resolved.operations...)
|
||||
after, _ := inheritanceLowerBoundCount(*node, -1)
|
||||
w.nodes += after - before
|
||||
}
|
||||
// Operations retain immutable enclosing templates. Detach just this
|
||||
// container before replacing descendants during destination resolution;
|
||||
// untouched subtrees remain shared with the retained evidence.
|
||||
working := **node
|
||||
working.Mapping = slices.Clone(working.Mapping)
|
||||
*node = &working
|
||||
indices := make([]int, len((*node).Mapping))
|
||||
for index := range indices {
|
||||
indices[index] = index
|
||||
}
|
||||
slices.SortFunc(indices, func(left, right int) int {
|
||||
return cmp.Compare((*node).Mapping[left].Key, (*node).Mapping[right].Key)
|
||||
})
|
||||
for _, index := range indices {
|
||||
entry := &(*node).Mapping[index]
|
||||
if problem := w.visit(&entry.Value, path.Key(entry.Key), false); problem != nil {
|
||||
return problem
|
||||
}
|
||||
}
|
||||
case document.Sequence:
|
||||
working := **node
|
||||
working.Sequence = slices.Clone(working.Sequence)
|
||||
*node = &working
|
||||
for index := range (*node).Sequence {
|
||||
child := &(*node).Sequence[index]
|
||||
if problem := w.visit(child, path.Index(index), (*child).Kind == document.Mapping); problem != nil {
|
||||
return problem
|
||||
}
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func scopeHasInheritanceDefinitions(scope *document.Node) bool {
|
||||
for _, entry := range scope.Mapping {
|
||||
if entry.Value.Kind != document.Mapping {
|
||||
continue
|
||||
}
|
||||
if _, found := inheritanceDirective(entry.Value); found {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// A directive may disappear during later destination resolution, so only
|
||||
// non-directive nodes form a safe early lower bound on final output size.
|
||||
func inheritanceLowerBoundWithin(root *document.Node, limit int) bool {
|
||||
_, within := inheritanceLowerBoundCount(root, limit)
|
||||
return within
|
||||
}
|
||||
|
||||
func inheritanceLowerBoundCount(root *document.Node, limit int) (int, bool) {
|
||||
if limit < 0 {
|
||||
limit = int(^uint(0) >> 1)
|
||||
}
|
||||
stack := []*document.Node{root}
|
||||
count := 0
|
||||
for len(stack) != 0 {
|
||||
node := stack[len(stack)-1]
|
||||
stack = stack[:len(stack)-1]
|
||||
if node == nil {
|
||||
continue
|
||||
}
|
||||
count++
|
||||
if count > limit {
|
||||
return count, false
|
||||
}
|
||||
switch node.Kind {
|
||||
case document.Mapping:
|
||||
for _, entry := range node.Mapping {
|
||||
if entry.Key == inheritanceKey {
|
||||
continue
|
||||
}
|
||||
count++ // mapping key
|
||||
stack = append(stack, entry.Value)
|
||||
}
|
||||
case document.Sequence:
|
||||
stack = append(stack, node.Sequence...)
|
||||
}
|
||||
if count > limit {
|
||||
return count, false
|
||||
}
|
||||
}
|
||||
return count, true
|
||||
}
|
||||
|
||||
// Count the materialized mapping shape before cloning a memoized parent.
|
||||
// Relationship fields are excluded because later destination resolution
|
||||
// consumes them; their omission makes this a safe lower bound.
|
||||
func inheritanceOverlayLowerBoundWithin(lower, upper *document.Node, limit int) bool {
|
||||
if limit < 0 {
|
||||
return false
|
||||
}
|
||||
index := new(mappingIndex)
|
||||
count := 0
|
||||
var visit func(*document.Node, *document.Node) bool
|
||||
visit = func(base, incoming *document.Node) bool {
|
||||
if base.Kind != document.Mapping || incoming.Kind != document.Mapping {
|
||||
added, within := inheritanceLowerBoundCount(incoming, limit-count)
|
||||
count += added
|
||||
return within
|
||||
}
|
||||
count++ // mapping node
|
||||
if count > limit {
|
||||
return false
|
||||
}
|
||||
for _, entry := range base.Mapping {
|
||||
if entry.Key == inheritanceKey {
|
||||
continue
|
||||
}
|
||||
count++ // mapping key
|
||||
if count > limit {
|
||||
return false
|
||||
}
|
||||
position := index.find(incoming, entry.Key)
|
||||
if position >= 0 {
|
||||
if !visit(entry.Value, incoming.Mapping[position].Value) {
|
||||
return false
|
||||
}
|
||||
} else {
|
||||
added, within := inheritanceLowerBoundCount(entry.Value, limit-count)
|
||||
count += added
|
||||
if !within {
|
||||
return false
|
||||
}
|
||||
}
|
||||
}
|
||||
for _, entry := range incoming.Mapping {
|
||||
if entry.Key == inheritanceKey || index.find(base, entry.Key) >= 0 {
|
||||
continue
|
||||
}
|
||||
count++ // mapping key
|
||||
if count > limit {
|
||||
return false
|
||||
}
|
||||
added, within := inheritanceLowerBoundCount(entry.Value, limit-count)
|
||||
count += added
|
||||
if !within || count > limit {
|
||||
return false
|
||||
}
|
||||
}
|
||||
return true
|
||||
}
|
||||
return visit(lower, upper)
|
||||
}
|
||||
|
||||
func inheritanceFinalBudget(root *document.Node, limits projectionLimits) *Diagnostic {
|
||||
type pending struct {
|
||||
node *document.Node
|
||||
path Path
|
||||
}
|
||||
stack := []pending{{node: root}}
|
||||
count := 0
|
||||
for len(stack) != 0 {
|
||||
item := stack[len(stack)-1]
|
||||
stack = stack[:len(stack)-1]
|
||||
if len(item.path.segments) > limits.depth {
|
||||
problem := inheritanceBudgetDiagnostic(item.path, item.node.Location)
|
||||
return &problem
|
||||
}
|
||||
count++
|
||||
if limits.nodes >= 0 && count > limits.nodes {
|
||||
problem := inheritanceBudgetDiagnostic(item.path, item.node.Location)
|
||||
return &problem
|
||||
}
|
||||
switch item.node.Kind {
|
||||
case document.Mapping:
|
||||
count += len(item.node.Mapping) // mapping keys
|
||||
for index := len(item.node.Mapping) - 1; index >= 0; index-- {
|
||||
entry := item.node.Mapping[index]
|
||||
stack = append(stack, pending{node: entry.Value, path: item.path.Key(entry.Key)})
|
||||
}
|
||||
case document.Sequence:
|
||||
for index := len(item.node.Sequence) - 1; index >= 0; index-- {
|
||||
stack = append(stack, pending{node: item.node.Sequence[index], path: item.path.Index(index)})
|
||||
}
|
||||
}
|
||||
if limits.nodes >= 0 && count > limits.nodes {
|
||||
problem := inheritanceBudgetDiagnostic(item.path, item.node.Location)
|
||||
return &problem
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func inheritanceBudgetDiagnostic(path Path, location document.Location) Diagnostic {
|
||||
primary := publicLocation(location)
|
||||
return Diagnostic{
|
||||
Code: CodeSchemaProjectionLimit, Severity: SeverityError, Path: path,
|
||||
Message: "expanded configuration exceeds the projection limit", Primary: &primary,
|
||||
}
|
||||
}
|
||||
|
||||
func inheritanceHistoryBudgetDiagnostic(path Path, location document.Location) Diagnostic {
|
||||
primary := publicLocation(location)
|
||||
return Diagnostic{
|
||||
Code: CodeExplainLimitExceeded, Severity: SeverityError, Path: path,
|
||||
Message: "retained inheritance history exceeds the node limit", Primary: &primary,
|
||||
}
|
||||
}
|
||||
219
internal/engine/inheritance_walk_test.go
Normal file
219
internal/engine/inheritance_walk_test.go
Normal file
@@ -0,0 +1,219 @@
|
||||
package engine
|
||||
|
||||
import (
|
||||
"reflect"
|
||||
"testing"
|
||||
|
||||
"gitea.maximumdirect.net/eric/ConfigKit/internal/document"
|
||||
)
|
||||
|
||||
func TestNestedInheritanceUsesEachDestinationScope(t *testing.T) {
|
||||
input := parseCompositionDocument(t, "source-1", `presets:
|
||||
base:
|
||||
settings:
|
||||
primary: {width: 10}
|
||||
secondary: {extends: primary}
|
||||
custom:
|
||||
extends: base
|
||||
settings:
|
||||
primary: {width: 20}
|
||||
`)
|
||||
before := document.Clone(input)
|
||||
result, diagnostics := expandInheritanceTree(input, defaultProjectionLimits())
|
||||
if len(diagnostics) != 0 {
|
||||
t.Fatal(diagnostics)
|
||||
}
|
||||
for path, want := range map[string]string{
|
||||
"/presets/base/settings/secondary/width": "10",
|
||||
"/presets/custom/settings/secondary/width": "20",
|
||||
} {
|
||||
parsed, _ := ParsePath(path)
|
||||
node, found := documentAtPath(result.root, parsed)
|
||||
if !found || node.Text != want {
|
||||
t.Errorf("%s = %#v, found %v", path, node, found)
|
||||
}
|
||||
}
|
||||
if len(result.transfers) != 3 || !reflect.DeepEqual(input, before) {
|
||||
t.Fatalf("transfers = %#v; input changed = %v", result.transfers, !reflect.DeepEqual(input, before))
|
||||
}
|
||||
if node, found := documentAtPath(result.root, Keys("presets", "custom", "settings", "secondary", inheritanceKey)); found {
|
||||
t.Fatalf("nested directive survived: %#v", node)
|
||||
}
|
||||
}
|
||||
|
||||
func TestNestedInheritanceSupportsChainsCancellationAndContainerReplacement(t *testing.T) {
|
||||
input := parseCompositionDocument(t, "source-1", `catalog:
|
||||
alpha:
|
||||
settings:
|
||||
first: {value: one}
|
||||
second: {extends: first}
|
||||
beta:
|
||||
extends: alpha
|
||||
settings:
|
||||
first: {value: two}
|
||||
gamma:
|
||||
extends: beta
|
||||
settings:
|
||||
first: {value: three}
|
||||
cancelled:
|
||||
extends: alpha
|
||||
settings:
|
||||
second: {extends: null, value: own}
|
||||
replaced:
|
||||
extends: alpha
|
||||
settings: none
|
||||
`)
|
||||
result, diagnostics := expandInheritanceTree(input, defaultProjectionLimits())
|
||||
if len(diagnostics) != 0 {
|
||||
t.Fatal(diagnostics)
|
||||
}
|
||||
checks := map[string]string{
|
||||
"/catalog/alpha/settings/second/value": "one",
|
||||
"/catalog/beta/settings/second/value": "two",
|
||||
"/catalog/gamma/settings/second/value": "three",
|
||||
"/catalog/cancelled/settings/second/value": "own",
|
||||
"/catalog/replaced/settings": "none",
|
||||
}
|
||||
for pointer, want := range checks {
|
||||
path, _ := ParsePath(pointer)
|
||||
node, found := documentAtPath(result.root, path)
|
||||
if !found || node.Text != want {
|
||||
t.Errorf("%s = %#v, found %v", pointer, node, found)
|
||||
}
|
||||
}
|
||||
if _, found := documentAtPath(result.root, Keys("catalog", "replaced", "settings", "second")); found {
|
||||
t.Fatal("replaced container retained an active child")
|
||||
}
|
||||
if _, found := documentAtPath(result.root, Keys("catalog", "cancelled", "settings", "second", inheritanceKey)); found {
|
||||
t.Fatal("cancelled nested directive survived")
|
||||
}
|
||||
}
|
||||
|
||||
func TestNestedInheritanceCanReplaceAnInheritedRelationship(t *testing.T) {
|
||||
input := parseCompositionDocument(t, "source-1", `base:
|
||||
settings:
|
||||
first: {value: old}
|
||||
other: {value: new}
|
||||
selected: {extends: first}
|
||||
custom:
|
||||
extends: base
|
||||
settings:
|
||||
selected: {extends: other}
|
||||
`)
|
||||
result, diagnostics := expandInheritanceTree(input, defaultProjectionLimits())
|
||||
if len(diagnostics) != 0 {
|
||||
t.Fatal(diagnostics)
|
||||
}
|
||||
base, found := documentAtPath(result.root, Keys("base", "settings", "selected", "value"))
|
||||
if !found || base.Text != "old" {
|
||||
t.Fatalf("base relationship = %#v, %v", base, found)
|
||||
}
|
||||
custom, found := documentAtPath(result.root, Keys("custom", "settings", "selected", "value"))
|
||||
if !found || custom.Text != "new" {
|
||||
t.Fatalf("replaced relationship = %#v, %v", custom, found)
|
||||
}
|
||||
}
|
||||
|
||||
func TestNestedInheritanceRejectsDestinationCycleAndInvalidPositions(t *testing.T) {
|
||||
tests := []struct {
|
||||
name, input, path string
|
||||
code Code
|
||||
}{
|
||||
{
|
||||
name: "destination-only cycle", input: `base:
|
||||
settings:
|
||||
a: {value: one}
|
||||
b: {extends: a}
|
||||
custom:
|
||||
extends: base
|
||||
settings:
|
||||
a: {extends: b}
|
||||
`, path: "/custom/settings/b/extends", code: CodeInheritanceCycle,
|
||||
},
|
||||
{"root reference", "extends: anything\nentry: {value: one}\n", "/extends", CodeInheritanceInvalid},
|
||||
{"list element reference", "items: [{extends: anything, value: one}]\n", "/items/0/extends", CodeInheritanceInvalid},
|
||||
}
|
||||
for _, test := range tests {
|
||||
t.Run(test.name, func(t *testing.T) {
|
||||
input := parseCompositionDocument(t, "source-1", test.input)
|
||||
result, diagnostics := expandInheritanceTree(input, defaultProjectionLimits())
|
||||
if result != nil || len(diagnostics) != 1 || diagnostics[0].Code != test.code ||
|
||||
diagnostics[0].Path.String() != test.path || diagnostics[0].Primary == nil {
|
||||
t.Fatalf("result = %#v, diagnostics = %#v", result, diagnostics)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestNestedInheritanceResolvesScopesInsideSequences(t *testing.T) {
|
||||
input := parseCompositionDocument(t, "source-1", `items:
|
||||
- primary: {value: first}
|
||||
secondary: {extends: primary}
|
||||
- extends: null
|
||||
zeta: {extends: alpha}
|
||||
alpha: {value: second}
|
||||
`)
|
||||
result, diagnostics := expandInheritanceTree(input, defaultProjectionLimits())
|
||||
if len(diagnostics) != 0 {
|
||||
t.Fatal(diagnostics)
|
||||
}
|
||||
first, found := documentAtPath(result.root, Keys("items").Index(0).Key("secondary").Key("value"))
|
||||
if !found || first.Text != "first" {
|
||||
t.Fatalf("first sequence scope = %#v, %v", first, found)
|
||||
}
|
||||
second, found := documentAtPath(result.root, Keys("items").Index(1).Key("zeta").Key("value"))
|
||||
if !found || second.Text != "second" {
|
||||
t.Fatalf("second sequence scope = %#v, %v", second, found)
|
||||
}
|
||||
if _, found := documentAtPath(result.root, Keys("items").Index(1).Key(inheritanceKey)); found {
|
||||
t.Fatal("sequence-element cancellation survived")
|
||||
}
|
||||
}
|
||||
|
||||
func TestInheritanceExpansionHonorsOutputNodeAndDepthBudgets(t *testing.T) {
|
||||
input := parseCompositionDocument(t, "source-1", "base: {value: one}\nchild: {extends: base}\n")
|
||||
for _, test := range []struct {
|
||||
name string
|
||||
limits projectionLimits
|
||||
valid bool
|
||||
}{
|
||||
{name: "at node boundary", limits: projectionLimits{nodes: 9, depth: 2}, valid: true},
|
||||
{name: "beyond node boundary", limits: projectionLimits{nodes: 8, depth: 2}},
|
||||
{name: "at depth boundary", limits: projectionLimits{nodes: 20, depth: 2}, valid: true},
|
||||
{name: "beyond depth boundary", limits: projectionLimits{nodes: 20, depth: 1}},
|
||||
} {
|
||||
t.Run(test.name, func(t *testing.T) {
|
||||
result, diagnostics := expandInheritanceTree(input, test.limits)
|
||||
if test.valid {
|
||||
if result == nil || len(diagnostics) != 0 {
|
||||
t.Fatalf("result = %#v, diagnostics = %#v", result, diagnostics)
|
||||
}
|
||||
} else if result != nil || len(diagnostics) != 1 || diagnostics[0].Code != CodeSchemaProjectionLimit {
|
||||
t.Fatalf("result = %#v, diagnostics = %#v", result, diagnostics)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestInheritanceBudgetIncludesOtherScopesAndSequenceItems(t *testing.T) {
|
||||
input := parseCompositionDocument(t, "source-1", `extends: null
|
||||
groups:
|
||||
- a: {x: one}
|
||||
b: {extends: a}
|
||||
- a: {x: two}
|
||||
b: {extends: a}
|
||||
unchanged: {x: three}
|
||||
`)
|
||||
// Root (1), groups key/list (2), two expanded scopes (18), and
|
||||
// unchanged key/mapping/key/value (4). Consumed controls cost nothing.
|
||||
for _, limit := range []int{24, 25, 26} {
|
||||
result, diagnostics := expandInheritanceTree(input, projectionLimits{nodes: limit, depth: 5})
|
||||
if limit < 25 {
|
||||
if result != nil || len(diagnostics) != 1 || diagnostics[0].Code != CodeSchemaProjectionLimit {
|
||||
t.Fatalf("limit %d accepted or wrong error: %#v", limit, diagnostics)
|
||||
}
|
||||
} else if result == nil || len(diagnostics) != 0 {
|
||||
t.Fatalf("limit %d rejected: %#v", limit, diagnostics)
|
||||
}
|
||||
}
|
||||
}
|
||||
50
internal/engine/layered.go
Normal file
50
internal/engine/layered.go
Normal file
@@ -0,0 +1,50 @@
|
||||
package engine
|
||||
|
||||
import "os"
|
||||
|
||||
// Input selects one source and its independent import policy.
|
||||
type Input struct {
|
||||
Source Source
|
||||
Imports bool
|
||||
Filesystem *FilesystemContext
|
||||
}
|
||||
|
||||
// Underlay contains strictly composed contributions at one precedence tier.
|
||||
type Underlay struct{ Inputs []Input }
|
||||
|
||||
// LayeredOptions selects lower-priority tiers and final application rules.
|
||||
type LayeredOptions[T any] struct {
|
||||
Definition Definition[T]
|
||||
Underlays []Underlay
|
||||
Overlay string
|
||||
OverlayFilesystem *FilesystemContext
|
||||
Overrides []Override
|
||||
}
|
||||
|
||||
// LoadLayered loads a main input and lower-priority grouped inputs through one
|
||||
// final decode, resolution, and validation lifecycle.
|
||||
func LoadLayered[T any](main Input, secrets SecretBinding, supplied ...LayeredOptions[T]) (*Loaded[T], error) {
|
||||
if len(supplied) > 1 {
|
||||
return nil, (&diagnosticAccumulator{diagnostics: []Diagnostic{optionDiagnostic(CodeOptionsInvalid, "LoadLayered accepts at most one options value")}}).loadError()
|
||||
}
|
||||
var options LayeredOptions[T]
|
||||
if len(supplied) != 0 {
|
||||
options = supplied[0]
|
||||
}
|
||||
request := loadRequest[T]{
|
||||
main: loadInput{source: main.Source, imports: main.Imports, filesystem: main.Filesystem},
|
||||
definition: options.Definition, overlay: options.Overlay,
|
||||
overlayFilesystem: options.OverlayFilesystem,
|
||||
overrides: append([]Override(nil), options.Overrides...),
|
||||
underlays: make([]loadGroup, len(options.Underlays)),
|
||||
}
|
||||
for index, underlay := range options.Underlays {
|
||||
request.underlays[index].inputs = make([]loadInput, len(underlay.Inputs))
|
||||
for inputIndex, input := range underlay.Inputs {
|
||||
request.underlays[index].inputs[inputIndex] = loadInput{
|
||||
source: input.Source, imports: input.Imports, filesystem: input.Filesystem,
|
||||
}
|
||||
}
|
||||
}
|
||||
return loadGroupedWithSecretLookup(request, secrets, os.LookupEnv)
|
||||
}
|
||||
@@ -33,6 +33,7 @@ type Options[T any] struct {
|
||||
type FilesystemContext struct {
|
||||
BaseDir string
|
||||
Root string
|
||||
FS *Filesystem
|
||||
}
|
||||
|
||||
// Override describes one explicitly supplied application override. Value uses
|
||||
@@ -77,6 +78,7 @@ type Loaded[T any] struct {
|
||||
type Inspection struct {
|
||||
metadata Metadata
|
||||
sources []SourceInfo
|
||||
contexts map[string]ResourceContext
|
||||
diagnostics []Diagnostic
|
||||
baseline *document.Node
|
||||
input *document.Node
|
||||
@@ -84,6 +86,7 @@ type Inspection struct {
|
||||
final *document.Node
|
||||
resolution []resolutionSnapshot
|
||||
applications []historyApplication
|
||||
inheritance []inheritanceOperation
|
||||
secrets map[string]bool
|
||||
secretReferences map[string]string
|
||||
}
|
||||
@@ -104,6 +107,16 @@ func (r *Inspection) Sources() []SourceInfo {
|
||||
return cloneSourceInfos(r.sources)
|
||||
}
|
||||
|
||||
// SourceContext returns a passive filesystem view for an acquired source.
|
||||
// Unknown IDs and sources without a filesystem context return false.
|
||||
func (r *Inspection) SourceContext(id string) (ResourceContext, bool) {
|
||||
if r == nil {
|
||||
return ResourceContext{}, false
|
||||
}
|
||||
context, ok := r.contexts[id]
|
||||
return context, ok
|
||||
}
|
||||
|
||||
// Diagnostics returns detached nonfatal diagnostics from the load.
|
||||
func (r *Inspection) Diagnostics() []Diagnostic {
|
||||
if r == nil {
|
||||
@@ -120,14 +133,27 @@ func Load[T any](source Source, secrets SecretBinding, supplied ...Options[T]) (
|
||||
}
|
||||
|
||||
func loadWithSecretLookup[T any](source Source, secrets SecretBinding, lookup secretLookup, supplied ...Options[T]) (*Loaded[T], error) {
|
||||
accumulator := diagnosticAccumulator{}
|
||||
options, diagnostics := selectOptions(source, supplied)
|
||||
accumulator.add(diagnostics...)
|
||||
if len(diagnostics) != 0 {
|
||||
return nil, accumulator.loadError()
|
||||
return nil, (&diagnosticAccumulator{diagnostics: diagnostics}).loadError()
|
||||
}
|
||||
request := loadRequest[T]{
|
||||
main: loadInput{source: source, imports: options.Imports, filesystem: options.Filesystem},
|
||||
definition: options.Definition, overlay: options.Overlay,
|
||||
overrides: options.Overrides, ordinary: true,
|
||||
}
|
||||
return executeLoadRequest(request, secrets, lookup)
|
||||
}
|
||||
|
||||
func executeLoadRequest[T any](request loadRequest[T], secrets SecretBinding, lookup secretLookup) (*Loaded[T], error) {
|
||||
accumulator := diagnosticAccumulator{}
|
||||
accounting := &importAccounting{}
|
||||
mainOptions := Options[T]{Definition: request.definition, Imports: request.main.imports, Filesystem: request.main.filesystem}
|
||||
if request.ordinary {
|
||||
mainOptions.Overlay = request.overlay
|
||||
}
|
||||
|
||||
input, diagnostics := assembleBaseInput(source, options)
|
||||
input, diagnostics := assembleBaseInputTracked(request.main.source, mainOptions, accounting, nil, nil)
|
||||
accumulator.sources = input.sources
|
||||
accumulator.add(diagnostics...)
|
||||
if len(diagnostics) != 0 {
|
||||
@@ -138,29 +164,68 @@ func loadWithSecretLookup[T any](source Source, secrets SecretBinding, lookup se
|
||||
accumulator.add(diagnosticFromSchemaIssue(schemaIssue))
|
||||
return nil, accumulator.loadError()
|
||||
}
|
||||
versionDiagnostics := checkVersion(options.Definition.Version, input.document)
|
||||
versionDiagnostics := checkVersion(request.definition.Version, input.document)
|
||||
accumulator.add(versionDiagnostics...)
|
||||
if hasErrors(versionDiagnostics) {
|
||||
return nil, accumulator.loadError()
|
||||
}
|
||||
input, diagnostics = applyInputLayers(source, options, input)
|
||||
if request.ordinary {
|
||||
input, diagnostics = applyInputLayers(request.main.source, Options[T]{Definition: request.definition, Overlay: request.overlay, Filesystem: request.main.filesystem, Overrides: request.overrides}, input)
|
||||
} else {
|
||||
input, diagnostics = assembleLayeredInput(request, input, accounting)
|
||||
}
|
||||
accumulator.sources = input.sources
|
||||
accumulator.add(diagnostics...)
|
||||
if len(diagnostics) != 0 {
|
||||
return nil, accumulator.loadError()
|
||||
}
|
||||
if request.ordinary && input.parsed.source != nil {
|
||||
baseSources := make([]SourceInfo, 0, len(input.sources))
|
||||
for _, source := range input.sources {
|
||||
if source.Layer == LayerBase {
|
||||
baseSources = append(baseSources, source)
|
||||
}
|
||||
}
|
||||
if len(baseSources) != 0 && input.contexts[baseSources[0].ID].FS == nil {
|
||||
contexts, diagnostic := contextsForAcquiredSources(request.main.source, request.main.filesystem, input.parsed.source, baseSources)
|
||||
if diagnostic != nil {
|
||||
accumulator.add(*diagnostic)
|
||||
return nil, accumulator.loadError()
|
||||
}
|
||||
mergeResourceContexts(&input.contexts, contexts)
|
||||
}
|
||||
}
|
||||
if input.document != nil && containsInheritanceDirective(input.document) {
|
||||
composition, problems := compositionForInput(input)
|
||||
if len(problems) != 0 {
|
||||
accumulator.add(problems...)
|
||||
return nil, accumulator.loadError()
|
||||
}
|
||||
expanded, problems := expandInheritanceWithOrigins(composition.root, composition.origins, defaultProjectionLimits())
|
||||
if len(problems) != 0 {
|
||||
accumulator.add(problems...)
|
||||
return nil, accumulator.loadError()
|
||||
}
|
||||
if problem := inheritanceVersionDiagnostic(request.definition.Version, expanded); problem != nil {
|
||||
accumulator.add(*problem)
|
||||
return nil, accumulator.loadError()
|
||||
}
|
||||
input.composition = composition
|
||||
input.inheritance = expanded
|
||||
input.document = expanded.root
|
||||
}
|
||||
|
||||
decoded, diagnostics := decodeLoadInput(schema, options.Definition, input)
|
||||
decoded, diagnostics := decodeLoadInput(schema, request.definition, input)
|
||||
accumulator.add(diagnostics...)
|
||||
if len(diagnostics) != 0 {
|
||||
return nil, accumulator.loadError()
|
||||
}
|
||||
resolved, diagnostics := resolveLoadInput(schema, options.Definition, input, decoded)
|
||||
resolved, diagnostics := resolveLoadInput(schema, request.definition, input, decoded)
|
||||
accumulator.add(diagnostics...)
|
||||
if hasErrors(diagnostics) {
|
||||
return nil, accumulator.loadError()
|
||||
}
|
||||
if !historyNodesWithinLimit(decoded.baseline, decoded.final, resolved.final, resolved.applications, resolved.snapshots) {
|
||||
if !historyNodesWithinLimit(decoded.baseline, decoded.final, resolved.final, resolved.applications, resolved.snapshots, input.inheritance) {
|
||||
accumulator.add(Diagnostic{Code: CodeExplainLimitExceeded, Severity: SeverityError, Message: "retained configuration history exceeds the node limit"})
|
||||
return nil, accumulator.loadError()
|
||||
}
|
||||
@@ -176,8 +241,8 @@ func loadWithSecretLookup[T any](source Source, secrets SecretBinding, lookup se
|
||||
if hasErrors(secretResolution.diagnostics) {
|
||||
return nil, accumulator.loadError()
|
||||
}
|
||||
if options.Definition.Validate != nil {
|
||||
diagnostics = runValidation(&configuration, options.Definition.Validate, schema, resolved.final, input.document, secretResolution.lookups)
|
||||
if request.definition.Validate != nil {
|
||||
diagnostics = runValidation(&configuration, request.definition.Validate, schema, resolved.final, input.document, secretResolution.lookups)
|
||||
accumulator.add(diagnostics...)
|
||||
if hasErrors(diagnostics) {
|
||||
return nil, accumulator.loadError()
|
||||
@@ -186,5 +251,5 @@ func loadWithSecretLookup[T any](source Source, secrets SecretBinding, lookup se
|
||||
resolved.configuration = configuration
|
||||
// Publication detaches equal-valued validator replacements, preserving the
|
||||
// validated secret state, then finalizes metadata and sensitivity.
|
||||
return publishLoadResult(schema, options.Definition, resolved, secretResolution.references, &accumulator)
|
||||
return publishLoadResult(schema, request.definition, resolved, secretResolution.references, &accumulator)
|
||||
}
|
||||
|
||||
@@ -1,24 +1,37 @@
|
||||
package engine
|
||||
|
||||
import "gitea.maximumdirect.net/eric/ConfigKit/internal/document"
|
||||
import (
|
||||
"strconv"
|
||||
|
||||
"gitea.maximumdirect.net/eric/ConfigKit/internal/document"
|
||||
)
|
||||
|
||||
// assembledInput is the source/document handoff across schema and version checks.
|
||||
type assembledInput struct {
|
||||
parsed parsedSource
|
||||
document *document.Node
|
||||
composition *composedDocument
|
||||
inheritance *inheritanceScope
|
||||
sources []SourceInfo
|
||||
contexts map[string]ResourceContext
|
||||
}
|
||||
|
||||
func assembleBaseInput[T any](source Source, options Options[T]) (assembledInput, []Diagnostic) {
|
||||
parsed, diagnostics := parseSource(source, "source-1", defaultParseLimits())
|
||||
return assembleBaseInputTracked(source, options, &importAccounting{}, nil, nil)
|
||||
}
|
||||
|
||||
func assembleBaseInputTracked[T any](source Source, options Options[T], accounting *importAccounting, visited map[resourceIdentity]importRoute, hostToken *namespaceToken) (assembledInput, []Diagnostic) {
|
||||
limits := defaultImportLimits()
|
||||
parseLimits := defaultParseLimits()
|
||||
parseLimits.source.remaining = limits.bytes - accounting.bytes
|
||||
parsed, diagnostics := parseSource(source, "source-"+strconv.Itoa(accounting.sources+1), parseLimits)
|
||||
input := assembledInput{parsed: parsed, document: parsed.document}
|
||||
if len(diagnostics) != 0 {
|
||||
input.sources = sourceInfos(parsed.source)
|
||||
return input, diagnostics
|
||||
}
|
||||
if options.Imports {
|
||||
input.composition, input.sources, diagnostics = loadImportedDocuments(source, options.Filesystem, parsed, options.Definition.Version, defaultImportLimits())
|
||||
input.composition, input.sources, diagnostics = loadImportedDocumentsTracked(source, options.Filesystem, parsed, options.Definition.Version, limits, accounting, visited, hostToken)
|
||||
if len(diagnostics) != 0 {
|
||||
return input, diagnostics
|
||||
}
|
||||
@@ -35,10 +48,44 @@ func assembleBaseInput[T any](source Source, options Options[T]) (assembledInput
|
||||
if diagnostics := disabledDirectiveDiagnostics(input.document, Path{}); len(diagnostics) != 0 {
|
||||
return input, diagnostics
|
||||
}
|
||||
if accounting.sources+1 > limits.sources || accounting.bytes+parsed.bytes > limits.bytes || accounting.nodes+countInputNodes(input.document) > limits.nodes {
|
||||
return input, []Diagnostic{sourceDiagnostic(CodeSourceLimitExceeded, parsed.source.ID, "configuration exceeds the aggregate input limit")}
|
||||
}
|
||||
if diagnostic := registerAcquiredSource(source, options.Filesystem, parsed.source, visited, hostToken); diagnostic != nil {
|
||||
return input, []Diagnostic{*diagnostic}
|
||||
}
|
||||
accounting.sources++
|
||||
accounting.bytes += parsed.bytes
|
||||
accounting.nodes += countInputNodes(input.document)
|
||||
}
|
||||
if !activeOverlay(options.Overlay) || options.Imports {
|
||||
contexts, diagnostic := contextsForAcquiredSources(source, options.Filesystem, parsed.source, input.sources)
|
||||
if diagnostic != nil {
|
||||
return input, []Diagnostic{*diagnostic}
|
||||
}
|
||||
input.contexts = contexts
|
||||
}
|
||||
return input, nil
|
||||
}
|
||||
|
||||
func registerAcquiredSource(source Source, context *FilesystemContext, info *SourceInfo, visited map[resourceIdentity]importRoute, hostToken *namespaceToken) *Diagnostic {
|
||||
if visited == nil || info == nil || info.Path == "" {
|
||||
return nil
|
||||
}
|
||||
identity := resourceIdentity{namespace: hostToken, path: info.Path}
|
||||
if source.kind == sourceFilesystem {
|
||||
identity.namespace = source.namespace.token
|
||||
} else if context != nil && context.FS != nil {
|
||||
identity.namespace = context.FS.namespace.token
|
||||
}
|
||||
if previous, repeated := visited[identity]; repeated {
|
||||
value := importRepeatedDiagnostic(Location{SourceID: info.ID}, previous, importRoute{})
|
||||
return &value
|
||||
}
|
||||
visited[identity] = importRoute{canonical: info.Path}
|
||||
return nil
|
||||
}
|
||||
|
||||
func applyInputLayers[T any](source Source, options Options[T], input assembledInput) (assembledInput, []Diagnostic) {
|
||||
if !activeOverlay(options.Overlay) && len(options.Overrides) == 0 {
|
||||
return input, nil
|
||||
@@ -47,25 +94,31 @@ func applyInputLayers[T any](source Source, options Options[T], input assembledI
|
||||
if len(diagnostics) != 0 {
|
||||
return input, diagnostics
|
||||
}
|
||||
assembly := newAssemblyAccumulator(composition)
|
||||
if activeOverlay(options.Overlay) {
|
||||
composition, input.sources, diagnostics = loadSelectedOverlayDocument(
|
||||
source, options.Filesystem, input.parsed, composition, input.sources,
|
||||
input.sources, diagnostics = loadSelectedOverlayInto(
|
||||
source, options.Filesystem, input.parsed, assembly, input.sources,
|
||||
options.Overlay, options.Definition.Version, defaultImportLimits(),
|
||||
)
|
||||
if len(diagnostics) != 0 {
|
||||
return input, diagnostics
|
||||
}
|
||||
contexts, diagnostic := contextsForAcquiredSources(source, options.Filesystem, input.parsed.source, input.sources[len(input.sources)-1:])
|
||||
if diagnostic != nil {
|
||||
return input, []Diagnostic{*diagnostic}
|
||||
}
|
||||
mergeResourceContexts(&input.contexts, contexts)
|
||||
}
|
||||
if len(options.Overrides) != 0 {
|
||||
composition, input.sources, diagnostics = applyOverrides(
|
||||
composition, input.sources, options.Overrides, options.Definition.Version, defaultImportLimits(),
|
||||
input.sources, diagnostics = assembly.applyOverrides(
|
||||
input.sources, options.Overrides, options.Definition.Version, defaultImportLimits(),
|
||||
)
|
||||
if len(diagnostics) != 0 {
|
||||
return input, diagnostics
|
||||
}
|
||||
}
|
||||
input.composition = composition
|
||||
input.document = composition.root
|
||||
input.composition = assembly.current
|
||||
input.document = assembly.current.root
|
||||
return input, nil
|
||||
}
|
||||
|
||||
|
||||
@@ -12,9 +12,10 @@ import (
|
||||
type resolutionInput struct {
|
||||
metadata Metadata
|
||||
sources map[string]SourceInfo
|
||||
contexts map[string]ResourceContext
|
||||
}
|
||||
|
||||
func newResolutionInput(metadata Metadata, final *document.Node, sources []SourceInfo) resolutionInput {
|
||||
func newResolutionInput(metadata Metadata, final *document.Node, sources []SourceInfo, contexts ...map[string]ResourceContext) resolutionInput {
|
||||
frozen := Metadata{
|
||||
fields: make(map[string]FieldInfo, len(metadata.fields)),
|
||||
effectivePaths: effectivePathIndex(final),
|
||||
@@ -32,7 +33,11 @@ func newResolutionInput(metadata Metadata, final *document.Node, sources []Sourc
|
||||
}
|
||||
indexed[source.ID] = cloneSourceInfo(source)
|
||||
}
|
||||
return resolutionInput{metadata: frozen, sources: indexed}
|
||||
result := resolutionInput{metadata: frozen, sources: indexed}
|
||||
if len(contexts) != 0 {
|
||||
result.contexts = cloneResourceContexts(contexts[0])
|
||||
}
|
||||
return result
|
||||
}
|
||||
|
||||
func (input resolutionInput) lookup(path Path) (FieldInfo, bool) {
|
||||
@@ -47,6 +52,11 @@ func (input resolutionInput) source(id string) (SourceInfo, bool) {
|
||||
return cloneSourceInfo(source), true
|
||||
}
|
||||
|
||||
func (input resolutionInput) sourceContext(id string) (ResourceContext, bool) {
|
||||
context, ok := input.contexts[id]
|
||||
return context, ok
|
||||
}
|
||||
|
||||
func metadataFromDecoded(decoded decodedConfiguration) Metadata {
|
||||
metadata := Metadata{
|
||||
fields: make(map[string]FieldInfo),
|
||||
@@ -115,9 +125,6 @@ func applyOverlayMetadata(metadata *Metadata, records []overlayRecord) {
|
||||
if err != nil {
|
||||
continue
|
||||
}
|
||||
if history := events[pointer]; len(history) != 0 && history[len(history)-1].Origin.Layer == info.Origin.Layer && history[len(history)-1].Origin.Operation == info.Origin.Operation {
|
||||
continue
|
||||
}
|
||||
appendInputOriginEvent(events, baseline, pointer, path, info.Origin)
|
||||
}
|
||||
metadata.history = make(map[string][]Change, len(baseline)+len(events))
|
||||
@@ -134,12 +141,37 @@ func appendInputOriginEvent(events map[string][]Change, baseline map[string][]Ch
|
||||
return
|
||||
}
|
||||
history := events[pointer]
|
||||
if len(history) != 0 && originContainedBy(history[len(history)-1].Origin, origin) {
|
||||
if len(history) != 0 && originRepresentedByChange(path, origin, history[len(history)-1]) {
|
||||
return
|
||||
}
|
||||
appendInputChangeEvent(events, baseline, pointer, Change{Path: path.clone(), Origin: cloneOrigin(origin)})
|
||||
}
|
||||
|
||||
func originRepresentedByChange(path Path, origin Origin, change Change) bool {
|
||||
if change.Path.String() == path.String() {
|
||||
return originContainedBy(origin, change.Origin)
|
||||
}
|
||||
if !pathContains(change.Path, path) || origin.Layer != change.Origin.Layer || origin.Operation != change.Origin.Operation {
|
||||
return false
|
||||
}
|
||||
// A subtree replacement already records its descendants' contributions.
|
||||
// Their coordinates differ from the container's, but their sources must be
|
||||
// covered by the operation; matching only the precedence role is insufficient.
|
||||
for _, location := range origin.Locations {
|
||||
found := false
|
||||
for _, recorded := range change.Origin.Locations {
|
||||
if location.SourceID == recorded.SourceID {
|
||||
found = true
|
||||
break
|
||||
}
|
||||
}
|
||||
if !found {
|
||||
return false
|
||||
}
|
||||
}
|
||||
return true
|
||||
}
|
||||
|
||||
func appendInputChangeEvent(events map[string][]Change, baseline map[string][]Change, pointer string, value Change) {
|
||||
change := cloneChange(value)
|
||||
history := events[pointer]
|
||||
@@ -169,6 +201,23 @@ func originContainedBy(candidate, container Origin) bool {
|
||||
return false
|
||||
}
|
||||
}
|
||||
if (len(candidate.contributors) == 0) != (len(container.contributors) == 0) {
|
||||
return false
|
||||
}
|
||||
for _, contributor := range candidate.contributors {
|
||||
found := false
|
||||
for _, available := range container.contributors {
|
||||
if contributor.location == available.location &&
|
||||
slices.Equal(contributor.source.segments, available.source.segments) &&
|
||||
sameInheritanceTrace(contributor.trace, available.trace) {
|
||||
found = true
|
||||
break
|
||||
}
|
||||
}
|
||||
if !found {
|
||||
return false
|
||||
}
|
||||
}
|
||||
return true
|
||||
}
|
||||
|
||||
@@ -205,8 +254,9 @@ func cloneMetadata(metadata Metadata) Metadata {
|
||||
return Metadata{}
|
||||
}
|
||||
cloned := Metadata{
|
||||
fields: make(map[string]FieldInfo, len(metadata.fields)),
|
||||
history: make(map[string][]Change, len(metadata.history)),
|
||||
fields: make(map[string]FieldInfo, len(metadata.fields)),
|
||||
history: make(map[string][]Change, len(metadata.history)),
|
||||
inheritance: append([]inheritanceOperation(nil), metadata.inheritance...),
|
||||
}
|
||||
cloned.effectivePaths = make(map[string]Path, len(metadata.effectivePaths))
|
||||
for pointer, path := range metadata.effectivePaths {
|
||||
|
||||
182
internal/engine/load_request.go
Normal file
182
internal/engine/load_request.go
Normal file
@@ -0,0 +1,182 @@
|
||||
package engine
|
||||
|
||||
import (
|
||||
"gitea.maximumdirect.net/eric/ConfigKit/internal/document"
|
||||
)
|
||||
|
||||
// loadRequest keeps source acquisition separate from precedence assembly.
|
||||
// The ordinary entry point and grouped entry point share the final lifecycle.
|
||||
type loadRequest[T any] struct {
|
||||
main loadInput
|
||||
underlays []loadGroup
|
||||
definition Definition[T]
|
||||
overlay string
|
||||
overlayFilesystem *FilesystemContext
|
||||
overrides []Override
|
||||
ordinary bool
|
||||
}
|
||||
|
||||
type loadInput struct {
|
||||
source Source
|
||||
imports bool
|
||||
filesystem *FilesystemContext
|
||||
}
|
||||
|
||||
type loadGroup struct{ inputs []loadInput }
|
||||
|
||||
func loadGroupedWithSecretLookup[T any](request loadRequest[T], secrets SecretBinding, lookup secretLookup) (*Loaded[T], error) {
|
||||
request.ordinary = false
|
||||
var diagnostics []Diagnostic
|
||||
_, mainDiagnostics := selectOptions(request.main.source, []Options[T]{{
|
||||
Definition: request.definition, Imports: request.main.imports, Filesystem: request.main.filesystem,
|
||||
}})
|
||||
diagnostics = append(diagnostics, mainDiagnostics...)
|
||||
for _, group := range request.underlays {
|
||||
for _, input := range group.inputs {
|
||||
_, inputDiagnostics := selectOptions(input.source, []Options[T]{{Imports: input.imports, Filesystem: input.filesystem}})
|
||||
diagnostics = append(diagnostics, inputDiagnostics...)
|
||||
}
|
||||
}
|
||||
if !validOverlayName(request.overlay) {
|
||||
diagnostics = append(diagnostics, optionDiagnostic(CodeOptionsInvalid, "overlay name must contain only ASCII letters, digits, underscore, or hyphen"))
|
||||
}
|
||||
if activeOverlay(request.overlay) {
|
||||
if request.overlayFilesystem == nil {
|
||||
diagnostics = append(diagnostics, optionDiagnostic(CodeOptionsInvalid, "selected overlay requires explicit filesystem context"))
|
||||
} else {
|
||||
diagnostics = append(diagnostics, filesystemOptionDiagnostics(Source{kind: sourceBytes}, request.overlayFilesystem)...)
|
||||
}
|
||||
}
|
||||
if len(diagnostics) != 0 {
|
||||
return nil, (&diagnosticAccumulator{diagnostics: canonicalDiagnostics(diagnostics)}).loadError()
|
||||
}
|
||||
return executeLoadRequest(request, secrets, lookup)
|
||||
}
|
||||
|
||||
func assembleLayeredInput[T any](request loadRequest[T], input assembledInput, accounting *importAccounting) (assembledInput, []Diagnostic) {
|
||||
groups := make([]*composedDocument, 0, len(request.underlays))
|
||||
for index, group := range request.underlays {
|
||||
composition, sources, contexts, diagnostics := acquireUnderlayGroup(group, index+1, request.definition.Version, accounting)
|
||||
input.sources = append(input.sources, sources...)
|
||||
mergeResourceContexts(&input.contexts, contexts)
|
||||
if len(diagnostics) != 0 {
|
||||
return input, diagnostics
|
||||
}
|
||||
if composition != nil {
|
||||
groups = append(groups, composition)
|
||||
}
|
||||
}
|
||||
|
||||
var assembly *assemblyAccumulator
|
||||
for _, group := range groups {
|
||||
if assembly == nil {
|
||||
assembly = newAssemblyAccumulator(group)
|
||||
continue
|
||||
}
|
||||
if diagnostics := assembly.applyOverlay(group.root, group.origins, group.applications, LayerUnderlay); len(diagnostics) != 0 {
|
||||
return input, diagnostics
|
||||
}
|
||||
}
|
||||
if input.document != nil {
|
||||
mainComposition, diagnostics := compositionForInput(input)
|
||||
if len(diagnostics) != 0 {
|
||||
return input, diagnostics
|
||||
}
|
||||
if assembly == nil {
|
||||
assembly = newAssemblyAccumulator(mainComposition)
|
||||
} else if diagnostics := assembly.applyOverlay(mainComposition.root, mainComposition.origins, mainComposition.applications, LayerBase); len(diagnostics) != 0 {
|
||||
return input, diagnostics
|
||||
}
|
||||
}
|
||||
if assembly == nil && (activeOverlay(request.overlay) || len(request.overrides) != 0) {
|
||||
composition, _ := compositionForInput(assembledInput{})
|
||||
assembly = newAssemblyAccumulator(composition)
|
||||
}
|
||||
if assembly == nil {
|
||||
return input, nil
|
||||
}
|
||||
assembly.current.inputBytes = accounting.bytes
|
||||
assembly.current.inputNodes = accounting.nodes
|
||||
if activeOverlay(request.overlay) {
|
||||
// The selected overlay context is independent of the main input. This
|
||||
// synthetic descriptor creates no source record or input contribution.
|
||||
selection := Source{kind: sourceBytes}
|
||||
selected := parsedSource{document: &document.Node{Kind: document.Mapping}, source: &SourceInfo{}}
|
||||
var diagnostics []Diagnostic
|
||||
input.sources, diagnostics = loadSelectedOverlayInto(selection, request.overlayFilesystem, selected, assembly, input.sources, request.overlay, request.definition.Version, defaultImportLimits())
|
||||
if len(diagnostics) != 0 {
|
||||
return input, diagnostics
|
||||
}
|
||||
contexts, diagnostic := contextsForAcquiredSources(selection, request.overlayFilesystem, selected.source, input.sources[len(input.sources)-1:])
|
||||
if diagnostic != nil {
|
||||
return input, []Diagnostic{*diagnostic}
|
||||
}
|
||||
mergeResourceContexts(&input.contexts, contexts)
|
||||
}
|
||||
if len(request.overrides) != 0 {
|
||||
var diagnostics []Diagnostic
|
||||
input.sources, diagnostics = assembly.applyOverrides(input.sources, request.overrides, request.definition.Version, defaultImportLimits())
|
||||
if len(diagnostics) != 0 {
|
||||
return input, diagnostics
|
||||
}
|
||||
}
|
||||
input.composition = assembly.current
|
||||
input.document = assembly.current.root
|
||||
return input, nil
|
||||
}
|
||||
|
||||
func acquireUnderlayGroup(group loadGroup, index int, rule *VersionRule, accounting *importAccounting) (*composedDocument, []SourceInfo, map[string]ResourceContext, []Diagnostic) {
|
||||
initialBytes, initialNodes := accounting.bytes, accounting.nodes
|
||||
visited := make(map[resourceIdentity]importRoute)
|
||||
hostToken := &namespaceToken{}
|
||||
var documents []*document.Node
|
||||
var sources []SourceInfo
|
||||
var contexts map[string]ResourceContext
|
||||
for _, contribution := range group.inputs {
|
||||
options := Options[struct{}]{Imports: contribution.imports, Filesystem: contribution.filesystem, Definition: Definition[struct{}]{Version: rule}}
|
||||
acquired, diagnostics := assembleBaseInputTracked(contribution.source, options, accounting, visited, hostToken)
|
||||
for _, source := range acquired.sources {
|
||||
source.Layer = LayerUnderlay
|
||||
source.UnderlayIndex = index
|
||||
sources = append(sources, source)
|
||||
}
|
||||
mergeResourceContexts(&contexts, acquired.contexts)
|
||||
if len(diagnostics) != 0 {
|
||||
return nil, sources, contexts, diagnostics
|
||||
}
|
||||
if acquired.document == nil {
|
||||
continue
|
||||
}
|
||||
if node := fragmentVersionClaim(acquired.document, rule); node != nil {
|
||||
location := publicLocation(node.Location)
|
||||
return nil, sources, contexts, []Diagnostic{{
|
||||
Code: CodeUnderlayVersion, Severity: SeverityError, Path: rule.Path.clone(),
|
||||
Message: "underlay inputs cannot contribute to the schema-version path", Primary: &location,
|
||||
}}
|
||||
}
|
||||
if acquired.composition != nil {
|
||||
for _, application := range acquired.composition.applications {
|
||||
documents = append(documents, application.value)
|
||||
}
|
||||
} else {
|
||||
documents = append(documents, acquired.document)
|
||||
}
|
||||
}
|
||||
if len(documents) == 0 {
|
||||
return nil, sources, contexts, nil
|
||||
}
|
||||
composition, diagnostics := composeDocuments(documents...)
|
||||
if len(diagnostics) != 0 {
|
||||
return nil, sources, contexts, diagnostics
|
||||
}
|
||||
composition.inputBytes = accounting.bytes - initialBytes
|
||||
composition.inputNodes = accounting.nodes - initialNodes
|
||||
for pointer, origin := range composition.origins {
|
||||
origin.Layer = LayerUnderlay
|
||||
composition.origins[pointer] = origin
|
||||
}
|
||||
for index := range composition.applications {
|
||||
composition.applications[index].layer = LayerUnderlay
|
||||
}
|
||||
return composition, sources, contexts, nil
|
||||
}
|
||||
280
internal/engine/load_request_test.go
Normal file
280
internal/engine/load_request_test.go
Normal file
@@ -0,0 +1,280 @@
|
||||
package engine
|
||||
|
||||
import (
|
||||
"reflect"
|
||||
"strings"
|
||||
"testing"
|
||||
"testing/fstest"
|
||||
)
|
||||
|
||||
type groupedConfig struct {
|
||||
Version string `yaml:"version"`
|
||||
Settings map[string]string `yaml:"settings"`
|
||||
Port int `yaml:"port"`
|
||||
Derived string `yaml:"derived"`
|
||||
}
|
||||
|
||||
func testGroupedLoad[T any](request loadRequest[T]) (*testResult[T], error) {
|
||||
loaded, err := loadGroupedWithSecretLookup(request, testSecretBinding(), func(string) (string, bool) { return "", false })
|
||||
return testLoaded(loaded), err
|
||||
}
|
||||
|
||||
func TestGroupedLoadPrecedenceSourceOrderAndSingleLifecycle(t *testing.T) {
|
||||
filesystem := NewFilesystem("bundle", fstest.MapFS{
|
||||
"main.yml": &fstest.MapFile{Data: []byte("configkit: {imports: [{path: imported.yml}]}\nversion: v1\nport: 5\nsettings: {shared: main}\n")},
|
||||
"imported.yml": &fstest.MapFile{Data: []byte("settings: {imported: included}\n")},
|
||||
"overlays/test.yml": &fstest.MapFile{Data: []byte("settings: {shared: overlay}\n")},
|
||||
})
|
||||
checks := struct{ version, defaults, resolution, validation int }{}
|
||||
request := loadRequest[groupedConfig]{
|
||||
main: loadInput{source: filesystem.File("main.yml"), imports: true},
|
||||
underlays: []loadGroup{
|
||||
{inputs: []loadInput{
|
||||
{source: Mapping("low-a", map[string]any{"settings": map[string]any{"shared": "low", "a": "one"}, "port": "invalid until replaced"})},
|
||||
{source: Bytes("low-b", []byte("settings: {b: two}\n"))},
|
||||
}},
|
||||
{inputs: []loadInput{{source: Mapping("middle", map[string]any{"settings": map[string]any{"shared": "middle", "c": "three"}})}}},
|
||||
},
|
||||
definition: Definition[groupedConfig]{
|
||||
Version: &VersionRule{Path: Keys("version"), Check: func(version Version, validation *Validation) error {
|
||||
checks.version++
|
||||
if version.Text != "v1" {
|
||||
t.Errorf("version = %#v", version)
|
||||
}
|
||||
return nil
|
||||
}},
|
||||
Defaults: func() groupedConfig { checks.defaults++; return groupedConfig{Derived: "initial"} },
|
||||
Resolve: func(config *groupedConfig, resolution *Resolution) error {
|
||||
checks.resolution++
|
||||
Default(resolution, Keys("derived"), &config.Derived, "resolved", "set derived")
|
||||
return nil
|
||||
},
|
||||
Validate: func(config *groupedConfig, validation *Validation) { checks.validation++ },
|
||||
},
|
||||
overlay: "test", overlayFilesystem: &FilesystemContext{Root: ".", BaseDir: ".", FS: &filesystem},
|
||||
overrides: []Override{{Path: Keys("settings", "shared"), Value: "cli", Flag: "--shared"}},
|
||||
}
|
||||
result, err := testGroupedLoad(request)
|
||||
if err != nil {
|
||||
t.Fatalf("load: %v, diagnostics %#v", err, err.(*LoadError).Diagnostics())
|
||||
}
|
||||
wantSettings := map[string]string{"shared": "cli", "a": "one", "b": "two", "c": "three", "imported": "included"}
|
||||
if result.Config.Port != 5 || !reflect.DeepEqual(result.Config.Settings, wantSettings) {
|
||||
t.Fatalf("config = %#v", result.Config)
|
||||
}
|
||||
if checks != (struct{ version, defaults, resolution, validation int }{1, 1, 1, 1}) {
|
||||
t.Fatalf("lifecycle calls = %#v", checks)
|
||||
}
|
||||
sources := result.Sources()
|
||||
if len(sources) != 7 {
|
||||
t.Fatalf("sources = %#v", sources)
|
||||
}
|
||||
wantLayers := []Layer{LayerBase, LayerBase, LayerUnderlay, LayerUnderlay, LayerUnderlay, LayerOverlay, LayerCLI}
|
||||
wantIndexes := []int{0, 0, 1, 1, 2, 0, 0}
|
||||
for index, source := range sources {
|
||||
if source.ID != "source-"+string(rune('1'+index)) || source.Layer != wantLayers[index] || source.UnderlayIndex != wantIndexes[index] {
|
||||
t.Errorf("source %d = %#v", index, source)
|
||||
}
|
||||
}
|
||||
info, ok := result.Metadata().Lookup(Keys("settings", "shared"))
|
||||
if !ok || info.Origin.Layer != LayerCLI {
|
||||
t.Fatalf("effective shared metadata = %#v, %v", info, ok)
|
||||
}
|
||||
explanation, ok := result.Explain(Keys("settings", "shared"))
|
||||
if !ok || len(explanation.History) < 5 {
|
||||
t.Fatalf("shared explanation = %#v, %v", explanation, ok)
|
||||
}
|
||||
var layers []Layer
|
||||
for _, change := range explanation.History {
|
||||
if change.Origin.Layer != LayerDefault && (len(layers) == 0 || layers[len(layers)-1] != change.Origin.Layer) {
|
||||
layers = append(layers, change.Origin.Layer)
|
||||
}
|
||||
}
|
||||
if !reflect.DeepEqual(layers, []Layer{LayerUnderlay, LayerBase, LayerOverlay, LayerCLI}) {
|
||||
t.Fatalf("history layers = %#v", layers)
|
||||
}
|
||||
}
|
||||
|
||||
func TestGroupedLoadAbsentMainAndVersionProtection(t *testing.T) {
|
||||
var observed Version
|
||||
request := loadRequest[groupedConfig]{
|
||||
main: loadInput{source: NoSource()},
|
||||
underlays: []loadGroup{{}, {inputs: []loadInput{{source: Bytes("underlay", []byte("port: 7\n"))}}}},
|
||||
definition: Definition[groupedConfig]{Version: &VersionRule{Path: Keys("version"), Check: func(version Version, validation *Validation) error {
|
||||
observed = version
|
||||
return nil
|
||||
}}},
|
||||
}
|
||||
result, err := testGroupedLoad(request)
|
||||
if err != nil || result.Config.Port != 7 || observed.Present || len(result.Sources()) != 1 || result.Sources()[0].UnderlayIndex != 2 {
|
||||
t.Fatalf("underlay-only load = %#v, version %#v, %v", result, observed, err)
|
||||
}
|
||||
request.underlays[1].inputs[0].source = Bytes("underlay", []byte("version: forbidden\n"))
|
||||
_, err = testGroupedLoad(request)
|
||||
if got := groupedErrorCode(t, err); got != CodeUnderlayVersion {
|
||||
t.Fatalf("underlay version code = %s", got)
|
||||
}
|
||||
filesystem := NewFilesystem("bundle", fstest.MapFS{
|
||||
"main.yml": &fstest.MapFile{Data: []byte("configkit: {imports: [{path: fragment.yml}]}\n")},
|
||||
"fragment.yml": &fstest.MapFile{Data: []byte("version: forbidden\n")},
|
||||
})
|
||||
request.underlays[1].inputs[0] = loadInput{source: filesystem.File("main.yml"), imports: true}
|
||||
_, err = testGroupedLoad(request)
|
||||
if got := groupedErrorCode(t, err); got != CodeImportVersion {
|
||||
t.Fatalf("import version code = %s", got)
|
||||
}
|
||||
request.underlays = []loadGroup{{}}
|
||||
request.definition.Defaults = func() groupedConfig { return groupedConfig{Port: 11} }
|
||||
result, err = testGroupedLoad(request)
|
||||
if err != nil || result.Config.Port != 11 || len(result.Sources()) != 0 {
|
||||
t.Fatalf("defaults-only load = %#v, %v", result, err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestGroupedLoadStrictConflictsRepetitionAndBudgets(t *testing.T) {
|
||||
main := loadInput{source: Bytes("main", []byte("port: 5\n"))}
|
||||
request := loadRequest[groupedConfig]{
|
||||
main: main,
|
||||
underlays: []loadGroup{{inputs: []loadInput{
|
||||
{source: Bytes("left", []byte("port: 1\n"))},
|
||||
{source: Bytes("right", []byte("port: 2\n"))},
|
||||
}}},
|
||||
}
|
||||
_, err := testGroupedLoad(request)
|
||||
if got := groupedErrorCode(t, err); got != CodeComposeConflict {
|
||||
t.Fatalf("strict conflict code = %s", got)
|
||||
}
|
||||
conflicting := NewFilesystem("bundle", fstest.MapFS{
|
||||
"main.yml": &fstest.MapFile{Data: []byte("configkit: {imports: [{path: fragment.yml}]}\nport: 1\n")},
|
||||
"fragment.yml": &fstest.MapFile{Data: []byte("port: 2\n")},
|
||||
})
|
||||
request.main = loadInput{source: conflicting.File("main.yml"), imports: true}
|
||||
request.underlays = []loadGroup{{inputs: []loadInput{{source: Bytes("lower", []byte("port: 3\n"))}}}}
|
||||
_, err = testGroupedLoad(request)
|
||||
if got := groupedErrorCode(t, err); got != CodeComposeConflict {
|
||||
t.Fatalf("main/import conflict code = %s", got)
|
||||
}
|
||||
request.main = main
|
||||
filesystem := NewFilesystem("bundle", fstest.MapFS{"same.yml": &fstest.MapFile{Data: []byte("port: 1\n")}})
|
||||
file := filesystem.File("same.yml")
|
||||
request.underlays = []loadGroup{{inputs: []loadInput{{source: file}, {source: file}}}}
|
||||
_, err = testGroupedLoad(request)
|
||||
if got := groupedErrorCode(t, err); got != CodeImportRepeated {
|
||||
t.Fatalf("same-group repetition code = %s", got)
|
||||
}
|
||||
request.underlays = []loadGroup{{inputs: []loadInput{{source: file}}}, {inputs: []loadInput{{source: file}}}}
|
||||
if result, err := testGroupedLoad(request); err != nil || result.Config.Port != 5 || len(result.Sources()) != 3 {
|
||||
t.Fatalf("cross-group reuse = %#v, %v", result, err)
|
||||
}
|
||||
|
||||
accounting := &importAccounting{sources: maxImportSources - 1}
|
||||
group := loadGroup{inputs: []loadInput{{source: Mapping("first", map[string]any{})}, {source: Mapping("second", map[string]any{})}}}
|
||||
_, _, _, diagnostics := acquireUnderlayGroup(group, 1, nil, accounting)
|
||||
if len(diagnostics) == 0 || diagnostics[0].Code != CodeSourceLimitExceeded {
|
||||
t.Fatalf("shared source budget = %#v", diagnostics)
|
||||
}
|
||||
accounting = &importAccounting{bytes: maxTotalBytes - 2}
|
||||
group.inputs = []loadInput{{source: Mapping("first", map[string]any{"port": "one"})}}
|
||||
_, _, _, diagnostics = acquireUnderlayGroup(group, 1, nil, accounting)
|
||||
if len(diagnostics) == 0 || diagnostics[0].Code != CodeSourceLimitExceeded {
|
||||
t.Fatalf("shared byte budget = %#v", diagnostics)
|
||||
}
|
||||
}
|
||||
|
||||
func TestGroupedLoadInvalidIntermediateTypeCanBeReplaced(t *testing.T) {
|
||||
request := loadRequest[groupedConfig]{
|
||||
main: loadInput{source: Bytes("main", []byte("port: 9\n"))},
|
||||
underlays: []loadGroup{{inputs: []loadInput{{source: Bytes("low", []byte("port: wrong\n"))}}}},
|
||||
}
|
||||
result, err := testGroupedLoad(request)
|
||||
if err != nil || result.Config.Port != 9 {
|
||||
t.Fatalf("final typed decode = %#v, %v", result, err)
|
||||
}
|
||||
request.main.source = Bytes("main", []byte("port: wrong\n"))
|
||||
_, err = testGroupedLoad(request)
|
||||
if got := groupedErrorCode(t, err); got != CodeDecodeType && got != CodeDecodeRange {
|
||||
t.Fatalf("final invalid type code = %s", got)
|
||||
}
|
||||
}
|
||||
|
||||
func groupedErrorCode(t *testing.T, err error) Code {
|
||||
t.Helper()
|
||||
loadError, ok := err.(*LoadError)
|
||||
if !ok || len(loadError.Diagnostics()) == 0 {
|
||||
t.Fatalf("error = %#v", err)
|
||||
}
|
||||
if strings.Contains(loadError.Diagnostics()[0].Message, "wrong") {
|
||||
t.Fatal("raw value escaped diagnostics")
|
||||
}
|
||||
return loadError.Diagnostics()[0].Code
|
||||
}
|
||||
|
||||
func TestGroupedAccountingChargesMixedOccurrencesOnce(t *testing.T) {
|
||||
const (
|
||||
one = "configkit: {imports: [{path: child-one.yml}]}\nleft: first\n"
|
||||
child = "extra: imported\n"
|
||||
two = "configkit: {imports: [{path: child-two.yml}]}\nright: second\n"
|
||||
)
|
||||
filesystem := NewFilesystem("bundle", fstest.MapFS{
|
||||
"one.yml": &fstest.MapFile{Data: []byte(one)},
|
||||
"child-one.yml": &fstest.MapFile{Data: []byte(child)},
|
||||
"two.yml": &fstest.MapFile{Data: []byte(two)},
|
||||
"child-two.yml": &fstest.MapFile{Data: []byte(child)},
|
||||
})
|
||||
accounting := &importAccounting{}
|
||||
first, sources, _, diagnostics := acquireUnderlayGroup(loadGroup{inputs: []loadInput{{source: filesystem.File("one.yml"), imports: true}}}, 1, nil, accounting)
|
||||
if len(diagnostics) != 0 || first == nil || len(sources) != 2 || accounting.sources != 2 || accounting.bytes != int64(len(one)+len(child)) {
|
||||
t.Fatalf("first acquisition = %#v, %#v, %#v, accounting %#v", first, sources, diagnostics, accounting)
|
||||
}
|
||||
second, sources, _, diagnostics := acquireUnderlayGroup(loadGroup{inputs: []loadInput{{source: filesystem.File("two.yml"), imports: true}}}, 2, nil, accounting)
|
||||
if len(diagnostics) != 0 || second == nil || len(sources) != 2 || sources[0].ID != "source-3" || sources[1].ID != "source-4" || accounting.sources != 4 || accounting.bytes != int64(len(one)+len(two)+2*len(child)) {
|
||||
t.Fatalf("second acquisition = %#v, %#v, %#v, accounting %#v", second, sources, diagnostics, accounting)
|
||||
}
|
||||
var wantNodes int
|
||||
for _, text := range []string{one, child, two, child} {
|
||||
parsed, diagnostics := parseSource(Bytes("fixture", []byte(text)), "source-1", defaultParseLimits())
|
||||
if len(diagnostics) != 0 {
|
||||
t.Fatal(diagnostics)
|
||||
}
|
||||
wantNodes += countInputNodes(parsed.document)
|
||||
}
|
||||
if accounting.nodes != wantNodes {
|
||||
t.Fatalf("charged nodes = %d, want %d", accounting.nodes, wantNodes)
|
||||
}
|
||||
if first.inputBytes != int64(len(one)+len(child)) || second.inputBytes != int64(len(two)+len(child)) {
|
||||
t.Fatalf("composition costs = %d, %d, accounting %d", first.inputBytes, second.inputBytes, accounting.bytes)
|
||||
}
|
||||
}
|
||||
|
||||
func TestGroupedGlobEntryBudgetIsShared(t *testing.T) {
|
||||
filesystem := NewFilesystem("bundle", fstest.MapFS{
|
||||
"main.yml": &fstest.MapFile{Data: []byte("configkit: {imports: [{glob: 'parts/*.yml'}]}\n")},
|
||||
"parts/a.yml": &fstest.MapFile{Data: []byte("left: one\n")},
|
||||
"parts/b.yml": &fstest.MapFile{Data: []byte("right: two\n")},
|
||||
})
|
||||
accounting := &importAccounting{entries: maxGlobEntries - 1}
|
||||
_, _, _, diagnostics := acquireUnderlayGroup(loadGroup{inputs: []loadInput{{source: filesystem.File("main.yml"), imports: true}}}, 1, nil, accounting)
|
||||
if len(diagnostics) == 0 || diagnostics[0].Code != CodeImportLimitExceeded || accounting.entries <= maxGlobEntries {
|
||||
t.Fatalf("shared glob budget = %#v, accounting %#v", diagnostics, accounting)
|
||||
}
|
||||
}
|
||||
|
||||
func TestGroupedMixedSourcesShareCLIRecordLimit(t *testing.T) {
|
||||
filesystem := NewFilesystem("bundle", fstest.MapFS{"main.yml": &fstest.MapFile{Data: []byte("value: base\n")}})
|
||||
inputs := make([]Input, maxImportSources-1)
|
||||
inputs[0] = Input{Source: Bytes("empty-bytes", []byte("{}\n"))}
|
||||
for index := 1; index < len(inputs); index++ {
|
||||
inputs[index] = Input{Source: Mapping("empty-map", map[string]any{})}
|
||||
}
|
||||
type config struct {
|
||||
Value string `yaml:"value"`
|
||||
}
|
||||
_, err := LoadLayered(Input{Source: filesystem.File("main.yml")}, testSecretBinding(), LayeredOptions[config]{
|
||||
Underlays: []Underlay{{Inputs: inputs}},
|
||||
Overrides: []Override{{Path: Keys("value"), Value: "cli", Flag: "--value"}},
|
||||
})
|
||||
loadError, ok := err.(*LoadError)
|
||||
if !ok || len(loadError.Diagnostics()) == 0 || loadError.Diagnostics()[0].Code != CodeOverrideLimitExceeded || len(loadError.Sources()) != maxImportSources {
|
||||
t.Fatalf("mixed source/CLI limit = %#v", err)
|
||||
}
|
||||
}
|
||||
@@ -15,6 +15,8 @@ type resolvedLoad[T any] struct {
|
||||
final *document.Node
|
||||
snapshots []resolutionSnapshot
|
||||
applications []historyApplication
|
||||
inheritance *inheritanceScope
|
||||
contexts map[string]ResourceContext
|
||||
}
|
||||
|
||||
func decodeLoadInput[T any](schema *schemaDescriptor, definition Definition[T], input assembledInput) (decodedConfiguration, []Diagnostic) {
|
||||
@@ -34,7 +36,9 @@ func decodeLoadInput[T any](schema *schemaDescriptor, definition Definition[T],
|
||||
}
|
||||
return decodedConfiguration{}, diagnostics
|
||||
}
|
||||
if input.composition != nil {
|
||||
if input.inheritance != nil {
|
||||
applyCompositionOrigins(&decoded, input.inheritance.origins)
|
||||
} else if input.composition != nil {
|
||||
applyCompositionOrigins(&decoded, input.composition.origins)
|
||||
}
|
||||
|
||||
@@ -45,13 +49,15 @@ func resolveLoadInput[T any](schema *schemaDescriptor, definition Definition[T],
|
||||
var diagnostics []Diagnostic
|
||||
configuration := decoded.value.Interface().(T)
|
||||
metadata := metadataFromDecoded(decoded)
|
||||
if input.composition != nil {
|
||||
if input.inheritance != nil {
|
||||
applyInheritanceMetadata(&metadata, input.composition.root, input.composition.origins, input.composition.overlays, input.inheritance)
|
||||
} else if input.composition != nil {
|
||||
applyOverlayMetadata(&metadata, input.composition.overlays)
|
||||
}
|
||||
final := decoded.final
|
||||
var resolutionSnapshots []resolutionSnapshot
|
||||
if definition.Resolve != nil {
|
||||
entry := newResolutionInput(metadata, decoded.final, input.sources)
|
||||
entry := newResolutionInput(metadata, decoded.final, input.sources, input.contexts)
|
||||
resolved := runResolution(&configuration, definition.Resolve, schema, decoded, input.document, entry)
|
||||
diagnostics = append(diagnostics, resolved.diagnostics...)
|
||||
if hasErrors(resolved.diagnostics) {
|
||||
@@ -67,7 +73,7 @@ func resolveLoadInput[T any](schema *schemaDescriptor, definition Definition[T],
|
||||
} else if input.document != nil {
|
||||
applications = []historyApplication{{layer: LayerBase, value: input.document}}
|
||||
}
|
||||
return resolvedLoad[T]{configuration: configuration, decoded: decoded, metadata: metadata, final: final, snapshots: resolutionSnapshots, applications: applications}, diagnostics
|
||||
return resolvedLoad[T]{configuration: configuration, decoded: decoded, metadata: metadata, final: final, snapshots: resolutionSnapshots, applications: applications, inheritance: input.inheritance, contexts: input.contexts}, diagnostics
|
||||
}
|
||||
|
||||
func detachLoadConfiguration[T any](schema *schemaDescriptor, configuration T, mode secretCloneMode) (T, *Diagnostic) {
|
||||
@@ -95,6 +101,7 @@ func publishLoadResult[T any](schema *schemaDescriptor, definition Definition[T]
|
||||
inspection := &Inspection{
|
||||
metadata: metadata,
|
||||
sources: cloneSourceInfos(accumulator.sources),
|
||||
contexts: cloneResourceContexts(resolved.contexts),
|
||||
diagnostics: canonicalDiagnostics(accumulator.diagnostics),
|
||||
baseline: decoded.baseline,
|
||||
input: decoded.final,
|
||||
@@ -102,6 +109,7 @@ func publishLoadResult[T any](schema *schemaDescriptor, definition Definition[T]
|
||||
final: final,
|
||||
resolution: resolutionSnapshots,
|
||||
applications: applications,
|
||||
inheritance: inheritanceOperations(resolved.inheritance),
|
||||
secrets: secrets,
|
||||
secretReferences: cloneSecretReferences(references),
|
||||
}
|
||||
|
||||
37
internal/engine/mapping_index.go
Normal file
37
internal/engine/mapping_index.go
Normal file
@@ -0,0 +1,37 @@
|
||||
package engine
|
||||
|
||||
import "gitea.maximumdirect.net/eric/ConfigKit/internal/document"
|
||||
|
||||
// mappingIndex keeps ordered document entries and their lookup positions in
|
||||
// sync while one composition or overlay is assembled.
|
||||
type mappingIndex struct {
|
||||
byNode map[*document.Node]map[string]int
|
||||
}
|
||||
|
||||
func (m *mappingIndex) entries(node *document.Node) map[string]int {
|
||||
if m.byNode == nil {
|
||||
m.byNode = make(map[*document.Node]map[string]int)
|
||||
}
|
||||
if entries, ok := m.byNode[node]; ok {
|
||||
return entries
|
||||
}
|
||||
entries := make(map[string]int, len(node.Mapping))
|
||||
for index, entry := range node.Mapping {
|
||||
entries[entry.Key] = index
|
||||
}
|
||||
m.byNode[node] = entries
|
||||
return entries
|
||||
}
|
||||
|
||||
func (m *mappingIndex) find(node *document.Node, key string) int {
|
||||
if index, ok := m.entries(node)[key]; ok {
|
||||
return index
|
||||
}
|
||||
return -1
|
||||
}
|
||||
|
||||
func (m *mappingIndex) append(node *document.Node, entry document.Entry) {
|
||||
entries := m.entries(node)
|
||||
entries[entry.Key] = len(node.Mapping)
|
||||
node.Mapping = append(node.Mapping, entry)
|
||||
}
|
||||
@@ -13,6 +13,7 @@ type Metadata struct {
|
||||
historicalPaths map[string][]Path
|
||||
fields map[string]FieldInfo
|
||||
history map[string][]Change
|
||||
inheritance []inheritanceOperation
|
||||
}
|
||||
|
||||
// Lookup returns information for an effective path. Unknown and invalid paths
|
||||
@@ -25,17 +26,19 @@ func (m Metadata) Lookup(path Path) (FieldInfo, bool) {
|
||||
if !ok {
|
||||
return FieldInfo{}, false
|
||||
}
|
||||
return cloneFieldInfo(info), true
|
||||
detached := cloneFieldInfo(info)
|
||||
detached.Origin.Inheritance = projectInheritanceSteps(detached.Origin)
|
||||
return detached, true
|
||||
}
|
||||
|
||||
// IsExplicit reports whether a surviving input contribution explicitly defined
|
||||
// path.
|
||||
// IsExplicit reports whether a surviving input contribution, including one
|
||||
// inherited from a sibling, explicitly defined path.
|
||||
func (m Metadata) IsExplicit(path Path) bool {
|
||||
info, ok := m.Lookup(path)
|
||||
return ok && info.Explicit
|
||||
}
|
||||
|
||||
// History returns detached definition and resolution records for path. A record's
|
||||
// History returns detached definition, inheritance, and resolution records for path. A record's
|
||||
// Path can be an ancestor when that container operation replaced or removed path.
|
||||
// Removed historical paths can have history even though Lookup returns false.
|
||||
// Unknown and invalid paths return nil.
|
||||
@@ -43,19 +46,12 @@ func (m Metadata) History(path Path) []Change {
|
||||
if !m.hasHistoricalPath(path) {
|
||||
return nil
|
||||
}
|
||||
history, ok := m.history[path.String()]
|
||||
if !ok {
|
||||
return nil
|
||||
}
|
||||
cloned := make([]Change, len(history))
|
||||
for i, change := range history {
|
||||
cloned[i] = cloneChange(change)
|
||||
}
|
||||
return cloned
|
||||
history := m.history[path.String()]
|
||||
return historyWithInheritance(path, history, m.inheritance)
|
||||
}
|
||||
|
||||
// HistoricalPaths returns the paths observed during loading, including effective
|
||||
// paths and paths removed by overlays or recorded resolution. It includes default
|
||||
// paths and paths removed by overlays, inheritance, or recorded resolution. It includes default
|
||||
// paths and the root, but not paths that never existed. The zero Metadata returns
|
||||
// an empty slice.
|
||||
//
|
||||
@@ -136,6 +132,16 @@ func finalizeMetadataPaths(m *Metadata, final *document.Node, applications []his
|
||||
}
|
||||
walkDocument(application.value, prefix, add)
|
||||
}
|
||||
for _, operation := range m.inheritance {
|
||||
if operation.control != nil {
|
||||
add(operation.control.path, nil)
|
||||
continue
|
||||
}
|
||||
if operation.transfer != nil {
|
||||
walkDocument(operation.parent, operation.transfer.to, add)
|
||||
walkDocument(operation.effective, operation.transfer.to, add)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// effectivePathIndex records document-derived paths with their contextual segment
|
||||
|
||||
284
internal/engine/native_filesystem_test.go
Normal file
284
internal/engine/native_filesystem_test.go
Normal file
@@ -0,0 +1,284 @@
|
||||
package engine
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"io/fs"
|
||||
"reflect"
|
||||
"strings"
|
||||
"testing"
|
||||
"testing/fstest"
|
||||
)
|
||||
|
||||
func TestNativeFilesystemImportsAndOverlay(t *testing.T) {
|
||||
filesystem := fstest.MapFS{
|
||||
"config/main.yml": &fstest.MapFile{Data: []byte(`configkit:
|
||||
imports:
|
||||
- path: ../shared.yml
|
||||
- glob: fragments/*.yml
|
||||
items: {main: main}
|
||||
`)},
|
||||
"shared.yml": &fstest.MapFile{Data: []byte("items: {shared: shared}\n")},
|
||||
"config/fragments/b.yml": &fstest.MapFile{Data: []byte("items: {beta: beta}\n")},
|
||||
"config/fragments/a.yml": &fstest.MapFile{Data: []byte("items: {alpha: alpha}\n")},
|
||||
"overlays/testing.yml": &fstest.MapFile{Data: []byte("items: {main: overlaid}\n")},
|
||||
}
|
||||
namespace := newNativeNamespace("bundle", filesystem)
|
||||
result, err := testLoad[importTestConfig](filesystemFile(namespace, "config/main.yml"), Options[importTestConfig]{Imports: true, Overlay: "testing"})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
want := map[string]string{"main": "overlaid", "shared": "shared", "alpha": "alpha", "beta": "beta"}
|
||||
if !reflect.DeepEqual(result.Config.Items, want) {
|
||||
t.Fatalf("items = %#v, want %#v", result.Config.Items, want)
|
||||
}
|
||||
sources := result.Sources()
|
||||
wantPaths := []string{"config/main.yml", "shared.yml", "config/fragments/a.yml", "config/fragments/b.yml", "overlays/testing.yml"}
|
||||
gotPaths := make([]string, len(sources))
|
||||
for index, source := range sources {
|
||||
gotPaths[index] = source.Path
|
||||
}
|
||||
if !reflect.DeepEqual(gotPaths, wantPaths) {
|
||||
t.Fatalf("source paths = %#v, want %#v", gotPaths, wantPaths)
|
||||
}
|
||||
}
|
||||
|
||||
func TestNativeFilesystemNamespaceAndPathFailures(t *testing.T) {
|
||||
first := fstest.MapFS{"main.yml": &fstest.MapFile{Data: []byte("items: {name: first}\n")}}
|
||||
second := fstest.MapFS{"main.yml": &fstest.MapFile{Data: []byte("items: {name: second}\n")}}
|
||||
firstNamespace := newNativeNamespace("same", first)
|
||||
secondNamespace := newNativeNamespace("same", second)
|
||||
if reflect.TypeOf(first).Comparable() {
|
||||
t.Fatal("test filesystem must have a non-comparable dynamic value")
|
||||
}
|
||||
if firstNamespace.token == secondNamespace.token {
|
||||
t.Fatal("independent filesystem constructors shared identity")
|
||||
}
|
||||
if newNativeAccess(first, ".", firstNamespace.token).identity("main.yml") == newNativeAccess(second, ".", secondNamespace.token).identity("main.yml") {
|
||||
t.Fatal("equal paths from separate filesystems have the same identity")
|
||||
}
|
||||
access := newNativeAccess(first, ".", firstNamespace.token)
|
||||
if access.syntheticIdentity() == access.syntheticIdentity() || access.syntheticIdentity() == access.identity("<selected-source>") {
|
||||
t.Fatal("independent synthetic sources shared a file identity")
|
||||
}
|
||||
for _, test := range []struct {
|
||||
name string
|
||||
namespace *nativeNamespace
|
||||
want string
|
||||
}{
|
||||
{name: "first", namespace: firstNamespace, want: "first"},
|
||||
{name: "second", namespace: secondNamespace, want: "second"},
|
||||
} {
|
||||
t.Run(test.name, func(t *testing.T) {
|
||||
result, err := testLoad[importTestConfig](filesystemFile(test.namespace, "main.yml"))
|
||||
if err != nil || result.Config.Items["name"] != test.want {
|
||||
t.Fatalf("result = %#v, %v", result, err)
|
||||
}
|
||||
})
|
||||
}
|
||||
for _, name := range []string{".", "../main.yml", "/main.yml", "missing.yml"} {
|
||||
_, diagnostics := readSource(filesystemFile(firstNamespace, name), "source-1", defaultSourceLimits())
|
||||
if len(diagnostics) != 1 {
|
||||
t.Errorf("%q diagnostics = %#v", name, diagnostics)
|
||||
}
|
||||
}
|
||||
missingImport := fstest.MapFS{"main.yml": &fstest.MapFile{Data: []byte("configkit: {imports: [{path: host-only.yml}]}\n")}}
|
||||
_, err := testLoad[importTestConfig](filesystemFile(newNativeNamespace("bundle", missingImport), "main.yml"), Options[importTestConfig]{Imports: true})
|
||||
if err == nil {
|
||||
t.Fatal("missing virtual import fell back to the host filesystem")
|
||||
}
|
||||
loadError, ok := err.(*LoadError)
|
||||
if !ok || loadError.Diagnostics()[0].Code != CodeImportMissing {
|
||||
t.Fatalf("missing import error = %#v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestNativeFilesystemClosesFilesOnSuccessAndFailure(t *testing.T) {
|
||||
for _, fallback := range []bool{false, true} {
|
||||
t.Run(map[bool]string{false: "ReadDirFile", true: "ReadDir fallback"}[fallback], func(t *testing.T) {
|
||||
filesystem := &observedFilesystem{files: fstest.MapFS{
|
||||
"main.yml": &fstest.MapFile{Data: []byte("configkit: {imports: [{glob: 'parts/*.yml'}]}\nitems: {main: main}\n")},
|
||||
"parts/one.yml": &fstest.MapFile{Data: []byte("items: {one: one}\n")},
|
||||
}, fallback: fallback}
|
||||
result, err := testLoad[importTestConfig](filesystemFile(newNativeNamespace("bundle", filesystem), "main.yml"), Options[importTestConfig]{Imports: true})
|
||||
if err != nil || result.Config.Items["one"] != "one" {
|
||||
t.Fatalf("result = %#v, %v", result, err)
|
||||
}
|
||||
filesystem.assertClosed(t)
|
||||
})
|
||||
}
|
||||
|
||||
for _, failure := range []struct {
|
||||
name string
|
||||
file string
|
||||
failRead string
|
||||
failStat string
|
||||
failDir string
|
||||
}{
|
||||
{name: "read", file: "items: {one: one}\n", failRead: "parts/one.yml"},
|
||||
{name: "stat", file: "items: {one: one}\n", failStat: "parts/one.yml"},
|
||||
{name: "parse", file: "items: [unterminated\n"},
|
||||
{name: "directory", file: "items: {one: one}\n", failDir: "parts"},
|
||||
} {
|
||||
t.Run(failure.name, func(t *testing.T) {
|
||||
filesystem := &observedFilesystem{files: fstest.MapFS{
|
||||
"main.yml": &fstest.MapFile{Data: []byte("configkit: {imports: [{glob: 'parts/*.yml'}]}\n")},
|
||||
"parts/one.yml": &fstest.MapFile{Data: []byte(failure.file)},
|
||||
}, failRead: failure.failRead, failStat: failure.failStat, failDir: failure.failDir}
|
||||
_, err := testLoad[importTestConfig](filesystemFile(newNativeNamespace("bundle", filesystem), "main.yml"), Options[importTestConfig]{Imports: true})
|
||||
if err == nil {
|
||||
t.Fatal("expected a load failure")
|
||||
}
|
||||
if strings.Contains(err.Error(), "tracking failure canary") {
|
||||
t.Fatal("unsafe filesystem error escaped")
|
||||
}
|
||||
filesystem.assertClosed(t)
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestNativeFilesystemRootAndOptionalImports(t *testing.T) {
|
||||
filesystem := fstest.MapFS{
|
||||
"config/sub/main.yml": &fstest.MapFile{Data: []byte(`configkit:
|
||||
imports:
|
||||
- path: ../shared.yml
|
||||
- path: absent.yml
|
||||
optional: true
|
||||
- glob: absent/*.yml
|
||||
optional: true
|
||||
`)},
|
||||
"config/shared.yml": &fstest.MapFile{Data: []byte("items: {shared: retained}\n")},
|
||||
"outside.yml": &fstest.MapFile{Data: []byte("items: {outside: blocked}\n")},
|
||||
}
|
||||
namespace := newNativeNamespace("bundle", filesystem)
|
||||
source := filesystemFile(namespace, "config/sub/main.yml")
|
||||
parsed, diagnostics := parseSource(source, "source-1", defaultParseLimits())
|
||||
if len(diagnostics) != 0 {
|
||||
t.Fatal(diagnostics)
|
||||
}
|
||||
access := newNativeAccess(filesystem, "config", namespace.token)
|
||||
composed, sources, diagnostics := loadImportedWithAccess(access, "config/sub", source.path, parsed, nil, defaultImportLimits(), &importAccounting{})
|
||||
if len(diagnostics) != 0 || composed == nil || len(sources) != 2 || sources[1].Path != "config/shared.yml" {
|
||||
t.Fatalf("contained imports = (%#v, %#v, %#v)", composed, sources, diagnostics)
|
||||
}
|
||||
|
||||
outside := fstest.MapFS{
|
||||
"config/sub/main.yml": &fstest.MapFile{Data: []byte("configkit: {imports: [{path: ../../outside.yml}]}\n")},
|
||||
"outside.yml": &fstest.MapFile{Data: []byte("items: {outside: blocked}\n")},
|
||||
}
|
||||
outsideNamespace := newNativeNamespace("bundle", outside)
|
||||
parsed, diagnostics = parseSource(filesystemFile(outsideNamespace, "config/sub/main.yml"), "source-1", defaultParseLimits())
|
||||
if len(diagnostics) != 0 {
|
||||
t.Fatal(diagnostics)
|
||||
}
|
||||
_, _, diagnostics = loadImportedWithAccess(newNativeAccess(outside, "config", outsideNamespace.token), "config/sub", "config/sub/main.yml", parsed, nil, defaultImportLimits(), &importAccounting{})
|
||||
if len(diagnostics) != 1 || diagnostics[0].Code != CodeImportOutsideRoot {
|
||||
t.Fatalf("escaping import diagnostics = %#v", diagnostics)
|
||||
}
|
||||
}
|
||||
|
||||
func TestNativeFilesystemDetectsRepeatedAndCyclicImports(t *testing.T) {
|
||||
for _, test := range []struct {
|
||||
name string
|
||||
files fstest.MapFS
|
||||
code Code
|
||||
}{
|
||||
{
|
||||
name: "repeated",
|
||||
files: fstest.MapFS{
|
||||
"main.yml": &fstest.MapFile{Data: []byte("configkit: {imports: [{path: child.yml}, {path: child.yml}]}\n")},
|
||||
"child.yml": &fstest.MapFile{Data: []byte("items: {child: child}\n")},
|
||||
},
|
||||
code: CodeImportRepeated,
|
||||
},
|
||||
{
|
||||
name: "cycle",
|
||||
files: fstest.MapFS{
|
||||
"main.yml": &fstest.MapFile{Data: []byte("configkit: {imports: [{path: child.yml}]}\n")},
|
||||
"child.yml": &fstest.MapFile{Data: []byte("configkit: {imports: [{path: main.yml}]}\n")},
|
||||
},
|
||||
code: CodeImportCycle,
|
||||
},
|
||||
} {
|
||||
t.Run(test.name, func(t *testing.T) {
|
||||
_, err := testLoad[importTestConfig](filesystemFile(newNativeNamespace("bundle", test.files), "main.yml"), Options[importTestConfig]{Imports: true})
|
||||
loadError, ok := err.(*LoadError)
|
||||
if !ok || len(loadError.Diagnostics()) == 0 || loadError.Diagnostics()[0].Code != test.code {
|
||||
t.Fatalf("load error = %#v, want %q", err, test.code)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
type observedFilesystem struct {
|
||||
files fstest.MapFS
|
||||
opened int
|
||||
closed int
|
||||
fallback bool
|
||||
failRead string
|
||||
failStat string
|
||||
failDir string
|
||||
}
|
||||
|
||||
func (f *observedFilesystem) Open(name string) (fs.File, error) {
|
||||
inner, err := f.files.Open(name)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
f.opened++
|
||||
file := &observedFile{File: inner, owner: f, name: name}
|
||||
if directory, ok := inner.(fs.ReadDirFile); ok && !f.fallback {
|
||||
return &observedDirectoryFile{observedFile: file, directory: directory}, nil
|
||||
}
|
||||
return file, nil
|
||||
}
|
||||
|
||||
func (f *observedFilesystem) ReadDir(name string) ([]fs.DirEntry, error) {
|
||||
if name == f.failDir {
|
||||
return nil, errors.New("tracking failure canary")
|
||||
}
|
||||
return fs.ReadDir(f.files, name)
|
||||
}
|
||||
|
||||
func (f *observedFilesystem) assertClosed(t *testing.T) {
|
||||
t.Helper()
|
||||
if f.opened == 0 || f.opened != f.closed {
|
||||
t.Fatalf("filesystem opens=%d closes=%d", f.opened, f.closed)
|
||||
}
|
||||
}
|
||||
|
||||
type observedFile struct {
|
||||
fs.File
|
||||
owner *observedFilesystem
|
||||
name string
|
||||
}
|
||||
|
||||
func (f *observedFile) Read(data []byte) (int, error) {
|
||||
if f.name == f.owner.failRead {
|
||||
return 0, errors.New("tracking failure canary")
|
||||
}
|
||||
return f.File.Read(data)
|
||||
}
|
||||
|
||||
func (f *observedFile) Stat() (fs.FileInfo, error) {
|
||||
if f.name == f.owner.failStat {
|
||||
return nil, errors.New("tracking failure canary")
|
||||
}
|
||||
return f.File.Stat()
|
||||
}
|
||||
|
||||
func (f *observedFile) Close() error {
|
||||
f.owner.closed++
|
||||
return f.File.Close()
|
||||
}
|
||||
|
||||
type observedDirectoryFile struct {
|
||||
*observedFile
|
||||
directory fs.ReadDirFile
|
||||
}
|
||||
|
||||
func (f *observedDirectoryFile) ReadDir(count int) ([]fs.DirEntry, error) {
|
||||
if f.name == f.owner.failDir {
|
||||
return nil, errors.New("tracking failure canary")
|
||||
}
|
||||
return f.directory.ReadDir(count)
|
||||
}
|
||||
@@ -22,8 +22,8 @@ func selectOptions[T any](source Source, supplied []Options[T]) (Options[T], []D
|
||||
if source.kind == sourceNone && options.Overlay != "" && options.Overlay != "default" {
|
||||
diagnostics = append(diagnostics, optionDiagnostic(CodeOptionsInvalid, "NoSource cannot select an active overlay"))
|
||||
}
|
||||
if options.Filesystem == nil && (source.kind == sourceBytes || source.kind == sourceReader) && (options.Imports || activeOverlay(options.Overlay)) {
|
||||
diagnostics = append(diagnostics, optionDiagnostic(CodeOptionsInvalid, "byte and reader imports or overlays require filesystem context"))
|
||||
if options.Filesystem == nil && (source.kind == sourceBytes || source.kind == sourceReader || source.kind == sourceMapping) && (options.Imports || activeOverlay(options.Overlay)) {
|
||||
diagnostics = append(diagnostics, optionDiagnostic(CodeOptionsInvalid, "synthetic-source imports or overlays require filesystem context"))
|
||||
}
|
||||
if definition.Version != nil {
|
||||
if !definition.Version.Path.valid() {
|
||||
|
||||
@@ -18,78 +18,105 @@ type displacedDefinition struct {
|
||||
}
|
||||
|
||||
func overlayDocuments(base *composedDocument, overlay *document.Node) (*composedDocument, []Diagnostic) {
|
||||
if base == nil || base.root == nil || base.root.Kind != document.Mapping || overlay == nil || overlay.Kind != document.Mapping {
|
||||
return nil, []Diagnostic{{
|
||||
Code: CodeOverlayInvalid, Severity: SeverityError,
|
||||
Message: "overlay requires mapping document roots",
|
||||
}}
|
||||
if base == nil {
|
||||
return nil, invalidOverlayDocument()
|
||||
}
|
||||
result := cloneComposedDocument(base)
|
||||
mergeOverlayOrigin(result.origins, Path{}, overlay.Location, LayerOverlay)
|
||||
mergeOverlayMappings(result, result.root, overlay, Path{}, LayerOverlay, true)
|
||||
result.applications = append(result.applications, historyApplication{layer: LayerOverlay, value: overlay})
|
||||
return result, nil
|
||||
assembly := newAssemblyAccumulator(cloneComposedDocument(base))
|
||||
if diagnostics := assembly.applyOverlay(overlay, nil, nil, LayerOverlay); len(diagnostics) != 0 {
|
||||
return nil, diagnostics
|
||||
}
|
||||
return assembly.current, nil
|
||||
}
|
||||
|
||||
func mergeOverlayMappings(result *composedDocument, target, overlay *document.Node, current Path, layer Layer, recordReplacements bool) {
|
||||
for _, incoming := range overlay.Mapping {
|
||||
path := current.Key(incoming.Key)
|
||||
index := documentEntryIndex(target.Mapping, incoming.Key)
|
||||
if index < 0 {
|
||||
cloned := incoming
|
||||
cloned.Value = document.Clone(incoming.Value)
|
||||
target.Mapping = append(target.Mapping, cloned)
|
||||
collectOverlayOrigins(result.origins, incoming.Value, path, layer)
|
||||
continue
|
||||
}
|
||||
func invalidOverlayDocument() []Diagnostic {
|
||||
return []Diagnostic{{
|
||||
Code: CodeOverlayInvalid, Severity: SeverityError,
|
||||
Message: "overlay requires mapping document roots",
|
||||
}}
|
||||
}
|
||||
|
||||
existing := target.Mapping[index].Value
|
||||
if existing.Kind == document.Mapping && incoming.Value.Kind == document.Mapping {
|
||||
mergeOverlayOrigin(result.origins, path, incoming.Value.Location, layer)
|
||||
mergeOverlayMappings(result, existing, incoming.Value, path, layer, recordReplacements)
|
||||
continue
|
||||
}
|
||||
|
||||
displaced := displacedOrigins(existing, result.origins, path)
|
||||
previous := cloneOrigin(result.origins[path.String()])
|
||||
deleteOriginSubtree(result.origins, path)
|
||||
target.Mapping[index] = incoming
|
||||
target.Mapping[index].Value = document.Clone(incoming.Value)
|
||||
collectOverlayOrigins(result.origins, incoming.Value, path, layer)
|
||||
if recordReplacements {
|
||||
result.overlays = append(result.overlays, overlayRecord{
|
||||
change: Change{
|
||||
Path: path.clone(), Origin: overlayOrigin(layer, incoming.Value.Location),
|
||||
Previous: previousOrigins(previous),
|
||||
},
|
||||
displaced: displaced,
|
||||
})
|
||||
func (a *assemblyAccumulator) applyOverlay(overlay *document.Node, incomingOrigins map[string]Origin, applications []historyApplication, layer Layer) []Diagnostic {
|
||||
if a == nil || a.current == nil || a.current.root == nil || a.current.root.Kind != document.Mapping || overlay == nil || overlay.Kind != document.Mapping {
|
||||
return invalidOverlayDocument()
|
||||
}
|
||||
mergeOriginContribution(a.current.origins, Path{}, incomingOrigin(incomingOrigins, Path{}, overlay, layer))
|
||||
mergeOverlayMappings(a.current, a.current.root, overlay, Path{}, layer, true, incomingOrigins, &a.index)
|
||||
if len(applications) == 0 {
|
||||
a.current.applications = append(a.current.applications, historyApplication{layer: layer, value: overlay})
|
||||
} else {
|
||||
for _, application := range applications {
|
||||
application.layer = layer
|
||||
a.current.applications = append(a.current.applications, application)
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func mergeOverlayMappings(result *composedDocument, target, overlay *document.Node, current Path, layer Layer, recordReplacements bool, incomingOrigins map[string]Origin, index *mappingIndex) {
|
||||
mergeRecursiveMappings(target, overlay, current, index, recursiveMergeHooks{
|
||||
added: func(path Path, incoming *document.Node) {
|
||||
collectOverlayOriginsFrom(result.origins, incomingOrigins, incoming, path, layer)
|
||||
},
|
||||
combined: func(path Path, incoming *document.Node) {
|
||||
mergeOriginContribution(result.origins, path, incomingOrigin(incomingOrigins, path, incoming, layer))
|
||||
},
|
||||
replaced: func(path Path, existing, incoming *document.Node) {
|
||||
displaced := displacedOrigins(existing, result.origins, path)
|
||||
previous := cloneOrigin(result.origins[path.String()])
|
||||
deleteOriginSubtree(result.origins, path, existing)
|
||||
collectOverlayOriginsFrom(result.origins, incomingOrigins, incoming, path, layer)
|
||||
if recordReplacements {
|
||||
result.overlays = append(result.overlays, overlayRecord{
|
||||
change: Change{
|
||||
Path: path.clone(), Origin: incomingOrigin(incomingOrigins, path, incoming, layer),
|
||||
Previous: previousOrigins(previous),
|
||||
},
|
||||
displaced: displaced,
|
||||
})
|
||||
}
|
||||
},
|
||||
})
|
||||
}
|
||||
|
||||
func collectOverlayOrigins(origins map[string]Origin, node *document.Node, path Path, layer Layer) {
|
||||
origins[path.String()] = overlayOrigin(layer, node.Location)
|
||||
collectOverlayOriginsFrom(origins, nil, node, path, layer)
|
||||
}
|
||||
|
||||
func collectOverlayOriginsFrom(origins, incomingOrigins map[string]Origin, node *document.Node, path Path, layer Layer) {
|
||||
origins[path.String()] = incomingOrigin(incomingOrigins, path, node, layer)
|
||||
switch node.Kind {
|
||||
case document.Mapping:
|
||||
for _, entry := range node.Mapping {
|
||||
collectOverlayOrigins(origins, entry.Value, path.Key(entry.Key), layer)
|
||||
collectOverlayOriginsFrom(origins, incomingOrigins, entry.Value, path.Key(entry.Key), layer)
|
||||
}
|
||||
case document.Sequence:
|
||||
for index, child := range node.Sequence {
|
||||
collectOverlayOrigins(origins, child, path.Index(index), layer)
|
||||
collectOverlayOriginsFrom(origins, incomingOrigins, child, path.Index(index), layer)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func mergeOverlayOrigin(origins map[string]Origin, path Path, location document.Location, layer Layer) {
|
||||
mergeOriginContribution(origins, path, overlayOrigin(layer, location))
|
||||
}
|
||||
|
||||
func mergeOriginContribution(origins map[string]Origin, path Path, incoming Origin) {
|
||||
pointer := path.String()
|
||||
origin := cloneOrigin(origins[pointer])
|
||||
origin.Layer = layer
|
||||
origin.Locations = append(origin.Locations, publicLocation(location))
|
||||
origin := origins[pointer]
|
||||
origin.Layer = incoming.Layer
|
||||
origin.Locations = append(origin.Locations, incoming.Locations...)
|
||||
origins[pointer] = origin
|
||||
}
|
||||
|
||||
func incomingOrigin(origins map[string]Origin, path Path, node *document.Node, layer Layer) Origin {
|
||||
if origin, ok := origins[path.String()]; ok {
|
||||
origin = cloneOrigin(origin)
|
||||
origin.Layer = layer
|
||||
return origin
|
||||
}
|
||||
return overlayOrigin(layer, node.Location)
|
||||
}
|
||||
|
||||
func overlayOrigin(layer Layer, location document.Location) Origin {
|
||||
return Origin{Layer: layer, Locations: []Location{publicLocation(location)}}
|
||||
}
|
||||
@@ -129,11 +156,16 @@ func collectDisplacedOrigins(displaced *[]displacedDefinition, node *document.No
|
||||
}
|
||||
}
|
||||
|
||||
func deleteOriginSubtree(origins map[string]Origin, path Path) {
|
||||
prefix := path.String()
|
||||
for pointer := range origins {
|
||||
if pointer == prefix || strings.HasPrefix(pointer, prefix+"/") {
|
||||
delete(origins, pointer)
|
||||
func deleteOriginSubtree(origins map[string]Origin, path Path, node *document.Node) {
|
||||
delete(origins, path.String())
|
||||
switch node.Kind {
|
||||
case document.Mapping:
|
||||
for _, entry := range node.Mapping {
|
||||
deleteOriginSubtree(origins, path.Key(entry.Key), entry.Value)
|
||||
}
|
||||
case document.Sequence:
|
||||
for index, child := range node.Sequence {
|
||||
deleteOriginSubtree(origins, path.Index(index), child)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -76,3 +76,44 @@ to_scalar: scalar
|
||||
t.Fatal("overlay result aliases an input document")
|
||||
}
|
||||
}
|
||||
|
||||
func TestOwnedOverlayKeepsCompositeContributorsAndAcquiredInputs(t *testing.T) {
|
||||
baseInput := parseCompositionDocument(t, "source-1", "catalog:\n base: true\n")
|
||||
base, diagnostics := composeDocuments(baseInput)
|
||||
if len(diagnostics) != 0 {
|
||||
t.Fatal(diagnostics)
|
||||
}
|
||||
first := parseCompositionDocument(t, "source-2", "catalog:\n first: true\n")
|
||||
second := parseCompositionDocument(t, "source-3", "catalog:\n second: true\n")
|
||||
firstBefore, secondBefore := document.Clone(first), document.Clone(second)
|
||||
incoming, diagnostics := composeDocuments(first, second)
|
||||
if len(diagnostics) != 0 {
|
||||
t.Fatal(diagnostics)
|
||||
}
|
||||
assembly := newAssemblyAccumulator(base)
|
||||
if diagnostics := assembly.applyOverlay(incoming.root, incoming.origins, incoming.applications, LayerOverlay); len(diagnostics) != 0 {
|
||||
t.Fatal(diagnostics)
|
||||
}
|
||||
for _, key := range []string{"base", "first", "second"} {
|
||||
if _, ok := documentAtPath(assembly.current.root, Keys("catalog", key)); !ok {
|
||||
t.Fatalf("catalog is missing %q", key)
|
||||
}
|
||||
}
|
||||
origin := assembly.current.origins[Keys("catalog").String()]
|
||||
if got, want := origin.Locations, []Location{{SourceID: "source-1", Line: 2, Column: 3}, {SourceID: "source-2", Line: 2, Column: 3}, {SourceID: "source-3", Line: 2, Column: 3}}; !reflect.DeepEqual(got, want) {
|
||||
t.Fatalf("catalog origin locations = %#v, want %#v", got, want)
|
||||
}
|
||||
if len(assembly.current.applications) != 3 {
|
||||
t.Fatalf("history applications = %d, want three source contributions", len(assembly.current.applications))
|
||||
}
|
||||
for _, application := range assembly.current.applications[1:] {
|
||||
if application.layer != LayerOverlay {
|
||||
t.Fatalf("incoming application layer = %q", application.layer)
|
||||
}
|
||||
}
|
||||
assembly.current.root.Mapping[0].Value.Mapping[1].Value.Text = "false"
|
||||
if !reflect.DeepEqual(first, firstBefore) || !reflect.DeepEqual(second, secondBefore) ||
|
||||
!reflect.DeepEqual(first, assembly.current.applications[1].value) || !reflect.DeepEqual(second, assembly.current.applications[2].value) {
|
||||
t.Fatal("working tree mutation changed an acquired input")
|
||||
}
|
||||
}
|
||||
|
||||
@@ -44,7 +44,7 @@ func prepareOverrides(base *composedDocument, sources []SourceInfo, overrides []
|
||||
allSources := cloneSourceInfos(sources)
|
||||
for index, override := range overrides {
|
||||
sourceID := "source-" + strconv.Itoa(len(sources)+index+1)
|
||||
info := SourceInfo{ID: sourceID, Label: override.Flag, Layer: LayerCLI}
|
||||
info := SourceInfo{ID: sourceID, Label: override.Flag, Layer: LayerCLI, Kind: SourceKindCLI}
|
||||
allSources = append(allSources, info)
|
||||
prepared[index].source = info
|
||||
}
|
||||
@@ -94,11 +94,12 @@ func prepareOverrides(base *composedDocument, sources []SourceInfo, overrides []
|
||||
}
|
||||
pathAccepted[index] = true
|
||||
|
||||
builder := overrideValueBuilder{sourceID: prepared[index].source.ID, active: make(map[valueVisit]bool)}
|
||||
builder := valueBuilder{sourceID: prepared[index].source.ID, active: make(map[valueVisit]bool)}
|
||||
value, issue := builder.build(override.Value, override.Path, len(override.Path.segments))
|
||||
if issue != nil {
|
||||
issue.Primary = location
|
||||
diagnostics = append(diagnostics, *issue)
|
||||
diagnostic := issue.overrideDiagnostic()
|
||||
diagnostic.Primary = location
|
||||
diagnostics = append(diagnostics, diagnostic)
|
||||
continue
|
||||
}
|
||||
if relative, ok := overrideDirectivePath(value, Path{}); ok {
|
||||
|
||||
@@ -11,31 +11,42 @@ func applyOverrides(base *composedDocument, sources []SourceInfo, overrides []Ov
|
||||
if len(overrides) == 0 {
|
||||
return base, cloneSourceInfos(sources), nil
|
||||
}
|
||||
prepared, allSources, diagnostics := prepareOverrides(base, sources, overrides, rule, limits)
|
||||
assembly := newAssemblyAccumulator(cloneComposedDocument(base))
|
||||
allSources, diagnostics := assembly.applyOverrides(sources, overrides, rule, limits)
|
||||
if len(diagnostics) != 0 {
|
||||
return nil, allSources, diagnostics
|
||||
}
|
||||
result := cloneComposedDocument(base)
|
||||
for _, override := range prepared {
|
||||
applyPreparedOverride(result, override)
|
||||
result.inputBytes += override.bytes
|
||||
result.inputNodes += override.nodes
|
||||
}
|
||||
return result, allSources, nil
|
||||
return assembly.current, allSources, nil
|
||||
}
|
||||
|
||||
func applyPreparedOverride(result *composedDocument, override preparedOverride) {
|
||||
func (a *assemblyAccumulator) applyOverrides(sources []SourceInfo, overrides []Override, rule *VersionRule, limits importLimits) ([]SourceInfo, []Diagnostic) {
|
||||
if len(overrides) == 0 {
|
||||
return cloneSourceInfos(sources), nil
|
||||
}
|
||||
prepared, allSources, diagnostics := prepareOverrides(a.current, sources, overrides, rule, limits)
|
||||
if len(diagnostics) != 0 {
|
||||
return allSources, diagnostics
|
||||
}
|
||||
for _, override := range prepared {
|
||||
applyPreparedOverride(a.current, override, &a.index)
|
||||
a.current.inputBytes += override.bytes
|
||||
a.current.inputNodes += override.nodes
|
||||
}
|
||||
return allSources, nil
|
||||
}
|
||||
|
||||
func applyPreparedOverride(result *composedDocument, override preparedOverride, index *mappingIndex) {
|
||||
location := document.Location{SourceID: override.source.ID}
|
||||
mergeOverlayOrigin(result.origins, Path{}, location, LayerCLI)
|
||||
current := result.root
|
||||
currentPath := Path{}
|
||||
for index, segment := range override.path.segments {
|
||||
for segmentIndex, segment := range override.path.segments {
|
||||
path := currentPath.Key(segment.key)
|
||||
position := documentEntryIndex(current.Mapping, segment.key)
|
||||
if index < len(override.path.segments)-1 {
|
||||
position := index.find(current, segment.key)
|
||||
if segmentIndex < len(override.path.segments)-1 {
|
||||
if position < 0 {
|
||||
child := &document.Node{Kind: document.Mapping, Location: location, Mapping: []document.Entry{}}
|
||||
current.Mapping = append(current.Mapping, document.Entry{Key: segment.key, KeyLocation: location, ValueLocation: location, Value: child})
|
||||
index.append(current, document.Entry{Key: segment.key, KeyLocation: location, ValueLocation: location, Value: child})
|
||||
collectOverlayOrigins(result.origins, child, path, LayerCLI)
|
||||
current = child
|
||||
} else {
|
||||
@@ -50,16 +61,16 @@ func applyPreparedOverride(result *composedDocument, override preparedOverride)
|
||||
var displaced []displacedDefinition
|
||||
if position < 0 {
|
||||
incoming := document.Clone(override.value)
|
||||
current.Mapping = append(current.Mapping, document.Entry{Key: segment.key, KeyLocation: location, ValueLocation: location, Value: incoming})
|
||||
index.append(current, document.Entry{Key: segment.key, KeyLocation: location, ValueLocation: location, Value: incoming})
|
||||
collectOverlayOrigins(result.origins, incoming, path, LayerCLI)
|
||||
} else {
|
||||
existing := current.Mapping[position].Value
|
||||
displaced = affectedOverrideOrigins(existing, override.value, result.origins, path)
|
||||
displaced = affectedOverrideOrigins(existing, override.value, result.origins, path, index)
|
||||
if existing.Kind == document.Mapping && override.value.Kind == document.Mapping {
|
||||
mergeOverlayOrigin(result.origins, path, location, LayerCLI)
|
||||
mergeOverlayMappings(result, existing, override.value, path, LayerCLI, false)
|
||||
mergeOverlayMappings(result, existing, override.value, path, LayerCLI, false, nil, index)
|
||||
} else {
|
||||
deleteOriginSubtree(result.origins, path)
|
||||
deleteOriginSubtree(result.origins, path, existing)
|
||||
current.Mapping[position] = document.Entry{Key: segment.key, KeyLocation: location, ValueLocation: location, Value: document.Clone(override.value)}
|
||||
collectOverlayOrigins(result.origins, current.Mapping[position].Value, path, LayerCLI)
|
||||
}
|
||||
@@ -74,18 +85,18 @@ func applyPreparedOverride(result *composedDocument, override preparedOverride)
|
||||
}
|
||||
}
|
||||
|
||||
func affectedOverrideOrigins(existing, incoming *document.Node, origins map[string]Origin, path Path) []displacedDefinition {
|
||||
func affectedOverrideOrigins(existing, incoming *document.Node, origins map[string]Origin, path Path, index *mappingIndex) []displacedDefinition {
|
||||
displaced := []displacedDefinition{{path: path.clone(), origin: cloneOrigin(origins[path.String()])}}
|
||||
if existing.Kind == document.Mapping && incoming.Kind == document.Mapping {
|
||||
for _, entry := range incoming.Mapping {
|
||||
position := documentEntryIndex(existing.Mapping, entry.Key)
|
||||
position := index.find(existing, entry.Key)
|
||||
if position < 0 {
|
||||
continue
|
||||
}
|
||||
childPath := path.Key(entry.Key)
|
||||
childExisting := existing.Mapping[position].Value
|
||||
if childExisting.Kind == document.Mapping && entry.Value.Kind == document.Mapping {
|
||||
displaced = append(displaced, affectedOverrideOrigins(childExisting, entry.Value, origins, childPath)...)
|
||||
displaced = append(displaced, affectedOverrideOrigins(childExisting, entry.Value, origins, childPath, index)...)
|
||||
} else {
|
||||
displaced = append(displaced, displacedOrigins(childExisting, origins, childPath)...)
|
||||
}
|
||||
|
||||
@@ -1,150 +1,21 @@
|
||||
package engine
|
||||
|
||||
import (
|
||||
"math"
|
||||
"reflect"
|
||||
"slices"
|
||||
"strconv"
|
||||
|
||||
"gitea.maximumdirect.net/eric/ConfigKit/internal/document"
|
||||
)
|
||||
|
||||
type overrideValueBuilder struct {
|
||||
sourceID string
|
||||
active map[valueVisit]bool
|
||||
bytes int64
|
||||
nodes int
|
||||
}
|
||||
|
||||
func (b *overrideValueBuilder) build(value any, path Path, depth int) (*document.Node, *Diagnostic) {
|
||||
if depth > maxDocumentDepth {
|
||||
return nil, overrideValueDiagnostic(CodeOverrideLimitExceeded, path, "configuration override exceeds the depth limit")
|
||||
}
|
||||
b.nodes++
|
||||
if b.nodes > maxDocumentNodes {
|
||||
return nil, overrideValueDiagnostic(CodeOverrideLimitExceeded, path, "configuration override exceeds the node limit")
|
||||
}
|
||||
location := document.Location{SourceID: b.sourceID}
|
||||
switch value := value.(type) {
|
||||
case nil:
|
||||
return &document.Node{Kind: document.Null, Location: location}, nil
|
||||
case string:
|
||||
if !b.addBytes(len(value)) {
|
||||
return nil, overrideValueDiagnostic(CodeOverrideLimitExceeded, path, "configuration override exceeds the record string-byte limit")
|
||||
}
|
||||
return &document.Node{Kind: document.String, Text: value, Location: location}, nil
|
||||
case bool:
|
||||
return &document.Node{Kind: document.Boolean, Text: strconv.FormatBool(value), Location: location}, nil
|
||||
case int:
|
||||
return overrideNumber(strconv.FormatInt(int64(value), 10), location), nil
|
||||
case int8:
|
||||
return overrideNumber(strconv.FormatInt(int64(value), 10), location), nil
|
||||
case int16:
|
||||
return overrideNumber(strconv.FormatInt(int64(value), 10), location), nil
|
||||
case int32:
|
||||
return overrideNumber(strconv.FormatInt(int64(value), 10), location), nil
|
||||
case int64:
|
||||
return overrideNumber(strconv.FormatInt(value, 10), location), nil
|
||||
case uint:
|
||||
return overrideNumber(strconv.FormatUint(uint64(value), 10), location), nil
|
||||
case uint8:
|
||||
return overrideNumber(strconv.FormatUint(uint64(value), 10), location), nil
|
||||
case uint16:
|
||||
return overrideNumber(strconv.FormatUint(uint64(value), 10), location), nil
|
||||
case uint32:
|
||||
return overrideNumber(strconv.FormatUint(uint64(value), 10), location), nil
|
||||
case uint64:
|
||||
return overrideNumber(strconv.FormatUint(value, 10), location), nil
|
||||
case float32:
|
||||
if math.IsNaN(float64(value)) || math.IsInf(float64(value), 0) {
|
||||
return nil, overrideValueDiagnostic(CodeOverrideInvalidValue, path, "configuration override float must be finite")
|
||||
}
|
||||
return overrideNumber(strconv.FormatFloat(float64(value), 'g', -1, 32), location), nil
|
||||
case float64:
|
||||
if math.IsNaN(value) || math.IsInf(value, 0) {
|
||||
return nil, overrideValueDiagnostic(CodeOverrideInvalidValue, path, "configuration override float must be finite")
|
||||
}
|
||||
return overrideNumber(strconv.FormatFloat(value, 'g', -1, 64), location), nil
|
||||
case []any:
|
||||
if value == nil {
|
||||
return &document.Node{Kind: document.Null, Location: location}, nil
|
||||
}
|
||||
leave, diagnostic := b.enter(reflect.ValueOf(value), path)
|
||||
if diagnostic != nil {
|
||||
return nil, diagnostic
|
||||
}
|
||||
defer leave()
|
||||
if len(value) > maxDocumentNodes-b.nodes {
|
||||
return nil, overrideValueDiagnostic(CodeOverrideLimitExceeded, path, "configuration override exceeds the node limit")
|
||||
}
|
||||
node := &document.Node{Kind: document.Sequence, Location: location, Sequence: make([]*document.Node, len(value))}
|
||||
for index, child := range value {
|
||||
built, diagnostic := b.build(child, path.Index(index), depth+1)
|
||||
if diagnostic != nil {
|
||||
return nil, diagnostic
|
||||
}
|
||||
node.Sequence[index] = built
|
||||
}
|
||||
return node, nil
|
||||
case map[string]any:
|
||||
if value == nil {
|
||||
return &document.Node{Kind: document.Null, Location: location}, nil
|
||||
}
|
||||
leave, diagnostic := b.enter(reflect.ValueOf(value), path)
|
||||
if diagnostic != nil {
|
||||
return nil, diagnostic
|
||||
}
|
||||
defer leave()
|
||||
if len(value) > (maxDocumentNodes-b.nodes)/2 {
|
||||
return nil, overrideValueDiagnostic(CodeOverrideLimitExceeded, path, "configuration override exceeds the node limit")
|
||||
}
|
||||
keys := make([]string, 0, len(value))
|
||||
for key := range value {
|
||||
keys = append(keys, key)
|
||||
}
|
||||
slices.Sort(keys)
|
||||
node := &document.Node{Kind: document.Mapping, Location: location, Mapping: make([]document.Entry, 0, len(keys))}
|
||||
for _, key := range keys {
|
||||
b.nodes++ // mapping key
|
||||
if !b.addBytes(len(key)) {
|
||||
return nil, overrideValueDiagnostic(CodeOverrideLimitExceeded, path.Key(key), "configuration override exceeds the record string-byte limit")
|
||||
}
|
||||
built, diagnostic := b.build(value[key], path.Key(key), depth+1)
|
||||
if diagnostic != nil {
|
||||
return nil, diagnostic
|
||||
}
|
||||
node.Mapping = append(node.Mapping, document.Entry{Key: key, KeyLocation: location, ValueLocation: location, Value: built})
|
||||
}
|
||||
return node, nil
|
||||
func (issue *valueIssue) overrideDiagnostic() Diagnostic {
|
||||
code := CodeOverrideInvalidValue
|
||||
var message string
|
||||
switch issue.kind {
|
||||
case valueDepthLimit:
|
||||
code, message = CodeOverrideLimitExceeded, "configuration override exceeds the depth limit"
|
||||
case valueNodeLimit:
|
||||
code, message = CodeOverrideLimitExceeded, "configuration override exceeds the node limit"
|
||||
case valueByteLimit:
|
||||
code, message = CodeOverrideLimitExceeded, "configuration override exceeds the record string-byte limit"
|
||||
case valueNonFinite:
|
||||
message = "configuration override float must be finite"
|
||||
case valueCycle:
|
||||
message = "configuration override values must not contain cycles"
|
||||
default:
|
||||
return nil, overrideValueDiagnostic(CodeOverrideInvalidValue, path, "configuration override value has an unsupported type")
|
||||
message = "configuration override value has an unsupported type"
|
||||
}
|
||||
}
|
||||
|
||||
func (b *overrideValueBuilder) addBytes(count int) bool {
|
||||
if count < 0 || int64(count) > maxSourceBytes-b.bytes {
|
||||
return false
|
||||
}
|
||||
b.bytes += int64(count)
|
||||
return true
|
||||
}
|
||||
|
||||
func (b *overrideValueBuilder) enter(value reflect.Value, path Path) (func(), *Diagnostic) {
|
||||
visit, ok := visitFor(value)
|
||||
if !ok {
|
||||
return func() {}, nil
|
||||
}
|
||||
if b.active[visit] {
|
||||
return nil, overrideValueDiagnostic(CodeOverrideInvalidValue, path, "configuration override values must not contain cycles")
|
||||
}
|
||||
b.active[visit] = true
|
||||
return func() { delete(b.active, visit) }, nil
|
||||
}
|
||||
|
||||
func overrideValueDiagnostic(code Code, path Path, message string) *Diagnostic {
|
||||
return &Diagnostic{Code: code, Severity: SeverityError, Path: path.clone(), Message: message}
|
||||
}
|
||||
|
||||
func overrideNumber(text string, location document.Location) *document.Node {
|
||||
return &document.Node{Kind: document.Number, Text: text, Location: location}
|
||||
return Diagnostic{Code: code, Severity: SeverityError, Path: issue.path.clone(), Message: message}
|
||||
}
|
||||
|
||||
@@ -53,6 +53,9 @@ func parseSource(source Source, sourceID string, limits parseLimits) (parsedSour
|
||||
if len(diagnostics) != 0 || read.info == nil {
|
||||
return parsedSource{source: read.info}, diagnostics
|
||||
}
|
||||
if read.document != nil {
|
||||
return parsedSource{document: read.document, source: read.info, bytes: read.bytes}, nil
|
||||
}
|
||||
|
||||
root, issues := yamladapter.Parse(bytes.NewReader(read.data), sourceID, limits.document)
|
||||
if len(issues) != 0 {
|
||||
|
||||
@@ -6,11 +6,24 @@ type Layer string
|
||||
const (
|
||||
LayerDefault Layer = "default"
|
||||
LayerBase Layer = "base"
|
||||
LayerUnderlay Layer = "underlay"
|
||||
LayerOverlay Layer = "overlay"
|
||||
LayerCLI Layer = "cli"
|
||||
LayerResolution Layer = "resolution"
|
||||
)
|
||||
|
||||
// SourceKind identifies source acquisition independently of precedence.
|
||||
type SourceKind string
|
||||
|
||||
const (
|
||||
SourceKindFile SourceKind = "file"
|
||||
SourceKindFilesystem SourceKind = "filesystem"
|
||||
SourceKindBytes SourceKind = "bytes"
|
||||
SourceKindReader SourceKind = "reader"
|
||||
SourceKindMapping SourceKind = "mapping"
|
||||
SourceKindCLI SourceKind = "cli"
|
||||
)
|
||||
|
||||
// ChangeKind identifies a recorded operation. The zero value is
|
||||
// used for input and default origins.
|
||||
type ChangeKind string
|
||||
@@ -18,7 +31,10 @@ type ChangeKind string
|
||||
const (
|
||||
// ChangeDecoding records the built-in transition to canonical typed values.
|
||||
// It is inspection metadata, not an operation accepted by Resolution.Record.
|
||||
ChangeDecoding ChangeKind = "decoding"
|
||||
ChangeDecoding ChangeKind = "decoding"
|
||||
// ChangeInheritance records structural input expansion. It is inspection
|
||||
// metadata, not an operation accepted by Resolution.Record.
|
||||
ChangeInheritance ChangeKind = "inheritance"
|
||||
ChangeDerivedDefault ChangeKind = "derived_default"
|
||||
ChangeNormalization ChangeKind = "normalization"
|
||||
)
|
||||
@@ -89,6 +105,7 @@ const (
|
||||
CodeImportRepeated Code = "import.repeated"
|
||||
CodeImportLimitExceeded Code = "import.limit_exceeded"
|
||||
CodeImportVersion Code = "import.version"
|
||||
CodeUnderlayVersion Code = "underlay.version"
|
||||
|
||||
CodeOverlayInvalid Code = "overlay.invalid"
|
||||
CodeOverlayMissing Code = "overlay.missing"
|
||||
@@ -146,6 +163,11 @@ const (
|
||||
CodeOverrideLimitExceeded Code = "override.limit_exceeded"
|
||||
|
||||
CodeExplainLimitExceeded Code = "explain.limit_exceeded"
|
||||
|
||||
CodeInheritanceInvalid Code = "inheritance.invalid"
|
||||
CodeInheritanceMissing Code = "inheritance.missing"
|
||||
CodeInheritanceCycle Code = "inheritance.cycle"
|
||||
CodeInheritanceVersion Code = "inheritance.version"
|
||||
)
|
||||
|
||||
// Location identifies a position in a configuration source. Zero line and column
|
||||
@@ -156,15 +178,19 @@ type Location struct {
|
||||
Column int
|
||||
}
|
||||
|
||||
// SourceInfo describes a source discovered during a load. Path is a canonical
|
||||
// local path when one exists. ImportedFrom identifies the declaration that
|
||||
// included this source.
|
||||
// SourceInfo describes a source discovered during a load. Kind identifies
|
||||
// acquisition independently of Layer. Path is a canonical host path or a
|
||||
// namespace-relative filesystem path; synthetic sources have no Path.
|
||||
// ImportedFrom identifies the declaration that included this source.
|
||||
type SourceInfo struct {
|
||||
ID string
|
||||
Label string
|
||||
Path string
|
||||
Layer Layer
|
||||
ImportedFrom *Location
|
||||
ID string
|
||||
Label string
|
||||
Path string
|
||||
Layer Layer
|
||||
Kind SourceKind
|
||||
UnderlayIndex int
|
||||
ImportedFrom *Location
|
||||
namespace *namespaceToken
|
||||
}
|
||||
|
||||
// Diagnostic is a safe structured problem report. Message and source labels must
|
||||
@@ -178,13 +204,25 @@ type Diagnostic struct {
|
||||
Related []Location
|
||||
}
|
||||
|
||||
// InheritanceStep identifies a concrete payload transfer controlled by a
|
||||
// relationship directive. Locations belong to that directive, not its value.
|
||||
type InheritanceStep struct {
|
||||
From Path
|
||||
To Path
|
||||
Reference Path
|
||||
Locations []Location
|
||||
}
|
||||
|
||||
// Origin describes where an effective definition came from. Multiple locations
|
||||
// represent a composite origin; resolution origins are synthetic and identify their
|
||||
// operation instead of a file location.
|
||||
// operation instead of a file location. Inheritance is projected from private
|
||||
// contributor traces at inspection boundaries.
|
||||
type Origin struct {
|
||||
Layer Layer
|
||||
Locations []Location
|
||||
Operation ChangeKind
|
||||
Layer Layer
|
||||
Locations []Location
|
||||
Operation ChangeKind
|
||||
Inheritance []InheritanceStep
|
||||
contributors []inheritanceContributor
|
||||
}
|
||||
|
||||
// FieldInfo describes an effective configuration path.
|
||||
@@ -238,9 +276,32 @@ func cloneSourceInfos(sources []SourceInfo) []SourceInfo {
|
||||
|
||||
func cloneOrigin(origin Origin) Origin {
|
||||
origin.Locations = cloneLocations(origin.Locations)
|
||||
origin.Inheritance = cloneInheritanceSteps(origin.Inheritance)
|
||||
if origin.contributors != nil {
|
||||
contributors := make([]inheritanceContributor, len(origin.contributors))
|
||||
for index, contributor := range origin.contributors {
|
||||
contributors[index] = contributor
|
||||
contributors[index].source = contributor.source.clone()
|
||||
}
|
||||
origin.contributors = contributors
|
||||
}
|
||||
return origin
|
||||
}
|
||||
|
||||
func cloneInheritanceSteps(steps []InheritanceStep) []InheritanceStep {
|
||||
if steps == nil {
|
||||
return nil
|
||||
}
|
||||
cloned := make([]InheritanceStep, len(steps))
|
||||
for index, step := range steps {
|
||||
cloned[index] = InheritanceStep{
|
||||
From: step.From.clone(), To: step.To.clone(), Reference: step.Reference.clone(),
|
||||
Locations: cloneLocations(step.Locations),
|
||||
}
|
||||
}
|
||||
return cloned
|
||||
}
|
||||
|
||||
func cloneFieldInfo(info FieldInfo) FieldInfo {
|
||||
info.Origin = cloneOrigin(info.Origin)
|
||||
return info
|
||||
|
||||
45
internal/engine/recursive_merge.go
Normal file
45
internal/engine/recursive_merge.go
Normal file
@@ -0,0 +1,45 @@
|
||||
package engine
|
||||
|
||||
import "gitea.maximumdirect.net/eric/ConfigKit/internal/document"
|
||||
|
||||
// recursiveMergeHooks lets an assembly owner account for the values changed by
|
||||
// a recursive mapping overlay without making source roles part of the merge.
|
||||
// Hooks run before replacement and receive the still-owned incoming subtree.
|
||||
type recursiveMergeHooks struct {
|
||||
added func(Path, *document.Node)
|
||||
combined func(Path, *document.Node)
|
||||
replaced func(Path, *document.Node, *document.Node)
|
||||
}
|
||||
|
||||
// mergeRecursiveMappings changes an owned mapping and never aliases incoming
|
||||
// values into it. Entry order follows the target, then new incoming entries.
|
||||
func mergeRecursiveMappings(target, incoming *document.Node, current Path, index *mappingIndex, hooks recursiveMergeHooks) {
|
||||
for _, entry := range incoming.Mapping {
|
||||
path := current.Key(entry.Key)
|
||||
position := index.find(target, entry.Key)
|
||||
if position < 0 {
|
||||
cloned := entry
|
||||
cloned.Value = document.Clone(entry.Value)
|
||||
index.append(target, cloned)
|
||||
if hooks.added != nil {
|
||||
hooks.added(path, entry.Value)
|
||||
}
|
||||
continue
|
||||
}
|
||||
|
||||
existing := target.Mapping[position].Value
|
||||
if existing.Kind == document.Mapping && entry.Value.Kind == document.Mapping {
|
||||
if hooks.combined != nil {
|
||||
hooks.combined(path, entry.Value)
|
||||
}
|
||||
mergeRecursiveMappings(existing, entry.Value, path, index, hooks)
|
||||
continue
|
||||
}
|
||||
|
||||
if hooks.replaced != nil {
|
||||
hooks.replaced(path, existing, entry.Value)
|
||||
}
|
||||
target.Mapping[position] = entry
|
||||
target.Mapping[position].Value = document.Clone(entry.Value)
|
||||
}
|
||||
}
|
||||
90
internal/engine/recursive_merge_test.go
Normal file
90
internal/engine/recursive_merge_test.go
Normal file
@@ -0,0 +1,90 @@
|
||||
package engine
|
||||
|
||||
import (
|
||||
"reflect"
|
||||
"testing"
|
||||
|
||||
"gitea.maximumdirect.net/eric/ConfigKit/internal/document"
|
||||
)
|
||||
|
||||
func TestRecursiveMappingMergePreservesValuesPathsAndInputs(t *testing.T) {
|
||||
type expectedNode struct {
|
||||
path Path
|
||||
kind document.Kind
|
||||
text string
|
||||
entries int
|
||||
}
|
||||
tests := []struct {
|
||||
name, base, incoming string
|
||||
want []expectedNode
|
||||
events []string
|
||||
}{
|
||||
{
|
||||
name: "nested mappings", base: "shared: {kept: base, changed: old}\n", incoming: "shared: {changed: new, added: child}\n",
|
||||
want: []expectedNode{
|
||||
{path: Keys("shared"), kind: document.Mapping, entries: 3},
|
||||
{path: Keys("shared", "kept"), kind: document.String, text: "base"},
|
||||
{path: Keys("shared", "changed"), kind: document.String, text: "new"},
|
||||
{path: Keys("shared", "added"), kind: document.String, text: "child"},
|
||||
},
|
||||
events: []string{"combine:/shared", "replace:/shared/changed", "add:/shared/added"},
|
||||
},
|
||||
{
|
||||
name: "sequence replacement", base: "value: [old]\n", incoming: "value: []\n",
|
||||
want: []expectedNode{{path: Keys("value"), kind: document.Sequence}},
|
||||
events: []string{"replace:/value"},
|
||||
},
|
||||
{
|
||||
name: "scalar replacement", base: "value: old\n", incoming: "value: new\n",
|
||||
want: []expectedNode{{path: Keys("value"), kind: document.String, text: "new"}},
|
||||
events: []string{"replace:/value"},
|
||||
},
|
||||
{
|
||||
name: "null replacement", base: "value: old\n", incoming: "value: null\n",
|
||||
want: []expectedNode{{path: Keys("value"), kind: document.Null}},
|
||||
events: []string{"replace:/value"},
|
||||
},
|
||||
{
|
||||
name: "empty mapping retains children", base: "shared: {kept: base}\n", incoming: "shared: {}\n",
|
||||
want: []expectedNode{
|
||||
{path: Keys("shared"), kind: document.Mapping, entries: 1},
|
||||
{path: Keys("shared", "kept"), kind: document.String, text: "base"},
|
||||
},
|
||||
events: []string{"combine:/shared"},
|
||||
},
|
||||
}
|
||||
for _, test := range tests {
|
||||
t.Run(test.name, func(t *testing.T) {
|
||||
base := parseCompositionDocument(t, "source-1", test.base)
|
||||
incoming := parseCompositionDocument(t, "source-2", test.incoming)
|
||||
baseBefore, incomingBefore := document.Clone(base), document.Clone(incoming)
|
||||
owned := document.Clone(base)
|
||||
var events []string
|
||||
mergeRecursiveMappings(owned, incoming, Path{}, new(mappingIndex), recursiveMergeHooks{
|
||||
added: func(path Path, _ *document.Node) { events = append(events, "add:"+path.String()) },
|
||||
combined: func(path Path, _ *document.Node) {
|
||||
events = append(events, "combine:"+path.String())
|
||||
},
|
||||
replaced: func(path Path, _, _ *document.Node) {
|
||||
events = append(events, "replace:"+path.String())
|
||||
},
|
||||
})
|
||||
if !reflect.DeepEqual(events, test.events) {
|
||||
t.Fatalf("merge events = %#v, want %#v", events, test.events)
|
||||
}
|
||||
for _, expected := range test.want {
|
||||
node, found := documentAtPath(owned, expected.path)
|
||||
if !found || node.Kind != expected.kind || node.Text != expected.text || len(node.Mapping) != expected.entries {
|
||||
t.Errorf("merged %s = %#v, found %v", expected.path.String(), node, found)
|
||||
}
|
||||
}
|
||||
if !reflect.DeepEqual(base, baseBefore) || !reflect.DeepEqual(incoming, incomingBefore) {
|
||||
t.Fatal("merge changed an input")
|
||||
}
|
||||
owned.Mapping[0].Value.Location.SourceID = "changed"
|
||||
if !reflect.DeepEqual(base, baseBefore) || !reflect.DeepEqual(incoming, incomingBefore) {
|
||||
t.Fatal("merged result aliases an input")
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
@@ -41,7 +41,7 @@ type resolutionResult struct {
|
||||
diagnostics []Diagnostic
|
||||
}
|
||||
|
||||
// IsExplicit reports whether surviving input explicitly defined path.
|
||||
// IsExplicit reports whether surviving direct or inherited input defined path.
|
||||
func (r *Resolution) IsExplicit(path Path) bool {
|
||||
state := r.activeState()
|
||||
if !state.validPath(path) {
|
||||
@@ -72,6 +72,12 @@ func (r *Resolution) Source(id string) (SourceInfo, bool) {
|
||||
return r.activeState().input.source(id)
|
||||
}
|
||||
|
||||
// SourceContext returns a passive context during the active resolution
|
||||
// callback. Retaining the callback context after return is unsupported.
|
||||
func (r *Resolution) SourceContext(id string) (ResourceContext, bool) {
|
||||
return r.activeState().input.sourceContext(id)
|
||||
}
|
||||
|
||||
// Record declares the intent for a resolver mutation. A container path covers
|
||||
// mutations to its descendants.
|
||||
func (r *Resolution) Record(path Path, kind ChangeKind, reason string, dependencies ...Path) {
|
||||
|
||||
133
internal/engine/resource_context.go
Normal file
133
internal/engine/resource_context.go
Normal file
@@ -0,0 +1,133 @@
|
||||
package engine
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"io/fs"
|
||||
"os"
|
||||
"path"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// ResourceContext is a passive copy of one acquired source's rooted filesystem
|
||||
// and slash-relative base directory. FS is borrowed and opens resources only
|
||||
// when the application explicitly requests them.
|
||||
type ResourceContext struct {
|
||||
FS fs.FS
|
||||
BaseDir string
|
||||
}
|
||||
|
||||
func cloneResourceContexts(contexts map[string]ResourceContext) map[string]ResourceContext {
|
||||
if contexts == nil {
|
||||
return nil
|
||||
}
|
||||
cloned := make(map[string]ResourceContext, len(contexts))
|
||||
for id, context := range contexts {
|
||||
cloned[id] = context
|
||||
}
|
||||
return cloned
|
||||
}
|
||||
|
||||
// rootedNativeFS borrows a namespace without invoking SubFS or opening it at
|
||||
// query time. All resource names remain relative to its configured root.
|
||||
type rootedNativeFS struct {
|
||||
filesystem fs.FS
|
||||
root string
|
||||
}
|
||||
|
||||
func (r rootedNativeFS) name(name string) (string, error) {
|
||||
if !fs.ValidPath(name) {
|
||||
return "", &fs.PathError{Op: "open", Path: name, Err: fs.ErrInvalid}
|
||||
}
|
||||
joined := path.Join(r.root, name)
|
||||
if !fs.ValidPath(joined) || r.root != "." && joined != r.root && !strings.HasPrefix(joined, r.root+"/") {
|
||||
return "", &fs.PathError{Op: "open", Path: name, Err: fs.ErrPermission}
|
||||
}
|
||||
return joined, nil
|
||||
}
|
||||
|
||||
func (r rootedNativeFS) Open(name string) (fs.File, error) {
|
||||
joined, err := r.name(name)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return r.filesystem.Open(joined)
|
||||
}
|
||||
|
||||
func (r rootedNativeFS) ReadDir(name string) ([]fs.DirEntry, error) {
|
||||
joined, err := r.name(name)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return fs.ReadDir(r.filesystem, joined)
|
||||
}
|
||||
|
||||
// rootedHostFS stores only a canonical root path. No os.Root is retained after
|
||||
// loading or created by a SourceContext query.
|
||||
type hostResourceRoot interface {
|
||||
Open(string) (*os.File, error)
|
||||
Close() error
|
||||
}
|
||||
|
||||
type rootedHostFS struct {
|
||||
root string
|
||||
openRoot func(string) (hostResourceRoot, error)
|
||||
}
|
||||
|
||||
func (r rootedHostFS) Open(name string) (fs.File, error) {
|
||||
localName, err := filepath.Localize(name)
|
||||
if err != nil {
|
||||
return nil, &fs.PathError{Op: "open", Path: name, Err: fs.ErrInvalid}
|
||||
}
|
||||
opener := r.openRoot
|
||||
if opener == nil {
|
||||
opener = func(name string) (hostResourceRoot, error) { return os.OpenRoot(name) }
|
||||
}
|
||||
root, err := opener(r.root)
|
||||
if err != nil {
|
||||
return nil, &fs.PathError{Op: "open", Path: name, Err: err}
|
||||
}
|
||||
canonical, err := canonicalExistingPath(filepath.Join(r.root, localName))
|
||||
if err != nil {
|
||||
_ = root.Close()
|
||||
return nil, &fs.PathError{Op: "open", Path: name, Err: err}
|
||||
}
|
||||
if !containedPath(r.root, canonical) {
|
||||
_ = root.Close()
|
||||
return nil, &fs.PathError{Op: "open", Path: name, Err: fs.ErrPermission}
|
||||
}
|
||||
relative, err := filepath.Rel(r.root, canonical)
|
||||
if err != nil {
|
||||
_ = root.Close()
|
||||
return nil, &fs.PathError{Op: "open", Path: name, Err: err}
|
||||
}
|
||||
file, err := root.Open(relative)
|
||||
if err != nil {
|
||||
_ = root.Close()
|
||||
return nil, &fs.PathError{Op: "open", Path: name, Err: err}
|
||||
}
|
||||
owned := rootedHostFile{File: file, root: root}
|
||||
var opened fs.File = file
|
||||
if directory, ok := opened.(fs.ReadDirFile); ok {
|
||||
return &rootedHostDirFile{rootedHostFile: owned, directory: directory}, nil
|
||||
}
|
||||
return &owned, nil
|
||||
}
|
||||
|
||||
type rootedHostFile struct {
|
||||
fs.File
|
||||
root hostResourceRoot
|
||||
}
|
||||
|
||||
func (f *rootedHostFile) Close() error {
|
||||
return errors.Join(f.File.Close(), f.root.Close())
|
||||
}
|
||||
|
||||
type rootedHostDirFile struct {
|
||||
rootedHostFile
|
||||
directory fs.ReadDirFile
|
||||
}
|
||||
|
||||
func (f *rootedHostDirFile) ReadDir(count int) ([]fs.DirEntry, error) {
|
||||
return f.directory.ReadDir(count)
|
||||
}
|
||||
169
internal/engine/resource_context_test.go
Normal file
169
internal/engine/resource_context_test.go
Normal file
@@ -0,0 +1,169 @@
|
||||
package engine
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"io/fs"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"runtime"
|
||||
"testing"
|
||||
"testing/fstest"
|
||||
)
|
||||
|
||||
func TestResourceContextsPreserveLiteralBackslashes(t *testing.T) {
|
||||
const directory = `assets\literal`
|
||||
const name = directory + `/text\file`
|
||||
check := func(t *testing.T, filesystem fs.FS) {
|
||||
t.Helper()
|
||||
data, err := fs.ReadFile(filesystem, name)
|
||||
if err != nil || string(data) != "content" {
|
||||
t.Fatalf("read = %q, %v", data, err)
|
||||
}
|
||||
entries, err := fs.ReadDir(filesystem, directory)
|
||||
if err != nil || len(entries) != 1 || entries[0].Name() != `text\file` {
|
||||
t.Fatalf("entries = %v, %v", entries, err)
|
||||
}
|
||||
if _, err := filesystem.Open("../outside"); !errors.Is(err, fs.ErrInvalid) {
|
||||
t.Fatalf("escape = %v", err)
|
||||
}
|
||||
}
|
||||
t.Run("native", func(t *testing.T) {
|
||||
check(t, rootedNativeFS{filesystem: fstest.MapFS{"bundle/" + name: &fstest.MapFile{Data: []byte("content")}}, root: "bundle"})
|
||||
})
|
||||
t.Run("host", func(t *testing.T) {
|
||||
if runtime.GOOS == "windows" {
|
||||
t.Skip("backslash is a host separator on Windows; native case remains covered")
|
||||
}
|
||||
root := t.TempDir()
|
||||
if err := os.Mkdir(filepath.Join(root, directory), 0o700); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := os.WriteFile(filepath.Join(root, filepath.FromSlash(name)), []byte("content"), 0o600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
check(t, rootedHostFS{root: root})
|
||||
})
|
||||
}
|
||||
|
||||
type trackedHostRoot struct {
|
||||
root *os.Root
|
||||
closed *int
|
||||
failOpen bool
|
||||
}
|
||||
|
||||
func (r *trackedHostRoot) Open(name string) (*os.File, error) {
|
||||
if r.failOpen {
|
||||
return nil, fs.ErrPermission
|
||||
}
|
||||
return r.root.Open(name)
|
||||
}
|
||||
func (r *trackedHostRoot) Close() error {
|
||||
(*r.closed)++
|
||||
return r.root.Close()
|
||||
}
|
||||
|
||||
func TestRootedHostFSClosesRootOnSuccessfulAndFailedOpen(t *testing.T) {
|
||||
rootPath := t.TempDir()
|
||||
if err := os.WriteFile(filepath.Join(rootPath, "resource.txt"), []byte("resource"), 0o600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
opened, closed := 0, 0
|
||||
failOpen := false
|
||||
filesystem := rootedHostFS{root: rootPath, openRoot: func(name string) (hostResourceRoot, error) {
|
||||
root, err := os.OpenRoot(name)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
opened++
|
||||
return &trackedHostRoot{root: root, closed: &closed, failOpen: failOpen}, nil
|
||||
}}
|
||||
if _, err := filesystem.Open("../resource.txt"); !errors.Is(err, fs.ErrInvalid) || opened != 0 {
|
||||
t.Fatalf("invalid path = %v; roots opened %d", err, opened)
|
||||
}
|
||||
if _, err := filesystem.Open("missing.txt"); !errors.Is(err, fs.ErrNotExist) || opened != 1 || closed != 1 {
|
||||
t.Fatalf("missing resource = %v; opened %d, closed %d", err, opened, closed)
|
||||
}
|
||||
failOpen = true
|
||||
if _, err := filesystem.Open("resource.txt"); !errors.Is(err, fs.ErrPermission) || opened != 2 || closed != 2 {
|
||||
t.Fatalf("failed inner open = %v; opened %d, closed %d", err, opened, closed)
|
||||
}
|
||||
failOpen = false
|
||||
file, err := filesystem.Open("resource.txt")
|
||||
if err != nil || opened != 3 || closed != 2 {
|
||||
t.Fatalf("resource open = %v; opened %d, closed %d", err, opened, closed)
|
||||
}
|
||||
if err := file.Close(); err != nil || closed != 3 {
|
||||
t.Fatalf("resource close = %v; opened %d, closed %d", err, opened, closed)
|
||||
}
|
||||
if _, err := file.Read(make([]byte, 1)); err == nil {
|
||||
t.Fatal("closed resource file remained readable")
|
||||
}
|
||||
directory, err := filesystem.Open(".")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
reader, ok := directory.(fs.ReadDirFile)
|
||||
if !ok {
|
||||
t.Fatal("rooted host directory did not expose ReadDirFile")
|
||||
}
|
||||
entries, err := reader.ReadDir(-1)
|
||||
if err != nil || len(entries) != 1 || entries[0].Name() != "resource.txt" {
|
||||
t.Fatalf("directory entries = %#v, %v", entries, err)
|
||||
}
|
||||
if err := directory.Close(); err != nil || opened != 4 || closed != 4 {
|
||||
t.Fatalf("directory close = %v; opened %d, closed %d", err, opened, closed)
|
||||
}
|
||||
}
|
||||
|
||||
func TestNativeResourceViewIsBorrowedAndRooted(t *testing.T) {
|
||||
filesystem := &observedFilesystem{files: fstest.MapFS{
|
||||
"bundle/config/resource.txt": &fstest.MapFile{Data: []byte("content")},
|
||||
}}
|
||||
view := rootedNativeFS{filesystem: filesystem, root: "bundle"}
|
||||
if filesystem.opened != 0 {
|
||||
t.Fatal("constructing a native resource view opened files")
|
||||
}
|
||||
data, err := fs.ReadFile(view, "config/resource.txt")
|
||||
if err != nil || string(data) != "content" {
|
||||
t.Fatalf("resource = %q, %v", data, err)
|
||||
}
|
||||
filesystem.assertClosed(t)
|
||||
if _, err := fs.ReadFile(view, "../outside.txt"); !errors.Is(err, fs.ErrInvalid) {
|
||||
t.Fatalf("resource escape = %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestLoadingClosesNativeFilesAcrossPipelineOutcomes(t *testing.T) {
|
||||
type config struct {
|
||||
Value string `yaml:"value"`
|
||||
Version string `yaml:"version"`
|
||||
}
|
||||
for _, test := range []struct {
|
||||
name string
|
||||
main string
|
||||
child string
|
||||
failRead string
|
||||
options Options[config]
|
||||
wantErr bool
|
||||
}{
|
||||
{name: "success", main: "value: retained\n"},
|
||||
{name: "acquisition", main: "value: retained\n", failRead: "main.yml", wantErr: true},
|
||||
{name: "parse", main: "value: [broken\n", wantErr: true},
|
||||
{name: "composition", main: "configkit: {imports: [{path: child.yml}]}\nvalue: first\n", child: "value: second\n", options: Options[config]{Imports: true}, wantErr: true},
|
||||
{name: "version", main: "version: wrong\n", options: Options[config]{Definition: Definition[config]{Version: &VersionRule{Path: Keys("version"), Check: func(Version, *Validation) error { return fs.ErrInvalid }}}}, wantErr: true},
|
||||
{name: "validation", main: "value: retained\n", options: Options[config]{Definition: Definition[config]{Validate: func(_ *config, validation *Validation) { validation.Error(Keys("value"), "rejected") }}}, wantErr: true},
|
||||
} {
|
||||
t.Run(test.name, func(t *testing.T) {
|
||||
files := fstest.MapFS{"main.yml": &fstest.MapFile{Data: []byte(test.main)}}
|
||||
if test.child != "" {
|
||||
files["child.yml"] = &fstest.MapFile{Data: []byte(test.child)}
|
||||
}
|
||||
filesystem := &observedFilesystem{files: files, failRead: test.failRead}
|
||||
_, err := testLoad(filesystemFile(newNativeNamespace("bundle", filesystem), "main.yml"), test.options)
|
||||
if (err != nil) != test.wantErr {
|
||||
t.Fatalf("load error = %v, want failure %v", err, test.wantErr)
|
||||
}
|
||||
filesystem.assertClosed(t)
|
||||
})
|
||||
}
|
||||
}
|
||||
@@ -34,6 +34,35 @@ func classifySecretPaths(schema *schemaDescriptor, predicate func(Path) bool, me
|
||||
}
|
||||
}
|
||||
}
|
||||
var inherited []secretDependency
|
||||
for _, operation := range metadata.inheritance {
|
||||
if operation.control != nil {
|
||||
control := operation.control.path
|
||||
paths[control.String()] = control.clone()
|
||||
for _, contributor := range operation.controlOrigin.contributors {
|
||||
paths[contributor.source.String()] = contributor.source.clone()
|
||||
inherited = append(inherited, secretDependency{target: control.clone(), dependency: contributor.source.clone()})
|
||||
}
|
||||
continue
|
||||
}
|
||||
if operation.transfer == nil {
|
||||
continue
|
||||
}
|
||||
for pointer := range operation.parentOrigins {
|
||||
source, err := ParsePath(pointer)
|
||||
if err != nil {
|
||||
continue
|
||||
}
|
||||
relative, ok := relativeHistoryPath(operation.transfer.from, source)
|
||||
if !ok {
|
||||
continue
|
||||
}
|
||||
destination := appendInheritancePath(operation.transfer.to, relative)
|
||||
paths[pointer] = source
|
||||
paths[destination.String()] = destination.clone()
|
||||
inherited = append(inherited, secretDependency{target: destination, dependency: source})
|
||||
}
|
||||
}
|
||||
|
||||
secret := make(map[string]bool)
|
||||
for _, tree := range append(projections, final) {
|
||||
@@ -62,6 +91,16 @@ func classifySecretPaths(schema *schemaDescriptor, predicate func(Path) bool, me
|
||||
changed = true
|
||||
}
|
||||
}
|
||||
for _, link := range inherited {
|
||||
if pathIsSecret(secret, link.dependency) && !secret[link.target.String()] {
|
||||
secret[link.target.String()] = true
|
||||
changed = true
|
||||
}
|
||||
if pathIsSecret(secret, link.target) && !secret[link.dependency.String()] {
|
||||
secret[link.dependency.String()] = true
|
||||
changed = true
|
||||
}
|
||||
}
|
||||
if !changed {
|
||||
return secret
|
||||
}
|
||||
|
||||
@@ -4,10 +4,7 @@ import (
|
||||
"bytes"
|
||||
"errors"
|
||||
"io/fs"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strconv"
|
||||
"strings"
|
||||
|
||||
yamladapter "gitea.maximumdirect.net/eric/ConfigKit/internal/yaml"
|
||||
)
|
||||
@@ -32,12 +29,47 @@ func validOverlayName(name string) bool {
|
||||
}
|
||||
|
||||
func loadSelectedOverlayDocument(source Source, context *FilesystemContext, main parsedSource, base *composedDocument, sources []SourceInfo, name string, rule *VersionRule, limits importLimits) (*composedDocument, []SourceInfo, []Diagnostic) {
|
||||
assembly := newAssemblyAccumulator(cloneComposedDocument(base))
|
||||
allSources, diagnostics := loadSelectedOverlayInto(source, context, main, assembly, sources, name, rule, limits)
|
||||
if len(diagnostics) != 0 {
|
||||
return nil, allSources, diagnostics
|
||||
}
|
||||
return assembly.current, allSources, nil
|
||||
}
|
||||
|
||||
func loadSelectedOverlayInto(source Source, context *FilesystemContext, main parsedSource, base *assemblyAccumulator, sources []SourceInfo, name string, rule *VersionRule, limits importLimits) ([]SourceInfo, []Diagnostic) {
|
||||
if nativeContext(source, context) {
|
||||
access, basePath, mainPath, diagnostic := nativeFilesystemAccess(source, context)
|
||||
if diagnostic != nil {
|
||||
return cloneSourceInfos(sources), []Diagnostic{*diagnostic}
|
||||
}
|
||||
if diagnostic := validateNativeDirectories(access, basePath, mainPath); diagnostic != nil {
|
||||
return cloneSourceInfos(sources), []Diagnostic{*diagnostic}
|
||||
}
|
||||
return loadSelectedOverlayWithAccess(access, base, sources, name, rule, limits)
|
||||
}
|
||||
rootPath, _, _, diagnostic := importFilesystem(source, context, main)
|
||||
if diagnostic != nil {
|
||||
return nil, cloneSourceInfos(sources), []Diagnostic{*diagnostic}
|
||||
return cloneSourceInfos(sources), []Diagnostic{*diagnostic}
|
||||
}
|
||||
overlayPath := filepath.Join(rootPath, "overlays", name+".yml")
|
||||
canonical, err := canonicalExistingPath(overlayPath)
|
||||
access, err := openHostAccess(rootPath)
|
||||
if err != nil {
|
||||
value := globFilesystemDiagnostic(err, "configuration root could not be opened for the selected overlay", Location{})
|
||||
value.Primary = nil
|
||||
return cloneSourceInfos(sources), []Diagnostic{value}
|
||||
}
|
||||
return loadSelectedOverlayWithAccess(access, base, sources, name, rule, limits)
|
||||
}
|
||||
|
||||
func loadSelectedOverlayWithAccess(access *filesystemAccess, base *assemblyAccumulator, sources []SourceInfo, name string, rule *VersionRule, limits importLimits) ([]SourceInfo, []Diagnostic) {
|
||||
closed := false
|
||||
defer func() {
|
||||
if !closed {
|
||||
_ = access.close()
|
||||
}
|
||||
}()
|
||||
overlayPath := access.join(access.rootPath, "overlays/"+name+".yml")
|
||||
canonical, err := access.canonical(overlayPath)
|
||||
if err != nil {
|
||||
code := CodeOverlayInvalid
|
||||
message := "selected overlay could not be resolved"
|
||||
@@ -46,83 +78,81 @@ func loadSelectedOverlayDocument(source Source, context *FilesystemContext, main
|
||||
} else if errors.Is(err, fs.ErrPermission) {
|
||||
code, message = CodeImportPermission, "selected overlay cannot be accessed"
|
||||
}
|
||||
return nil, cloneSourceInfos(sources), []Diagnostic{importDiagnostic(code, message, nil)}
|
||||
return cloneSourceInfos(sources), []Diagnostic{importDiagnostic(code, message, nil)}
|
||||
}
|
||||
if !containedPath(rootPath, canonical) {
|
||||
return nil, cloneSourceInfos(sources), []Diagnostic{importDiagnostic(CodeOverlayOutsideRoot, "selected overlay resolves outside the configuration root", nil)}
|
||||
if !access.contains(canonical) {
|
||||
return cloneSourceInfos(sources), []Diagnostic{importDiagnostic(CodeOverlayOutsideRoot, "selected overlay resolves outside the configuration root", nil)}
|
||||
}
|
||||
for _, existing := range sources {
|
||||
if existing.Path != "" && existing.Path == canonical {
|
||||
return nil, cloneSourceInfos(sources), []Diagnostic{importDiagnostic(CodeOverlayRepeated, "selected overlay is already present in the base source graph", nil)}
|
||||
if existing.Path != "" && existing.Path == canonical &&
|
||||
((access.native == nil && existing.Kind == SourceKindFile) ||
|
||||
(access.native != nil && existing.Kind == SourceKindFilesystem && existing.namespace == access.token)) {
|
||||
return cloneSourceInfos(sources), []Diagnostic{importDiagnostic(CodeOverlayRepeated, "selected overlay is already present in the base source graph", nil)}
|
||||
}
|
||||
}
|
||||
if len(sources)+1 > limits.sources {
|
||||
return nil, cloneSourceInfos(sources), []Diagnostic{importDiagnostic(CodeOverlayLimitExceeded, "selected overlay exceeds the source-count limit", nil)}
|
||||
return cloneSourceInfos(sources), []Diagnostic{importDiagnostic(CodeOverlayLimitExceeded, "selected overlay exceeds the source-count limit", nil)}
|
||||
}
|
||||
|
||||
root, err := os.OpenRoot(rootPath)
|
||||
if err != nil {
|
||||
value := globFilesystemDiagnostic(err, "configuration root could not be opened for the selected overlay", Location{})
|
||||
value.Primary = nil
|
||||
return nil, cloneSourceInfos(sources), []Diagnostic{value}
|
||||
}
|
||||
rootClosed := false
|
||||
defer func() {
|
||||
if !rootClosed {
|
||||
_ = root.Close()
|
||||
}
|
||||
}()
|
||||
relative, err := filepath.Rel(rootPath, canonical)
|
||||
if err != nil || filepath.IsAbs(relative) || relative == ".." || strings.HasPrefix(relative, ".."+string(filepath.Separator)) {
|
||||
return nil, cloneSourceInfos(sources), []Diagnostic{importDiagnostic(CodeOverlayOutsideRoot, "selected overlay could not be addressed inside the configuration root", nil)}
|
||||
if _, err := access.relative(canonical); err != nil {
|
||||
return cloneSourceInfos(sources), []Diagnostic{importDiagnostic(CodeOverlayOutsideRoot, "selected overlay could not be addressed inside the configuration root", nil)}
|
||||
}
|
||||
|
||||
sourceID := "source-" + strconv.Itoa(len(sources)+1)
|
||||
overlayLocation := &Location{SourceID: sourceID}
|
||||
overlaySource := SourceInfo{ID: sourceID, Label: filepath.ToSlash(filepath.Join("overlays", name+".yml")), Path: canonical, Layer: LayerOverlay}
|
||||
kind := SourceKindFile
|
||||
label := "overlays/" + name + ".yml"
|
||||
if access.native != nil {
|
||||
kind = SourceKindFilesystem
|
||||
label = access.label + "/" + canonical
|
||||
}
|
||||
overlaySource := SourceInfo{ID: sourceID, Label: label, Path: canonical, Layer: LayerOverlay, Kind: kind}
|
||||
if kind == SourceKindFilesystem {
|
||||
overlaySource.namespace = access.token
|
||||
}
|
||||
sources = append(cloneSourceInfos(sources), overlaySource)
|
||||
data, diagnostic := readOverlayFile(root, relative, sourceID, limits.bytes-base.inputBytes)
|
||||
data, diagnostic := readOverlayFile(access, canonical, sourceID, limits.bytes-base.current.inputBytes)
|
||||
if diagnostic != nil {
|
||||
return nil, sources, []Diagnostic{*diagnostic}
|
||||
return sources, []Diagnostic{*diagnostic}
|
||||
}
|
||||
overlayRoot, issues := yamladapter.Parse(bytes.NewReader(data), sourceID, yamladapter.Limits{Depth: maxDocumentDepth, Nodes: maxDocumentNodes})
|
||||
if len(issues) != 0 {
|
||||
return nil, sources, parserDiagnostics(sourceID, issues)
|
||||
return sources, parserDiagnostics(sourceID, issues)
|
||||
}
|
||||
nodeCount := countInputNodes(overlayRoot)
|
||||
if base.inputNodes+nodeCount > limits.nodes {
|
||||
return nil, sources, []Diagnostic{importDiagnostic(CodeOverlayLimitExceeded, "selected overlay exceeds the aggregate node limit", overlayLocation)}
|
||||
if base.current.inputNodes+nodeCount > limits.nodes {
|
||||
return sources, []Diagnostic{importDiagnostic(CodeOverlayLimitExceeded, "selected overlay exceeds the aggregate node limit", overlayLocation)}
|
||||
}
|
||||
if diagnostics := overlayDirectiveDiagnostics(overlayRoot, Path{}); len(diagnostics) != 0 {
|
||||
return nil, sources, diagnostics
|
||||
return sources, diagnostics
|
||||
}
|
||||
if versionNode := fragmentVersionClaim(overlayRoot, rule); versionNode != nil {
|
||||
location := publicLocation(versionNode.Location)
|
||||
return nil, sources, []Diagnostic{{
|
||||
return sources, []Diagnostic{{
|
||||
Code: CodeOverlayVersion, Severity: SeverityError, Path: rule.Path.clone(),
|
||||
Message: "overlays cannot contribute to or replace the schema-version path", Primary: &location,
|
||||
}}
|
||||
}
|
||||
closeErr := root.Close()
|
||||
rootClosed = true
|
||||
closeErr := access.close()
|
||||
closed = true
|
||||
if closeErr != nil {
|
||||
return nil, sources, []Diagnostic{importDiagnostic(CodeImportRead, "configuration root could not be closed after reading the selected overlay", nil)}
|
||||
return sources, []Diagnostic{importDiagnostic(CodeImportRead, "configuration root could not be closed after reading the selected overlay", nil)}
|
||||
}
|
||||
|
||||
overlayd, diagnostics := overlayDocuments(base, overlayRoot)
|
||||
if overlayd != nil {
|
||||
overlayd.inputBytes = base.inputBytes + int64(len(data))
|
||||
overlayd.inputNodes = base.inputNodes + nodeCount
|
||||
if diagnostics := base.applyOverlay(overlayRoot, nil, nil, LayerOverlay); len(diagnostics) != 0 {
|
||||
return sources, diagnostics
|
||||
}
|
||||
return overlayd, sources, diagnostics
|
||||
base.current.inputBytes += int64(len(data))
|
||||
base.current.inputNodes += nodeCount
|
||||
return sources, nil
|
||||
}
|
||||
|
||||
func readOverlayFile(root *os.Root, relative, sourceID string, remaining int64) ([]byte, *Diagnostic) {
|
||||
func readOverlayFile(access *filesystemAccess, name, sourceID string, remaining int64) ([]byte, *Diagnostic) {
|
||||
limit := min(int64(maxSourceBytes), remaining)
|
||||
if limit < 0 {
|
||||
limit = 0
|
||||
}
|
||||
data, failure := readRegularFile(relative, root.Stat, root.Open, limit)
|
||||
data, failure := readRegularFile(name, access.stat, access.open, limit)
|
||||
if failure != nil {
|
||||
value := sourceReadDiagnostic(failure, sourceID, CodeOverlayLimitExceeded, "selected overlay exceeds a source or aggregate byte limit")
|
||||
return nil, &value
|
||||
|
||||
@@ -9,6 +9,8 @@ import (
|
||||
"path/filepath"
|
||||
"reflect"
|
||||
"sync/atomic"
|
||||
|
||||
"gitea.maximumdirect.net/eric/ConfigKit/internal/document"
|
||||
)
|
||||
|
||||
const (
|
||||
@@ -16,14 +18,19 @@ const (
|
||||
maxTotalBytes = 64 << 20
|
||||
)
|
||||
|
||||
// Source is an opaque configuration input. Construct sources with File, Bytes,
|
||||
// Reader, or NoSource; the zero value is invalid.
|
||||
// Source is an opaque configuration input. Construct sources with File,
|
||||
// Filesystem.File, Bytes, Reader, or NoSource; the zero value is invalid.
|
||||
type Source struct {
|
||||
kind sourceKind
|
||||
path string
|
||||
label string
|
||||
namespace *nativeNamespace
|
||||
data []byte
|
||||
reader *readerSource
|
||||
mapping *document.Node
|
||||
mapIssue *valueIssue
|
||||
mapBytes int64
|
||||
mapNodes int
|
||||
createErr sourceCreationError
|
||||
}
|
||||
|
||||
@@ -32,8 +39,10 @@ type sourceKind uint8
|
||||
const (
|
||||
sourceInvalid sourceKind = iota
|
||||
sourceFile
|
||||
sourceFilesystem
|
||||
sourceBytes
|
||||
sourceReader
|
||||
sourceMapping
|
||||
sourceNone
|
||||
)
|
||||
|
||||
@@ -45,8 +54,53 @@ const (
|
||||
creationInvalidLabel
|
||||
creationInvalidReader
|
||||
creationTooLarge
|
||||
creationInvalidFilesystem
|
||||
creationInvalidFilesystemLabel
|
||||
creationInvalidFilesystemPath
|
||||
creationInvalidMappingRoot
|
||||
)
|
||||
|
||||
// Filesystem is a reusable opaque namespace for caller-owned fs.FS inputs.
|
||||
// Copies preserve the namespace identity used for explicit contexts.
|
||||
type Filesystem struct {
|
||||
namespace *nativeNamespace
|
||||
createErr sourceCreationError
|
||||
}
|
||||
|
||||
// NewFilesystem records a borrowed filesystem without opening it. Invalid
|
||||
// arguments are reported when a File source is consumed.
|
||||
func NewFilesystem(label string, filesystem fs.FS) Filesystem {
|
||||
result := Filesystem{}
|
||||
if !validOverrideLabel(label) {
|
||||
result.createErr = creationInvalidFilesystemLabel
|
||||
} else if nilFilesystem(filesystem) {
|
||||
result.createErr = creationInvalidFilesystem
|
||||
} else {
|
||||
result.namespace = newNativeNamespace(label, filesystem)
|
||||
}
|
||||
return result
|
||||
}
|
||||
|
||||
// File selects a non-root slash-relative file within this namespace.
|
||||
func (f Filesystem) File(name string) Source {
|
||||
if f.createErr != creationValid {
|
||||
return Source{kind: sourceFilesystem, createErr: f.createErr}
|
||||
}
|
||||
return filesystemFile(f.namespace, name)
|
||||
}
|
||||
|
||||
func nilFilesystem(filesystem fs.FS) bool {
|
||||
if filesystem == nil {
|
||||
return true
|
||||
}
|
||||
value := reflect.ValueOf(filesystem)
|
||||
switch value.Kind() {
|
||||
case reflect.Chan, reflect.Func, reflect.Interface, reflect.Map, reflect.Pointer, reflect.Slice:
|
||||
return value.IsNil()
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
type readerSource struct {
|
||||
input io.Reader
|
||||
used atomic.Bool
|
||||
@@ -60,6 +114,16 @@ func File(path string) Source {
|
||||
return Source{kind: sourceFile, path: path, label: path}
|
||||
}
|
||||
|
||||
func filesystemFile(namespace *nativeNamespace, name string) Source {
|
||||
if namespace == nil || nilFilesystem(namespace.filesystem) {
|
||||
return Source{kind: sourceFilesystem, createErr: creationInvalidFilesystem}
|
||||
}
|
||||
if name == "." || !fs.ValidPath(name) {
|
||||
return Source{kind: sourceFilesystem, createErr: creationInvalidFilesystemPath}
|
||||
}
|
||||
return Source{kind: sourceFilesystem, path: name, label: namespace.label + "/" + name, namespace: namespace}
|
||||
}
|
||||
|
||||
// Bytes snapshots data immediately. Label must be nonempty and safe to display;
|
||||
// constructor problems are reported when the source is consumed.
|
||||
func Bytes(label string, data []byte) Source {
|
||||
@@ -92,6 +156,24 @@ func Reader(label string, input io.Reader) Source {
|
||||
return source
|
||||
}
|
||||
|
||||
// Mapping snapshots supported built-in values into an owned document at
|
||||
// construction. Invalid values are retained as safe deferred diagnostics.
|
||||
func Mapping(label string, values map[string]any) Source {
|
||||
source := Source{kind: sourceMapping, label: label}
|
||||
if !validOverrideLabel(label) {
|
||||
source.createErr = creationInvalidLabel
|
||||
return source
|
||||
}
|
||||
if values == nil {
|
||||
source.createErr = creationInvalidMappingRoot
|
||||
return source
|
||||
}
|
||||
builder := valueBuilder{active: make(map[valueVisit]bool)}
|
||||
source.mapping, source.mapIssue = builder.build(values, Path{}, 0)
|
||||
source.mapBytes, source.mapNodes = builder.bytes, builder.nodes
|
||||
return source
|
||||
}
|
||||
|
||||
// NoSource explicitly selects a defaults-only input. It creates no source record.
|
||||
func NoSource() Source {
|
||||
return Source{kind: sourceNone}
|
||||
@@ -107,14 +189,24 @@ func defaultSourceLimits() sourceLimits {
|
||||
}
|
||||
|
||||
type sourceRead struct {
|
||||
data []byte
|
||||
info *SourceInfo
|
||||
data []byte
|
||||
document *document.Node
|
||||
bytes int64
|
||||
info *SourceInfo
|
||||
}
|
||||
|
||||
func readSource(source Source, sourceID string, limits sourceLimits) (sourceRead, []Diagnostic) {
|
||||
if diagnostic := sourceConstructorDiagnostic(source); diagnostic != nil {
|
||||
if source.kind == sourceMapping {
|
||||
diagnostic.Primary = &Location{SourceID: sourceID}
|
||||
return sourceRead{info: mappingSourceInfo(source, sourceID)}, []Diagnostic{*diagnostic}
|
||||
}
|
||||
return sourceRead{}, []Diagnostic{*diagnostic}
|
||||
}
|
||||
if source.kind == sourceMapping && source.mapIssue != nil {
|
||||
diagnostic := source.mapIssue.sourceDiagnostic(sourceID)
|
||||
return sourceRead{info: mappingSourceInfo(source, sourceID)}, []Diagnostic{diagnostic}
|
||||
}
|
||||
if source.kind == sourceNone {
|
||||
return sourceRead{}, nil
|
||||
}
|
||||
@@ -123,7 +215,7 @@ func readSource(source Source, sourceID string, limits sourceLimits) (sourceRead
|
||||
if limit < 0 {
|
||||
limit = 0
|
||||
}
|
||||
info := &SourceInfo{ID: sourceID, Label: source.label, Layer: LayerBase}
|
||||
info := &SourceInfo{ID: sourceID, Label: source.label, Layer: LayerBase, Kind: sourceSourceKind(source.kind)}
|
||||
|
||||
var input io.Reader
|
||||
switch source.kind {
|
||||
@@ -143,6 +235,20 @@ func readSource(source Source, sourceID string, limits sourceLimits) (sourceRead
|
||||
return sourceRead{info: info}, []Diagnostic{sourceReadDiagnostic(failure, sourceID, CodeSourceLimitExceeded, "source exceeds the configured byte limit")}
|
||||
}
|
||||
return sourceRead{data: bytes.Clone(data), info: info}, nil
|
||||
case sourceFilesystem:
|
||||
access := newNativeAccess(source.namespace.filesystem, ".", source.namespace.token)
|
||||
info.Path = source.path
|
||||
info.namespace = source.namespace.token
|
||||
canonical, err := access.canonical(source.path)
|
||||
if err != nil {
|
||||
return sourceRead{info: info}, []Diagnostic{classifySourceError(err, sourceID)}
|
||||
}
|
||||
info.Path = canonical
|
||||
data, failure := readRegularFile(canonical, access.stat, access.open, limit)
|
||||
if failure != nil {
|
||||
return sourceRead{info: info}, []Diagnostic{sourceReadDiagnostic(failure, sourceID, CodeSourceLimitExceeded, "source exceeds the configured byte limit")}
|
||||
}
|
||||
return sourceRead{data: bytes.Clone(data), info: info}, nil
|
||||
case sourceBytes:
|
||||
input = bytes.NewReader(source.data)
|
||||
case sourceReader:
|
||||
@@ -150,6 +256,13 @@ func readSource(source Source, sourceID string, limits sourceLimits) (sourceRead
|
||||
return sourceRead{info: info}, []Diagnostic{sourceDiagnostic(CodeSourceReaderUsed, sourceID, "reader source has already been consumed")}
|
||||
}
|
||||
input = source.reader.input
|
||||
case sourceMapping:
|
||||
if source.mapBytes > limit || source.mapNodes > maxDocumentNodes {
|
||||
return sourceRead{info: info}, []Diagnostic{sourceDiagnostic(CodeSourceLimitExceeded, sourceID, "mapping source exceeds the configured input limit")}
|
||||
}
|
||||
owned := document.Clone(source.mapping)
|
||||
setDocumentSourceID(owned, sourceID)
|
||||
return sourceRead{document: owned, bytes: source.mapBytes, info: info}, nil
|
||||
default:
|
||||
return sourceRead{info: info}, []Diagnostic{sourceDiagnostic(CodeSourceInvalid, sourceID, "source is invalid")}
|
||||
}
|
||||
@@ -160,6 +273,14 @@ func readSource(source Source, sourceID string, limits sourceLimits) (sourceRead
|
||||
return sourceRead{data: bytes.Clone(data), info: info}, nil
|
||||
}
|
||||
|
||||
func mappingSourceInfo(source Source, sourceID string) *SourceInfo {
|
||||
label := source.label
|
||||
if !validOverrideLabel(label) {
|
||||
label = "mapping"
|
||||
}
|
||||
return &SourceInfo{ID: sourceID, Label: label, Layer: LayerBase, Kind: SourceKindMapping}
|
||||
}
|
||||
|
||||
func sourceConstructorDiagnostic(source Source) *Diagnostic {
|
||||
var message string
|
||||
code := CodeSourceInvalid
|
||||
@@ -173,16 +294,60 @@ func sourceConstructorDiagnostic(source Source) *Diagnostic {
|
||||
message = "source path must not be empty"
|
||||
case creationInvalidLabel:
|
||||
message = "source label must not be empty"
|
||||
if source.kind == sourceMapping {
|
||||
message = "mapping source label must be nonempty and safe to display"
|
||||
}
|
||||
case creationInvalidReader:
|
||||
message = "source reader must not be nil"
|
||||
case creationTooLarge:
|
||||
code = CodeSourceLimitExceeded
|
||||
message = "source exceeds the configured byte limit"
|
||||
case creationInvalidFilesystem:
|
||||
message = "filesystem must not be nil"
|
||||
case creationInvalidFilesystemLabel:
|
||||
message = "filesystem label must be nonempty and safe to display"
|
||||
case creationInvalidFilesystemPath:
|
||||
message = "filesystem source path must be a non-root slash-relative valid path"
|
||||
case creationInvalidMappingRoot:
|
||||
message = "mapping source root must be a non-nil mapping"
|
||||
}
|
||||
diagnostic := sourceDiagnostic(code, "", message)
|
||||
return &diagnostic
|
||||
}
|
||||
|
||||
func sourceSourceKind(kind sourceKind) SourceKind {
|
||||
switch kind {
|
||||
case sourceFile:
|
||||
return SourceKindFile
|
||||
case sourceFilesystem:
|
||||
return SourceKindFilesystem
|
||||
case sourceBytes:
|
||||
return SourceKindBytes
|
||||
case sourceReader:
|
||||
return SourceKindReader
|
||||
case sourceMapping:
|
||||
return SourceKindMapping
|
||||
default:
|
||||
return ""
|
||||
}
|
||||
}
|
||||
|
||||
func setDocumentSourceID(node *document.Node, sourceID string) {
|
||||
if node == nil {
|
||||
return
|
||||
}
|
||||
node.Location.SourceID = sourceID
|
||||
for index := range node.Mapping {
|
||||
entry := &node.Mapping[index]
|
||||
entry.KeyLocation.SourceID = sourceID
|
||||
entry.ValueLocation.SourceID = sourceID
|
||||
setDocumentSourceID(entry.Value, sourceID)
|
||||
}
|
||||
for _, child := range node.Sequence {
|
||||
setDocumentSourceID(child, sourceID)
|
||||
}
|
||||
}
|
||||
|
||||
func classifySourceError(err error, sourceID string) Diagnostic {
|
||||
switch {
|
||||
case errors.Is(err, fs.ErrNotExist):
|
||||
|
||||
112
internal/engine/source_context.go
Normal file
112
internal/engine/source_context.go
Normal file
@@ -0,0 +1,112 @@
|
||||
package engine
|
||||
|
||||
import (
|
||||
"io/fs"
|
||||
"path"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
)
|
||||
|
||||
func contextsForAcquiredSources(source Source, context *FilesystemContext, baseSource *SourceInfo, sources []SourceInfo) (map[string]ResourceContext, *Diagnostic) {
|
||||
if len(sources) == 0 {
|
||||
return nil, nil
|
||||
}
|
||||
var rootPath, basePath string
|
||||
var filesystem fs.FS
|
||||
if source.kind == sourceFilesystem {
|
||||
filesystem = source.namespace.filesystem
|
||||
rootPath, basePath = ".", path.Dir(source.path)
|
||||
if context != nil && context.Root != "" {
|
||||
rootPath = context.Root
|
||||
}
|
||||
} else if context != nil && context.FS != nil {
|
||||
filesystem = context.FS.namespace.filesystem
|
||||
rootPath, basePath = context.Root, context.BaseDir
|
||||
} else if source.kind == sourceFile {
|
||||
filePath := sources[0].Path
|
||||
if baseSource != nil {
|
||||
filePath = baseSource.Path
|
||||
}
|
||||
basePath = filepath.Dir(filePath)
|
||||
rootPath = basePath
|
||||
if context != nil && context.Root != "" {
|
||||
canonical, err := canonicalExistingPath(context.Root)
|
||||
if err != nil {
|
||||
value := importDiagnostic(CodeImportInvalid, "configuration root could not be canonicalized", nil)
|
||||
return nil, &value
|
||||
}
|
||||
rootPath = canonical
|
||||
}
|
||||
} else if context != nil {
|
||||
var err error
|
||||
rootPath, err = canonicalExistingPath(context.Root)
|
||||
if err != nil {
|
||||
value := importDiagnostic(CodeImportInvalid, "configuration root could not be canonicalized", nil)
|
||||
return nil, &value
|
||||
}
|
||||
basePath, err = canonicalExistingPath(context.BaseDir)
|
||||
if err != nil {
|
||||
value := importDiagnostic(CodeImportInvalid, "import base directory could not be canonicalized", nil)
|
||||
return nil, &value
|
||||
}
|
||||
} else {
|
||||
return nil, nil
|
||||
}
|
||||
var rooted fs.FS
|
||||
if filesystem != nil {
|
||||
rooted = rootedNativeFS{filesystem: filesystem, root: rootPath}
|
||||
} else {
|
||||
rooted = rootedHostFS{root: rootPath}
|
||||
}
|
||||
result := make(map[string]ResourceContext, len(sources))
|
||||
for _, acquired := range sources {
|
||||
base := basePath
|
||||
if acquired.Path != "" {
|
||||
if filesystem != nil {
|
||||
base = path.Dir(acquired.Path)
|
||||
} else {
|
||||
base = filepath.Dir(acquired.Path)
|
||||
}
|
||||
}
|
||||
var relative string
|
||||
if filesystem != nil {
|
||||
if !fs.ValidPath(rootPath) || !fs.ValidPath(base) || rootPath != "." && base != rootPath && !strings.HasPrefix(base, rootPath+"/") {
|
||||
value := importDiagnostic(CodeImportOutsideRoot, "source base is outside the configuration root", nil)
|
||||
return nil, &value
|
||||
}
|
||||
relative = base
|
||||
if rootPath != "." {
|
||||
relative = strings.TrimPrefix(strings.TrimPrefix(base, rootPath), "/")
|
||||
if relative == "" {
|
||||
relative = "."
|
||||
}
|
||||
}
|
||||
} else {
|
||||
if !containedPath(rootPath, base) {
|
||||
value := importDiagnostic(CodeImportOutsideRoot, "source base is outside the configuration root", nil)
|
||||
return nil, &value
|
||||
}
|
||||
var err error
|
||||
relative, err = filepath.Rel(rootPath, base)
|
||||
if err != nil {
|
||||
value := importDiagnostic(CodeImportInvalid, "source base could not be addressed in the configuration root", nil)
|
||||
return nil, &value
|
||||
}
|
||||
relative = filepath.ToSlash(relative)
|
||||
}
|
||||
result[acquired.ID] = ResourceContext{FS: rooted, BaseDir: relative}
|
||||
}
|
||||
return result, nil
|
||||
}
|
||||
|
||||
func mergeResourceContexts(target *map[string]ResourceContext, incoming map[string]ResourceContext) {
|
||||
if len(incoming) == 0 {
|
||||
return
|
||||
}
|
||||
if *target == nil {
|
||||
*target = make(map[string]ResourceContext, len(incoming))
|
||||
}
|
||||
for id, context := range incoming {
|
||||
(*target)[id] = context
|
||||
}
|
||||
}
|
||||
@@ -4,7 +4,6 @@ import (
|
||||
"errors"
|
||||
"io"
|
||||
"io/fs"
|
||||
"os"
|
||||
)
|
||||
|
||||
type sourceReadFailureKind uint8
|
||||
@@ -26,7 +25,7 @@ type sourceReadFailure struct {
|
||||
// readRegularFile borrows stat/open operations so root-bound callers retain their
|
||||
// containment policy. Preflight rejects FIFOs before open can block. The opened
|
||||
// handle is checked again and always closed; read-only close errors are ignored.
|
||||
func readRegularFile(name string, stat func(string) (fs.FileInfo, error), open func(string) (*os.File, error), limit int64) ([]byte, *sourceReadFailure) {
|
||||
func readRegularFile[F fs.File](name string, stat func(string) (fs.FileInfo, error), open func(string) (F, error), limit int64) ([]byte, *sourceReadFailure) {
|
||||
info, err := stat(name)
|
||||
if err != nil {
|
||||
return nil, &sourceReadFailure{cause: err, missingTarget: errors.Is(err, fs.ErrNotExist)}
|
||||
|
||||
19
internal/engine/source_value.go
Normal file
19
internal/engine/source_value.go
Normal file
@@ -0,0 +1,19 @@
|
||||
package engine
|
||||
|
||||
func (issue *valueIssue) sourceDiagnostic(sourceID string) Diagnostic {
|
||||
code := CodeSourceInvalid
|
||||
message := "mapping source has an unsupported value"
|
||||
switch issue.kind {
|
||||
case valueDepthLimit:
|
||||
code, message = CodeSourceLimitExceeded, "mapping source exceeds the depth limit"
|
||||
case valueNodeLimit:
|
||||
code, message = CodeSourceLimitExceeded, "mapping source exceeds the node limit"
|
||||
case valueByteLimit:
|
||||
code, message = CodeSourceLimitExceeded, "mapping source exceeds the string-byte limit"
|
||||
case valueNonFinite:
|
||||
message = "mapping source float must be finite"
|
||||
case valueCycle:
|
||||
message = "mapping source values must not contain cycles"
|
||||
}
|
||||
return Diagnostic{Code: code, Severity: SeverityError, Path: issue.path.clone(), Message: message, Primary: &Location{SourceID: sourceID}}
|
||||
}
|
||||
156
internal/engine/underlay_bench_test.go
Normal file
156
internal/engine/underlay_bench_test.go
Normal file
@@ -0,0 +1,156 @@
|
||||
package engine
|
||||
|
||||
import (
|
||||
"io"
|
||||
"strconv"
|
||||
"testing"
|
||||
"testing/fstest"
|
||||
)
|
||||
|
||||
type benchmarkCatalogConfig struct {
|
||||
Catalog map[string]string `yaml:"catalog"`
|
||||
}
|
||||
|
||||
type benchmarkNodeConfig struct {
|
||||
Node string `yaml:"node"`
|
||||
}
|
||||
|
||||
func BenchmarkUnderlayBroadCatalog(b *testing.B) {
|
||||
for _, width := range []int{128, 256, 512} {
|
||||
b.Run(strconv.Itoa(width), func(b *testing.B) {
|
||||
inputs := make([]Input, 2)
|
||||
for half := range inputs {
|
||||
catalog := make(map[string]any, width/2)
|
||||
for index := range width / 2 {
|
||||
catalog["item-"+strconv.Itoa(half*width/2+index)] = "value"
|
||||
}
|
||||
inputs[half] = Input{Source: Mapping("catalog-"+strconv.Itoa(half), map[string]any{"catalog": catalog})}
|
||||
}
|
||||
options := LayeredOptions[benchmarkCatalogConfig]{Underlays: []Underlay{{Inputs: inputs}}}
|
||||
b.ReportAllocs()
|
||||
b.ResetTimer()
|
||||
for range b.N {
|
||||
if _, err := LoadLayered(Input{Source: NoSource()}, testSecretBinding(), options); err != nil {
|
||||
b.Fatal(err)
|
||||
}
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func BenchmarkUnderlayManySmallTiers(b *testing.B) {
|
||||
for _, count := range []int{32, 64, 128} {
|
||||
b.Run(strconv.Itoa(count), func(b *testing.B) {
|
||||
groups := make([]Underlay, count)
|
||||
for index := range groups {
|
||||
groups[index].Inputs = []Input{{Source: Mapping("item", map[string]any{
|
||||
"catalog": map[string]any{"item-" + strconv.Itoa(index): "value"},
|
||||
})}}
|
||||
}
|
||||
options := LayeredOptions[benchmarkCatalogConfig]{Underlays: groups}
|
||||
b.ReportAllocs()
|
||||
b.ResetTimer()
|
||||
for range b.N {
|
||||
if _, err := LoadLayered(Input{Source: NoSource()}, testSecretBinding(), options); err != nil {
|
||||
b.Fatal(err)
|
||||
}
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func BenchmarkUnderlaySubtreeReplacement(b *testing.B) {
|
||||
for _, count := range []int{32, 64, 128} {
|
||||
b.Run(strconv.Itoa(count), func(b *testing.B) {
|
||||
groups := make([]Underlay, count)
|
||||
for index := range groups {
|
||||
var node any = "temporary"
|
||||
if index%2 == 0 {
|
||||
node = map[string]any{"left": "one", "right": "two", "nested": map[string]any{"enabled": false}}
|
||||
}
|
||||
groups[index].Inputs = []Input{{Source: Mapping("replacement", map[string]any{"node": node})}}
|
||||
}
|
||||
options := LayeredOptions[benchmarkNodeConfig]{Underlays: groups}
|
||||
main := Input{Source: Mapping("main", map[string]any{"node": "final"})}
|
||||
b.ReportAllocs()
|
||||
b.ResetTimer()
|
||||
for range b.N {
|
||||
if _, err := LoadLayered(main, testSecretBinding(), options); err != nil {
|
||||
b.Fatal(err)
|
||||
}
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func BenchmarkUnderlayNativeImports(b *testing.B) {
|
||||
for _, count := range []int{16, 32, 64} {
|
||||
b.Run(strconv.Itoa(count), func(b *testing.B) {
|
||||
files := fstest.MapFS{
|
||||
"main.yml": &fstest.MapFile{Data: []byte("configkit: {imports: [{glob: 'defaults/*.yml'}]}\n")},
|
||||
}
|
||||
for index := range count {
|
||||
name := "item-" + strconv.Itoa(index)
|
||||
files["defaults/"+name+".yml"] = &fstest.MapFile{Data: []byte("catalog: {" + name + ": value}\n")}
|
||||
}
|
||||
filesystem := NewFilesystem("bundle", files)
|
||||
main := Input{Source: filesystem.File("main.yml"), Imports: true}
|
||||
b.ReportAllocs()
|
||||
b.ResetTimer()
|
||||
for range b.N {
|
||||
if _, err := LoadLayered[benchmarkCatalogConfig](main, testSecretBinding()); err != nil {
|
||||
b.Fatal(err)
|
||||
}
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func BenchmarkUnderlayRetainedHistory(b *testing.B) {
|
||||
for _, count := range []int{16, 32, 64} {
|
||||
b.Run(strconv.Itoa(count), func(b *testing.B) {
|
||||
groups := make([]Underlay, count)
|
||||
for index := range groups {
|
||||
groups[index].Inputs = []Input{{Source: Mapping("value", map[string]any{
|
||||
"catalog": map[string]any{"shared": strconv.Itoa(index)},
|
||||
})}}
|
||||
}
|
||||
options := LayeredOptions[benchmarkCatalogConfig]{Underlays: groups}
|
||||
b.ReportAllocs()
|
||||
b.ResetTimer()
|
||||
for range b.N {
|
||||
result, err := LoadLayered(Input{Source: NoSource()}, testSecretBinding(), options)
|
||||
if err != nil {
|
||||
b.Fatal(err)
|
||||
}
|
||||
if _, ok := result.Inspection.Explain(Keys("catalog", "shared")); !ok {
|
||||
b.Fatal("retained path was not explained")
|
||||
}
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func BenchmarkUnderlayRenderedHistory(b *testing.B) {
|
||||
for _, count := range []int{32, 64, 128} {
|
||||
b.Run(strconv.Itoa(count), func(b *testing.B) {
|
||||
groups := make([]Underlay, count)
|
||||
for index := range groups {
|
||||
groups[index].Inputs = []Input{{Source: Mapping("value", map[string]any{
|
||||
"catalog": map[string]any{"shared": strconv.Itoa(index)},
|
||||
})}}
|
||||
}
|
||||
result, err := LoadLayered(Input{Source: NoSource()}, testSecretBinding(), LayeredOptions[benchmarkCatalogConfig]{Underlays: groups})
|
||||
if err != nil {
|
||||
b.Fatal(err)
|
||||
}
|
||||
b.ReportAllocs()
|
||||
b.ResetTimer()
|
||||
for range b.N {
|
||||
if err := result.Inspection.WriteExplanation(io.Discard, Keys("catalog", "shared")); err != nil {
|
||||
b.Fatal(err)
|
||||
}
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
166
internal/engine/value.go
Normal file
166
internal/engine/value.go
Normal file
@@ -0,0 +1,166 @@
|
||||
package engine
|
||||
|
||||
import (
|
||||
"math"
|
||||
"reflect"
|
||||
"slices"
|
||||
"strconv"
|
||||
|
||||
"gitea.maximumdirect.net/eric/ConfigKit/internal/document"
|
||||
)
|
||||
|
||||
type valueBuilder struct {
|
||||
sourceID string
|
||||
active map[valueVisit]bool
|
||||
bytes int64
|
||||
nodes int
|
||||
}
|
||||
|
||||
type valueIssueKind uint8
|
||||
|
||||
const (
|
||||
valueDepthLimit valueIssueKind = iota
|
||||
valueNodeLimit
|
||||
valueByteLimit
|
||||
valueNonFinite
|
||||
valueUnsupported
|
||||
valueCycle
|
||||
)
|
||||
|
||||
type valueIssue struct {
|
||||
kind valueIssueKind
|
||||
path Path
|
||||
}
|
||||
|
||||
func newValueIssue(kind valueIssueKind, path Path) *valueIssue {
|
||||
return &valueIssue{kind: kind, path: path.clone()}
|
||||
}
|
||||
|
||||
func (b *valueBuilder) build(value any, path Path, depth int) (*document.Node, *valueIssue) {
|
||||
if depth > maxDocumentDepth {
|
||||
return nil, newValueIssue(valueDepthLimit, path)
|
||||
}
|
||||
b.nodes++
|
||||
if b.nodes > maxDocumentNodes {
|
||||
return nil, newValueIssue(valueNodeLimit, path)
|
||||
}
|
||||
location := document.Location{SourceID: b.sourceID}
|
||||
switch value := value.(type) {
|
||||
case nil:
|
||||
return &document.Node{Kind: document.Null, Location: location}, nil
|
||||
case string:
|
||||
if !b.addBytes(len(value)) {
|
||||
return nil, newValueIssue(valueByteLimit, path)
|
||||
}
|
||||
return &document.Node{Kind: document.String, Text: value, Location: location}, nil
|
||||
case bool:
|
||||
return &document.Node{Kind: document.Boolean, Text: strconv.FormatBool(value), Location: location}, nil
|
||||
case int:
|
||||
return valueNumber(strconv.FormatInt(int64(value), 10), location), nil
|
||||
case int8:
|
||||
return valueNumber(strconv.FormatInt(int64(value), 10), location), nil
|
||||
case int16:
|
||||
return valueNumber(strconv.FormatInt(int64(value), 10), location), nil
|
||||
case int32:
|
||||
return valueNumber(strconv.FormatInt(int64(value), 10), location), nil
|
||||
case int64:
|
||||
return valueNumber(strconv.FormatInt(value, 10), location), nil
|
||||
case uint:
|
||||
return valueNumber(strconv.FormatUint(uint64(value), 10), location), nil
|
||||
case uint8:
|
||||
return valueNumber(strconv.FormatUint(uint64(value), 10), location), nil
|
||||
case uint16:
|
||||
return valueNumber(strconv.FormatUint(uint64(value), 10), location), nil
|
||||
case uint32:
|
||||
return valueNumber(strconv.FormatUint(uint64(value), 10), location), nil
|
||||
case uint64:
|
||||
return valueNumber(strconv.FormatUint(value, 10), location), nil
|
||||
case float32:
|
||||
if math.IsNaN(float64(value)) || math.IsInf(float64(value), 0) {
|
||||
return nil, newValueIssue(valueNonFinite, path)
|
||||
}
|
||||
return valueNumber(strconv.FormatFloat(float64(value), 'g', -1, 32), location), nil
|
||||
case float64:
|
||||
if math.IsNaN(value) || math.IsInf(value, 0) {
|
||||
return nil, newValueIssue(valueNonFinite, path)
|
||||
}
|
||||
return valueNumber(strconv.FormatFloat(value, 'g', -1, 64), location), nil
|
||||
case []any:
|
||||
if value == nil {
|
||||
return &document.Node{Kind: document.Null, Location: location}, nil
|
||||
}
|
||||
leave, diagnostic := b.enter(reflect.ValueOf(value), path)
|
||||
if diagnostic != nil {
|
||||
return nil, diagnostic
|
||||
}
|
||||
defer leave()
|
||||
if len(value) > maxDocumentNodes-b.nodes {
|
||||
return nil, newValueIssue(valueNodeLimit, path)
|
||||
}
|
||||
node := &document.Node{Kind: document.Sequence, Location: location, Sequence: make([]*document.Node, len(value))}
|
||||
for index, child := range value {
|
||||
built, diagnostic := b.build(child, path.Index(index), depth+1)
|
||||
if diagnostic != nil {
|
||||
return nil, diagnostic
|
||||
}
|
||||
node.Sequence[index] = built
|
||||
}
|
||||
return node, nil
|
||||
case map[string]any:
|
||||
if value == nil {
|
||||
return &document.Node{Kind: document.Null, Location: location}, nil
|
||||
}
|
||||
leave, diagnostic := b.enter(reflect.ValueOf(value), path)
|
||||
if diagnostic != nil {
|
||||
return nil, diagnostic
|
||||
}
|
||||
defer leave()
|
||||
if len(value) > (maxDocumentNodes-b.nodes)/2 {
|
||||
return nil, newValueIssue(valueNodeLimit, path)
|
||||
}
|
||||
keys := make([]string, 0, len(value))
|
||||
for key := range value {
|
||||
keys = append(keys, key)
|
||||
}
|
||||
slices.Sort(keys)
|
||||
node := &document.Node{Kind: document.Mapping, Location: location, Mapping: make([]document.Entry, 0, len(keys))}
|
||||
for _, key := range keys {
|
||||
b.nodes++ // mapping key
|
||||
if !b.addBytes(len(key)) {
|
||||
return nil, newValueIssue(valueByteLimit, path.Key(key))
|
||||
}
|
||||
built, diagnostic := b.build(value[key], path.Key(key), depth+1)
|
||||
if diagnostic != nil {
|
||||
return nil, diagnostic
|
||||
}
|
||||
node.Mapping = append(node.Mapping, document.Entry{Key: key, KeyLocation: location, ValueLocation: location, Value: built})
|
||||
}
|
||||
return node, nil
|
||||
default:
|
||||
return nil, newValueIssue(valueUnsupported, path)
|
||||
}
|
||||
}
|
||||
|
||||
func (b *valueBuilder) addBytes(count int) bool {
|
||||
if count < 0 || int64(count) > maxSourceBytes-b.bytes {
|
||||
return false
|
||||
}
|
||||
b.bytes += int64(count)
|
||||
return true
|
||||
}
|
||||
|
||||
func (b *valueBuilder) enter(value reflect.Value, path Path) (func(), *valueIssue) {
|
||||
visit, ok := visitFor(value)
|
||||
if !ok {
|
||||
return func() {}, nil
|
||||
}
|
||||
if b.active[visit] {
|
||||
return nil, newValueIssue(valueCycle, path)
|
||||
}
|
||||
b.active[visit] = true
|
||||
return func() { delete(b.active, visit) }, nil
|
||||
}
|
||||
|
||||
func valueNumber(text string, location document.Location) *document.Node {
|
||||
return &document.Node{Kind: document.Number, Text: text, Location: location}
|
||||
}
|
||||
78
layered.go
Normal file
78
layered.go
Normal file
@@ -0,0 +1,78 @@
|
||||
package configkit
|
||||
|
||||
import "gitea.maximumdirect.net/eric/ConfigKit/internal/engine"
|
||||
|
||||
// Input selects one source, whether its imports are enabled, and any explicit
|
||||
// filesystem context. A zero Source is invalid; use NoSource() for an omitted
|
||||
// main input. Sources keep their documented reuse and reader ownership rules.
|
||||
type Input struct {
|
||||
Source Source
|
||||
Imports bool
|
||||
Filesystem *FilesystemContext
|
||||
}
|
||||
|
||||
// Underlay contains inputs that must compose strictly within one tier. Tiers
|
||||
// appear in lowest-to-highest precedence order; an empty tier is a no-op.
|
||||
type Underlay struct{ Inputs []Input }
|
||||
|
||||
// LayeredOptions selects lower-priority tiers and final application rules.
|
||||
// An active Overlay requires an explicit OverlayFilesystem; an inactive overlay
|
||||
// ignores it. Overrides remain explicit CLI records, not general mappings.
|
||||
type LayeredOptions[T any] struct {
|
||||
Definition Definition[T]
|
||||
Underlays []Underlay
|
||||
Overlay string
|
||||
OverlayFilesystem *FilesystemContext
|
||||
Overrides []Override
|
||||
}
|
||||
|
||||
// LoadLayered acquires the main input first and strictly composes enabled
|
||||
// imports with each input's tier. It strictly composes contributions within
|
||||
// each underlay, then recursively combines underlays in listed order beneath
|
||||
// main, a selected overlay, and CLI overrides. Defaults, decoding, resolution,
|
||||
// and validation run once after sibling `extends` expansion of the assembled
|
||||
// input. Source IDs follow acquisition order, not precedence order. Zero or
|
||||
// one options value is accepted; any error
|
||||
// returns a nil result.
|
||||
func LoadLayered[T any](main Input, supplied ...LayeredOptions[T]) (*Result[T], error) {
|
||||
options := make([]engine.LayeredOptions[T], len(supplied))
|
||||
for index := range supplied {
|
||||
options[index] = toEngineLayeredOptions(supplied[index])
|
||||
}
|
||||
loaded, err := engine.LoadLayered(toEngineInput(main), secretBinding(), options...)
|
||||
return fromEngineLoaded(loaded, err)
|
||||
}
|
||||
|
||||
func toEngineInput(input Input) engine.Input {
|
||||
return engine.Input{
|
||||
Source: input.Source.toEngine(), Imports: input.Imports,
|
||||
Filesystem: toEngineFilesystemContext(input.Filesystem),
|
||||
}
|
||||
}
|
||||
|
||||
func toEngineLayeredOptions[T any](options LayeredOptions[T]) engine.LayeredOptions[T] {
|
||||
result := engine.LayeredOptions[T]{
|
||||
Definition: toEngineDefinition(options.Definition), Overlay: options.Overlay,
|
||||
OverlayFilesystem: toEngineFilesystemContext(options.OverlayFilesystem),
|
||||
}
|
||||
if options.Underlays != nil {
|
||||
result.Underlays = make([]engine.Underlay, len(options.Underlays))
|
||||
for index, group := range options.Underlays {
|
||||
if group.Inputs != nil {
|
||||
result.Underlays[index].Inputs = make([]engine.Input, len(group.Inputs))
|
||||
for inputIndex, input := range group.Inputs {
|
||||
result.Underlays[index].Inputs[inputIndex] = toEngineInput(input)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
if options.Overrides != nil {
|
||||
result.Overrides = make([]engine.Override, len(options.Overrides))
|
||||
for index, override := range options.Overrides {
|
||||
result.Overrides[index] = engine.Override{
|
||||
Path: toEnginePath(override.Path), Value: override.Value, Flag: override.Flag,
|
||||
}
|
||||
}
|
||||
}
|
||||
return result
|
||||
}
|
||||
31
lifecycle.go
31
lifecycle.go
@@ -28,10 +28,14 @@ type Options[T any] struct {
|
||||
Overrides []Override
|
||||
}
|
||||
|
||||
// FilesystemContext supplies filesystem roots for imports and overlays.
|
||||
// FilesystemContext supplies roots for imports and overlays. With FS nil,
|
||||
// BaseDir and Root use absolute host paths. With FS set, they use slash-relative
|
||||
// paths in that namespace ("." denotes its root). File sources derive BaseDir
|
||||
// from the selected file; a native file's explicit Root limits its scope.
|
||||
type FilesystemContext struct {
|
||||
BaseDir string
|
||||
Root string
|
||||
FS *Filesystem
|
||||
}
|
||||
|
||||
// Override describes one explicitly supplied application override. Value uses
|
||||
@@ -75,14 +79,19 @@ type Result[T any] struct {
|
||||
}
|
||||
|
||||
// Load parses and strictly decodes zero or one selected source, explicitly enabled
|
||||
// imports, and one selected overlay. It accepts at most one Options value and
|
||||
// returns no result when any error is reported.
|
||||
// imports, and one selected overlay. After source precedence, built-in sibling
|
||||
// `extends` relationships resolve before defaults and typed decoding. It accepts
|
||||
// at most one Options value and returns no result when any error is reported.
|
||||
func Load[T any](source Source, supplied ...Options[T]) (*Result[T], error) {
|
||||
options := make([]engine.Options[T], len(supplied))
|
||||
for index := range supplied {
|
||||
options[index] = toEngineOptions(supplied[index])
|
||||
}
|
||||
loaded, err := engine.Load(source.toEngine(), secretBinding(), options...)
|
||||
return fromEngineLoaded(loaded, err)
|
||||
}
|
||||
|
||||
func fromEngineLoaded[T any](loaded *engine.Loaded[T], err error) (*Result[T], error) {
|
||||
if err != nil {
|
||||
var loadError *engine.LoadError
|
||||
if errors.As(err, &loadError) {
|
||||
@@ -133,7 +142,7 @@ func (r *Result[T]) Diagnostics() []Diagnostic {
|
||||
return result
|
||||
}
|
||||
|
||||
// IsExplicit reports whether surviving input explicitly defined path.
|
||||
// IsExplicit reports whether surviving direct or inherited input defined path.
|
||||
func (r *Resolution) IsExplicit(path Path) bool {
|
||||
return r.engineContext().IsExplicit(toEnginePath(path))
|
||||
}
|
||||
@@ -222,7 +231,7 @@ func toEngineOptions[T any](options Options[T]) engine.Options[T] {
|
||||
Overlay: options.Overlay,
|
||||
}
|
||||
if options.Filesystem != nil {
|
||||
result.Filesystem = &engine.FilesystemContext{BaseDir: options.Filesystem.BaseDir, Root: options.Filesystem.Root}
|
||||
result.Filesystem = toEngineFilesystemContext(options.Filesystem)
|
||||
}
|
||||
if options.Overrides != nil {
|
||||
result.Overrides = make([]engine.Override, len(options.Overrides))
|
||||
@@ -235,6 +244,18 @@ func toEngineOptions[T any](options Options[T]) engine.Options[T] {
|
||||
return result
|
||||
}
|
||||
|
||||
func toEngineFilesystemContext(context *FilesystemContext) *engine.FilesystemContext {
|
||||
if context == nil {
|
||||
return nil
|
||||
}
|
||||
result := &engine.FilesystemContext{BaseDir: context.BaseDir, Root: context.Root}
|
||||
if context.FS != nil {
|
||||
filesystem := context.FS.value
|
||||
result.FS = &filesystem
|
||||
}
|
||||
return result
|
||||
}
|
||||
|
||||
func toEngineDefinition[T any](definition Definition[T]) engine.Definition[T] {
|
||||
result := engine.Definition[T]{Defaults: definition.Defaults}
|
||||
if definition.Resolve != nil {
|
||||
|
||||
@@ -13,13 +13,13 @@ func (m Metadata) Lookup(path Path) (FieldInfo, bool) {
|
||||
return fromEngineFieldInfo(value), ok
|
||||
}
|
||||
|
||||
// IsExplicit reports whether a surviving input contribution explicitly defined
|
||||
// path.
|
||||
// IsExplicit reports whether a surviving input contribution, including one
|
||||
// inherited from a sibling, explicitly defined path.
|
||||
func (m Metadata) IsExplicit(path Path) bool {
|
||||
return m.value.IsExplicit(toEnginePath(path))
|
||||
}
|
||||
|
||||
// History returns detached definition and resolution records for path. A record's
|
||||
// History returns detached definition, inheritance, and resolution records for path. A record's
|
||||
// Path can be an ancestor when that container operation replaced or removed path.
|
||||
// Removed historical paths can have history even though Lookup returns false.
|
||||
// Unknown and invalid paths return nil.
|
||||
@@ -36,7 +36,7 @@ func (m Metadata) History(path Path) []Change {
|
||||
}
|
||||
|
||||
// HistoricalPaths returns the paths observed during loading, including effective
|
||||
// paths and paths removed by overlays or recorded resolution. It includes default
|
||||
// paths and paths removed by overlays, inheritance, or recorded resolution. It includes default
|
||||
// paths and the root, but not paths that never existed. The zero Metadata returns
|
||||
// an empty slice.
|
||||
//
|
||||
|
||||
75
records.go
75
records.go
@@ -6,11 +6,25 @@ type Layer string
|
||||
const (
|
||||
LayerDefault Layer = "default"
|
||||
LayerBase Layer = "base"
|
||||
LayerUnderlay Layer = "underlay"
|
||||
LayerOverlay Layer = "overlay"
|
||||
LayerCLI Layer = "cli"
|
||||
LayerResolution Layer = "resolution"
|
||||
)
|
||||
|
||||
// SourceKind identifies how a configuration source was acquired, independent
|
||||
// of its precedence layer.
|
||||
type SourceKind string
|
||||
|
||||
const (
|
||||
SourceKindFile SourceKind = "file"
|
||||
SourceKindFilesystem SourceKind = "filesystem"
|
||||
SourceKindBytes SourceKind = "bytes"
|
||||
SourceKindReader SourceKind = "reader"
|
||||
SourceKindMapping SourceKind = "mapping"
|
||||
SourceKindCLI SourceKind = "cli"
|
||||
)
|
||||
|
||||
// ChangeKind identifies a recorded operation. The zero value is
|
||||
// used for input and default origins.
|
||||
type ChangeKind string
|
||||
@@ -18,7 +32,10 @@ type ChangeKind string
|
||||
const (
|
||||
// ChangeDecoding records the built-in transition to canonical typed values.
|
||||
// It is inspection metadata, not an operation accepted by Resolution.Record.
|
||||
ChangeDecoding ChangeKind = "decoding"
|
||||
ChangeDecoding ChangeKind = "decoding"
|
||||
// ChangeInheritance records structural input expansion. It is inspection
|
||||
// metadata, not an operation accepted by Resolution.Record.
|
||||
ChangeInheritance ChangeKind = "inheritance"
|
||||
ChangeDerivedDefault ChangeKind = "derived_default"
|
||||
ChangeNormalization ChangeKind = "normalization"
|
||||
)
|
||||
@@ -89,6 +106,7 @@ const (
|
||||
CodeImportRepeated Code = "import.repeated"
|
||||
CodeImportLimitExceeded Code = "import.limit_exceeded"
|
||||
CodeImportVersion Code = "import.version"
|
||||
CodeUnderlayVersion Code = "underlay.version"
|
||||
|
||||
CodeOverlayInvalid Code = "overlay.invalid"
|
||||
CodeOverlayMissing Code = "overlay.missing"
|
||||
@@ -146,6 +164,11 @@ const (
|
||||
CodeOverrideLimitExceeded Code = "override.limit_exceeded"
|
||||
|
||||
CodeExplainLimitExceeded Code = "explain.limit_exceeded"
|
||||
|
||||
CodeInheritanceInvalid Code = "inheritance.invalid"
|
||||
CodeInheritanceMissing Code = "inheritance.missing"
|
||||
CodeInheritanceCycle Code = "inheritance.cycle"
|
||||
CodeInheritanceVersion Code = "inheritance.version"
|
||||
)
|
||||
|
||||
// Location identifies a position in a configuration source. Zero line and column
|
||||
@@ -156,15 +179,19 @@ type Location struct {
|
||||
Column int
|
||||
}
|
||||
|
||||
// SourceInfo describes a source discovered during a load. Path is a canonical
|
||||
// local path when one exists. ImportedFrom identifies the declaration that
|
||||
// included this source.
|
||||
// SourceInfo describes a source discovered during a load. Kind identifies its
|
||||
// acquisition, independent of Layer. Path is canonical for a host file and
|
||||
// namespace-relative for a filesystem file; synthetic sources have no Path.
|
||||
// UnderlayIndex is the one-based original underlay position, or zero for other
|
||||
// sources. ImportedFrom identifies the declaration that included this source.
|
||||
type SourceInfo struct {
|
||||
ID string
|
||||
Label string
|
||||
Path string
|
||||
Layer Layer
|
||||
ImportedFrom *Location
|
||||
ID string
|
||||
Label string
|
||||
Path string
|
||||
Layer Layer
|
||||
Kind SourceKind
|
||||
UnderlayIndex int
|
||||
ImportedFrom *Location
|
||||
}
|
||||
|
||||
// Diagnostic is a safe structured problem report. Message and source labels must
|
||||
@@ -178,13 +205,25 @@ type Diagnostic struct {
|
||||
Related []Location
|
||||
}
|
||||
|
||||
// InheritanceStep identifies a concrete payload transfer. From and To name
|
||||
// semantic value paths; Reference names the controlling destination `extends`
|
||||
// path. Locations belong to that directive, not the transferred value.
|
||||
type InheritanceStep struct {
|
||||
From Path
|
||||
To Path
|
||||
Reference Path
|
||||
Locations []Location
|
||||
}
|
||||
|
||||
// Origin describes where an effective definition came from. Multiple locations
|
||||
// represent a composite origin; resolution origins are synthetic and identify their
|
||||
// operation instead of a file location.
|
||||
// operation instead of a file location. Inheritance records the concrete
|
||||
// derivation path while Layer and Locations continue to identify value sources.
|
||||
type Origin struct {
|
||||
Layer Layer
|
||||
Locations []Location
|
||||
Operation ChangeKind
|
||||
Layer Layer
|
||||
Locations []Location
|
||||
Operation ChangeKind
|
||||
Inheritance []InheritanceStep
|
||||
}
|
||||
|
||||
// FieldInfo describes an effective configuration path.
|
||||
@@ -238,6 +277,16 @@ func cloneSourceInfos(sources []SourceInfo) []SourceInfo {
|
||||
|
||||
func cloneOrigin(origin Origin) Origin {
|
||||
origin.Locations = cloneLocations(origin.Locations)
|
||||
if origin.Inheritance != nil {
|
||||
steps := make([]InheritanceStep, len(origin.Inheritance))
|
||||
for index, step := range origin.Inheritance {
|
||||
steps[index] = InheritanceStep{
|
||||
From: step.From.clone(), To: step.To.clone(), Reference: step.Reference.clone(),
|
||||
Locations: cloneLocations(step.Locations),
|
||||
}
|
||||
}
|
||||
origin.Inheritance = steps
|
||||
}
|
||||
return origin
|
||||
}
|
||||
|
||||
|
||||
33
resource_context.go
Normal file
33
resource_context.go
Normal file
@@ -0,0 +1,33 @@
|
||||
package configkit
|
||||
|
||||
import "io/fs"
|
||||
|
||||
// ResourceContext provides a passive rooted filesystem view and a
|
||||
// slash-relative BaseDir for one acquired source. FS is borrowed: caller-owned
|
||||
// filesystems remain under caller control, while a host context opens a fresh
|
||||
// contained root for each resource open rather than retaining a loading handle.
|
||||
// Querying this record performs no I/O. Resource contents may change after
|
||||
// loading; callers must close files they open through FS.
|
||||
type ResourceContext struct {
|
||||
FS fs.FS
|
||||
BaseDir string
|
||||
}
|
||||
|
||||
// SourceContext returns the context of an acquired source ID. Unknown IDs and
|
||||
// sources without context return false. It is nil-safe and performs no I/O.
|
||||
// Follow individual origin locations when an origin has multiple sources.
|
||||
func (r *Result[T]) SourceContext(id string) (ResourceContext, bool) {
|
||||
if r == nil || r.inspection == nil {
|
||||
return ResourceContext{}, false
|
||||
}
|
||||
context, ok := r.inspection.SourceContext(id)
|
||||
return ResourceContext{FS: context.FS, BaseDir: context.BaseDir}, ok
|
||||
}
|
||||
|
||||
// SourceContext returns a passive source context during the active resolution
|
||||
// callback. Like other Resolution queries, it cannot be used after callback
|
||||
// return and performs no filesystem operations itself.
|
||||
func (r *Resolution) SourceContext(id string) (ResourceContext, bool) {
|
||||
context, ok := r.engineContext().SourceContext(id)
|
||||
return ResourceContext{FS: context.FS, BaseDir: context.BaseDir}, ok
|
||||
}
|
||||
30
source.go
30
source.go
@@ -2,14 +2,31 @@ package configkit
|
||||
|
||||
import (
|
||||
"io"
|
||||
"io/fs"
|
||||
|
||||
"gitea.maximumdirect.net/eric/ConfigKit/internal/engine"
|
||||
)
|
||||
|
||||
// Source is an opaque configuration input. Construct sources with File, Bytes,
|
||||
// Reader, or NoSource; the zero value is invalid.
|
||||
// Source is an opaque configuration input. Construct sources with File,
|
||||
// Filesystem.File, Bytes, Reader, Mapping, or NoSource; the zero value is invalid.
|
||||
type Source struct{ value engine.Source }
|
||||
|
||||
// Filesystem is a reusable opaque namespace for a caller-owned fs.FS. Copies
|
||||
// preserve identity; separately constructed values are distinct even if their
|
||||
// labels or underlying filesystems match. The filesystem is never closed.
|
||||
type Filesystem struct{ value engine.Filesystem }
|
||||
|
||||
// NewFilesystem borrows filesystem without opening it. Label must be nonempty
|
||||
// and safe to display; a nil filesystem or invalid label is reported when a
|
||||
// source from the returned namespace is consumed.
|
||||
func NewFilesystem(label string, filesystem fs.FS) Filesystem {
|
||||
return Filesystem{value: engine.NewFilesystem(label, filesystem)}
|
||||
}
|
||||
|
||||
// File selects a non-root fs.ValidPath name in this filesystem. Each load
|
||||
// reopens the file; constructor problems are reported when the source is used.
|
||||
func (f Filesystem) File(name string) Source { return Source{value: f.value.File(name)} }
|
||||
|
||||
// File returns a source that reopens path for every load attempt.
|
||||
func File(path string) Source { return Source{value: engine.File(path)} }
|
||||
|
||||
@@ -21,6 +38,15 @@ func Bytes(label string, data []byte) Source { return Source{value: engine.Bytes
|
||||
// Copies of the returned Source share the same single-use claim.
|
||||
func Reader(label string, input io.Reader) Source { return Source{value: engine.Reader(label, input)} }
|
||||
|
||||
// Mapping snapshots supported built-in values immediately. A nil root map is
|
||||
// invalid; an empty map is a valid empty document. Values may be nil, built-in
|
||||
// scalars, []any, or map[string]any recursively. Unsupported values and cycles
|
||||
// are reported when the source is consumed. Label must be nonempty and safe to
|
||||
// display. Later caller mutation cannot change the captured document.
|
||||
func Mapping(label string, values map[string]any) Source {
|
||||
return Source{value: engine.Mapping(label, values)}
|
||||
}
|
||||
|
||||
// NoSource explicitly selects a defaults-only input. It creates no source record.
|
||||
func NoSource() Source { return Source{value: engine.NoSource()} }
|
||||
|
||||
|
||||
5
testdata/filesystem/config/main.yml
vendored
Normal file
5
testdata/filesystem/config/main.yml
vendored
Normal file
@@ -0,0 +1,5 @@
|
||||
configkit:
|
||||
imports:
|
||||
- path: ../shared.yml
|
||||
items:
|
||||
main: base
|
||||
2
testdata/filesystem/overlays/testing.yml
vendored
Normal file
2
testdata/filesystem/overlays/testing.yml
vendored
Normal file
@@ -0,0 +1,2 @@
|
||||
items:
|
||||
main: selected
|
||||
2
testdata/filesystem/shared.yml
vendored
Normal file
2
testdata/filesystem/shared.yml
vendored
Normal file
@@ -0,0 +1,2 @@
|
||||
items:
|
||||
shared: default
|
||||
Reference in New Issue
Block a user