eric/ConfigKit
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Add redacted effective rendering

Browse Source
This commit is contained in:
Eric Rakestraw
2026-09-06 21:21:11 +00:00
parent 26cece6d53
commit 828fdd38eb
14 changed files with 662 additions and 62 deletions
Download Patch File Download Diff File Expand all files Collapse all files

6
README.md
View File

@@ -8,8 +8,10 @@ imports, strictly composes their disjoint definitions, applies typed defaults, a
can overlay one selected fixed profile and check an application schema version. It
returns detached metadata and safe diagnostics.
Applications may perform audited defaulting and normalization through a resolver,
then report domain errors and warnings through read-only semantic validation.
Overrides, secrets, and rendering are still under development.
then report domain errors and warnings through read-only semantic validation. Secret
tags and a path predicate drive deterministic redacted effective YAML.
Overrides, secret references/bootstrap, and value explanation are still under
development.
```go
type Config struct {

63
docs/config.md
View File

@@ -1,9 +1,9 @@
# Configuration Input Contract
This document owns ConfigKit's currently implemented source-input, YAML syntax,
schema, typed-decoding, loading, metadata, and ownership contracts. The parser,
owned document representation, and decoder remain private behind the public loading
lifecycle.
schema, typed-decoding, loading, metadata, redaction, rendering, and ownership
contracts. The parser, owned document representation, and decoder remain private
behind the public loading lifecycle.
## Sources
@@ -42,7 +42,7 @@ The current lifecycle is:
validate options → parse the selected source → traverse/compose enabled imports
→ build schema → check the optional application version → apply a selected profile
→ create and clone defaults → strictly decode → recorded application resolution
→ semantic validation → snapshot configuration and metadata
→ semantic validation → classify secrets → snapshot configuration and metadata
```
`File`, `Bytes`, and `Reader` each contribute the first base source with deterministic
@@ -53,17 +53,18 @@ missing file remains an error; it is never interpreted as no input.
The current `Options` boundary accepts a defaults factory, `VersionRule`, resolver,
semantic validator, explicit path and glob imports, one selected profile, and
filesystem context. Overrides and dynamic secret classification return
`options.unavailable` because their lifecycle stages are not implemented. `NoSource`
rejects imports, active profiles, and filesystem context as invalid combinations.
Input containing a reserved `_configkit` mapping key is rejected while imports are
disabled.
filesystem context. It also accepts static secret tags and a dynamic path predicate.
Overrides return `options.unavailable` because their lifecycle stage is not
implemented. `NoSource` rejects imports, active profiles, and filesystem context as
invalid combinations. Input containing a reserved `_configkit` mapping key is
rejected while imports are disabled.
Every source, options, schema, version, defaults, decode, resolution, or validation
failure returns a nil result and a `*LoadError`. Successful results contain caller-owned `Config`, detached
`Metadata`, a defensive source list, and defensive nonfatal diagnostics. Mutating
the returned configuration or inspection records cannot alter ConfigKit's private
snapshot. ConfigKit performs no logging and exposes no effective writer yet.
snapshot. ConfigKit performs no logging. `Result.WriteEffective` exposes only the
frozen redacted projection.
### Application schema versions
@@ -88,8 +89,7 @@ available source coordinates are retained. `History` records the Go-zero or appl
default baseline followed by exact base or profile definitions, including
equal-valued input. Profile replacement removes displaced descendants from effective
metadata while retaining their base definition and the parent replacement operation
in history.
All metadata queries and returned nested records are detached.
in history. All metadata queries and returned nested records are detached.
History is chronological: a path can have a synthetic default record, its exact base
definition, a profile replacement, and a recorded resolution operation. A descendant
@@ -108,6 +108,42 @@ operation. Reasons and dependency paths retain first occurrence order with dupli
removed. Source records remain ordered as selected source, depth-first imports, then
the selected profile.
### Secret classification and effective rendering
Secret classification combines `configkit:"secret"` field tags with the optional
pure `Definition.Secret(Path) bool` predicate. A tagged or predicate-classified
container makes all descendants sensitive. ConfigKit evaluates final and historical
paths, so replacing or deleting a definition does not erase its classification. The
predicate can add sensitivity but cannot declassify a tagged path.
Recorded resolution dependencies propagate sensitivity to their derived target.
Propagation reaches a fixed point, so record/map iteration order does not affect the
result and dependency cycles terminate. A dependency on a classified descendant or
on a container that contains one is treated conservatively as sensitive. Applications
must classify secret-producing transformations whose dependencies are not recorded.
`Result.WriteEffective` writes the frozen resolved snapshot as deterministic redacted
YAML. Mappings sort by decoded key string, sequences retain order, indentation is two
spaces with noncompact sequence indentation, line endings are LF, the document has no
start or end marker, and output ends with a newline. Nil pointers/maps/slices remain
`null`; non-nil empty maps and sequences remain `{}` and `[]`. A secret root is
replaced by the quoted string `'[REDACTED]'`. Redacted output is intended for
inspection and is not promised to be reloadable.
Rendering uses only a safe adapter tree built from the already completed canonical
typed projection. It does not invoke application YAML marshal hooks or text codecs;
all text-codec projection has succeeded before `Load` returns. Caller mutation of
`Result.Config` cannot change later output. Writer failures, including short writes,
are returned and may leave partial redacted output; internal dumper failures are
translated to a fixed safe error. There is no unredacted rendering option and
ConfigKit performs no implicit logging.
`Result.Config` is the explicit trusted boundary and contains ordinary typed values,
including secrets. Metadata, histories, diagnostics, and load errors contain no raw
configuration values. Applications must still keep source labels, resolution
reasons, and diagnostic messages value-free because ConfigKit cannot infer and
sanitize secrets embedded in arbitrary application prose.
### Recorded application resolution
An optional `Definition.Resolve` callback runs after successful typed decoding. It
@@ -356,7 +392,8 @@ or pointer-to-struct, or designates at most one string-keyed inline map for othe
unmatched names. Ambiguous names, named inline fields, multiple inline maps, and
unsupported or malformed tag options are schema errors. Embedded fields are ordinary
named fields unless explicitly inlined. `configkit:"secret"` is the only current
ConfigKit field tag and records sensitivity for later lifecycle work.
ConfigKit field tag and marks its bound field or container for secret classification
and redacted inspection.
Schema path traversal uses these same bindings. Explicit key segments address
mappings, explicit index segments address sequences, and parsed numeric pointer

4
docs/development.md
View File

@@ -4,8 +4,8 @@ This is the contributor entry point for ConfigKit. Read the task-specific owners
below before making changes. The repository is in early implementation: its public
loader currently covers strict selected sources, explicit path/glob import graphs,
composition, selected profile overlays, typed defaults, schema-version checks,
recorded application resolution,
semantic validation, and metadata. The
recorded application resolution, semantic validation, layered metadata, secret
classification, and redacted effective rendering. The
[component overview](internal/overview.md) owns the implementation inventory.
## Initial orientation

12
docs/internal/extraction.md
View File

@@ -200,6 +200,18 @@ snapshot agreement, defensive nested copies, and concurrent load isolation. Valu
rendering, redaction, explanation, commands, generated attribution, diffs, and
digests remain assigned to their later owning work.
Effective-rendering implementation record (2026-09-06): ConfigKit adapts Narratio's
deterministic effective-output and side-effect-free inspection cases in
`classifySecretPaths`, `redactedDocument`, `Result.WriteEffective`, and the private
`internal/yaml.Write` adapter. Tests cover stable sorted mappings, preserved sequence
order, explicit indentation/line/document formatting, tagged containers, dynamic
current and historical paths, fixed-point dependency cycles, nil/empty distinctions,
resolved values, text-scalar snapshot reuse, application YAML-hook isolation,
non-disclosure through failures and record formatting, caller mutation, and writer
failures. Unlike Narratio's marshal/reparse path, ConfigKit renders only its owned
frozen tree and substitutes redaction before the upstream dumper boundary. Historical
value explanation and secret references remain assigned to later work.
### Defaults, presence, and domain resolution
Assigned across schema, decoding, resolution, validation, and consumer examples:

8
docs/internal/overview.md
View File

@@ -30,11 +30,15 @@ module and package documentation boundary. Its implemented public primitives are
explicitness queries, helper verification, structural mutation audit, coalesced
intent, private before/after snapshots, and resolution metadata integration; and
- post-resolution schema-aware report collection and validator mutation detection in
`validation.go`, completing the public callback lifecycle after composed input.
`validation.go`, completing the public callback lifecycle after composed input;
- fixed-point tag/predicate/dependency sensitivity classification in `secret.go`; and
- frozen-snapshot redaction and canonical effective output in `render.go`, with the
only upstream dumper integration isolated in `internal/yaml/write.go`.
The [configuration contract](../config.md) owns implemented input, schema, decoding,
loading lifecycle, path imports, composition, profile overlay, version checking,
recorded resolution, semantic validation, metadata, and ownership behavior.
recorded resolution, semantic validation, metadata, secret classification, redacted
effective rendering, and ownership behavior.
The source-extraction inventory records the application foundations assigned to
future implementation work:

33
docs/roadmap/implementation.md
View File

@@ -2,7 +2,7 @@
## Status and ownership
Status: implementation in progress; Stages 1–14 are complete and later stages are
Status: implementation in progress; Stages 1–15 are complete and later stages are
pending.
Audience: a `gpt-5.6-terra` coding agent implementing one stage per prompt, in
@@ -1092,7 +1092,36 @@ Acceptance:
## Stage 15 — Implement secret classification and effective rendering
Status: pending.
Status: complete.
Completion evidence (2026-09-06):
- Removed the dynamic-secret gate and combined descriptor `configkit:"secret"` roots
with a pure application path predicate over final and historical paths. Tagged
containers cover descendants, replaced/deleted definitions stay classified, and
resolution dependencies propagate sensitivity to a fixed point, including
container dependencies and cycles independent of record-map iteration order.
- Added `Result.WriteEffective` over the frozen resolved document. It clones and
redacts before crossing the adapter boundary, sorts mappings by decoded key,
preserves sequence order and nil/empty distinctions, and never invokes application
YAML hooks or text codecs during rendering. Caller mutation cannot change output;
there is no unredacted path or implicit logging.
- Added a private v4 dumper adapter with explicit two-space/noncompact-sequence
indentation, unlimited line width, Unicode, unique keys, block collections,
single-quote preference, LF endings, one document, and disabled start/end markers.
Secret roots render as the quoted string `'[REDACTED]'`; writer errors and short
writes propagate while internal dumper errors collapse to a fixed safe error.
- Tests cover tagged containers, dynamic current and overwritten paths, dependency
cycles, container dependencies, custom text scalars, application marshal-hook
isolation, resolved values, sorted/stable YAML, sequence order, nil/empty forms,
snapshot mutation isolation, writer failures, and canary non-disclosure through
parsing, decoding, returned hook errors, warnings, metadata, and ordinary/Go-syntax
record formatting. The
[current redaction contract](../config.md#secret-classification-and-effective-rendering)
owns the behavior. Historical-value explanation and secret references/bootstrap
remain later work. Verified `go test ./...`, `go test -race ./...`, `go vet ./...`,
`go build ./...`, `go mod tidy -diff`, and `git diff --check` with Go 1.26.7 on
linux/amd64.
Read: feature [redaction](initial.md#provenance-diagnostics-and-redaction),
[secret predicate](initial.md#schema-checking-and-secret-classification), result API.

57
docs/roadmap/initial.md
View File

@@ -470,7 +470,8 @@ The [current application schema contract](../config.md#application-schema-bindin
owns root/type support, field and tag binding, inlining, codec pairing, and
schema-aware path rules. The [current defaults and typed-decoding contract](../config.md#defaults-and-typed-decoding)
owns strict conversions, container application, presence, safe failures, and initial
origins. Secret handling and rendering remain target work below.
origins. Secret classification and effective rendering are owned by the
[current redaction contract](../config.md#secret-classification-and-effective-rendering).
## Recorded resolution and validation
@@ -479,7 +480,7 @@ Application-owned recorded resolution is implemented. The
explicitness queries, helpers and manual records, structural mutation auditing,
coalescing, diagnostics, origins, history, and snapshot ownership. Layered input
history is owned by the [current metadata contract](../config.md#layered-metadata);
redacted explanation and secret dependency propagation remain target work.
redacted historical-value explanation remains target work.
Canonical typed projection, detached cloning, structural comparison, and publishing
independently captured resolved results are implemented. The
@@ -499,7 +500,9 @@ Safe diagnostic/provenance value records, canonical diagnostic rendering, and
Default, base, profile, and resolution records are attached to the loading lifecycle;
their source ordering, composite/synthetic origins, replacement history, and detached
queries are owned by the [current metadata contract](../config.md#layered-metadata).
Safe historical values, redaction, and rendering remain target work below.
Secret classification and effective rendering are owned by the
[current redaction contract](../config.md#secret-classification-and-effective-rendering).
Safe historical values and explanation remain target work below.
Diagnostics have stable codes, severity, semantic path when available, a safe
message, primary location, and related locations. Sort deterministically by
@@ -520,20 +523,6 @@ Applications likewise must supply value-free validation messages and resolution
reasons. ConfigKit cannot sanitize secrets embedded in arbitrary application
prose reliably.
Use `configkit:"secret"` on fields and an application path-classification hook
for dynamic entries. A secret container makes descendants secret. Secret
classification applies conservatively across current and historical definitions
of a path: replacing a secret does not declassify its history. It propagates from
recorded secret dependencies to derived outputs. Applications remain responsible
for marking secret-producing transformations whose dependencies are not recorded.
Provide source reporting and redacted effective rendering in the initial target.
Render from a canonical projection of the final typed snapshot, including zero
values and null/empty distinctions supported by the schema. Redacted output is
for inspection and is not promised to be reloadable. ConfigKit renderers must
not expose raw secrets; returning typed configuration to the application is an
intentional trusted boundary. ConfigKit does not log configuration on its own.
Provide full safe historical-value explanation in the initial release, as defined
[below](#historical-value-explanation). Defer config diffs and fingerprints. A future fingerprint design
must specify canonicalization, secret handling, and disclosure risks before
@@ -568,9 +557,9 @@ and verification against consumer models.
`Load`, `Definition`, `Options`, `FilesystemContext`, `Override`, `VersionRule`,
`Version`, `Validation`, and `Result` are declared. The current loader supports the
callback and path-import subset described by the [current contract](../config.md);
later override, secret, and rendering behavior below remains target work and
unavailable options fail explicitly rather than being ignored.
implemented boundary described by the [current contract](../config.md). Explicit
overrides, secret references/bootstrap, and historical-value explanation remain
target work; unavailable options fail explicitly rather than being ignored.
```go
func Load[T any](source Source, options ...Options[T]) (*Result[T], error)
@@ -863,8 +852,9 @@ potentially sensitive configured values.
`VersionRule`, `Version`, and the version-checking use of `Validation` are
implemented. The [current version contract](../config.md#application-schema-versions)
owns absent/null/string handling, report locations, warnings, failure behavior, and
raw-error suppression. Secret classification and later composed-source placement
checks remain target work.
raw-error suppression. The
[current redaction contract](../config.md#secret-classification-and-effective-rendering)
owns secret classification.
```go
type VersionRule struct {
@@ -918,19 +908,15 @@ versions remain strings under ConfigKit's convention; adopting consumers with
integer versions must migrate their syntax rather than enable a second scalar
policy.
`Definition.Secret` is an optional pure path predicate for dynamic entries,
combined with secret field tags. It can add sensitivity but cannot override a
tag to declassify a value. It is applied to historical as well as final paths.
Recorded dependencies propagate sensitivity under the existing redaction rules;
applications own secret-producing transformations not described by those records.
`Definition.Secret` and `configkit:"secret"` behavior are implemented and owned by
the [current redaction contract](../config.md#secret-classification-and-effective-rendering).
### Results, errors, and inspection
`Result.Config`, `Metadata`, `Sources`, and `Diagnostics`, together with
`LoadError` and `WriteDiagnostics`, are implemented at the current loading boundary.
`Result.Config`, `Metadata`, `Sources`, `Diagnostics`, and `WriteEffective`, together
with `LoadError` and `WriteDiagnostics`, are implemented at the current loading boundary.
The [current loading contract](../config.md#single-source-loading) and Go reference
comments own their behavior. Effective rendering and richer layered inspection
below remain target work.
comments own their behavior. Historical-value explanation below remains target work.
```go
type Result[T any] struct {
@@ -961,12 +947,9 @@ locations as described [above](#provenance-diagnostics-and-redaction). No raw
parser error or unrestricted secret-bearing history is exposed. No automatic
logging occurs. Inspection methods returning slices provide defensive copies.
`Config` is caller-owned typed data. Metadata and rendering use an independently
captured final snapshot, so later caller mutation cannot rewrite the recorded
result. `WriteEffective` emits deterministic redacted YAML for inspection, with
no unredacted switch. It returns output errors rather than a new configuration validation result.
Custom projection failures must already have been rejected before a load
succeeds. Output can be partial on writer failure.
Caller-owned configuration, the frozen result snapshot, secret classification, and
`WriteEffective` behavior are owned by the
[current redaction contract](../config.md#secret-classification-and-effective-rendering).
An application can report errors and inspect a successful load without rebuilding
diagnostic formatting. This example uses `errors` and `io` and the pipeline

103
internal/yaml/write.go Normal file
View File

@@ -0,0 +1,103 @@
package yaml
import (
"errors"
"io"
"strings"
"go.yaml.in/yaml/v4"
"gitea.maximumdirect.net/eric/ConfigKit/internal/document"
)
var errWrite = errors.New("configkit: effective YAML rendering failed")
// Write serializes one owned document tree with fully specified stable formatting.
func Write(output io.Writer, root *document.Node) error {
if output == nil || root == nil {
return errWrite
}
dumper, err := yaml.NewDumper(strictWriter{output},
yaml.WithIndent(2),
yaml.WithCompactSeqIndent(false),
yaml.WithLineWidth(-1),
yaml.WithUnicode(true),
yaml.WithUniqueKeys(true),
yaml.WithCanonical(false),
yaml.WithLineBreak(yaml.LineBreakLN),
yaml.WithExplicitStart(false),
yaml.WithExplicitEnd(false),
yaml.WithFlowSimpleCollections(false),
yaml.WithQuotePreference(yaml.QuoteSingle),
yaml.WithAllDocuments(false),
)
if err != nil {
return errWrite
}
if err := dumper.Dump(adapterNode(root)); err != nil {
return safeWriteError(err)
}
if err := dumper.Close(); err != nil {
return safeWriteError(err)
}
return nil
}
func adapterNode(node *document.Node) *yaml.Node {
result := &yaml.Node{}
switch node.Kind {
case document.Mapping:
result.Kind, result.Tag = yaml.MappingNode, "!!map"
result.Content = make([]*yaml.Node, 0, len(node.Mapping)*2)
for _, entry := range node.Mapping {
result.Content = append(result.Content,
&yaml.Node{Kind: yaml.ScalarNode, Tag: "!!str", Value: entry.Key},
adapterNode(entry.Value),
)
}
case document.Sequence:
result.Kind, result.Tag = yaml.SequenceNode, "!!seq"
result.Content = make([]*yaml.Node, len(node.Sequence))
for index, child := range node.Sequence {
result.Content[index] = adapterNode(child)
}
case document.String:
result.Kind, result.Tag, result.Value = yaml.ScalarNode, "!!str", node.Text
if node.Text == "[REDACTED]" {
result.Style = yaml.SingleQuotedStyle
}
case document.Boolean:
result.Kind, result.Tag, result.Value = yaml.ScalarNode, "!!bool", node.Text
case document.Number:
result.Kind, result.Value = yaml.ScalarNode, node.Text
result.Tag = "!!int"
if strings.ContainsAny(node.Text, ".eE") {
result.Tag = "!!float"
}
case document.Null:
result.Kind, result.Tag, result.Value = yaml.ScalarNode, "!!null", "null"
}
return result
}
func safeWriteError(err error) error {
var dumpError *yaml.DumpError
if errors.As(err, &dumpError) && dumpError.Stage == yaml.WriterStage {
if cause := errors.Unwrap(dumpError); cause != nil {
return cause
}
}
return errWrite
}
type strictWriter struct {
output io.Writer
}
func (writer strictWriter) Write(data []byte) (int, error) {
written, err := writer.output.Write(data)
if err == nil && written != len(data) {
err = io.ErrShortWrite
}
return written, err
}

59
internal/yaml/write_test.go Normal file
View File

@@ -0,0 +1,59 @@
package yaml
import (
"bytes"
"errors"
"io"
"testing"
"gitea.maximumdirect.net/eric/ConfigKit/internal/document"
)
func TestWriteUsesStableExplicitFormatting(t *testing.T) {
t.Parallel()
root := &document.Node{Kind: document.Mapping, Mapping: []document.Entry{
{Key: "empty_map", Value: &document.Node{Kind: document.Mapping, Mapping: []document.Entry{}}},
{Key: "empty_sequence", Value: &document.Node{Kind: document.Sequence, Sequence: []*document.Node{}}},
{Key: "number", Value: &document.Node{Kind: document.Number, Text: "1.25e+2"}},
{Key: "redacted", Value: &document.Node{Kind: document.String, Text: "[REDACTED]"}},
{Key: "sequence", Value: &document.Node{Kind: document.Sequence, Sequence: []*document.Node{
{Kind: document.String, Text: "first"},
{Kind: document.String, Text: "true"},
}}},
{Key: "nil_value", Value: &document.Node{Kind: document.Null}},
}}
var output bytes.Buffer
if err := Write(&output, root); err != nil {
t.Fatal(err)
}
want := "empty_map: {}\nempty_sequence: []\nnumber: 1.25e+2\nredacted: '[REDACTED]'\nsequence:\n - first\n - 'true'\nnil_value: null\n"
if output.String() != want {
t.Fatalf("output:\n%s\nwant:\n%s", output.String(), want)
}
}
func TestWritePropagatesWriterErrorsAndShortWrites(t *testing.T) {
t.Parallel()
root := &document.Node{Kind: document.Mapping, Mapping: []document.Entry{{Key: "value", Value: &document.Node{Kind: document.String, Text: "safe"}}}}
want := errors.New("write failed")
if err := Write(failingWriter{err: want}, root); !errors.Is(err, want) {
t.Fatalf("writer error = %v", err)
}
if err := Write(shortOutput{}, root); !errors.Is(err, io.ErrShortWrite) {
t.Fatalf("short writer error = %v", err)
}
}
type failingWriter struct {
err error
}
func (writer failingWriter) Write(data []byte) (int, error) {
return min(2, len(data)), writer.err
}
type shortOutput struct{}
func (shortOutput) Write(data []byte) (int, error) {
return max(0, len(data)-1), nil
}

7
load.go
View File

@@ -7,6 +7,7 @@ import (
)
// Definition contains reusable application rules for a configuration type.
// Secret is a pure predicate that can add sensitivity to semantic paths.
// Definitions should be treated as read-only after being shared.
type Definition[T any] struct {
Defaults func() T
@@ -68,6 +69,7 @@ type Result[T any] struct {
diagnostics []Diagnostic
final *document.Node
resolution []resolutionSnapshot
secrets map[string]bool
}
// Metadata returns detached effective presence, origin, and history records.
@@ -213,6 +215,7 @@ func Load[T any](source Source, supplied ...Options[T]) (*Result[T], error) {
return nil, accumulator.loadError()
}
}
secrets := classifySecretPaths(schema, options.Definition.Secret, metadata, final)
return &Result[T]{
Config: configuration,
metadata: metadata,
@@ -220,6 +223,7 @@ func Load[T any](source Source, supplied ...Options[T]) (*Result[T], error) {
diagnostics: canonicalDiagnostics(accumulator.diagnostics),
final: document.Clone(final),
resolution: cloneResolutionSnapshots(resolutionSnapshots),
secrets: secrets,
}, nil
}
@@ -233,9 +237,6 @@ func selectOptions[T any](source Source, supplied []Options[T]) (Options[T], []D
}
var diagnostics []Diagnostic
definition := options.Definition
if definition.Secret != nil {
diagnostics = append(diagnostics, optionDiagnostic(CodeOptionsUnavailable, "dynamic secret classification is not available in this release boundary"))
}
if !validProfileName(options.Profile) {
diagnostics = append(diagnostics, optionDiagnostic(CodeOptionsInvalid, "profile name must contain only ASCII letters, digits, underscore, or hyphen"))
}

1
load_test.go
View File

@@ -222,7 +222,6 @@ func TestLoadRejectsUnavailableOptionsAndDisabledDirectives(t *testing.T) {
code Code
}{
{name: "multiple options", source: validSource(), options: []Options[config]{{}, {}}, code: CodeOptionsInvalid},
{name: "secret classifier", source: validSource(), options: []Options[config]{{Definition: Definition[config]{Secret: func(Path) bool { return false }}}}, code: CodeOptionsUnavailable},
{name: "imports without filesystem context", source: validSource(), options: []Options[config]{{Imports: true}}, code: CodeOptionsInvalid},
{name: "incomplete filesystem", source: validSource(), options: []Options[config]{{Filesystem: &FilesystemContext{}}}, code: CodeOptionsInvalid},
{name: "overrides", source: validSource(), options: []Options[config]{{Overrides: []Override{{Path: Keys("name"), Value: "other", Flag: "--name"}}}}, code: CodeOptionsUnavailable},

70
render.go Normal file
View File

@@ -0,0 +1,70 @@
package configkit
import (
"errors"
"io"
"slices"
"strings"
"gitea.maximumdirect.net/eric/ConfigKit/internal/document"
yamladapter "gitea.maximumdirect.net/eric/ConfigKit/internal/yaml"
)
var errEffectiveUnavailable = errors.New("configkit: effective configuration is unavailable")
// WriteEffective writes deterministic redacted YAML from the frozen resolved
// snapshot. The output is intended for inspection and is not promised reloadable.
// Writer errors, including short writes, are returned and may leave partial output.
func (r *Result[T]) WriteEffective(output io.Writer) error {
if r == nil || r.final == nil || output == nil {
return errEffectiveUnavailable
}
safe := redactedDocument(r.final, r.secrets, Path{})
return yamladapter.Write(output, safe)
}
func redactedDocument(node *document.Node, secret map[string]bool, path Path) *document.Node {
if node == nil {
return nil
}
if pathIsSecret(secret, path) {
return &document.Node{Kind: document.String, Text: "[REDACTED]"}
}
result := *node
switch node.Kind {
case document.Mapping:
result.Mapping = make([]document.Entry, len(node.Mapping))
copy(result.Mapping, node.Mapping)
slices.SortFunc(result.Mapping, func(left, right document.Entry) int {
return strings.Compare(left.Key, right.Key)
})
for index := range result.Mapping {
entry := &result.Mapping[index]
entry.Value = redactedDocument(entry.Value, secret, path.Key(entry.Key))
}
case document.Sequence:
result.Sequence = make([]*document.Node, len(node.Sequence))
for index, child := range node.Sequence {
result.Sequence[index] = redactedDocument(child, secret, path.Index(index))
}
}
return &result
}
func pathIsSecret(secret map[string]bool, path Path) bool {
if secret[""] {
return true
}
pointer := path.String()
for pointer != "" {
if secret[pointer] {
return true
}
position := strings.LastIndexByte(pointer, '/')
if position <= 0 {
break
}
pointer = pointer[:position]
}
return false
}

177
render_test.go Normal file
View File

@@ -0,0 +1,177 @@
package configkit
import (
"bytes"
"errors"
"fmt"
"io"
"strings"
"sync/atomic"
"testing"
)
const (
taggedCanary = "TAGGED_CANARY_7c65"
dynamicCanary = "DYNAMIC_CANARY_6a12"
historicalCanary = "HISTORICAL_CANARY_91ef"
derivedACanary = "DERIVED_A_CANARY_4c20"
derivedBCanary = "DERIVED_B_CANARY_831d"
)
var renderTextMarshalCalls atomic.Int64
var renderYAMLMarshalCalls atomic.Int64
type renderText string
func (value renderText) MarshalText() ([]byte, error) {
renderTextMarshalCalls.Add(1)
return []byte(value), nil
}
func (value *renderText) UnmarshalText(text []byte) error {
*value = renderText(text)
return nil
}
type renderCredentials struct {
Token string `yaml:"token"`
Nested []string `yaml:"nested"`
}
type renderConfig struct {
Credentials renderCredentials `yaml:"credentials" configkit:"secret"`
Custom renderText `yaml:"custom"`
DerivedA string `yaml:"derived_a"`
DerivedB string `yaml:"derived_b"`
Dynamic any `yaml:"dynamic"`
EmptyMap map[string]string `yaml:"empty_map"`
EmptySequence []string `yaml:"empty_sequence"`
Labels map[string]string `yaml:"labels"`
NilMap map[string]string `yaml:"nil_map"`
NilSequence []string `yaml:"nil_sequence"`
Sequence []string `yaml:"sequence"`
}
func (renderConfig) MarshalYAML() (any, error) {
renderYAMLMarshalCalls.Add(1)
return map[string]string{"forbidden": "application YAML hook ran"}, nil
}
func TestWriteEffectiveRedactsTagsDynamicHistoryAndDependencyCycles(t *testing.T) {
renderTextMarshalCalls.Store(0)
renderYAMLMarshalCalls.Store(0)
directory := t.TempDir()
mainPath := writeImportFile(t, directory, "main.yml", `credentials:
token: TAGGED_CANARY_7c65
nested: [nested-secret]
custom: custom value
dynamic: {retired: HISTORICAL_CANARY_91ef}
empty_map: {}
empty_sequence: []
labels: {hidden: DYNAMIC_CANARY_6a12, public: visible}
sequence: [second, first]
`)
writeImportFile(t, directory, "profiles/testing.yml", "dynamic: selected\n")
definition := Definition[renderConfig]{
Secret: func(path Path) bool {
return path.String() == "/labels/hidden" || path.String() == "/dynamic/retired"
},
Resolve: func(configuration *renderConfig, resolution *Resolution) error {
Normalize(resolution, Keys("derived_a"), &configuration.DerivedA, derivedACanary,
"derived through cycle", Keys("derived_b"))
Normalize(resolution, Keys("derived_b"), &configuration.DerivedB, derivedBCanary,
"derived from retired secret", Keys("derived_a"), Keys("dynamic"))
resolution.Warning(Keys("derived_a"), "derived output is configured")
return nil
},
}
result, err := Load[renderConfig](File(mainPath), Options[renderConfig]{Definition: definition, Profile: "testing"})
if err != nil {
t.Fatal(err)
}
if result.Config.Credentials.Token != taggedCanary || result.Config.Labels["hidden"] != dynamicCanary ||
result.Config.DerivedA != derivedACanary || result.Config.DerivedB != derivedBCanary {
t.Fatal("typed configuration did not retain trusted values")
}
if !result.secrets["/dynamic/retired"] ||
!result.secrets["/derived_a"] || !result.secrets["/derived_b"] {
t.Fatalf("secret closure = %#v", result.secrets)
}
if renderYAMLMarshalCalls.Load() != 0 {
t.Fatal("loading called an application YAML marshal hook")
}
textCalls := renderTextMarshalCalls.Load()
var output bytes.Buffer
if err := result.WriteEffective(&output); err != nil {
t.Fatal(err)
}
want := "credentials: '[REDACTED]'\ncustom: custom value\nderived_a: '[REDACTED]'\nderived_b: '[REDACTED]'\ndynamic: selected\nempty_map: {}\nempty_sequence: []\nlabels:\n hidden: '[REDACTED]'\n public: visible\nnil_map: null\nnil_sequence: null\nsequence:\n - second\n - first\n"
if output.String() != want {
t.Fatalf("effective output:\n%s\nwant:\n%s", output.String(), want)
}
if renderTextMarshalCalls.Load() != textCalls || renderYAMLMarshalCalls.Load() != 0 {
t.Fatal("rendering called an application marshal hook")
}
for _, canary := range []string{taggedCanary, dynamicCanary, historicalCanary, derivedACanary, derivedBCanary, "nested-secret"} {
if strings.Contains(output.String(), canary) {
t.Fatalf("effective output contains %q", canary)
}
formatted := fmt.Sprintf("%v %#v %v %#v", result.Metadata(), result.Metadata(), result.Diagnostics(), result.Diagnostics())
if strings.Contains(formatted, canary) {
t.Fatalf("metadata formatting contains %q", canary)
}
}
beforeMutation := output.String()
result.Config.Credentials.Token = "MUTATED_SECRET"
result.Config.Labels["hidden"] = "MUTATED_DYNAMIC_SECRET"
result.Config.Sequence[0] = "mutated"
output.Reset()
if err := result.WriteEffective(&output); err != nil || output.String() != beforeMutation {
t.Fatalf("frozen output after caller mutation = %q, %v", output.String(), err)
}
wantWriteError := errors.New("write failed")
if err := result.WriteEffective(errorWriter{err: wantWriteError}); !errors.Is(err, wantWriteError) {
t.Fatalf("writer error = %v", err)
}
if err := result.WriteEffective(shortWriter{}); !errors.Is(err, io.ErrShortWrite) {
t.Fatalf("short writer error = %v", err)
}
var nilResult *Result[renderConfig]
if err := nilResult.WriteEffective(io.Discard); !errors.Is(err, errEffectiveUnavailable) {
t.Fatalf("nil result error = %v", err)
}
}
func TestSecretValuesStayOutOfLoadFailuresAndWarnings(t *testing.T) {
tests := []struct {
name string
input string
definition Definition[renderConfig]
}{
{name: "parse", input: "credentials: [" + taggedCanary + "\n"},
{name: "decode", input: "credentials: " + taggedCanary + "\n"},
{name: "returned hook error", input: "credentials: {token: " + taggedCanary + "}\n", definition: Definition[renderConfig]{
Resolve: func(*renderConfig, *Resolution) error { return errors.New(taggedCanary) },
}},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
test.definition.Secret = func(path Path) bool { return path.String() == "/credentials" }
result, err := Load[renderConfig](Bytes("secret-failure.yml", []byte(test.input)), Options[renderConfig]{Definition: test.definition})
if result != nil || err == nil {
t.Fatalf("result, error = %#v, %v", result, err)
}
var loadError *LoadError
if !errors.As(err, &loadError) {
t.Fatalf("error type = %T", err)
}
formatted := fmt.Sprintf("%v %#v %v %#v %v %#v", err, err, loadError.Diagnostics(), loadError.Diagnostics(), loadError.Sources(), loadError.Sources())
if strings.Contains(formatted, taggedCanary) {
t.Fatalf("failure formatting disclosed a secret: %s", formatted)
}
})
}
}

124
secret.go Normal file
View File

@@ -0,0 +1,124 @@
package configkit
import "gitea.maximumdirect.net/eric/ConfigKit/internal/document"
type secretDependency struct {
target Path
dependency Path
}
func classifySecretPaths(schema *schemaDescriptor, predicate func(Path) bool, metadata Metadata, final *document.Node) map[string]bool {
paths := make(map[string]Path)
finalPaths := make(map[string]*document.Node)
indexDocument(final, Path{}, finalPaths)
for pointer := range finalPaths {
addSecretPath(paths, pointer)
}
var dependencies []secretDependency
for pointer, history := range metadata.history {
addSecretPath(paths, pointer)
for _, change := range history {
paths[change.Path.String()] = change.Path.clone()
for _, dependency := range change.Dependencies {
paths[dependency.String()] = dependency.clone()
dependencies = append(dependencies, secretDependency{
target: change.Path.clone(), dependency: dependency.clone(),
})
}
}
}
secret := make(map[string]bool)
for pointer, path := range paths {
if root, ok := schemaSecretRoot(schema, path); ok {
secret[root.String()] = true
}
if predicate != nil && predicate(path.clone()) {
secret[pointer] = true
}
}
for {
changed := false
for _, dependency := range dependencies {
if secretIntersectsPath(paths, secret, dependency.dependency) && !secret[dependency.target.String()] {
secret[dependency.target.String()] = true
changed = true
}
}
if !changed {
return secret
}
}
}
func addSecretPath(paths map[string]Path, pointer string) {
if path, err := ParsePath(pointer); err == nil {
paths[pointer] = path
}
}
func secretIntersectsPath(paths map[string]Path, secret map[string]bool, candidate Path) bool {
for pointer := range secret {
classified, ok := paths[pointer]
if !ok {
continue
}
if pathContains(classified, candidate) || pathContains(candidate, classified) {
return true
}
}
return false
}
func schemaSecretRoot(schema *schemaDescriptor, path Path) (Path, bool) {
if schema == nil || schema.root == nil || !path.valid() {
return Path{}, false
}
current := schema.root
walked := Path{}
for _, segment := range path.segments {
walked.segments = append(walked.segments, segment)
for current.kind == descriptorPointer {
current = current.element
}
switch current.kind {
case descriptorStruct:
if segment.kind == pathSegmentIndex {
return Path{}, false
}
position, ok := current.fieldByName[segment.key]
var field *fieldDescriptor
if ok {
field = &current.fields[position]
} else {
field = current.inlineMap
}
if field == nil {
return Path{}, false
}
if field.secret {
return walked.clone(), true
}
if field.inlineMap {
current = field.descriptor.element
} else {
current = field.descriptor
}
case descriptorMap:
if segment.kind == pathSegmentIndex {
return Path{}, false
}
current = current.element
case descriptorSlice, descriptorArray:
if _, ok := schemaIndex(segment); !ok {
return Path{}, false
}
current = current.element
case descriptorDynamic:
return Path{}, false
default:
return Path{}, false
}
}
return Path{}, false
}