diff --git a/docs/adr/0004-secret-references-and-explicit-bootstrap.md b/docs/adr/0004-secret-references-and-explicit-bootstrap.md index ddc97a3..47c0c7a 100644 --- a/docs/adr/0004-secret-references-and-explicit-bootstrap.md +++ b/docs/adr/0004-secret-references-and-explicit-bootstrap.md @@ -1,6 +1,6 @@ # ADR 0004: Use secret references and explicit environment bootstrap -Status: accepted +Status: superseded by [ADR 0023](0023-explicit-configured-secrets-bootstrap.md). Date: 2026-09-06 diff --git a/docs/adr/0023-explicit-configured-secrets-bootstrap.md b/docs/adr/0023-explicit-configured-secrets-bootstrap.md new file mode 100644 index 0000000..cdae6fc --- /dev/null +++ b/docs/adr/0023-explicit-configured-secrets-bootstrap.md @@ -0,0 +1,60 @@ +# ADR 0023: Keep Configured Secrets Bootstrap Explicit + +## Status + +Accepted. Supersedes [ADR 0004](0004-secret-references-and-explicit-bootstrap.md). + +## Date + +2026-11-18 + +## Context + +Applications may discover their main configuration file during startup and keep +secret files beside it. Requiring applications to discover and parse that file +separately duplicates ConfigKit's work. Some deployments also need secret files +to replace environment values that are already set. + +## Decision + +Keep secrets as typed references to environment-variable names. Resolve surviving +references and keep credential values out of configuration trees, history, and +inspection. Applications own credential use. + +Directory bootstrap remains a separate startup operation that the application +must call explicitly. Ordinary configuration loading never writes the process +environment. The application may supply the directory directly or authorize +bootstrap from settings in the selected main document. Imports, underlays, +overlays, and later application processing do not choose those settings. + +When bootstrap uses the main document, ConfigKit selects and captures that +document and its filesystem context for subsequent loading. This keeps bootstrap +and loading tied to the same main contents and location. It does not snapshot +later imports, resources, or the environment. + +Preserve existing environment variables by default, including empty values. +Allow an explicit overwrite choice. An application-supplied choice takes +precedence over the main document's choice, including an explicit choice to +preserve existing values. + +Keep bootstrap reports separate from later secret-lookup evidence. Report +writes, skips, and partial failures without credential values. Bootstrap offers +no rollback or transaction across environment variables and no protection +against concurrent environment mutation. General environment-variable +configuration overlays remain outside this model. + +## Alternatives Considered + +- Leave discovery and parsing to applications. This preserves duplicate startup + infrastructure and can select different main contents for bootstrap and loading. +- Run bootstrap inside ordinary loading. This hides process mutation in an + operation applications may repeat or run concurrently. +- Always preserve existing environment values. This keeps one precedence rule + but prevents deployments from explicitly choosing provisioned files instead. + +## Consequences + +Applications retain control of startup ordering and environment writes. +ConfigKit owns main-source capture, bounded secret-file loading, and safe +reporting. A later configuration failure leaves completed bootstrap writes in +place. Existing callers retain the preservation default. diff --git a/docs/development.md b/docs/development.md index 0c3578b..2762999 100644 --- a/docs/development.md +++ b/docs/development.md @@ -54,7 +54,7 @@ Accepted decisions: - [Check source versions with one shared policy](adr/0013-shared-source-version-checks.md) - [Own YAML semantics behind a private parser boundary](adr/0002-own-yaml-semantics.md) - [Preserve presence and record application resolution](adr/0003-preserve-presence-and-resolution-history.md) -- [Secret references and explicit bootstrap](adr/0004-secret-references-and-explicit-bootstrap.md) +- [Explicit configured secrets bootstrap](adr/0023-explicit-configured-secrets-bootstrap.md) - [Explicit overrides and safe explanation](adr/0005-explicit-overrides-and-safe-explanation.md) - [Keep the public facade root-owned](adr/0006-public-facade.md) - [Compare frozen effective configuration snapshots](adr/0010-effective-configuration-diff.md) diff --git a/docs/policy/architecture.md b/docs/policy/architecture.md index 0ee7e91..74fbd10 100644 --- a/docs/policy/architecture.md +++ b/docs/policy/architecture.md @@ -405,9 +405,14 @@ Secret values enter configuration through typed environment-name references; literal credentials are not a supported configuration convention. Only an explicit accessor exposes a resolved credential to application code. Credential bytes must never enter composition trees, history, or inspection. General environment-variable -configuration overlays are an explicit non-goal. Bootstrap must preserve already -set variables, including empty ones, and report blocked files without contents. -See [ADR 0004](../adr/0004-secret-references-and-explicit-bootstrap.md). +configuration overlays are an explicit non-goal. Bootstrap preserves already +set variables by default, including empty ones, and reports blocked files without +contents. [ADR 0023](../adr/0023-explicit-configured-secrets-bootstrap.md) permits +explicit overwrite and main-document-directed bootstrap as a separate startup +operation. Its implementation is tracked in the +[secrets-directory roadmap](../roadmap/secrets-directory.md); the +[configuration contract](../config.md#explicit-secrets-directory-bootstrap) +owns current behavior. Secrets must be considered throughout error handling and inspection. Library renderers must apply secret classification, and parser errors must not be diff --git a/docs/roadmap/secrets-directory.md b/docs/roadmap/secrets-directory.md index b18ea97..29560db 100644 --- a/docs/roadmap/secrets-directory.md +++ b/docs/roadmap/secrets-directory.md @@ -1,5 +1,10 @@ # Main-relative secrets directories +Status: planned, not implemented. [ADR 0023](../adr/0023-explicit-configured-secrets-bootstrap.md) +supersedes ADR 0004 and records explicit configured bootstrap and optional +overwrite. The decisions below preserve ordinary loading without environment +writes and the existing direct-directory helper's Go signature. + ## Motivation and baseline Applications can use `SearchConfig` to select their main configuration file @@ -63,8 +68,10 @@ declaration and any diagnostics in source-local coordinates. [Document-root cont ### Companion helper and prepared main source The companion accepts the main `Source`, including a `SearchConfig` source, -and returns a secrets report and a prepared main source. The public name and -option type below are illustrative. +and returns a secrets report and a prepared main source. Its options accept an +explicit host `FilesystemContext` for main preparation and an optional overwrite +choice. Represent overwrite as `*bool`: nil defers to YAML, while a supplied +false or true takes precedence. The public names below are illustrative. ```go main := configkit.SearchConfig("demo", search) @@ -83,6 +90,15 @@ contents with the selected file's identity and filesystem context so the later load consumes that same main document. Keep its remaining controls, including imports and collection declarations, available to normal loading. +Apply the supplied host context using the existing main-file root rules during +preparation. Without an explicit context, derive the usual context from the +selected main file. Capture the resolved canonical base and containment root +with the document. Later loading uses that captured context when no explicit +context is supplied. An explicit later context must resolve to the same context; +diagnose an incompatible context rather than silently rebinding the prepared +source. A caller needing a different root prepares the original source again. +This rule applies to ordinary loading options and layered main `Input` alike. + The helper prepares only the main document. Composition, import acquisition, inheritance, typed decoding, application resolution, secret lookup, and validation run through the subsequent ordinary or layered load. Main @@ -91,27 +107,57 @@ Source records, diagnostics, placements, imports, and main-relative companions use the selected main's identity and context. Release library-owned file handles before the helper returns, including on -error. Prepared sources retain captured document data and passive filesystem -context and follow the source reuse contract without sharing mutable load -state. Later imports and resources observe the filesystem when accessed. A -later load reads the environment at its normal secret-resolution step. +error. Prepared sources are reusable snapshots of the main contents, with +passive filesystem context and no shared mutable load state. Reusing one neither +rediscovers nor rereads the main file and does not repeat bootstrap. This differs +from reusing the original `File` or `SearchConfig` source. Later imports and +resources observe the filesystem when accessed. A later load reads the environment +at its normal secret-resolution step. Each load accounts for the captured main +under its normal source limits and creates its own source and inspection records. [Complete-on-return loading](../adr/0008-complete-on-return-loading.md) · [Source resource contexts](../config.md#source-resource-contexts) +### Preparation failures and return values + +Use the existing error families at their respective boundaries. Discovery, +main-file acquisition, YAML parsing, bootstrap-control validation, and preparation +context failures return `LoadError` with applicable source records and source-local +diagnostics. Secret-directory inspection, reads, and environment writes return +`SecretsError` with the existing safe diagnostics. Do not introduce another error +or reporting framework. + +Return a secrets report on every attempt. Before directory processing begins, +it has no entries; after processing begins, it retains the entries and outcomes +observed so far. On any error, return a zero, unusable prepared `Source`. Earlier +environment writes remain reported and are not rolled back. A caller must handle +the failure before attempting configuration loading. + +On success, return a reusable prepared source, including when no directory was +declared. Optional search exhaustion returns the absent-main source and an empty +report. Preparation validates YAML and the bootstrap settings needed for its +operation; it does not claim the later configuration load will succeed. Normal +version checks, composition, decoding, and application validation remain in that +load. A later incompatible-context error is a loading error and does not undo +bootstrap writes. + ### Shared bootstrap and overwrite behavior -Extend the existing direct-directory helper with an optional named Boolean -option, keeping existing calls valid. The option type name in this example is -provisional. +Keep `LoadSecretsIntoEnvironment(string) (*SecretsReport, error)` unchanged, +including its nonvariadic function type and preservation behavior. Add a separately +named options-bearing helper for explicit overwrite. Changing the existing helper +to accept variadic options would preserve ordinary calls but break assignments to +its existing function type. The new public names below are provisional. ```go report, err := configkit.LoadSecretsIntoEnvironment("/etc/demo/secrets") -report, err = configkit.LoadSecretsIntoEnvironment( +report, err = configkit.LoadSecretsIntoEnvironmentWithOptions( "/etc/demo/secrets", configkit.SecretsOptions{Overwrite: true}, ) ``` -Both helpers use the same directory-bootstrap implementation. With overwrite +The existing helper, options-bearing helper, and main-config companion use the +same directory-bootstrap implementation. The direct helper's Boolean option +defaults to false; it has no YAML policy to override. With overwrite disabled, existing variables win, including empty ones, with presence checked again immediately before application as today. With overwrite enabled, eligible files are read even for existing variables and their values replace @@ -136,7 +182,7 @@ environment transaction guarantees continues to apply. Keep bootstrap reporting separate from the subsequent configuration's secret lookup evidence. Each operation describes what it observed and performed. -[Secrets bootstrap](../config.md#explicit-secrets-directory-bootstrap) · [Secret references and explicit bootstrap](../adr/0004-secret-references-and-explicit-bootstrap.md) +[Secrets bootstrap](../config.md#explicit-secrets-directory-bootstrap) · [Explicit configured bootstrap](../adr/0023-explicit-configured-secrets-bootstrap.md) ### Filesystem scope @@ -152,23 +198,29 @@ later extension requiring decisions about file-type and symlink capabilities. bootstrap. Retain source-local positions for diagnostics. - Factor main-file selection and capture for use by the companion helper and normal acquisition. Add a prepared-source handoff with retained main contents, - identity, context, and ownership. Select an API for supplying applicable - main-root constraints and representing an absent or explicit Go overwrite - choice, including `false`. -- Add the optional overwrite setting to direct-directory bootstrap and reuse - that operation from the companion helper, with matching reports and failures. + identity, context, and ownership. Supply the preparation filesystem context and + optional Go overwrite choice as described above. Preserve per-load accounting, + diagnose incompatible later contexts, and implement the failure/return contract. +- Preserve the current direct-directory signature and add a separately named + options-bearing helper. Reuse directory bootstrap from all three entry points, + with matching reports and failures. Finalize public names during implementation. - Verify discovery from a different working directory, canonical main paths, optional search exhaustion, declaration scope, default and explicit - overwrite choices, Go-over-YAML precedence, and both helper interfaces. + overwrite choices, Go-over-YAML precedence, and all helper interfaces. Include + assignment of the unchanged helper to its existing function type. Cover empty values, dot entries, read failures before writes, and partial write failures. Verify bootstrap includes entries unused by typed fields and that later load failures preserve bootstrap effects. -- Verify prepared-source reuse, handle cleanup, normal loading without +- Verify error types, empty and partial reports, and unusable prepared sources on + failure. Cover absent-main success and failed later loads without rollback. +- Verify prepared-source reuse after main-file changes and discovery changes, + live later imports and environment lookup, equivalent and incompatible later + contexts, handle cleanup, normal loading without environment writes, and agreement with selected-file provenance, placement, relative imports, and main-relative companions. Use disposable credential values and deterministic failure seams in line with the [testing policy](../policy/testing.md). -- Document the startup workflow and add a consumer example. Record the prepared - main-source handoff and optional overwrite policy in an ADR referencing - ADR 0004, and update the canonical configuration contract and affected - architecture and component documentation when implemented. +- Document the startup workflow and add a consumer example. Implement the + ownership decision in ADR 0023 and update the canonical configuration contract, + public source documentation, and affected component documentation when the + behavior lands. Retire the roadmap reference in architecture policy at that time.