ConfigKit/docs/roadmap/implementation.md

Public facade: implementation plan

Status: not started. Baseline: 28e7fc5f690b36c60e4d03586afcff00fb1dee90. The feature roadmap owns the target architecture and user intent. This plan owns implementation decisions, ordered work, validation, and handoffs. Each numbered stage is sized for one gpt-5.6-terra coding-agent assignment; no stage requires subagents or changes to downstream repositories.

Historical navigation: this filename previously held the completed source-origins plan, including its Stage 5 completion record. That record is preserved at the baseline above: git show 28e7fc5:docs/roadmap/implementation.md. Existing incoming links from the source-origins roadmap, remediation record, and development guide refer to that historical plan, not the stages below. Stage 1 repairs those links.

Working rules

Before each stage read development, the feature roadmap, the preceding completion records, and all applicable policies: architecture, documentation, testing, and release. Read the owning configuration/API contracts and implementation before editing. Prefer the code graph for discovery, verifying its results against current code. Preserve unrelated work and the baseline's already-fixed resolution-history copy.

Complete stages in order. Every handoff must build and pass its relevant tests; do not leave placeholder public methods, skipped regressions, or known failures for a later stage. Run ./scripts/check.sh at every code-stage handoff, in addition to the focused gates below. Keep default tests deterministic and offline. Follow the development guide for dependency acquisition and supported toolchains.

Record completion beneath each stage: changed owners, tests and actual toolchain, results, deviations, remaining risks, and the next handoff. Update current documentation with actual changes, not promised completion. Commit stage work when authorized; this plan does not authorize pushes, tags, release selection, or publication. Release notes and native-platform release evidence remain part of the separate release procedure.

Fixed implementation decisions

Package and migration strategy

Final production packages are the public root, internal/engine, and the existing internal/document and internal/yaml. Keep the engine cohesive; do not create separate packages for every phase. Root imports engine; engine imports its supporting packages; no internal production package imports root. Only the YAML adapter imports the upstream YAML dependency.

Public named types remain defined in root. Use named private fields for delegated state, explicit forwarding methods, and ordinary public record declarations. Do not alias public types to engine types, anonymously embed engine types, expose engine types in public signatures/fields, or add public bridge methods. Internal type aliases used only inside implementation are not public aliases.

Because private-state references span most root implementation files, use a bounded relocation candidate rather than attempting a live package split in every stage. Stage 2 copies the existing implementation into internal/engine; Stages 4–7 build the eventual root facade in temporary internal/facade (package name configkit). Root remains the sole consumer entry point until Stage 8. The temporary facade is staging space, not a second supported API/package.

This temporary duplication is a migration mechanism only: no feature development, runtime selector, fallback engine, or new consumer import is introduced. Baseline semantics remain fixed; necessary corrections must be reconciled explicitly. Stage 8 moves the facade to root and removes the old implementation and temporary directory in the same handoff. There is one production implementation at completion. This makes the large textual move mechanical while separating it from the more demanding ownership and callback work. Do not keep two engines as a final design.

Type and state ownership

Public family	Selected representation and adaptation
Path	Root struct with named private engine.Path value; constructors and methods delegate. Preserve semantic segment kinds and invalid state by direct private wrapping, never stringify/reparse.
Layer, ChangeKind, Severity, Code, SecretFileStatus	Root-defined string types and explicitly typed constants with unchanged values. Convert to engine equivalents at the boundary; engine constants may supply values without making public aliases.
Location, SourceInfo, Diagnostic, Origin, FieldInfo, Change	Root-defined records with unchanged exported fields. Explicit private conversions cover nested pointers, slices, and paths.
ValueSnapshot, ValueChange, SecretReferenceInfo, Explanation, SecretFileResult	Root-defined output records. Convert detached engine records; preserve nil/empty distinctions and field semantics.
Source	Root struct with private engine.Source value. Engine owns bytes, constructor validity, and the shared reader claim; wrapper copies must not create new claims or re-read/copy source input.
Metadata	Root struct with private engine metadata value; explicit query methods convert detached results. Engine retains effective and historical indexes.
Resolution, Validation	Root structs with private pointers to their engine contexts. Engine alone owns activity and audit state; wrappers never recreate that state.
Definition[T], Options[T], FilesystemContext, Override, VersionRule, Version	Root records retaining all current fields and signatures. Private option/callback conversion into engine records; no extra configuration clone or reflection-based generic record mapper.
Result[T]	Root struct retaining Config T, plus private pointer to a non-generic engine inspection result. Engine load returns configuration and inspection separately; facade stores configuration exactly once.
Secret	Root-owned opaque scalar retaining value-copy semantics and existing public methods. Private exact-type adapter lets engine read/build it; no public accessor for plumbing.
LoadError, SecretsError, SecretsReport	Root structs with private engine backing state and explicit methods. Translate engine typed failures at entry points; never return or unwrap an engine error as the public structured error.

The inventory in Stage 1 verifies every baseline exported declaration, not just these grouped examples. Public type identity stays rooted in configkit. Preserve receivers, generics, supported zero/nil behavior, interface satisfaction, error assertions, and supported construction/copy behavior. Private field layout and incidental formatting are not new compatibility promises.

Use handwritten, local conversion helpers. For detached outputs, transfer already detached data where safe instead of deep-copying everything again. Never expose engine-owned mutable slices/maps through public records. Preserve order and missing/null distinctions. Do not round-trip records through JSON/YAML or use unsafe, registration globals, or reflection to bypass private fields.

Exact-type secret bridge

Introduce an engine-internal SecretBinding containing Type reflect.Type, Read func(reflect.Value) SecretState, and Make func(SecretState) reflect.Value. SecretState carries the existing name, value, specified, and resolved fields. It is operation data, never inspection data. The root constructs the binding privately for exactly reflect.TypeFor[Secret](); Read and Make use ordinary type assertions/construction in the owning package.

Supply the binding explicitly to the engine load and schema construction. Store it in the per-operation schema/value-operation context, available to recursive decoding, projection, cloning, dynamic-value checks, resolution, and validation. Keep the application configuration type T unchanged: no mirror configuration structs and no conversion of arbitrary application objects.

Replace all direct engine assertions/constructions of the old Secret, not just schema recognition. Include decoding, unresolved versus validated clone modes, projection, required/optional references, lookup caching, and validator mutation audit. Recognition remains exact-type recognition, not acceptance of any type implementing an interface. Bindings are fixed by the facade, not configurable by consumers, and perform no environment access themselves. Environment lookup remains in its existing lifecycle phase and its existing private test seam.

Engine unit tests use a test-local secret type and binding. Root integration tests exercise the actual public Secret. There is no production engine-owned replacement secret type that consumers must put in their configuration.

Callback, option, result, and error bridges

Keep the engine lifecycle generic in the real T. Convert each supplied options value without selecting/validating options in the facade: engine retains original validation order and diagnostics, including multiple-options rejection. Preserve nil callbacks, version pointers, defaults behavior, and override values without eagerly invoking callbacks or evaluating any values.

Adapt Resolve/Validate/Version.Check by closures wrapping the exact engine context pointer and passing the same *T received from engine. Convert version records and semantic paths at those boundaries. The secret-classification predicate similarly wraps the engine path. Do not recover, retry, close, or extend callbacks in the facade; engine retains the existing lifetime and failure behavior.

Generic public Default/Normalize forward to generic engine helper functions with the original target pointer and value. Do not implement them as just assignment plus Record: existing target validation and projected expected-value audit matter. Nil wrapper handling must preserve current inactive-context behavior.

Refactor engine publication to return a short-lived generic loaded value containing Config T and an inspection pointer. Inspection owns frozen trees, metadata, history, sources, diagnostics, and sensitivity; it does not retain T or callback bindings. Public Result[T] holds Config and that pointer, not another complete generic engine result. Result inspection must not reread caller-mutated Config. Existing caller-constructed zero results and nil result receivers retain behavior.

Translate engine load/bootstrap failures into root *LoadError/*SecretsError, respectively. Their backing state can be delegated privately, but no public Unwrap exposes internal failures. Retain bootstrap partial reports and ordinary writer errors/short-write behavior. Do not wrap arbitrary render/write errors into new structured load errors or add error-disclosure guarantees.

Stage 1 — Record the boundary and establish compatibility evidence

Status: not started.

1. Run baseline everyday checks. Inventory the root's exported types, constants, fields, functions, receiver method sets, generics, and interface implementations with Go tooling. Record the baseline revision and concise ownership mapping in the stage handoff; do not create a second normative API specification.
2. Add the next available ADR (currently 0006-public-facade.md) recording wrappers, private delegation, the dependency direction, and the alias-first alternative. Record the temporary migration mechanism here, not as durable architecture policy.
3. Update architecture policy's public API/dependency section for the agreed boundary, preserving proportionality and existing behavior contracts. Explain that this is accepted design, not completed implementation. Add ADR navigation.
4. Repair the three historical incoming references described at the top of this plan to identify the old plan by Git revision. Preserve the completed provenance evidence; do not label this facade plan as its completion record.
5. Retain/add a compact external-package facade compatibility test covering the realistic boundary risks: root named-type identity, exported construction, generic helper use, structured error assertions, and relevant interfaces. Use existing tests for behavior already protected. Inspect the complete exported surface separately; do not add golden dumps of private struct layouts.

Acceptance: baseline passes, public surface is understood, ADR/policy accurately distinguish decision from implementation, and documentation links pass. Run go test . -run 'LocalDocumentationLinks|DependencyImportBoundary|Example|Facade' and the everyday check. No engine migration in this stage.

Completion

Status: complete (2026-09-12).

Baseline behavior was verified at 28e7fc5f690b36c60e4d03586afcff00fb1dee90; this plan was introduced by eb7de8e40ab944f022a2b6984f4d4e0086f1484d. The complete root surface was inventoried with go doc -all .: lifecycle and generic entry points; root value/input types (Path, Source, Secret); callback and option records; metadata, provenance, diagnostics, history, and rendering records; typed errors and bootstrap reports; and the scalar enum families. The inventory also covered receiver methods, generic instantiations, and relevant interfaces (error, fmt.Formatter, fmt.Stringer, json.Marshaler, and io.WriterTo). Existing behavioral tests remain the source of detailed API semantics; facade_compatibility_test.go supplies compact external-package coverage for the migration boundary.

Added ADR 0006 and the architecture rule for root-owned public wrappers and inward-only implementation dependencies. They describe the accepted target, not an already-completed extraction. Repaired the development guide, source-origins record, and remediation record to identify the old provenance plan by its Git revision instead of linking to this plan. No engine or lifecycle migration was introduced.

Validation on Linux with Go 1.26.7 passed: the focused documentation, import-boundary, executable-example, and facade checks, followed by ./scripts/check.sh. There were no deviations. The next handoff copies the unchanged root implementation into the bounded internal relocation candidate; it must retain the documented public behavior while proving engine independence.

Stage 2 — Establish the internal relocation candidate

Status: not started.

1. Copy root production implementation except public package orientation into internal/engine, changing package declarations/import references mechanically. Preserve file divisions and existing document/YAML packages. Keep internal functions private unless a later facade connection actually needs an export.
2. Copy behavior/mechanism tests and needed fixtures for engine validation, excluding repository documentation, release-script, import-boundary, public examples, and consumer integration tests. Keep root unchanged and passing. Repair fixture paths explicitly; do not make tests depend on arbitrary working directories.
3. Avoid blind replacements inside user-facing strings, diagnostic codes, fixture YAML, or documentation comments. The package rename is not permission to rename messages or behavior. Record the candidate's source baseline.
4. Update the component overview with an explicitly temporary candidate entry. Do not claim root loads delegate yet.

Acceptance: go test ./internal/engine and the everyday check pass. Engine imports no root code, existing internal/parser boundaries remain intact, and root consumers still execute the unchanged original implementation. This stage is deliberately mechanical; do not mix ownership redesign into the relocation diff.

Completion

Status: complete (2026-09-12).

Copied the 39 root production files from the Stage 1 baseline f81def9 into internal/engine, excluding only root package orientation in doc.go, and changed package declarations mechanically. Copied the 29 applicable root behavior and fuzz tests. Repository documentation, release-script and import-boundary checks, root external-package/API tests, and maintained consumer integration tests remain at their existing owners; root has not been changed to delegate to the candidate. No root testdata fixtures exist to copy.

docs/internal/overview.md now labels the candidate as temporary and unused. go list -deps confirms that the engine has no dependency on the root module package; its existing dependencies remain the private document and YAML boundaries. Linux validation with Go 1.26.7 passed go test ./internal/engine and ./scripts/check.sh. There were no behavioral deviations or package-renamed diagnostics. The next handoff replaces the copied engine's concrete public secret assumption with an explicit per-operation exact-type binding.

Stage 3 — Decouple reflection from the public secret type

Status: not started.

1. Implement the exact-type SecretBinding/SecretState design above in engine. Thread it through schema and recursive value operations explicitly; do not use a global binding or make it an application option.
2. Replace concrete-secret assumptions in schema.go, decode.go, clone.go, projection.go, value_operation.go, secret_reference.go, and their callers. Find all additional references using the graph/text search; the list is not a reason to leave a direct assertion behind.
3. Move engine tests to their local secret type/binding and remove the candidate's production concrete Secret and public-style constructor. Preserve named-type rejection/acceptance semantics and dynamic-value checks from the baseline.
4. Exercise unresolved/default secrets, pointer and container shapes, borrowed callback values, lookup-once behavior, and validator replacements without disclosing canary credentials in diagnostic expectations.

Acceptance: engine secret/schema/decode/clone/projection/validation tests and race tests pass; binding instances cannot affect each other. Run go test ./internal/engine -run 'Secret|Schema|Decode|Clone|Projection|Validation|Callback', go test -race ./internal/engine, and the everyday check.

Completion

Status: complete (2026-09-12).

internal/engine now receives a per-operation SecretBinding at schema and load construction. Its exact reflect.Type, Read, and Make functions carry the facade-owned SecretState through decoding, projection, unresolved and validated cloning, dynamic-value rejection, secret lookup, resolution helpers, and validator mutation auditing. The binding is retained by each schema only; there is no package-global registration or engine-owned replacement secret.

Removed the candidate's production Secret and SecretEnv declarations. secret_binding_test.go provides a local test secret type, fresh binding for every test load, and exact-type/isolation coverage using a second local secret type. Existing secret tests continue to cover defaults, pointers and containers, resolution-selected names, lookup-once behavior, borrowed callback values, redaction, and validator replacement without credential disclosure. The root implementation remains unchanged for this migration handoff.

Linux validation with Go 1.26.7 passed the focused secret/schema/decode/clone/ projection/validation/callback suite, go test -race ./internal/engine, and ./scripts/check.sh. There were no deviations. The next handoff creates the temporary facade's root-owned value, source, diagnostic, and secret wrappers.

Stage 4 — Build candidate value, source, and diagnostic wrappers

Status: not started.

1. Create temporary internal/facade, package configkit, importing engine. Implement root-owned Path, Source, Secret, scalar enums/constants, and provenance records according to the ownership table. Preserve declaration comments for their eventual root placement. There must be no aliases or embedded engine types.
2. Implement private path/record conversions and the real facade secret binding. Keep public Secret's small access/format/JSON methods local; its private scalar state remains value-based, not a shared mutable reference.
3. Add LoadError and WriteDiagnostics delegation/conversion. Candidate engine may export narrow constructors/accessors as needed internally, not public bridge functions on the candidate facade. Preserve nil handling and typed errors.
4. Test candidate public operations from its external test package. Verify that numeric-key versus index versus parsed paths are not collapsed; source copies share the same reader claim; bytes are snapshotted at construction; formatting and JSON do not expose resolved secret values.

Acceptance: candidate wrapper tests and engine tests pass. Run go test ./internal/facade ./internal/engine and the everyday check. Lifecycle entry points are added in Stage 6; do not add nonfunctional stubs here.

Completion

Status: complete (2026-09-12).

Created the temporary internal/facade package (named configkit) with root-owned Path, Source, and Secret wrappers; root-owned scalar and provenance records; explicit private conversions; LoadError; and WriteDiagnostics. Public fields and signatures contain only facade types. Path carries its engine value privately without text round-tripping, Source retains the engine's source value so copied reader sources preserve their shared claim, and the facade builds the exact-type secret binding locally. No lifecycle entry point or placeholder was added.

Candidate tests cover semantic path distinctions, source byte snapshotting and copied-reader ownership, external-package type ownership and interfaces, and safe secret formatting/JSON. Linux validation with Go 1.26.7 passed go test ./internal/facade ./internal/engine and ./scripts/check.sh. There were no deviations. The next handoff separates engine publication from its generic result before lifecycle delegation begins.

Stage 5 — Separate engine publication and inspection

Status: complete (2026-09-12).

1. Split generic engine Result into the short-lived loaded configuration carrier and non-generic inspection state specified above. Move inspection methods to the inspection receiver without changing their snapshots or algorithms.
2. Adapt load_result.go, load_metadata.go, metadata.go, explain.go, explain_render.go, render.go, history.go, and snapshot.go as necessary. Keep the fixed entry-state view independent of final history, including the baseline fix that avoids copying unused resolution history.
3. Update engine tests for the internal receiver/return changes while preserving observable expectations. Do not broaden nil/result-construction contracts.

Acceptance: engine loading, historical provenance, detached metadata, explanation, and rendering behavior pass. Verify publication still performs its existing configuration detachment but does not store Config in inspection. Run go test ./internal/engine -run 'Load|Metadata|Provenance|Historical|Explain|Effective|Render|Independence', go test -race ./internal/engine, and the everyday check.

Completion

Replaced the engine's generic result with Loaded[T], a short-lived carrier of the detached configuration and a non-generic Inspection. Inspection now owns the frozen metadata, sources, diagnostics, document snapshots, history, resolution snapshots, and sensitivity state; its query, explanation, and rendering methods retain their previous behavior. The published configuration is detached as before and is not retained by Inspection.

Engine tests use a test-only adapter while exercising the moved inspection receiver, and a direct Loaded test verifies that mutating Config cannot change its frozen effective view. Linux validation with Go 1.26.7 passed go test ./internal/engine -run 'Load|Metadata|Provenance|Historical|Explain|Effective|Render|Independence', go test -race ./internal/engine, and ./scripts/check.sh. There were no deviations or remaining risks. The next handoff wires the candidate facade's lifecycle and callback wrappers to the split engine result.

Stage 6 — Wire candidate lifecycle and callback wrappers

Status: not started.

1. Add candidate Definition, Options, VersionRule, Version, FilesystemContext, Override, Resolution, and Validation with unchanged public declarations. Implement the private option and callback bridges specified above.
2. Add candidate Load using the engine and facade secret binding. Add Result[T] with Config and private inspection, initially with Metadata, Sources, and Diagnostics. Implement Metadata's complete public query methods.
3. Forward all resolution methods, including InputInfo and Source, and the generic Default/Normalize helpers. Engine owns context activity, helper verification, resolution snapshots, and validation mutation detection.
4. Add focused external candidate tests using actual public wrapper types inside application configuration. Exercise version callbacks, defaulting/normalization, retained/copy contexts, root typed error assertions, and public Secret recognition. Private engine secret tests alone are insufficient evidence for this bridge.

Acceptance: real candidate loads with callbacks succeed/fail identically to the baseline, no callback is called by option conversion, no extra lookup occurs, and detached entry provenance stays fixed. Run go test ./internal/facade ./internal/engine -run 'Load|Resolution|Validation|Version|Metadata|Secret|Callback|Facade', go test -race ./internal/facade ./internal/engine, and the everyday check.

Stage 7 — Complete candidate inspection and bootstrap surface

Status: not started.

1. Add the public explanation/value records and Result Explain, WriteEffective, and WriteExplanation delegation. Inspection uses the frozen engine state, not the mutable public Config field. Complete every remaining public method.
2. Add SecretsReport, SecretsError, bootstrap records/status constants, and LoadSecretsDir. Preserve opt-in writes, partial reports, detached entries, writer counts/errors, and non-unwrapping typed errors.
3. Exercise complete load-to-inspection workflows through the candidate. Prove that caller mutation of Config or returned records does not rewrite inspection history or affect a later query/load. Include malformed paths and nil receivers where the baseline already defines behavior.
4. Compare the complete candidate exported surface with the baseline inventory, normalizing only the temporary import path. Resolve all signature/method/field omissions before cutover. No placeholder methods or ignored API mismatches.

Acceptance: the candidate implements the entire supported API and has real integration evidence for inspection, credentials, bootstrap, and writer failures. Run go test ./internal/facade ./internal/engine, race tests for both packages, and the everyday check. Root remains unchanged until Stage 8.

Stage 8 — Cut over the root and remove migration duplication

Status: not started.

1. Replace root production plumbing with the tested candidate facade, retaining root package documentation and the root named types. Move files; do not leave forwarding aliases to the temporary facade. Remove internal/facade entirely.
2. Root-owned unit/integration tests remain at root; engine mechanism tests remain with engine. Remove duplicated baseline copies after confirming coverage owners. Split mixed tests by boundary: assertions requiring private engine state move internally, while public lifecycle/consumer assertions still exercise root. Do not move all tests inward simply to avoid adapting public wrappers.
3. Run existing executable API examples and maintained consumers unchanged in their imports/configuration types. Fix wrapper defects rather than migrating consumers to a different API. Test the actual root Secret binding after the move.
4. Keep repository check tests at root. Repair documentation/fixture references, fuzz target locations, and development task commands as files move. For links formerly pointing at mixed implementation files, point API references at root and implementation references at engine as appropriate.
5. Remove old root algorithms, temporary candidate tests superseded by root tests, and staging documentation. Preserve useful narrow private test seams, but no root accessors solely exposing engine internals to tests.

Acceptance: one engine serves the sole root facade; no candidate directory, parallel implementation, or root import from internal production code remains. Run the everyday check, go test ./examples/consumers, root examples, and go test -race ./.... All gates must pass in this handoff; do not leave a broken root suite for the final audit.

Stage 9 — Audit the public boundary and finalize documentation

Status: not started.

1. Compare the final complete exported API against the baseline. Audit public signatures, fields, aliases, embedded types, method sets, and error returns for internal-type exposure. Review root code for remaining substantive algorithms; explain small value operations deliberately retained there.
2. Extend the existing import-boundary check for inward production dependencies. Add a narrowly scoped public-boundary check that detects exported aliases to, embedded exposure of, and exported signatures/fields involving internal types. Use standard Go tooling and avoid private-field/layout snapshots. Account for generic declarations and public records referencing other public records. Keep behavior tests as the evidence for adapter correctness.
3. Verify source ownership, secret binding isolation, zero/nil handling, helper audit semantics, typed errors, final Config versus frozen inspection, and the baseline resolution-entry optimization. Add tests only for uncovered risks.
4. Finalize architecture-policy wording, ADR navigation, component overview, development test/fuzz commands, and public declaration comments. Add a concise internal facade mechanism document if needed for the secret/callback/record connections; do not spread implementation recipes across consumer docs.
5. Update consumer orientation only where useful. Keep YAML behavior in its existing contract. Convert the feature roadmap to an implemented summary with canonical links and evidence; record remaining release preparation separately.

Acceptance: documented target matches code and no unapproved API/behavior change is hidden by the refactor. Run the everyday check, full race suite, consumer examples, documentation/import/public-boundary checks, and tests/builds using the supported baseline and newer Go toolchains recorded in the release procedure. Record actual versions/platforms; do not treat Linux tests or cross-builds as native macOS release evidence. Do not publish or select a version in this stage.

Open Questions

None requiring a user decision were identified. Wrappers, public type ownership, one final engine, per-operation exact-type secret adaptation, and a single cutover are specified above. Internal helper names and file grouping remain ordinary implementation choices. A newly discovered conflict with an existing contract must be surfaced with evidence and alternatives rather than resolved by changing consumer behavior or reverting to public aliases.