701 lines
40 KiB
Markdown
701 lines
40 KiB
Markdown
# ConfigKit Adoption Implementation Plan
|
||
|
||
## Purpose And Execution Contract
|
||
|
||
Implement the target in [ConfigKit Adoption](configkit.md). That document owns
|
||
the feature scope and operator choices; this document owns the implementation
|
||
sequence. Each numbered stage is one implementation prompt for a
|
||
gpt-5.6-terra coding agent. Execute stages in order and finish the named checks
|
||
before proceeding. Do not combine the configuration cutover with unrelated
|
||
pipeline or security refactoring.
|
||
|
||
Read `AGENTS.md`, `docs/development.md`, all three documents under
|
||
`docs/policy/`, and the relevant focused internal documentation before acting.
|
||
Apply the implementation-discipline policy: use the dependency's actual APIs,
|
||
preserve the requested flexibility, and do not add application restrictions
|
||
under the guise of this migration. Tests protect behavior, not exact helper
|
||
structure, incidental formatting, or a duplicated implementation of ConfigKit.
|
||
|
||
The selected approach requires an upstream extension: ConfigKit v0.2.0 cannot
|
||
expose source origins during its resolution callback. See **Upstream
|
||
Prerequisite**. Stages 1–6 are independent of that missing capability. Stage 7
|
||
and all later stages have an explicit gate; the sprint is not ready for
|
||
unattended execution through that gate until the upstream release is available.
|
||
Do not mark a gated stage complete by weakening the feature contract.
|
||
|
||
Temporary coexistence of the old production loader and the new loader during
|
||
these stages is implementation scaffolding only. There is no supported dual
|
||
version loader in the final product and no release should be tagged midway
|
||
through the cutover. Each stage must compile and pass its scoped tests; the
|
||
final stage establishes repository-wide readiness.
|
||
|
||
## Repository Baseline
|
||
|
||
Planning examined Notarius at `946fa10` plus the outstanding policy and roadmap
|
||
changes. The sibling ConfigKit checkout is tagged `v0.2.0`.
|
||
|
||
Relevant current owners are:
|
||
|
||
| Area | Repository evidence |
|
||
| --- | --- |
|
||
| Version 4 YAML and conversion | `internal/core/config/file_config.go`, including `FileConfig`, custom `UnmarshalYAML` methods, `ParseFileConfigYAML`, `LoadFileConfig`, and `ApplyFileConfig` |
|
||
| Defaults, runtime ownership, domain validation | `internal/core/config/config.go`, `validation.go`, `effective_config.go`, and their contract tests |
|
||
| Field-level environment overrides | `internal/core/config/env.go` and `env_contract_test.go` |
|
||
| Configuration discovery and command overrides | `internal/cli/run.go`: `loadConfig`, `runConfigValidate`, `runPipelinesList`, and `runPipelineCommand` |
|
||
| Catalog checks and configured/CLI reference behavior | `internal/framework/pipeline/profile.go`, `references.go`, and CLI reference contract tests |
|
||
| Dynamic integer consumers | `internal/modules/generic/chunk/units/chunker.go` and `internal/modules/generic/output/json/encoder.go` currently require native `int` |
|
||
| Debug configuration and resolved-pipeline output | `internal/core/config/redaction.go`, `internal/core/debugbundle/summary.go`, and the CLI debug handoff |
|
||
| Maintained examples | `examples/dnd-minimal.config.yml`, `examples/dnd-complete.config.yml`, `examples/profiles/dnd-extraction.yml`, and CLI production contract tests |
|
||
| Direct YAML test dependency | `internal/cli/production_contract_test.go` also parses embedded prompt definitions using `yaml.v3` |
|
||
| Upstream contract | ConfigKit `docs/config.md`, `docs/consumers/README.md`, `load.go`, `resolution.go`, `metadata.go`, `records.go`, `path.go`, and consumer examples at v0.2.0 |
|
||
|
||
The sibling checkout is reference material, not a production dependency path.
|
||
Do not commit a local `replace`, modify upstream, or invent a future API as part
|
||
of these Notarius stages.
|
||
|
||
## Decisions Applied Throughout
|
||
|
||
### Configuration And Dependency Boundary
|
||
|
||
- Initially pin `gitea.maximumdirect.net/eric/ConfigKit v0.2.0`. Stage 7 may
|
||
require a reviewed release providing the agreed upstream extension. A later
|
||
pin must be explicit, never an unreviewed `main` revision.
|
||
- Use a string-backed document version and a `/version` rule accepting exactly
|
||
text `5`; both numeric `version: 5` and quoted `version: "5"` are accepted.
|
||
Maintained examples use the numeric form. Reject missing/null/unsupported
|
||
versions. Numeric v4 fails the application checker with migration guidance.
|
||
- Use plain YAML-tagged document structs in `internal/core/config`, without
|
||
custom YAML hooks. Keep runtime framework types independent of ConfigKit.
|
||
- Keep the existing runtime `Config`, `ResolveInput`, and `EffectiveConfig`
|
||
boundary where it is useful. A new `LoadedConfig` owns the private ConfigKit
|
||
result and exposes runtime configuration plus safe writer methods. No
|
||
caller may mutate a returned `Result.Config` and expect its frozen inspection
|
||
output to change.
|
||
- ConfigKit owns composition, strict decoding, metadata, and safe rendering.
|
||
Notarius owns typed defaults, field semantics, recorded normalization,
|
||
conversion, catalog resolution, and physical run preparation.
|
||
- Application paths use typed ConfigKit key/index constructors; never build
|
||
JSON Pointers by concatenating unescaped pipeline IDs or reference keys.
|
||
|
||
### Document Shape And Presence
|
||
|
||
Use a distinct finite `ValidatorDocument` containing only `module`,
|
||
`llm_profile`, `structured_output_repair_attempts`, `retries`, and `options`.
|
||
The producer `BindingDocument` contains those fields plus references,
|
||
validation policy, and a validator list. This avoids recursive Go schema types
|
||
and makes forbidden nested validator fields strict unknown-field errors.
|
||
|
||
Use pointer scalars only when nil means inheritance or omission, and ordinary
|
||
structs/maps/slices elsewhere as appropriate to the contract. Use ConfigKit
|
||
presence before normalization changes paths. Remove custom parser-presence
|
||
flags. In particular:
|
||
|
||
- omitted validators use the production chain; explicit `[]` selects an empty
|
||
chain; present `null` is invalid, matching the existing array contract;
|
||
- omitted repair attempts inherit; explicit zero disables repair; present
|
||
null is invalid for this integer setting;
|
||
- omitted LLM-profile fields inherit; explicitly blank or null selections are
|
||
invalid;
|
||
- omitted optional module bindings permit existing catalog defaults; a present
|
||
binding must be a valid object with a nonempty module;
|
||
- reference objects have exactly one selected source: nonempty `path` or a
|
||
complete `artifact` object with step and lane; explicit invalid branches must
|
||
not disappear during pointer conversion;
|
||
- preserve the single-step `artifacts` form and the ordered `steps` form;
|
||
reject simultaneous declarations and explicitly empty/null `steps`;
|
||
- retain optional null behavior where the existing contract permits clearing
|
||
an optional object or collection; do not impose a universal no-null rule;
|
||
- remove the lane-level reference alias. Keep the existing treatment of the
|
||
deprecated lane-level validator list during migration (empty allowed,
|
||
nonempty rejected) rather than adding an unrelated contract removal.
|
||
|
||
Static defaults stay as currently documented. Derive omitted extraction workers
|
||
from the final configured total LLM limit. ConfigKit profile null replacement
|
||
is subject to the destination field's existing domain validity; overlay support
|
||
does not make null valid for every setting.
|
||
|
||
### CLI And Path Ownership
|
||
|
||
| Input | Owner / treatment |
|
||
| --- | --- |
|
||
| `--config`, `NOTARIUS_CONFIG`, installed fallback | Existing CLI discovery; ConfigKit receives the selected file |
|
||
| `--config-profile` | ConfigKit profile selection on all five configuration-loading command forms |
|
||
| Explicit `--output-dir` | ConfigKit override at `/output/directory` |
|
||
| Explicit `--debug-dir` | ConfigKit override at `/debug/directory`; preserve `--debug` requirement |
|
||
| Explicit `--chunk_cache` | ConfigKit override at `/cache/chunk_plans/mode` |
|
||
| `--llm-profile` | Existing runtime selection overriding all selected LLM bindings; not a replacement for a single YAML field |
|
||
| Input, lanes, session ID, reasoning effort, references, resume/recompute, debug enablement, JSON output | Existing invocation actions outside ConfigKit |
|
||
| Seven retired operational environment variables | No longer read or applied; no deprecation-warning scan of process environment |
|
||
| PromptKit credential environment variables | Existing PromptKit responsibility |
|
||
|
||
Only explicitly supplied flags create override records, with safe flag names
|
||
as provenance. Preserve current blank-value checks and invocation error classes.
|
||
For `--config-profile`, pass the supplied name through ConfigKit's selection
|
||
rules; empty and exact `default` select no overlay. Do not invent stricter name
|
||
validation or whitespace normalization than the upstream profile contract.
|
||
|
||
Enable imports without a separate Notarius enablement flag. The canonical
|
||
selected file's directory is the configuration root. Relative file references
|
||
and PromptKit profile sources use the surviving declaration's directory;
|
||
absolute values remain absolute. Resolve these to absolute paths before
|
||
runtime conversion, without reading the referenced contents. Output/cache/debug
|
||
relative directories retain working-directory semantics. CLI reference paths
|
||
retain the existing working-directory semantics and generated-reference
|
||
protection. Configuration-root confinement applies only to ConfigKit's imported
|
||
configuration and overlay files.
|
||
|
||
## Stage 1 — Add ConfigKit And The Version 5 Document Types
|
||
|
||
**Goal:** Establish the new file contract without switching production commands.
|
||
|
||
**Work:**
|
||
|
||
1. Pin v0.2.0 in `go.mod`/`go.sum` and inspect its actual exported API.
|
||
2. Add `document.go` and `load.go` in `internal/core/config` (or equivalent
|
||
cohesive files) with the complete plain document types, finite validator
|
||
type, explicit reference union, and YAML-tagged validation-policy fields.
|
||
3. Provide a private reusable definition factory with fresh static defaults and
|
||
the exact version rule. Define a small application `LoadOptions` for profile
|
||
selection and explicit configuration overrides; avoid a generic loader
|
||
abstraction or exposing parser types.
|
||
4. Keep the new load entry point unconnected to production until its
|
||
normalization and validation stages are complete. Mark it internal
|
||
scaffolding; do not advertise partially implemented v5 support.
|
||
5. Add the next available ADR (expected `0018-adopt-configkit.md`) recording
|
||
the accepted ownership, v5 break, imports/profiles, and environment policy.
|
||
Record the source-origin dependency without claiming it is implemented.
|
||
|
||
**Acceptance:** v5 numeric/quoted versions decode; unsupported versions fail
|
||
before document decoding; representative canonical bindings and references
|
||
decode while shorthand/unknown fields fail. Separate loads do not alias
|
||
defaults. The current CLI still works through its existing loader.
|
||
|
||
**Validation:** `go test ./internal/core/config`; `go test ./...` to confirm the
|
||
new dependency and schema types do not break the existing build. Keep tests
|
||
small; generic YAML exhaustiveness remains upstream.
|
||
|
||
**Prompt size:** Small to medium; types, dependency, and version boundary only.
|
||
|
||
## Stage 2 — Implement Operational Defaults And Normalization
|
||
|
||
**Goal:** Make operational configuration behavior correct and inspectable.
|
||
|
||
**Work:**
|
||
|
||
1. Express existing concurrency, output, cache, debug, and PromptKit-local-backend
|
||
defaults and validations through the definition callbacks.
|
||
2. Use recorded `Default`/`Normalize` operations for derived extraction workers,
|
||
path whitespace, cache mode, and endpoint whitespace. Derived defaults must
|
||
follow profile/CLI overlays and preserve explicit zero for validation.
|
||
3. Use presence-aware checks for explicit empty/null PromptKit source selectors
|
||
and required fields in a present local-backend object. Preserve mutually
|
||
exclusive profile sources and the existing endpoint/range constraints.
|
||
4. Keep filesystem checks and PromptKit profile catalog lookup out of callbacks.
|
||
This stage trims profile-source paths but does not yet resolve their bases.
|
||
5. Extract small reusable value checks where needed so runtime validation and
|
||
document validation do not separately implement the same rule. Collect
|
||
independent errors through ConfigKit with safe messages and field paths.
|
||
|
||
**Acceptance:** defaults and explicit values retain their meanings; worker
|
||
derivation follows effective total concurrency; multiple independent field
|
||
errors identify their source locations; normalization appears in explanation
|
||
without generating warnings.
|
||
|
||
**Validation:** `go test ./internal/core/config`. Add load-boundary cases for
|
||
profiles overriding concurrency, CLI overrides, invalid output versus empty
|
||
cache paths, and local backend presence. No live profile resolution.
|
||
|
||
**Prompt size:** Medium; operational fields only.
|
||
|
||
## Stage 3 — Implement Pipeline Document Rules
|
||
|
||
**Goal:** Validate and normalize the complete pipeline file shape before
|
||
catalog resolution.
|
||
|
||
**Work:**
|
||
|
||
1. Implement one typed traversal of pipelines, steps, lanes, bindings,
|
||
validators, and reference sources. Prefer shared binding helpers and
|
||
explicit walks over reflective generic validation.
|
||
2. Perform all presence-sensitive checks against original input paths before
|
||
renaming map keys. Detect whitespace collisions deterministically before
|
||
writing normalized keys. Report both paths where useful.
|
||
3. Normalize identifiers, selected profiles, and generated selectors with
|
||
recorded changes. For map-key moves, record the parent operation and relevant
|
||
source dependencies; preserve a load-local original-path mapping for Stage
|
||
7. Do not guess origin from a normalized key after it has moved.
|
||
4. Enforce the field and shape contracts above, including global lane-ID
|
||
uniqueness across steps, valid retry values, validation-policy fields, and
|
||
allowed binding-local reference/validator placements.
|
||
5. Keep stage/module compatibility and actual option meaning in the existing
|
||
catalog resolver. Do not reject a module based on an embedded allowlist in
|
||
the document schema.
|
||
6. Factor shared semantic value checks with `Config.Validate` where necessary.
|
||
Keep file presence/shape checks in the document callback and direct runtime
|
||
construction checks at the runtime boundary; avoid parallel complete
|
||
validators with independently copied rules.
|
||
|
||
**Acceptance:** single-step and ordered pipelines behave as specified;
|
||
explicit empty validator chains and zero repair budgets survive; invalid
|
||
references, profile selectors, normalized collisions, and duplicate lane IDs
|
||
fail with useful paths. No mutation is hidden from ConfigKit's change recorder.
|
||
|
||
**Validation:** `go test ./internal/core/config ./internal/framework/pipeline`.
|
||
Migrate meaningful rule cases from the old parser tests to the new load
|
||
boundary as they gain an owner; retain old-loader coverage until cutover.
|
||
|
||
**Prompt size:** Medium to large, bounded to pipeline document semantics; do
|
||
not combine it with source-origin or CLI work.
|
||
|
||
## Stage 4 — Convert Documents Into Runtime Configuration
|
||
|
||
**Goal:** Produce the existing typed runtime composition from the new document.
|
||
|
||
**Work:**
|
||
|
||
1. Add a narrow explicit conversion after successful ConfigKit loading. Build
|
||
independently owned `Config` and `pipeline.PipelineProfile` values, preserving
|
||
map identities, step order, inheritance pointers, empty chains, and reference
|
||
source forms. Convert the finite validator document to runtime bindings.
|
||
2. Wrap the result in `LoadedConfig`, retaining the private ConfigKit snapshot.
|
||
Provide safe effective/explanation writers and the runtime configuration;
|
||
avoid a public unredacted serialization method.
|
||
3. Stop re-deriving workers on the already finalized new-load path. Retain
|
||
programmatic `Config` validation and ownership checks. Any legacy
|
||
recomputation shim needed by the old loader is temporary and removed at
|
||
Stage 12. Runtime callers needing defaults should use the same default values
|
||
as the document definition, not a second literal default table.
|
||
4. Keep `Config.Resolve`/`pipeline.ResolvePipeline` responsible for selection,
|
||
capabilities, concrete validation-policy inheritance, options, references,
|
||
and digest generation. Do not add configuration profile name or source
|
||
history to semantic pipeline identity.
|
||
|
||
**Acceptance:** equivalent canonical documents produce equivalent runtime
|
||
pipelines; changing a semantic setting changes identity, while merely importing
|
||
an equivalent definition does not. Mutating caller-owned maps/slices cannot
|
||
alter the snapshot or resolved pipeline. ConfigKit changes do not force its
|
||
types into module interfaces.
|
||
|
||
**Validation:** `go test ./internal/core/config ./internal/framework/pipeline`.
|
||
Use a real test catalog and representative reference/validator chains; resolve
|
||
absolute reference fixtures until Stage 7 supplies relative path resolution.
|
||
|
||
**Prompt size:** Medium; conversion and runtime ownership only.
|
||
|
||
## Stage 5 — Support ConfigKit Dynamic Integers In Modules
|
||
|
||
**Goal:** Ensure the module boundary accepts valid ConfigKit options.
|
||
|
||
**Work:**
|
||
|
||
1. Inspect every production numeric `map[string]any` option consumer. The known
|
||
sites are generic unit chunking and JSON evidence `window_units`.
|
||
2. Add one narrow shared option conversion helper in the existing pipeline
|
||
option-helper area. Accept `int64` from ConfigKit and native `int` from
|
||
programmatically constructed module bindings. Range-check `int64` before
|
||
conversion; keep domain positivity/nonnegativity at the module owner.
|
||
3. Preserve rejection of floats and strings. Do not recursively rewrite all
|
||
options, change their serialized representation, or add general numeric
|
||
coercion. Keep options in ConfigKit form through config conversion.
|
||
4. Update the old test that rejects `int64` to protect the newly accepted
|
||
boundary. Add representative real-loader-to-module tests, including nested
|
||
JSON evidence options.
|
||
|
||
**Acceptance:** integral options work from YAML and programmatic bindings;
|
||
fractional/string inputs and invalid ranges still fail before preparation.
|
||
No unchecked narrowing or YAML round-trip remains at these consumers.
|
||
|
||
**Validation:** `go test ./internal/modules/generic/... ./internal/framework/pipeline ./internal/core/config`.
|
||
|
||
**Prompt size:** Small; two known consumers and a narrow shared helper.
|
||
|
||
## Stage 6 — Prepare Debug Rendering For The ConfigKit Result
|
||
|
||
**Goal:** Prepare the publication boundary before changing command loading.
|
||
|
||
**Work:**
|
||
|
||
1. Add a small writer interface such as `WriteEffective(io.Writer) error` to
|
||
the debug summary boundary. `LoadedConfig` satisfies it without exposing a
|
||
concrete ConfigKit result to debug storage.
|
||
2. Add publication of `effective-config.yml` through the existing confined,
|
||
atomic summary file writer. Buffer redacted output before publishing if the
|
||
current file API takes bytes; propagate rendering and write errors.
|
||
3. Preserve the existing redacted resolved-pipeline writer and all reference
|
||
content exclusion. The old `effective-config.json` writer remains only until
|
||
the run command switches in Stage 10.
|
||
4. Reuse `sensitiveConfigKey` for the definition's dynamic secret predicate at
|
||
nested options paths, including validator options, containers, and moved
|
||
map keys. Keep scalar classification policy in one owner. Do not log raw
|
||
configuration or metadata snapshots that contain values.
|
||
|
||
**Acceptance:** redacted effective YAML publishes atomically; failure leaves no
|
||
partial artifact; nested sensitive values never appear in effective output or
|
||
explanation. Resolved runtime summaries still omit materialized references.
|
||
|
||
**Validation:** `go test ./internal/core/config ./internal/core/debugbundle`.
|
||
Test rendering failure and caller mutation at these real local boundaries.
|
||
|
||
**Prompt size:** Small to medium; rendering interface and publication only.
|
||
|
||
## Stage 7 — Resolve Declaration-Relative Paths
|
||
|
||
**Goal:** Normalize file paths using their effective source origins and retain
|
||
correct ConfigKit inspection history.
|
||
|
||
**Prerequisite gate:** A reviewed ConfigKit release must expose surviving input origin and canonical
|
||
source information during resolution. Confirm the actual API and pin that
|
||
release before implementing this stage. If only v0.2.0 is available, stop here
|
||
and report the unmet dependency. Do not implement an upstream fork, reload the
|
||
configuration twice, parse YAML independently, or silently change the promised
|
||
effective output.
|
||
|
||
**Work after the gate:**
|
||
|
||
1. Resolve path origins at original input paths before map-key normalization
|
||
and before recording changes can replace a source origin. Use the scalar
|
||
origin and source ID to get the canonical declaring file; do not pick the
|
||
first origin of a composite parent mapping.
|
||
2. Join nonempty relative external reference paths and PromptKit profile-source
|
||
paths to that file's directory. Preserve absolute values. Do not read or
|
||
require existence of referenced files in this callback.
|
||
3. Record normalized paths and map moves so the frozen effective view matches
|
||
runtime values and explanation can trace back to the original declaration.
|
||
Keep map collision checks ahead of mutation; process stable sorted keys.
|
||
4. Leave destination directories unchanged except existing whitespace
|
||
normalization. CLI reference paths remain runtime-owned. No configuration
|
||
root confinement or symlink policy is added to referenced application files.
|
||
5. Enable imports and selected profiles on the public new loader, using
|
||
`File` and the selected file's canonical directory as the root. Use
|
||
ConfigKit's composition and profile behavior directly.
|
||
|
||
**Acceptance:** root, fragment, and overlay values resolve from their own files;
|
||
inherited values retain their base origin; profile-replaced sequences use the
|
||
new declarations; normalized enclosing IDs do not lose attribution. Effective
|
||
and explanation output agree with the actual paths. Removed/overridden file
|
||
values are not opened or resolved. Root containment affects only ConfigKit
|
||
configuration sources.
|
||
|
||
**Validation:** `go test ./internal/core/config ./internal/framework/pipeline`.
|
||
Use temporary configuration trees with same-named reference files in different
|
||
directories, aliases/symlinks accepted by ConfigKit, profile replacements,
|
||
normalized keys, and absolute external paths outside the config root. Use real
|
||
reference materialization for one case to prove the intended bytes are chosen.
|
||
|
||
**Prompt size:** Medium, with a firm upstream gate; no CLI or example migration.
|
||
|
||
## Stage 8 — Prepare Shared CLI Loading And Diagnostics
|
||
|
||
**Goal:** Make one command-facing configuration entry point ready for cutover.
|
||
|
||
**Work:**
|
||
|
||
1. Add a CLI helper that discovers the file, passes profile selection and
|
||
explicit override records to the new loader, and returns `LoadedConfig`
|
||
plus the selected config path. Keep discovery order and injected test lookup.
|
||
2. Add a shared load-error reporter using `errors.As` for ConfigKit load errors
|
||
and `WriteDiagnostics`; supply safe source context using the dependency's
|
||
source records. Do not flatten multi-error output into repeated wrappers or
|
||
print raw parser/source contents. Preserve status 1 for valid invocations
|
||
whose configuration cannot load.
|
||
3. Add shared flag registration/parsing only where it reduces actual repeated
|
||
code. Use existing CLI parsing conventions, including flags surrounding
|
||
positional arguments. Preserve missing-value and unexpected-argument checks.
|
||
4. Route successful ConfigKit process diagnostics through one reporting owner.
|
||
Define no new application warnings for successful imports, defaulting,
|
||
normalization, or ignored retired environment names. Current application
|
||
rules should ordinarily return no nonfatal configuration diagnostics.
|
||
|
||
**Acceptance:** the helper returns a fully validated loaded value with safe
|
||
diagnostics and uses the requested source/profile. No production command has
|
||
switched accidentally; existing command tests remain green.
|
||
|
||
**Validation:** `go test ./internal/cli ./internal/core/config`. Add direct
|
||
helper tests for explicit flags versus parser defaults, discovery, safe
|
||
multi-error reporting, and profile selection. Use the real ConfigKit boundary.
|
||
|
||
**Prompt size:** Medium; helper and diagnostic wiring only.
|
||
|
||
## Stage 9 — Prepare Validation And Pipeline Listing Commands
|
||
|
||
**Goal:** Finish the read-only command implementations before the shared
|
||
production cutover.
|
||
|
||
**Work:**
|
||
|
||
1. Implement v5 handlers for `config validate` and `pipelines list` using the
|
||
new helper and `--config-profile`. Preserve `--pipeline`, `--only`, JSON
|
||
listing, sorting, and exit/stream behavior. Keep production dispatch on the
|
||
old handlers until Stage 10 so all configuration commands switch together.
|
||
Use temporary explicit handler names, not a runtime version switch.
|
||
2. Plain validation is configuration-only. Selected-pipeline validation still
|
||
performs catalog and explicit PromptKit profile checks at their existing
|
||
owners, without provider calls or credential requirements beyond existing
|
||
offline profile validation.
|
||
3. Test these handlers with dedicated v5 fixtures and production registries.
|
||
Keep existing dispatch and maintained-example tests intact until Stage 10.
|
||
Do not weaken tests to accept both versions or advertise the unconnected
|
||
handlers as implemented CLI features.
|
||
|
||
**Acceptance:** the new handlers see imported pipelines and overlay changes,
|
||
reject v4, and retain useful module/profile failure context. Existing public
|
||
command behavior stays coherent until all three handlers switch in Stage 10.
|
||
|
||
**Validation:** `go test ./internal/cli ./internal/core/config`; run the focused
|
||
production validation cases using v5 fixtures, including unknown modules and
|
||
incompatible generated references.
|
||
|
||
**Prompt size:** Medium; two closely related read-only commands.
|
||
|
||
## Stage 10 — Switch Run And The Maintained V5 Configurations
|
||
|
||
**Goal:** Complete production cutover and make all runs use one ConfigKit load.
|
||
|
||
**Work:**
|
||
|
||
1. Switch `runPipelineCommand` to the new loader. Translate only the three
|
||
config-valued flags in the decision table to ConfigKit overrides; remove
|
||
their direct post-load mutations. Add `--config-profile` consistently to
|
||
argument reordering and missing-value checks. Activate Stage 9's validation
|
||
and listing handlers in the same change and remove superseded handlers.
|
||
2. Preserve runtime LLM override, references, input, session identity,
|
||
resume/recompute, scheduler setup, and output receipt semantics. Config
|
||
failures must still occur before source parsing and physical run allocation.
|
||
3. Pass the retained loaded result to the effective YAML debug writer. Continue
|
||
to write the separately redacted resolved pipeline after reference
|
||
materialization. Add effective configuration profile selection to debug
|
||
invocation metadata (`config_profile`), with no overlay represented as empty.
|
||
Do not add profile-selection text to checkpoint or pipeline identity.
|
||
4. Convert both maintained D&D example configurations to v5 object syntax in
|
||
place, still as self-contained files in this stage. Change the complete
|
||
example's PromptKit source path for declaration-relative interpretation.
|
||
5. Migrate run/reference/state tests and the maintained-example loader to v5.
|
||
Split mechanical fixture updates from behavioral assertions within the
|
||
diff so regressions remain reviewable. Retired env-based tests now supply
|
||
YAML or explicit flags; keep one test proving those variables have no effect.
|
||
6. Update current CLI, configuration, operations, and consumer references for
|
||
v5, profile selection, environment retirement, and `effective-config.yml`.
|
||
|
||
**Acceptance:** all production commands use the new helper; imported/profile
|
||
configuration and explicit overrides agree with execution and debug output;
|
||
the maintained minimal and full fake-provider workflows still complete with
|
||
the same artifact families and valid reference handoffs.
|
||
|
||
**Validation:** `go test ./internal/cli ./internal/core/config ./internal/core/debugbundle ./internal/framework/pipeline`.
|
||
Use the existing complete D&D fake LLM test instead of a live backend. Verify
|
||
JSON run receipt and stderr warning behavior remain unchanged.
|
||
|
||
**Prompt size:** Largest cutover prompt, limited to wiring, fixture migration,
|
||
and its current documentation. Schema, paths, numeric adapters, diagnostic
|
||
helpers, and debug writer must already be complete in earlier stages.
|
||
|
||
## Stage 11 — Add Effective And Explanation Commands
|
||
|
||
**Goal:** Expose the accepted operator inspection workflow.
|
||
|
||
**Work:**
|
||
|
||
1. Add `config effective` and `config explain` with the syntax in the feature
|
||
roadmap, including `--config` and `--config-profile`.
|
||
2. Effective writes only `LoadedConfig.WriteEffective` output. Explain accepts
|
||
exactly one JSON Pointer positional argument, parses it with ConfigKit, and
|
||
calls the safe explanation writer. Accept the empty pointer for the root
|
||
when explicitly supplied; reject malformed escapes/argument shape with
|
||
status 2. A well-formed unknown path returns status 1 and no success output.
|
||
3. Use ConfigKit's contextual sequence-index rules. Signed numeric mapping
|
||
keys remain valid keys; `/steps/+0` cannot identify a sequence element.
|
||
Do not add a blanket numeric-token restriction that rejects legal map keys.
|
||
4. Preserve writer failure handling. Explanations are human-oriented text;
|
||
effective configuration is YAML. Neither output is promised to be a
|
||
reloadable config or a new durable JSON API. Do not print success banners.
|
||
5. Avoid LLM client construction, module preparation, reference materialization,
|
||
state allocation, or pipeline selection. Existing catalog registration at
|
||
CLI composition is acceptable; it must not trigger provider work.
|
||
6. Update `docs/cli.md`, the configuration inspection guidance, and help.
|
||
|
||
**Acceptance:** both commands work against imports/overlays, redaction applies
|
||
to history and effective values, removed keys remain explainable, malformed
|
||
and unknown paths fail appropriately, and stdout contains only requested data.
|
||
|
||
**Validation:** `go test ./internal/cli ./internal/core/config`. Use real
|
||
ConfigKit with temporary files and a provider factory that fails if called.
|
||
|
||
**Prompt size:** Medium; two commands sharing the completed load boundary.
|
||
|
||
## Stage 12 — Remove The Legacy Configuration Machinery
|
||
|
||
**Goal:** Finish the simplification promised by the adoption.
|
||
|
||
**Work:**
|
||
|
||
1. Remove the unused v4 parser, YAML hooks, `FileConfig` application APIs,
|
||
environment-overlay code, parser-only presence flags, and old loading helper.
|
||
2. Remove `extractWorkersConfigured`/`defaultedExtractWorkers` inference once
|
||
file defaults are finalized by ConfigKit. Programmatically constructed
|
||
configs treat a supplied worker entry as explicit and derive only an absent
|
||
entry; update internal construction tests accordingly. Keep runtime checks
|
||
through `Config.Resolve` for direct callers, without mutating shared maps.
|
||
3. Delete the superseded `effective-config.json` publication method and raw
|
||
configuration summary/redaction functions. Retain runtime ownership cloning
|
||
and resolved-pipeline redaction that still serve real callers.
|
||
4. Remove obsolete parser/env tests; retain behavioral coverage at the new
|
||
owner. Remove `yaml.v3` from the embedded-prompt test in
|
||
`production_contract_test.go`: exercise the production PromptKit loader or
|
||
existing asset-loading seam, rather than adopting ConfigKit to parse
|
||
PromptKit assets or adding another YAML dependency for the test.
|
||
5. Run `go mod tidy`; remove the direct `gopkg.in/yaml.v3` requirement and verify
|
||
Notarius Go files no longer import a YAML parser. Dependencies may retain
|
||
their private YAML implementation.
|
||
|
||
**Acceptance:** no alternate loader, environment application layer, redundant
|
||
configuration redactor, or parser-only clone remains. Legitimate pipeline and
|
||
artifact serialization code is unaffected. Programmatic config behavior has
|
||
an explicit tested ownership/default contract.
|
||
|
||
**Validation:** `go test ./...`; `go vet ./...`; search for `UnmarshalYAML`,
|
||
`ApplyFileConfig`, `ApplyEnvOverrides`, `gopkg.in/yaml`, and old effective-config
|
||
artifact references. Inspect matches rather than treating historical docs or
|
||
negative migration tests as failures.
|
||
|
||
**Prompt size:** Medium; deletion and ownership consolidation after all callers
|
||
have moved.
|
||
|
||
## Stage 13 — Demonstrate Imports And Development Profiles
|
||
|
||
**Goal:** Supply two maintained operator examples covering simple and composed
|
||
configuration, with a clear distinction between the two profile systems.
|
||
|
||
**Work:**
|
||
|
||
1. Keep `examples/dnd-minimal.config.yml` self-contained.
|
||
2. Keep `examples/dnd-complete.config.yml` as the complete root and move its
|
||
whole `dnd-session` pipeline into
|
||
`examples/pipelines/dnd-session.yml`, imported through `configkit.imports`.
|
||
Only the root declares version. Fix fragment-relative campaign/catalog
|
||
references to `../dnd-party.txt`, `../dnd-glossary.txt`, and
|
||
`../dnd-spell-catalog.json` where applicable.
|
||
3. Move the existing PromptKit example to
|
||
`examples/llm-profiles/dnd-extraction.yml` and point the root's
|
||
`promptkit.profile_file` to `./llm-profiles/dnd-extraction.yml`.
|
||
4. Add `examples/profiles/development.yml` as a ConfigKit overlay. Set
|
||
`concurrency.total_llm: 1` and `stage_workers.extract: 1`, plus a development
|
||
output directory. Include a short comment explaining how an operator can
|
||
change the pipeline's `llm_profile` independently; do not invent an
|
||
unavailable model profile just to demonstrate the field. The overlay has
|
||
no version or import directive.
|
||
5. Extend the existing maintained-example test to load base and development
|
||
configurations through production code, validate catalogs/profiles offline,
|
||
and exercise complete fake-provider reference materialization from a
|
||
working directory other than the example directory.
|
||
6. Update `docs/config.md`, `docs/operations.md`, `docs/consumers/subprocess.md`,
|
||
and `docs/consumers/dnd-pipeline.md` at their canonical scopes. Link the
|
||
examples; explain disjoint composition and whole-sequence replacement
|
||
instead of duplicating complete configs in prose.
|
||
|
||
**Acceptance:** all example paths resolve, the base and development selections
|
||
validate, the full pipeline still contains every implemented D&D lane, and
|
||
operators can distinguish `--config-profile` from `--llm-profile`.
|
||
|
||
**Validation:** `go test ./internal/cli`; production CLI validation for both
|
||
roots and the development overlay. No API key or live generation is needed;
|
||
use only the documented placeholder if offline PromptKit validation requires
|
||
one. Verify all moved example links.
|
||
|
||
**Prompt size:** Medium; examples and their operator-facing documentation.
|
||
|
||
## Stage 14 — Final Architecture And Contract Verification
|
||
|
||
**Goal:** Verify the completed architecture and reconcile canonical docs with
|
||
the implemented boundaries.
|
||
|
||
**Work:**
|
||
|
||
1. Review the complete diff against `configkit.md` and this plan. Check that no
|
||
stage was bypassed, no source-origin workaround weakened the selected
|
||
contract, and no unsupported step-composition semantics slipped in.
|
||
2. Finalize `docs/internal/configuration.md`, `docs/internal/cli.md`,
|
||
`docs/internal/state.md`, and `docs/internal/overview.md`. Add
|
||
`docs/integrations/pkg-configkit.md` for the pinned API boundary and relevant
|
||
upstream semantics; link upstream for general library behavior.
|
||
3. Update architecture's configuration ownership/precedence text and finalize
|
||
the adoption ADR. Document public behavior in its canonical owner; do not
|
||
retroactively edit historical release notes or prior accepted ADR decisions.
|
||
4. Ensure ConfigKit warning/diagnostic handling, safe rendering, source-relative
|
||
paths, runtime option validation, direct construction, and digest semantics
|
||
all have sufficient focused coverage. Do not duplicate upstream's test
|
||
suite or create exact-output change detectors.
|
||
5. Verify current docs no longer recommend v4 syntax, retired environment
|
||
overlays, the former PromptKit example path, or `effective-config.json`.
|
||
Migration explanations and historical records may legitimately name them.
|
||
6. Complete the checks below and report any remaining issue explicitly. Do not
|
||
tag a release, push, or retire the active roadmap during implementation
|
||
without a separate request.
|
||
|
||
**Acceptance:** every feature acceptance criterion holds, current docs and
|
||
examples agree with runtime behavior, all required checks pass, and the
|
||
remaining configuration code expresses Notarius policy rather than rebuilding
|
||
ConfigKit machinery.
|
||
|
||
**Validation:**
|
||
|
||
```sh
|
||
go test ./...
|
||
go vet ./...
|
||
go build ./cmd/notarius
|
||
go run ./cmd/notarius config validate --config examples/dnd-minimal.config.yml --pipeline dnd-session
|
||
go run ./cmd/notarius config validate --config examples/dnd-complete.config.yml --pipeline dnd-session
|
||
go run ./cmd/notarius config validate --config examples/dnd-complete.config.yml --config-profile development --pipeline dnd-session
|
||
go run ./cmd/notarius pipelines list --config examples/dnd-complete.config.yml --config-profile development --json
|
||
go run ./cmd/notarius config effective --config examples/dnd-complete.config.yml --config-profile development
|
||
go run ./cmd/notarius config explain --config examples/dnd-complete.config.yml --config-profile development /concurrency/total_llm
|
||
git diff --check
|
||
```
|
||
|
||
Use the project's normal link-check mechanism if present; otherwise check
|
||
changed relative links and heading anchors manually. Live model evaluation is
|
||
unnecessary for a configuration migration.
|
||
|
||
**Prompt size:** Medium; verification and focused documentation reconciliation,
|
||
with no planned production redesign.
|
||
|
||
## Upstream Prerequisite
|
||
|
||
### Source Origins During ConfigKit Resolution
|
||
|
||
**Evidence:** In v0.2.0, `Resolution` exports `IsExplicit`, `Record`, `Error`,
|
||
and `Warning`, with generic `Default` and `Normalize` helpers. It does not
|
||
expose source metadata. `Result.Metadata()` and `Result.Sources()` become
|
||
available only after `Load` returns. Mutating `Result.Config` afterward does
|
||
not change `WriteEffective` or `WriteExplanation`, which use frozen snapshots.
|
||
Therefore a single v0.2.0 load cannot both use declaration origins to normalize
|
||
paths and record that normalization in ConfigKit's inspection history.
|
||
|
||
**Selected approach:** Request a small upstream API exposing surviving
|
||
input origins and canonical source records during the existing resolution
|
||
callback. It should distinguish base/profile/CLI/default origins, identify the
|
||
declaring file for a scalar, and permit lookup at original paths before
|
||
application map-key normalization. Keep the existing recorded mutation API.
|
||
Exact method names and the containing release must be confirmed upstream;
|
||
none are assumed to exist by this plan. This preserves one load, ConfigKit
|
||
ownership of composition/provenance, and the agreed effective-output contract.
|
||
The tradeoff is an upstream release dependency at Stage 7. This plan does not
|
||
authorize changes to the sibling repository.
|
||
|
||
**Alternative considered:** Stay on v0.2.0 and resolve paths in the narrow Notarius
|
||
runtime converter using completed-result metadata. Preserve original document
|
||
paths for origin lookup. Present ConfigKit's effective configuration as a
|
||
logical configuration view, and expose resolved runtime paths separately in
|
||
debug/inspection output. This is implementable without an upstream change and
|
||
keeps composition centralized, but changes the agreed inspection contract,
|
||
requires Notarius to own the extra path report, and cannot make ConfigKit's
|
||
explanation record the post-load normalization. It requires explicit approval
|
||
and revision of Stage 7 and the feature roadmap before dependent work begins;
|
||
the user selected the upstream extension instead.
|
||
|
||
Do not resolve this by silently dropping imports/profiles, changing all file
|
||
paths to working-directory-relative, reparsing YAML, or performing a second
|
||
load that might observe different files. All other product decisions in this
|
||
plan are settled; the upstream release is the sole remaining gate. No product
|
||
questions remain open. Once the API is released, review its exact signatures,
|
||
update the pinned version and this stage's API mapping, and then resume Stage 7.
|