16 Commits
v0.8.1 ... main

Author SHA1 Message Date
aae01a59ce Add operator-facing PromptKit inspection to the roadmap 2026-10-04 11:13:33 +00:00
71cf084e8b Adopt native PromptKit configuration capabilities in the roadmap 2026-10-04 11:11:40 +00:00
e327a5eef7 Plan the PromptKit configuration upgrade 2026-10-04 00:34:43 +00:00
af62b3bacc Include transcript input in the development command example 2026-10-04 00:24:04 +00:00
e1097cc7a5 Make inherited path tests distinguish source directories 2026-10-04 00:22:13 +00:00
3f226c91d0 Record ConfigKit inheritance review findings 2026-10-04 00:18:32 +00:00
64509947df Complete ConfigKit inheritance implementation 2026-10-03 18:17:28 +00:00
29c7c165a5 Add development pipeline inheritance example 2026-10-03 18:15:12 +00:00
4ef69314e1 Exercise inherited pipeline CLI behavior 2026-10-03 18:08:07 +00:00
1d1434b118 Retain inherited configuration inspection provenance 2026-10-03 18:03:40 +00:00
f4779fede9 Preserve inherited reference path origins 2026-10-03 17:58:48 +00:00
01c08ec8e4 Preserve inherited presence and validation 2026-10-03 17:55:58 +00:00
d7ea0fd32d Add pipeline inheritance coverage and guidance 2026-10-03 17:49:28 +00:00
ec99d48619 Upgrade ConfigKit to v0.5.0 2026-10-03 17:44:58 +00:00
9e74df0b6e Plan the ConfigKit inheritance upgrade 2026-10-03 17:33:13 +00:00
6e5e277426 Plan ConfigKit inheritance support 2026-10-03 17:16:41 +00:00
15 changed files with 2193 additions and 30 deletions

View File

@@ -40,6 +40,21 @@ have no effect; `NOTARIUS_CONFIG` remains a source-discovery input.
[LLM profile](../examples/llm-profiles/dnd-extraction.yml) and imports its
pipeline from [a fragment](../examples/pipelines/dnd-session.yml).
The complete fragment also defines `dnd-session-development`, which inherits
the complete pipeline and profile while setting
`structured_output_repair_attempts: 0`. Select it with:
```sh
notarius run dnd-session-development \
--config /path/to/dnd-complete.config.yml \
--input /path/to/transcript.json
```
See the [CLI reference](cli.md#run) for the full invocation contract. This
variant disables PromptKit structural repairs without changing configured producer
retry budgets. When using another profile, operators can instead select
`dnd-session` and set its `llm_profile` to that profile.
Use these complete files as starting points rather than combining the
illustrative fragments in this reference.
@@ -80,10 +95,12 @@ umask, setgid inheritance, and default ACLs; see [Operations](operations.md#cach
Every nonempty relative configured destination or reference path resolves from
the canonical directory of the file that supplies its surviving declaration.
This includes imported fragments and selected configuration overlays. Built-in
relative destinations use the selected main configuration file's directory;
explicit `--output-dir` and `--debug-dir` values use the invocation working
directory. Absolute values remain unchanged.
This includes imported fragments, selected configuration overlays, and values
copied through inheritance: the copied value keeps its original declaring
file. A child-supplied replacement resolves from the child's declaring file.
Built-in relative destinations use the selected main configuration file's
directory; explicit `--output-dir` and `--debug-dir` values use the invocation
working directory. Absolute values remain unchanged.
The root file can use ConfigKit imports to compose disjoint definitions. Imports
and selected overlays remain under the configuration root; they do not merge or
@@ -91,6 +108,50 @@ append a pipeline's `steps` sequence. An overlay replaces sequences as defined
by ConfigKit, so changing an individual ordered binding requires supplying the
whole replacement sequence.
## Sibling Inheritance
ConfigKit v0.5.0 supports native `extends` composition in mappings. A directive
names exactly one immediate sibling in the same mapping; it is not a path or a
cross-scope lookup. Imports, the selected overlay, and explicit overrides are
composed before inheritance, which runs before typed decoding and normal
Notarius validation. Every definition, including a parent and definitions not
selected for a run, remains subject to ordinary validation.
Child mappings recursively overlay parent mappings. A child sequence, scalar,
or null replaces the inherited value. This makes pipeline inheritance useful
for reusing a complete pipeline while replacing whole ordered lists:
~~~yaml
pipelines:
production:
input: {module: seriatim}
llm_profile: dnd-extraction
# Other complete pipeline settings and ordered steps...
development:
extends: production
structured_output_repair_attempts: 0
~~~
Pipeline names are siblings within `pipelines`; nested mappings such as named
artifact lanes also have their own sibling namespace. Step entries are a
sequence, so a step cannot extend another step. Replacing `steps` or a validator
chain supplies the complete replacement sequence; entries are not appended or
merged by identifier.
Inherited values count as explicit input for Notarius's existing
presence-sensitive rules: zero, false, and empty-list overrides stay explicit
instead of restoring a default. Null also replaces the inherited value, but
each field's ordinary type and validation rules still apply; null is not
universally valid. Omitting `extends` preserves a relationship supplied by a
lower-priority source, while `extends: null` cancels that relationship. It
does not independently clear other inherited fields.
`extends` is now a reserved key in every input mapping, including dynamic
module options and named maps. Existing ordinary data or entry names using that
key must be renamed along with any consumer that reads them. Configuration
overlays, pipeline inheritance, and PromptKit profile `base_profile`
inheritance are separate mechanisms.
## PromptKit Profiles
The optional **promptkit** object selects one source of profile definitions and

View File

@@ -2,17 +2,18 @@
Notarius uses the public root package
[`gitea.maximumdirect.net/eric/ConfigKit`](https://gitea.maximumdirect.net/eric/ConfigKit)
at v0.4.2 for version 5 configuration acquisition, imports, overlays, explicit
overrides, diagnostics, and safe inspection rendering.
at v0.5.0 for version 5 configuration acquisition, imports, overlays, explicit
overrides, native sibling inheritance, diagnostics, and safe inspection
rendering. ConfigKit composes source precedence and inheritance before typed
decoding; Notarius does not perform a second inheritance merge.
Notarius owns the typed document, domain defaults and validation, runtime
conversion, catalog-dependent pipeline resolution, source-aware configured
paths, and command discovery. It uses only ConfigKit's public path, result,
diagnostic, and resolution-source APIs. Upstream documentation owns general
ConfigKit semantics; [Configuration](../config.md) owns Notarius's file and
operator contract.
operator contract. See the pinned upstream [v0.5.0 release notes](https://gitea.maximumdirect.net/eric/ConfigKit/src/commit/dc09959cca4e105f49ec316c6d4ba585bd2340bd/docs/releases/v0.5.0.md)
and [configuration contract](https://gitea.maximumdirect.net/eric/ConfigKit/src/commit/dc09959cca4e105f49ec316c6d4ba585bd2340bd/docs/config.md).
The v0.4.2 patch updates upstream release tooling and policy/licensing
documentation without changing configuration behavior or the public API. See
the canonical [ConfigKit attribution](../oss/attribution.md#configkit) for its
license and notices.
See the canonical [ConfigKit attribution](../oss/attribution.md#configkit) for
its license and notices.

View File

@@ -15,7 +15,7 @@ runs anything.
| Boundary | Inputs | Outputs | Does not own |
| --- | --- | --- | --- |
| Loading | Selected file path, profile, overrides, and invocation directory | **LoadedConfig** with ConfigKit inspection metadata and a populated **Config** | Choosing the file path or reporting a command result. |
| Loading | Selected file path, overlay, overrides, and invocation directory | **LoadedConfig** with ConfigKit inspection metadata and a populated **Config** | Choosing the file path or reporting a command result. |
| Validation | **Config** | Structural configuration errors with pipeline, lane, or binding context | Module availability, capabilities, or construction. |
| Resolution | Valid **Config**, selected pipeline and lanes, runtime reference changes, LLM override, and module catalog | **EffectiveConfig** with a **ResolvedPipeline** | Materializing reference bytes, preparing modules, execution, or filesystem state. |
| Inspection | **LoadedConfig** and an optional JSON Pointer | Redacted effective YAML or explanation text | Pipeline selection, module preparation, or provider work. |
@@ -28,10 +28,21 @@ at their respective [CLI](cli.md), [pipeline](pipeline.md), and
## Loading And Validation
ConfigKit acquires the selected root, imports, optional selected overlay, and
explicit CLI overrides in precedence order. Its document definition supplies
typed defaults, source-aware path normalization, and structural validation
before conversion to the independently owned runtime **Config**. Only
ConfigKit composes the selected root, imports, optional selected overlay, and
explicit CLI overrides in precedence order, then expands native sibling
inheritance before typed decoding. The document definition supplies typed
defaults and structural validation; Notarius's recorded resolution changes
normalize configured paths after decode. Path normalization follows the source
of the surviving value, including a value copied through inheritance, rather
than the inheritance directive's location. Inherited input remains explicit
for presence-sensitive behavior, including zero, false, and empty values;
existing field validation still governs whether the resulting values are valid.
ConfigKit's inspection history retains inheritance transfers alongside later
normalization, and effective configuration renders the normalized values used
by runtime conversion. The private typed document consumes the directive, so
no upstream inheritance type or unresolved directive reaches runtime pipeline
or module contracts. See [Configuration](../config.md#sibling-inheritance)
for the operator syntax and field rules. Only
`NOTARIUS_CONFIG` participates in discovery; retired field-level environment
variables do not participate in loading. Version 5 decoding rejects scalar
module bindings and scalar external references. ConfigKit retains redacted,

View File

@@ -18,7 +18,7 @@ License and notices: [CC BY-SA 4.0](licenses/CC-BY-SA-4.0.txt) |
## ConfigKit
Selected module: `gitea.maximumdirect.net/eric/ConfigKit` at `v0.4.2`.
Selected module: `gitea.maximumdirect.net/eric/ConfigKit` at `v0.5.0`.
Copyright (C) 2026 Eric Rakestraw
@@ -26,10 +26,10 @@ SPDX-License-Identifier: LGPL-3.0-or-later
License and notices: [LGPL 3.0](licenses/LGPL-3.0.txt) |
[GPL 3.0](licenses/GPL-3.0.txt) |
[COPYRIGHT](https://gitea.maximumdirect.net/eric/ConfigKit/src/commit/67c03690181c5af8dde6be325abac241dae6732f/COPYRIGHT) |
[LICENSE](https://gitea.maximumdirect.net/eric/ConfigKit/src/commit/67c03690181c5af8dde6be325abac241dae6732f/LICENSE).
[COPYRIGHT](https://gitea.maximumdirect.net/eric/ConfigKit/src/commit/dc09959cca4e105f49ec316c6d4ba585bd2340bd/COPYRIGHT) |
[LICENSE](https://gitea.maximumdirect.net/eric/ConfigKit/src/commit/dc09959cca4e105f49ec316c6d4ba585bd2340bd/LICENSE).
[Source](https://gitea.maximumdirect.net/eric/ConfigKit/src/commit/67c03690181c5af8dde6be325abac241dae6732f).
[Source](https://gitea.maximumdirect.net/eric/ConfigKit/src/commit/dc09959cca4e105f49ec316c6d4ba585bd2340bd).
## PromptKit

229
docs/roadmap/configkit.md Normal file
View File

@@ -0,0 +1,229 @@
# ConfigKit 0.5 And Configuration Inheritance
## Purpose
Adopt ConfigKit v0.5.0 and make its sibling inheritance useful and supported in
Notarius configuration, particularly for deriving development or testing
pipelines from a complete D&D pipeline without duplicating the pipeline body.
ConfigKit owns source composition and inheritance. Notarius continues to own
typed configuration, domain defaults and validation, pipeline resolution, and
execution. This work does not introduce a second configuration-merging engine.
The [implementation plan](implementation.md) owns the ordered work and
validation sequence. Release preparation and publication remain separate.
## Reviewed Baseline
Reviewed on 2026-10-03. Notarius pins ConfigKit v0.4.2 and uses its ordinary
`Load` API in `internal/core/config/load.go`. Configuration version 5 supports
imports, selected configuration overlays, explicit command overrides, recorded
normalization, and redacted effective/explain output. Runtime configuration is
detached from the ConfigKit inspection snapshot.
Named pipelines and artifact lanes are mappings; pipeline `steps` and validator
chains are sequences. The complete example imports
`examples/pipelines/dnd-session.yml`, while the existing development overlay
changes operational settings. Relative configured paths are resolved from the
file supplying their surviving declaration. The current path resolver uses
ConfigKit origin/source metadata and host filesystem paths.
Target ConfigKit v0.5.0 at full source commit
`dc09959cca4e105f49ec316c6d4ba585bd2340bd`. Consult the pinned
[release guide](https://gitea.maximumdirect.net/eric/ConfigKit/src/tag/v0.5.0/docs/releases/v0.5.0.md)
and [configuration contract](https://gitea.maximumdirect.net/eric/ConfigKit/src/tag/v0.5.0/docs/config.md).
The release adds inheritance, layered loading, filesystem-backed sources,
mapping sources, and passive resource contexts. Its Go requirement and YAML
dependency are unchanged. The upgrade has not yet been trialed against
Notarius's test suite at this review baseline.
## Target End State
### Native Sibling Inheritance
Operators can define a named pipeline with `extends: parent-pipeline` and
override only the settings that differ. The parent remains an ordinary named,
valid pipeline; no abstract-template flag, template registry, or special
validation exemption is introduced. Derived pipelines use the existing run,
listing, validation, effective-configuration, and explanation commands.
Use ConfigKit's native semantics without restricting inheritance to pipelines
or implementing a Notarius-specific subset:
- `extends` selects one exact key among the mapping's immediate siblings.
It is not a path, cross-scope lookup, or multiple-parent expression.
- Source precedence is assembled before inheritance, and inheritance runs
before typed decoding and application resolution. Inherited input is explicit
for presence-sensitive defaults, including zero, false, empty, and null.
- Child mappings recursively overlay parent mappings; sequences, scalars,
and null replace inherited values. Existing typed/domain validation still
determines whether the resulting value is valid.
- Omitting `extends` retains a lower-priority relationship; `extends: null`
cancels the relationship, not arbitrary inherited fields independently.
- Root and sequence-element mappings have no sibling namespace for their own
string relationship. Nested mappings may have one. In particular, individual
entries of the `steps` sequence cannot extend other step entries.
- ConfigKit rejects missing/non-mapping parents, invalid directive types, and
cycles, and resolves effective relationships even in unselected entries.
- Parent definitions receive normal typed decoding and Notarius validation.
Inheritance is not a way to hide invalid partial pipeline definitions.
- Inherited nested relationships resolve at their destination according to
ConfigKit's rules; do not pre-expand or reinterpret them in Notarius.
Names are matched by ConfigKit before Notarius's existing identifier trimming.
Document exact-key lookup rather than introducing pre-inheritance name rewriting.
The principal documented workflow is whole-pipeline reuse. Inheritance does not
append steps, merge sequences by `id`, or allow a child to override one list
element without supplying a replacement list. The same applies to validator
chains. Do not disguise this boundary through custom list-merging code.
### Paths, Provenance, And Runtime Boundaries
An inherited relative reference or resource path remains relative to its
original declaring file, not the child's file or the invocation directory.
Imported parents, overlays, and nested inheritance must retain that behavior.
Direct child declarations use their own source context. Built-in relative
destinations and CLI-supplied paths keep their existing bases.
Review `documentPathBase` and related original-path/normalization handling
against the new origin metadata. Make narrowly required corrections if needed;
do not assume that successful typed decoding proves correct path resolution.
Effective output must agree with the runtime paths, and explanations must retain
the upstream derivation evidence through recorded Notarius normalization.
Keep the ConfigKit boundary private: no upstream inheritance objects or
unresolved directives reach runtime pipeline/module types. ConfigKit expands
relationships once during loading. Notarius's existing binding-profile,
validation-policy, and module-default resolution still operate on the resulting
configuration; they are not replaced with generic sibling inheritance.
Configuration overlays, pipeline inheritance, and PromptKit `base_profile`
inheritance remain distinct mechanisms. The latter still belongs to PromptKit.
### Compatibility And Upgrade Policy
Keep Notarius configuration `version: 5`. The typed runtime shape is unchanged;
the new capability belongs to configuration composition. No new CLI flag or
inheritance opt-in setting is needed or available upstream.
The significant compatibility change is that `extends` becomes a reserved key
in every input mapping, including dynamic module-option maps and named maps.
It is consumed before typed decoding, so it can no longer be ordinary data or
an ordinary entry name there. Audit maintained configuration/fixtures for such
usage and document that affected operator configurations must rename the key
or identifier and any associated consumer field. Do not silently strip, alias,
or bypass the directive to preserve an incompatible interpretation.
Configurations without that collision should retain their existing meaning.
Preserve discovery, import rules, overlay selection, explicit CLI override
precedence, strict decoding, redaction, defaults, and validation. Existing
configuration-stage inheritance does not alter the fixed execution pipeline,
artifact contracts, warning policy, or checkpoint identity design.
The eventual Notarius release should be a new minor version, explaining both
the feature and the newly reserved input key. Release preparation/publication
is separate from this work; no exact release number is selected here.
## Required Work
### Dependency And Configuration Integration
Upgrade `go.mod` and `go.sum` to the exact target. Keep PromptKit and unrelated
dependencies unchanged unless dependency resolution demonstrably requires it.
Continue using ordinary `configkit.Load`, the existing typed defaults factory,
and source-aware resolution/validation callbacks.
Inspect the load, operational-document, path, conversion, inspection, and CLI
boundaries. Add only integration adjustments demonstrated necessary by the
upstream change. There should be no `Extends` field added to Notarius runtime
types and no homegrown inheritance algorithm.
Update ConfigKit's factual attribution version and COPYRIGHT/LICENSE/source
links to the target commit after inspecting its actual notices. Preserve the
selected project grant and reuse unchanged local license texts.
### Maintained Example
Extend the complete example's imported pipeline definitions with a small
`dnd-session-development` sibling deriving from `dnd-session`, demonstrating a
pipeline-level setting override such as a smaller structural-repair budget.
Use already supported values and existing profile sources so the example
validates without an invented model/profile dependency. Explain that
`llm_profile` is another useful override when the operator supplies that profile.
Retain the runnable parent, minimal example, and existing operational overlay.
Do not duplicate the complete steps list or add a third standalone configuration
merely to demonstrate inheritance. Update example-loading/listing expectations
for the additional pipeline, checking inherited behavior rather than snapshots
of incidental formatting or total line counts.
### Canonical Documentation
- `docs/config.md`: own syntax, composition order, exact sibling lookup,
replacement/null semantics, path bases, reserved-key migration, and limits
on step/validator-list composition. Link to the maintained example and pinned
upstream rules rather than duplicating the entire upstream manual.
- `docs/internal/configuration.md`: describe expansion before typed decoding,
presence and source-aware normalization, and the unchanged runtime boundary.
- `docs/integrations/pkg-configkit.md`: update the supported version, public
contract links, and integration scope.
- `docs/oss/attribution.md`: update the dependency's verified source/notice links.
- `docs/cli.md` and consumer guides: update only where navigation or examples
benefit from showing selection of the derived pipeline. Do not invent new
invocation syntax or duplicate the configuration reference.
The existing ConfigKit architecture decision remains applicable. No new ADR is
required solely for consuming the library's composition feature; do not rewrite
an accepted ADR or turn new private implementation details into invariants.
## Verification And Acceptance
Use focused Notarius boundary tests, not a duplicate of ConfigKit's full suite:
- Existing maintained configurations continue loading and resolving.
- A derived pipeline retains its own identifier and inherits the complete
parent pipeline while applying its explicit overrides.
- Imports and selected overlays participate in composition before inheritance;
explicit CLI overrides keep their documented precedence.
- Representative zero/false/null inputs exercise existing presence-sensitive
defaulting or validation correctly; null is not universally valid merely
because inheritance can represent it.
- Replacing a steps or validator sequence does not concatenate or merge it.
- References inherited from a parent in another file retain their declaring
directory, while child-supplied replacements use the child's directory.
- Effective and explanation output remain redacted and expose useful source
and inheritance information after application normalization.
- Representative invalid inheritance produces ConfigKit load diagnostics through
the normal Notarius CLI error path, without a parallel validation engine.
- Maintained examples and their imported fragments validate offline without
credentials or provider calls.
Configuration and CLI boundary coverage should protect document/path/operational,
effective/explanation, and maintained-example behavior. Framework-resolution
coverage is appropriate where a distinct handoff risk requires it. Acceptance
includes the offline default suite, vet, and build with `GOWORK=off`, dependency
tidiness, and documentation diff/link checks appropriate to the changes. The
implementation plan owns their sequencing; these checks are not additional
permanent release gates.
## Deferred Capabilities And Non-Goals
- Do not adopt `LoadLayered` or add a new underlay precedence tier now. Existing
imports, overlays, and typed defaults remain the loading model. Bundled
pipeline presets or site-wide default layers need a separate operator-facing
design if selected later.
- Do not move Go defaults into embedded YAML or mapping sources solely because
v0.5.0 provides those APIs.
- Do not generalize references to arbitrary `fs.FS` resources in this work.
`SourceContext` may be used if it materially helps the required path behavior,
but its filesystem access rules must not silently replace Notarius's existing
host-path contract or introduce new path restrictions.
- No abstract templates, multiple inheritance, list-element composition,
new template schema, new pipeline scheduler, or extra configuration knobs.
- No opt-in gate or Notarius-only prohibition on other valid native inheritance
locations; use upstream semantics and existing domain validation.
There are no significant open design decisions at this baseline. Implementation
should verify the new metadata behavior and correct concrete integration gaps
without broadening the agreed feature scope.

View File

@@ -0,0 +1,528 @@
# ConfigKit Inheritance Implementation Plan
## Purpose And Scope
Implement [ConfigKit 0.5 And Configuration Inheritance](configkit.md). That
feature roadmap owns the intended behavior and policy choices. This document
owns the ordered implementation work. Each numbered stage is one prompt for a
Luna coding agent; complete its acceptance criteria before starting the next.
The starting baseline pins ConfigKit v0.4.2 and PromptKit v0.11.2, uses ordinary
`configkit.Load`, and accepts Notarius configuration version 5. Upgrade only
ConfigKit to v0.5.0, source commit
`dc09959cca4e105f49ec316c6d4ba585bd2340bd`. The release tag is annotated: use
`git rev-parse 'v0.5.0^{commit}'` when verifying the source commit in a checkout.
The implementation adds supported native configuration inheritance and a
maintained example. It does not require changing pipeline execution, artifact
schemas, or release tooling. Release preparation and publication remain a
separate task; this feature belongs in a future minor release.
## Working Instructions For Every Stage
- Read `docs/development.md`, `docs/policy/agents.md`, and the feature roadmap.
Use the development guide to select additional applicable policies, including
architecture, testing, documentation, and OSS for the changes being made.
- Inspect the named code and tests before editing. Prefer the repository's
code knowledge graph for code discovery. Inspect the pinned upstream release
and public API when an integration detail needs confirmation; do not assume
a sibling checkout's current branch equals the released version.
- Keep ordinary `configkit.Load`, typed defaults, and the existing private
configuration boundary. ConfigKit handles `extends` before decoding; add no
`Extends` field, second merge pass, application template registry, or opt-in
flag. Accept valid native inheritance locations under existing domain rules.
- Make production changes only where a concrete integration failure requires
them. A stage may consist of focused tests and documentation when the
existing adapter already behaves correctly. Fix a regression rather than
weakening an established expectation to accommodate the upgrade.
- Test Notarius's boundary with the real ConfigKit dependency and small,
deterministic fixtures. Reuse existing helpers and coverage; do not replicate
ConfigKit's complete test matrix, add diagnostic snapshots, or assert
incidental output formatting. Keep the default suite offline and independent
of credentials and model calls.
- Use `GOWORK=off` for Go commands so a sibling checkout cannot substitute for
the pinned modules. Update canonical documentation as each behavior lands.
Review the diff, run the stage's checks, and commit the coherent result under
the agent policy. Record any unresolved failure in the handoff rather than
marking that stage complete. Do not push or publish a release as part of this
plan.
## Stage 1 — Upgrade The Dependency And Its Notices
> Completion: complete.
Goal: establish the released dependency baseline without redesigning the
configuration adapter.
Inspect:
- `go.mod`, `go.sum`, and `internal/core/config/load.go`.
- `docs/integrations/pkg-configkit.md` and the ConfigKit entry in
`docs/oss/attribution.md`.
- ConfigKit's pinned `docs/releases/v0.5.0.md`, `go.mod`, `COPYRIGHT`, and
`LICENSE`; consult the sibling checkout with `git show v0.5.0:<path>`.
Work:
1. Run `GOWORK=off go get gitea.maximumdirect.net/eric/ConfigKit@v0.5.0`, then
`GOWORK=off go mod tidy`. Review the selected dependency graph. Retain
PromptKit v0.11.2 and unrelated pins unless module resolution requires a
change; explain any such change in the handoff.
2. Search maintained configuration, fixtures, YAML tags, and documentation for
ordinary uses of the key or entry name `extends`. For example, use
`rg -n '\bextends\b' examples internal docs assets`. Classify any matches:
new inheritance instructions are valid; ordinary field/key uses collide
with the new reserved directive. Rename actual maintained collisions and
their consumers together, without an alias or bypass. Do not rename the
unrelated PromptKit `base_profile` setting.
3. Run focused existing tests. Address compile/API compatibility failures with
the smallest necessary adapter changes. Native inheritance needs no
registration or load option.
4. Update the integration document's pin and brief description to v0.5.0;
replace the v0.4.2 patch-only explanation with the new composition boundary.
Link to pinned upstream release/configuration documentation.
5. Update only ConfigKit's attribution version and exact source/notice links
to the target commit, preserving the verified copyright and
`LGPL-3.0-or-later` expression. Reuse the existing LGPL/GPL texts when
unchanged. Do not refresh unrelated component attribution or policies.
Acceptance criteria:
- The declared and selected ConfigKit module is v0.5.0, without a workspace or
local `replace` dependency.
- Existing configuration and CLI tests pass; maintained ordinary `extends`
collisions, if any, have been resolved explicitly.
- Integration and attribution documents agree with the actual dependency and
imported source commit. The adapter still uses ordinary `Load`.
Validation:
```sh
GOWORK=off go list -m gitea.maximumdirect.net/eric/ConfigKit
GOWORK=off go test ./internal/core/config ./internal/cli
git diff --check
```
Prompt size: one dependency upgrade, its notices, and demonstrated compatibility
adjustments. Leave new inheritance coverage to the following stages.
## Stage 2 — Prove Pipeline Composition And Runtime Conversion
> Completion: complete.
Goal: establish whole-pipeline reuse through the real configuration boundary.
Inspect `load.go`, `document.go`, `loaded_config.go`, `pipeline_document.go`,
`document_test.go`, `pipeline_document_test.go`, and
`effective_config_contract_test.go` under `internal/core/config`. Reuse the
existing test catalog for runtime resolution.
Work:
1. Add focused loading coverage in `internal/core/config/inheritance_test.go`.
Define a valid parent pipeline and a child with `extends: parent` plus a
pipeline-level override. Assert the inherited input, bindings, and ordered
steps survive conversion, while the child retains its own pipeline ID and
the parent keeps its own settings.
Keep the fixture small rather than copying the complete D&D example.
2. Resolve the resulting child with the existing catalog. Assert its selected
ID and overridden setting reach the resolved pipeline. Use the catalog's
registered module keys and avoid generated-reference requirements in this
small fixture; the full generated-reference handoff is covered in Stage 7.
Reuse existing resolution tests for unrelated defaults and digests.
3. Add one replacement case: the child supplies a complete different `steps`
list. Assert replacement and ordering, with no parent steps appended or
merged by ID. Both resulting pipelines must satisfy ordinary validation.
4. Add one native nested-mapping case using sibling artifact lanes in an
`artifacts` mapping: a child lane inherits a valid parent lane and overrides
a binding value. This proves that Notarius does not limit inheritance to
pipeline mappings. Other upstream-supported mapping shapes need no custom
permission mechanism.
5. In `docs/config.md`, add the canonical inheritance explanation: exact
immediate-sibling lookup, one parent, source composition before expansion,
recursive mapping combination, sequence/scalar/null replacement, normal
validation of parent and child, and all effective relationships being
resolved. State that step-sequence entries cannot extend each other.
Include a small illustrative pipeline snippet and the reserved `extends`
migration note. Distinguish configuration overlays and PromptKit profiles.
Acceptance criteria:
- Loading and resolving a child yields an ordinary independently identified
pipeline with the expected inherited composition and explicit override.
- Replacement lists retain only the child list, in its configured order.
- Valid nested artifact-lane inheritance works through native ConfigKit
behavior without extra runtime types or application merging.
- Configuration version remains 5 and the user reference describes the
implemented syntax and reserved-key compatibility impact.
Validation:
```sh
GOWORK=off go test ./internal/core/config
git diff --check
```
Prompt size: small composition fixtures, runtime handoff, and one canonical
syntax section. Presence-sensitive cases belong to Stage 3.
## Stage 3 — Preserve Presence And Existing Domain Validation
> Completion: complete.
Goal: ensure inherited input remains explicit through Notarius normalization,
defaulting, and validator-chain conversion.
Inspect `operational_document.go`, `pipeline_document.go`,
`document_normalizer.go`, `loaded_config.go`, and their existing presence tests.
Extend `inheritance_test.go` rather than adding a parallel fixture framework.
Work:
1. Cover inherited `structured_output_repair_attempts: 0`: it remains a
populated zero override after loading and resolution, rather than reverting
to a pipeline or prompt default. Also cover a child explicitly overriding a
nonzero parent budget with zero.
2. Cover an inherited supported `validators: []` on a chunk or extraction
binding. Its runtime override retains `Set=true` and an empty chain. Cover
a child replacing a nonempty parent chain with `[]`; do not concatenate it
or reinstall default validators because the child list is empty.
3. Cover an inherited false module-option value, such as
`output.options.evidence_context.enabled: false`, remaining false in runtime
configuration. Inspect the decoded document's input metadata to establish
explicit presence where needed, rather than relying on truthiness.
4. Cover inherited or child-supplied null for a field already rejected when
explicit, such as `structured_output_repair_attempts`. Require the existing
contextual load failure. Inheritance's support for null does not make that
field nullable. Reuse existing direct-input null coverage.
5. Use a small overlay fixture to demonstrate the sequencing: an overlay that
changes the parent before expansion changes the child's inherited value;
a child's own explicit replacement wins over the parent's value. Verify
`extends: null` cancellation using a child with its own valid input binding,
and verify omission of the directive in an overlay preserves the base
relationship. An explicit override record for the parent value must be
reflected in an inheriting child as well.
6. If any integration defect appears, correct Notarius's existing presence or
conversion handling, using ConfigKit's metadata. Add no application
inheritance evaluator. Refine `docs/config.md` to explain explicit
zero/false/empty/null and cancellation without promising universal validity.
Acceptance criteria:
- Inherited and direct replacements retain their existing explicit-value
semantics, including zero repair budgets and empty validator overrides.
- Existing null restrictions are neither relaxed nor expanded.
- Overlay/explicit-override composition precedes inheritance, and cancellation
behaves as described by the pinned upstream contract.
Validation:
```sh
GOWORK=off go test ./internal/core/config
git diff --check
```
Prompt size: presence and precedence at one package boundary. This stage does
not change defaults or add CLI flags for arbitrary configuration overrides.
## Stage 4 — Verify Declaration-Relative Paths Through Inheritance
> Completion: complete.
Goal: keep inherited resource paths anchored to the original declaration.
Inspect `document_paths.go`, `document_paths_test.go`, the original-path map in
`pipeline_document.go`, and sequence mutation recording in
`document_normalizer.go`. Consult ConfigKit's public `InputInfo`, `Origin`, and
`Source` contracts at v0.5.0.
Work:
1. Extend real-file tests using `t.TempDir()`: import parent and child pipeline
definitions from different subdirectories beneath one configuration root.
Give the parent pipeline a relative external reference and one reference
inside an inherited ordered step. Assert both inherited paths resolve from
the parent's file; a direct child reference replacement resolves from the
child's file. Reference files may be minimal public fixture data.
2. Add an overlay replacement case where the parent reference is declared by
the overlay and subsequently inherited. Its base must be the overlay's
directory. A nested reference-mapping inheritance case must preserve its
original contributor when copied into a derived pipeline.
3. Combine one inherited path case with whitespace-normalized pipeline or
reference keys. The existing `originalPath` mechanism must still find the
input's origin. ConfigKit exact sibling matching happens before Notarius
trims names; do not pre-normalize the inheritance lookup.
4. Include a legitimate `../` reference, retaining the current host-path
behavior. Reuse existing coverage for built-in destination bases, explicit
CLI destination bases, absolute paths, and empty cache roots unless the
upgrade reveals a distinct regression.
5. Fix `documentPathBase` or related origin handling only if these tests expose
a failure. Query the surviving value's contributor, not the `extends`
control location or an arbitrary sibling. Keep the existing host-path
resolution model; do not replace it with resource opening through
`ResourceContext.FS`, which could introduce containment restrictions.
6. Clarify inherited declaring-file behavior in `docs/config.md`. Update the
loading/path explanation in `docs/internal/configuration.md` to reflect
expansion before decoding and source-aware recorded normalization.
Acceptance criteria:
- Imported, nested, and overlaid inherited paths use the file supplying the
surviving value; direct child replacements use their own declaring file.
- Normalized identifiers and inherited ordered-step references retain correct
bases without unrecorded sequence mutation failures.
- Existing path access and destination policies are preserved. No new path
containment, existence, or resource-format restriction is introduced.
Validation:
```sh
GOWORK=off go test ./internal/core/config
git diff --check
```
Prompt size: one path-resolution seam and focused real-file regressions.
Inspection rendering is checked separately in Stage 5.
## Stage 5 — Preserve Inspection, Derivation, And Ownership
> Completion: complete.
Goal: prove inherited configuration remains correctly inspectable after
Notarius's recorded normalization and runtime conversion.
Inspect `loaded_config.go`, `loaded_config_test.go`, `document_normalizer.go`,
`redaction_test.go`, and the new inherited path fixtures. Review upstream
metadata/history APIs rather than inventing a new provenance format.
Work:
1. Through `LoadedConfig.WriteEffective`, check a derived pipeline's expanded
values and resolved reference paths agree with its runtime configuration.
Verify consumed `extends` directives are absent from the effective typed
configuration. Assert selected values/paths, not the full YAML document.
2. Through `WriteExplanation`, inspect one inherited scalar and one inherited
path that Notarius normalizes. Assert useful parent/source and inheritance
evidence remains available alongside the final normalization result.
Use structured upstream metadata within the configuration package for
precise provenance checks, and minimal stable fragments for rendered text.
Do not require a new Notarius renderer or exact diagnostic wording.
3. Add a synthetic credential-shaped option to a parent and inherit it into a
child. Verify effective and explanation views redact its value, including
inherited/displaced history where present. Use only fake secret strings and
the existing sensitivity predicate; add no new credential rules.
4. Extend the existing ownership test: mutating a child's runtime nested map
must not mutate the parent's runtime map or the frozen inspection snapshot.
Retain the `RuntimeConfig()` detached-copy behavior without a new cloning
mechanism unless the test demonstrates a conversion defect.
5. Update `docs/internal/configuration.md` to describe inherited explicit input,
retained derivation, normalization, and the detached runtime boundary. Repair
any affected stale loading terminology, such as calling a configuration
overlay a profile. Link to `docs/config.md` for syntax and field rules.
Acceptance criteria:
- Effective paths agree with runtime paths; explanations retain useful source
and transfer evidence after normalization.
- Existing redaction applies to inherited values and history.
- Parent, child, returned copies, and inspection snapshots have the established
ownership behavior. Upstream inspection types stay private to configuration.
Validation:
```sh
GOWORK=off go test ./internal/core/config
git diff --check
```
Prompt size: configuration inspection and ownership, with focused tests and its
internal documentation.
## Stage 6 — Exercise CLI Selection And Failure Reporting
> Completion: complete.
Goal: make derived pipelines first-class through the existing commands and
preserve the normal load-error path for invalid inheritance.
Inspect `internal/cli/configkit_commands_test.go`, `command_contract_test.go`,
and the configuration command/load helpers. Use the existing injected catalog
and fake collaborators; do not construct providers or make LLM calls.
Work:
1. Add a small valid parent/child fixture. Exercise `pipelines list` and
`config validate --pipeline child`; assert the child can be selected by its
own ID and listing remains sorted. Reuse existing command rendering tests
rather than duplicating every output mode.
2. Exercise `config effective` and `config explain` for that child to prove CLI
wiring reaches the safe upstream inspection views. Assert successful status
and relevant inherited evidence, not a whole-output snapshot.
3. Add representative invalid load cases for a missing parent, a cycle, and an
invalid directive type. Keep one invalid relationship in an unselected
pipeline to prove selecting a different valid pipeline cannot hide it.
Assert runtime-error status, no success output, and meaningful safe
configuration diagnostics. Reuse package-level diagnostics for more detailed
path checks; ConfigKit owns the full malformed-directive matrix.
4. Check exact-key behavior with a whitespace-bearing parent name: the matching
declaration works and existing identifier normalization follows expansion.
A mismatch should produce the upstream missing-parent failure. No automatic
trimming, aliases, or application-specific inheritance restrictions are
added.
5. Change production CLI code only if these tests expose a wiring defect.
`docs/cli.md` may gain a short link to configuration inheritance where useful,
but commands and flags remain the same.
Acceptance criteria:
- Derived pipelines use the ordinary selection, listing, validation, and
inspection commands.
- Invalid relationships reach the existing load-error reporting path, even
when their entries are not selected for execution.
- Read-only command tests require neither credentials nor provider work.
Validation:
```sh
GOWORK=off go test ./internal/cli ./internal/core/config
git diff --check
```
Prompt size: command-boundary behavior and representative error propagation.
## Stage 7 — Publish And Test The Development Pipeline Example
> Completion: complete.
Goal: give operators one complete, runnable demonstration without duplicating
the full pipeline.
Inspect `examples/dnd-complete.config.yml`,
`examples/pipelines/dnd-session.yml`, `examples/overlays/development.yml`,
`internal/cli/example_contract_test.go`, and `maintainedExampleFiles` in
`internal/cli/production_contract_test.go`.
Work:
1. Add this sibling to `examples/pipelines/dnd-session.yml`:
```yaml
dnd-session-development:
extends: dnd-session
structured_output_repair_attempts: 0
```
Keep it beneath `pipelines`, beside the complete parent. Explain with a short
comment that this development variant disables PromptKit structural repairs
while inheriting the steps and configured producer-stage retry budgets.
It uses the parent's existing `dnd-extraction` profile.
2. Update the complete example's expected pipeline IDs to include the child in
sorted order. Keep the minimal example and existing operational overlay.
Do not add a third standalone configuration or duplicate the parent's steps.
3. Extend maintained-example coverage to load, resolve, and materialize external
references for both complete-example pipelines. Compare the child to the
parent for step/lane composition and generated-reference bindings, and assert
the selected zero structural-repair setting reaches eligible resolved
bindings. Do not assert equal pipeline digests: their IDs/settings differ.
4. Update `TestMaintainedExamplesValidateEffectiveProfilesOffline` to validate
every expected pipeline in each maintained example, including the new child,
with credentials absent. Retain the existing two-root-example inventory.
5. Update `docs/config.md`'s maintained-example guidance to show selection of the
variant and explain that operators may instead override `llm_profile` when
supplying another profile. Link to the inherited fragment. Add only a short
configuration link in `docs/consumers/dnd-pipeline.md` if useful; keep its
complete parent workflow and artifact instructions intact.
Acceptance criteria:
- The existing complete root imports both valid pipelines, and the derived
pipeline contains only the relationship and its intentional override.
- Both variants resolve and validate offline using the maintained profile,
external references, generated bindings, and full module catalog.
- Maintained examples and navigation accurately explain the variant without
confusing it with the operational overlay or PromptKit profile inheritance.
Validation:
```sh
GOWORK=off go test ./internal/cli -run 'TestMaintained'
GOWORK=off go test ./internal/core/config ./internal/cli
git diff --check
```
Prompt size: one example fragment, its existing test inventory, and focused
operator guidance. No full live extraction run is needed.
## Stage 8 — Consolidate Documentation And Verify The Complete Upgrade
> Completion: complete.
Goal: finish with an internally consistent, validated upgrade ready for review.
Inspect all changed files, `docs/config.md`,
`docs/internal/configuration.md`, `docs/integrations/pkg-configkit.md`,
`docs/oss/attribution.md`, and affected CLI/consumer navigation. Compare against
the feature roadmap's acceptance criteria.
Work:
1. Confirm canonical ownership: configuration defines syntax/precedence/path
rules; internals explain the adapter; integration documentation identifies
the supported upstream contract; attribution records the imported version
and notices. Remove duplicate definitions introduced during earlier stages.
2. Check the documentation covers the reserved key in every input mapping,
exact sibling naming, normal validation of all definitions, replacement of
step/validator sequences, source-relative inherited paths, explicit values,
and cancellation. Do not describe deferred `LoadLayered`, filesystem source,
mapping source, or underlay workflows as available Notarius features.
3. Search for stale ConfigKit v0.4.2/current-pin claims in active documentation
and dependency files, ordinary reserved-key collisions, and confusing profile
terminology. Leave historical releases and archived roadmaps intact. Verify
relative document links locally and pinned external links where accessible;
no new documentation-checking tool or recurring network gate is required.
4. Run the full offline default suite, vet, and native build. Build to a
`mktemp -d` destination so no binary is left in the repository. Run
`GOWORK=off go mod tidy -diff`; an unexpected diff must be resolved and
reviewed rather than accepted silently.
5. Review the final implementation for unintended restrictions or guarantees:
no new schema version, inheritance whitelist, abstract-template exemption,
custom list merge, host-path restriction, configuration source tier,
additional CLI knobs, or upstream types leaking into runtime contracts.
6. Report the implemented behavior, successful checks, and any concrete gaps.
Keep roadmap retirement and release publication for a later request.
Acceptance criteria:
- Every required outcome in `configkit.md` is implemented and covered at the
appropriate boundary, with no unexplained scope expansion.
- Maintained examples and canonical documents agree with the selected module
and observed loading behavior. Dependency metadata is tidy.
- Full offline tests, vet, build, and relevant diff/link checks pass. Any
unavailable optional external-link verification is reported accurately.
Validation:
```sh
GOWORK=off go test ./...
GOWORK=off go vet ./...
build_dir=$(mktemp -d)
GOWORK=off go build -o "$build_dir/notarius" ./cmd/notarius
GOWORK=off go mod tidy -diff
git diff --check
git status --short
```
These are development checks for this upgrade, not additional permanent release
gates. Do not repeat successful checks for unchanged content solely to satisfy
another handoff step; rerun the affected checks after any correction.
Prompt size: final review, small documentation consolidation, and aggregate
verification. A newly discovered substantial defect must be reported with its
evidence and required follow-up rather than folded into an unrelated refactor.
## Open Questions
None at this baseline. The dependency target, native inheritance semantics,
path policy, example override, documentation owners, and deferred features are
specified. Whether an adapter needs a small correction is determined by the
integration tests above, rather than a new product-policy decision.

409
docs/roadmap/promptkit.md Normal file
View File

@@ -0,0 +1,409 @@
# PromptKit 0.12 Configuration Migration
## Purpose
Adopt PromptKit v0.12.0 after the completed ConfigKit upgrade, replacing the
removed configuration APIs and asset formats with PromptKit's ConfigKit-backed
configuration boundary. Make the new composition, inheritance, presence, and
captured-configuration capabilities useful without introducing a second loader
or changing Notarius's domain extraction and pipeline acceptance policies.
This is a feature roadmap, not an execution sequence. The operator-source and
native-overlay decisions were accepted on 2026-10-04 and are incorporated below.
A separate implementation plan will define the execution sequence. Release
preparation and publication are separate work.
## Reviewed Baseline
Reviewed on 2026-10-04 against Notarius commit
`af62b3b`, with ConfigKit v0.5.0 and PromptKit v0.11.2 selected in `go.mod`.
The remote PromptKit v0.12.0 tag resolves to full source commit
`4b8169da286917fde7875425bb32b799b12c6af3`, matching the sibling checkout.
Primary upstream references are the
[release guide](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.12.0/docs/releases/v0.12.0.md),
[consumer migration guide](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.12.0/docs/consumers/pkg-promptkit.md#migrate-from-the-previous-configuration-api),
and [format reference](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.12.0/docs/formats.md).
These were reviewed in the local checkout; the remote tag was verified directly.
The current integration has these migration-sensitive boundaries:
- `internal/framework/llm/asset_registry.go` flattens module-owned prompt,
schema, and fallback-profile filesystems and constructs `WithPromptFS`,
`WithSchemaFS`, and `WithFallbackProfileFS` options.
- `promptkit_profiles.go` shares profile-source construction between the
execution client and credential-free explicit-profile inspection. It uses
`Config.ProfileDir`, `WithProfileFile`, and the backend registration option.
The inspection engine supplies `PromptDir: "."` without preparing a prompt.
- `promptkit_client.go` constructs the engine and uses prepared execution,
semantic correction messages, structural repair overrides, observations,
direct sessions, reasoning overrides, and provider-neutral error adaptation.
- `promptkit_profile_fingerprint.go` combines a pinned catalog marker with
reread operator profile files and fallback asset bytes. The optional local
endpoint contributes a separate identity; local concurrency is excluded.
- Embedded prompt definitions throughout `assets/dnd/` and
`assets/generic/normalize/deduplication/` use top-level `id` and `version`.
`assets/dnd/profiles/dnd-extraction.yaml` and the maintained deployment file
`examples/llm-profiles/dnd-extraction.yml` use top-level `id`.
- The public profile contract says a matching operator profile completely
replaces a fallback or catalog profile. It documents `base_profile`
inheritance. These claims do not match v0.12.0's native semantics.
The current focused LLM, CLI, and module-integration tests pass with
`GOWORK=off`. This establishes the baseline only: no trial dependency upgrade,
live inference, or production model-quality evaluation has been performed.
## Upstream Changes Relevant To Notarius
PromptKit v0.12.0 is an incompatible configuration/API release:
- `Config.Sources` accepts ordered ConfigKit underlays and an optional main
input. Legacy prompt/profile directory fields and configuration-source
options are removed. Schema and injected-runtime options remain available.
- Documents use keyed `backends`, `profiles`, and `prompts` mappings, with a
second version mapping beneath each prompt ID. Embedded `id`, `version`, and
`base_profile` fields are invalid; profile inheritance uses `extends`.
- Files within one source group compose strictly, not by filename precedence.
Later groups overlay earlier groups recursively. Explicit null, zero,
false, and empty values can differ from omission; `{}` does not clear a map.
- Engine construction validates every final effective definition, including
unused definitions, and captures message content and static templates.
Ordinary execution and inspection do not reload them. Schemas, request
artifacts, credentials, and provider operations retain their runtime timing.
- `InspectConfiguration` exposes captured presence, sources, diagnostics,
explanations, and redacted effective output. These reports are not
automatically safe to publish: ordinary prompt/provider content remains data.
- Maintained catalogs move to `/v3`, both at `v3.20260930.0`.
PromptKit and Notarius already share the required ConfigKit v0.5.0 and Go
1.25.5 baseline; no additional Go-version upgrade is required by this release.
## Target End State
### One Native Configuration Boundary
PromptKit owns the effective backend/profile/prompt configuration and all
upstream composition and inheritance. Notarius selects its sources and adapts
the resulting engine to the existing provider-neutral completion contract.
ConfigKit and PromptKit types remain inside the LLM adapter's explicit
integration boundary, not in domain modules or pipeline contracts.
Retain Notarius configuration `version: 5`, `promptkit.profile_file`,
`promptkit.profile_dir`, their mutual exclusion and declaring-file-relative
path handling, `transient_retries`, and the optional conventional local backend.
Do not add replacement Notarius configuration fields merely because the
upstream constructor has changed. Document the changed external file format.
Use one shared source-construction path for execution and profile preflight.
The source precedence is:
1. PromptKit's maintained catalog layer, supplied by PromptKit itself.
2. Registered application prompt and fallback-profile assets.
3. The explicitly selected operator file or directory.
4. A programmatic `local` backend contribution when Notarius's
`promptkit.local_backend` is configured, so those explicit Notarius values
win over conflicting file declarations of those fields.
The existing selected file/directory accepts native combined `profiles`,
`backends`, and `prompts` documents. Enable explicit ConfigKit imports on these
operator inputs. Describe the familiar `profile_file` and `profile_dir` fields
as PromptKit configuration sources whose principal workflow is profile
selection; do not add a section whitelist or a custom merge language.
Domain modules remain responsible for interpreting and validating model output,
even when an operator supplies a prompt overlay.
Inputs within each group use native strict composition. Directory discovery is
recursive over YAML files and has no hidden per-file priority. No unselected
directory is scanned. Construct filesystem-backed inputs with usable resource
contexts so referenced message files retain their declaring-source base.
Do not combine host profile files with embedded content under a fabricated
common filesystem root.
Document import layouts that do not accidentally select the same definitions
through both directory discovery and imports: a file-root import layout is
often the simplest choice. Use native import/resource rules rather than
inventing deduplication, filename override order, or broader host access.
Use `FilesystemInputs` for the existing flattened embedded views,
`DirectoryInputs` or an individual ConfigKit file input for the operator
source, and `BackendSource` with the presence-aware local-backend helper for
programmatic backend configuration. Keep schemas on the existing schema
filesystem boundary. There is no need to deserialize catalog assets, register
catalog backends individually, or recreate PromptKit inheritance.
Profile preflight does not need dummy prompt directories or synthetic prompts.
Both construction paths include the same catalog, application prompt/fallback,
operator, and local-backend layers. This matters for partial operator prompt
overlays: inspecting a layer without its application base would validate a
different effective definition. Call `InspectProfile` on the constructed engine
without preparing a synthetic request. Construction still captures and
validates the effective prompt definitions; preflight does not resolve
credential values, prepare request artifacts, or contact providers.
### Keyed Assets Without Prompt Behavior Changes
Every maintained prompt declaration becomes a keyed document of this shape:
```yaml
prompts:
dnd.spells:
v1:
# Existing definition body, without embedded id/version fields.
```
Profiles become keyed entries beneath `profiles`. Preserve literal prompt IDs,
the `v1` version, logical profile IDs, selected response schemas, content paths,
message order, roles, shared fragment bytes, cache-control metadata, and
configured structural repair budgets. Do not conflate this migration with
prompt rewriting, schema changes, or durable artifact version changes.
The application-owned `dnd-extraction` fallback continues to explicitly select
`openrouter`, `openai/gpt-5.6-luna`, timeout 240, and service tier `flex`, with
reasoning effort unset. Do not replace that choice with an upstream catalog
alias or adopt unrelated model defaults. Migrate the maintained operator
example and relevant inline Go/YAML test fixtures as well as production assets.
Module ownership, ordered asset manifests, scoped mounting, and selected-asset
hashing remain intact. Check virtual filenames and source-relative
`content_file` resolution after wrapping definitions; do not move business
logic into the root `assets` package.
### Presence, Inheritance, And Operator Migration
The adopted policy is native recursive overlay, not a compatibility shim
for complete-definition replacement. Operators can override only the fields
they want. Omitted fields retain lower-layer values, and `extends` uses native
literal sibling-key lookup. Changing a backend or model does not erase
explicit inherited settings; backend-derived defaults are applied to the
final backend after inheritance.
Document intentional clearing with `null`, including reasoning/service tier,
sampling controls, credential lookup, and provider maps where supported.
Distinguish null from zero/false/blank and from an empty mapping. For example,
switching an operator's `dnd-extraction` overlay to a local model may also need
`service_tier: null` to remove the fallback's explicit `flex` setting. Do not
automatically scrub settings based on model-name heuristics.
Translate `base_profile` to `extends` in maintained examples and explain that
operators must migrate their external files. Do not retain legacy aliases or
an alternate parser. Preserve exact PromptKit keys; do not introduce a new
identifier normalization pass. Update tests that assumed whole replacement
or selection-time validation deliberately, rather than treating those
intentional upstream changes as regressions.
Native eager validation applies to final effective values, not independently
to each superseded contribution. Do not add a source-history audit or raw-layer
validation that prevents legitimate overrides from repairing lower layers.
### Native Operator Capabilities
Support the capabilities of the unified format directly, without new Notarius
configuration knobs or model-specific adaptation:
- Custom or multiple backend definitions, including endpoint, credential
environment name, provider parameters, concurrency, and queue capacity.
Operators may also overlay maintained backend settings, for example changing
OpenRouter's admission limit. Backend IDs are configuration keys, not a
closed registration list. Notarius's global scheduler still applies.
- Profile inheritance, partial overrides, and recursive provider-parameter
composition. Explicit null suppression and zero/false values follow native
field rules. Native sibling inheritance is not limited by a Notarius
profiles-only whitelist; prompt/backend mappings follow upstream rules too.
- File-configured `api_key_required`, with PromptKit's exact credential
precedence and runtime checks. In particular, true suppresses a backend's
implicit environment source; an explicitly configured profile environment
source remains usable. This is an operator choice, not a new mandatory
credential policy or a reason to require credentials during preflight.
- Deployment prompt overlays and declaring-source-relative message files,
including imported content. Prompt version selection remains caller-owned;
this upgrade does not infer a new module version from file order or add
automatic latest-version selection.
The existing `local_backend` object remains a convenient highest-priority
contribution for that conventional backend. Other backend settings remain
available through the unified operator source; do not copy every upstream field
into a parallel Notarius backend schema.
Keep the two maintained top-level configuration examples. Use the complete
example's existing PromptKit deployment file and small canonical documentation
snippets to demonstrate native inheritance/partial overrides, custom backend
configuration, imports, null clearing, and prompt editing without adding a
large unrelated example suite. Not every capability needs its own runnable
pipeline.
### Captured Execution And Checkpoint Identity
One engine represents one captured configuration for its lifetime. Editing
operator definitions or message files requires constructing a new engine;
there is no watcher or automatic hot reload in this scope. Preserve the normal
CLI lifecycle without promising that independently constructed preflight and
execution engines are one transactional filesystem snapshot.
Compute and retain checkpoint identity from the configuration actually
captured by the execution engine. Do not identify an old captured engine by
files reread after it was constructed. Identity must account for effective
profile/backend semantics and the pinned PromptKit/catalog revisions. It must
also account for operator prompt definitions and their
captured prompt content, not just `content_file` path strings.
Use public captured inspection and prompt-definition equality values where
appropriate; do not reach into PromptKit internals. `InspectPrompt.PromptHash`
includes captured content and is an opaque equality value. Inspection reports
alone do not embed referenced message-file bodies. Preserve application
module/schema fingerprints, actual-used profile provenance, and the existing
exclusion of concurrency-only local settings from semantic checkpoint identity.
A conservative identity across effective profiles is acceptable; per-selected-
profile optimization is not a prerequisite for this upgrade.
Raw sources, configuration history, prompt bodies, endpoints, and credential
values must not be added to ordinary manifests or checkpoint metadata. The
new inspection API supports both private captured identity and the explicitly
requested operator reports below. Do not automatically dump reports into run
output or debug bundles. Hash-only identity is distinct from publishing an
inspection report, and redaction is not a substitute for covering execution
semantics.
### Operator-Facing PromptKit Inspection
Provide explicit CLI operations for three complementary views:
- **Effective configuration:** write the captured, ConfigKit-redacted PromptKit
configuration, including effective keyed backends, profiles, and prompts.
Use `Engine.InspectConfiguration().WriteEffective` rather than reconstructing
YAML from Notarius's profile declarations or runtime manifests.
- **Setting explanation:** explain one canonical JSON Pointer in that captured
configuration, including source history, layer overrides, and inheritance.
Use the captured inspection's explanation API. Literal IDs containing `/`
or `~` use ordinary JSON Pointer escaping, not a dot-path interpretation.
- **Resolved profile:** inspect one explicit profile ID with `InspectProfile`
and present its backend-completed execution settings, including target,
effective reasoning/provider controls, credential environment name and
requirement, and selected backend identity. Endpoint-only profiles remain
identifiable as such. Do not conflate configured profile fields with values
derived from backend defaults.
Use the same selected Notarius configuration/overlay, registered application
assets, operator sources, and local-backend contribution as execution and
preflight. A report describes the engine captured for that invocation, not a
live view of a separate running process. Constructing the engine performs
native definition/content validation, but these commands do not need transcript
input, execute pipeline stages, prepare requests, resolve credential values,
admit provider capacity, or contact a model. Required-credential profiles remain
inspectable when the named environment variable is unset.
Keep existing Notarius `config effective` and `config explain` behavior intact;
the PromptKit view must be an explicit selection, not a replacement or an
automatic second document appended to their output. Define the concrete command
syntax in the implementation plan and maintain it in `docs/cli.md` when it
lands. Standard output contains the selected report only; ordinary diagnostics
and failures follow the existing CLI conventions. Unknown paths/profiles and
invalid pointers must be distinguishable without leaking raw configuration
through error messages.
Reports are operator-requested application data. Preserve upstream credential
redaction and Notarius's existing secret-handling boundary; include credential
variable names, never their values. Configuration explanations can contain
overridden values and inline prompt/provider content, and effective YAML is not
a transcript-free or generally sanitized report. Document these properties and
operator-owned output handling without adding permission enforcement, new
confirmation gates, automatic report persistence, or blanket content removal.
Effective configuration, backend-completed profile settings, and prepared
request details are different views. These commands do not render a prompt
with transcript inputs or return captured message-file bodies merely because
they appear in the configuration as `content_file` references. Keep complete
request/response inspection on the existing explicitly requested debug path.
### Existing Execution Policies Remain Unchanged
Preserve prepared execution, the exact latest-response semantic correction
pair, prompt-prefix caching, direct sessions, reasoning overrides, cancellation,
the shared scheduler, local/backend admission, observer accounting, error
classification, and credential redaction. Structural repairs, transient HTTP
retries, outer producer attempts, and validator execution budgets remain
separate. The existing one-repair maintained prompt default remains unchanged.
The migration does not change extraction/normalization semantics, warning
classification, artifact contracts, subprocess receipts, or reusable-state
eligibility. Construction errors may occur earlier under native eager
validation; adapt them through the existing safe adapter/CLI error boundary,
not by exposing arbitrary upstream configuration content.
## Required Work
- Update `go.mod` and `go.sum` to PromptKit v0.12.0 and its selected `/v3`
catalogs. Keep ConfigKit v0.5.0 and unrelated dependencies unless module
resolution requires a change. Remove obsolete `/v2` requirements after tidy.
- Replace removed constructor/source/backend APIs in the LLM adapter and
shared profile-preflight construction; retain schema and injected-client
options. Remove obsolete adapter helpers rather than layering a parallel
configuration system beside them.
- Migrate every registered prompt and fallback profile, the maintained
deployment profile, and affected test fixtures to keyed definitions.
- Adopt the unified source policy, explicit operator imports, presence/null
behavior, native operator capabilities, and captured
lifetime; replace live-file checkpoint fingerprinting with captured identity.
- Add explicit effective-configuration, setting-explanation, and resolved-
profile CLI inspection using the shared captured-engine construction and
public PromptKit inspection APIs. Keep upstream inspection values within
the adapter boundary and preserve ordinary Notarius configuration reports.
- Update behavioral tests at the adapter, CLI preflight, asset registry, and
representative assembled-module boundaries. Reuse real ConfigKit/PromptKit
loading and offline injected clients or local HTTP fixtures.
- Update canonical documentation when implementation lands:
`docs/config.md` owns selected sources, precedence, profile syntax and
migration; `docs/operations.md` owns deployment and reload guidance;
`docs/internal/llm.md` owns construction, capture, and checkpoint mechanics;
`docs/integrations/pkg-promptkit.md` owns the supported upstream boundary and
pinned links; `docs/cli.md` owns the new inspection operations and output/error
behavior. Add a short troubleshooting workflow in `docs/operations.md` that
links to those references. Revise consumer examples only where affected.
- Update `docs/oss/attribution.md` for PromptKit and the two `/v3` catalogs,
checking their actual notices and exact source commits. PromptKit remains
LGPL-3.0-or-later; the catalogs remain MIT at the reviewed revisions. Reuse
unchanged standard texts and preserve the project's own license grant.
Native layered overrides change an existing documented operator contract.
Add a concise Notarius ADR recording the accepted native composition and
unified-source ownership/compatibility decision rather than duplicating the
upstream manual. Existing historical ADRs need not be rewritten.
## Verification And Acceptance
Acceptance is behavioral, not an inventory of private helper names:
- All registered D&D and generic prompts construct and prepare with migrated
assets, resolving shared/module content in the correct source context.
Request identities, message content/order, schemas, and cache controls retain
their intended behavior. Do not add exact shared-prefix-length detector tests.
- Operator overrides, inheritance, explicit zero/null, duplicate definitions
within a group, local backend precedence, endpoint-only profiles, and invalid
unused final definitions follow the native configuration policy. Cover
representative imported combined definitions, custom/backend catalog
overlays, partial prompt overlays with their embedded base, and optional
versus explicitly required credentials. Credential-free preflight and
execution resolve the same source ordering and application definitions.
- Engine/file mutation tests demonstrate captured settings and content;
fingerprints stay tied to that engine and change when a new engine captures
different execution-relevant settings or operator prompt content.
- CLI inspection reports effective values, setting origins, and backend-
completed profiles from the same source policy without transcript input,
credentials, or provider calls. Cover representative inheritance/null
overrides, escaped literal IDs, required-credential inspection with an unset
variable, and invalid/unknown selectors. Verify redaction and unchanged
ordinary Notarius reports without pinning incidental report formatting.
- Structural repair, semantic feedback, sessions/reasoning, capacity failure,
transient errors, safe diagnostics, and observations retain existing
provider-neutral behavior using deterministic boundary tests.
- Maintained minimal, complete, and inherited development pipelines validate
without credentials or provider calls. External legacy profile files receive
clear migration guidance, not silent reinterpretation.
Use focused LLM, CLI, and module-integration tests during implementation, then
`GOWORK=off go test ./...`, `GOWORK=off go vet ./...`, a temporary native build,
and `GOWORK=off go mod tidy -diff` for this shared-boundary migration. Verify
selected module versions, attribution links, maintained examples, and stale
API/format claims. These are migration checks, not new permanent release gates.
Live provider testing is optional follow-up evidence, not required acceptance.
This work warrants a minor Notarius release because the accepted external
profile format and override behavior change. Keep Notarius configuration at
version 5; select the actual release number during separate release preparation.

108
docs/roadmap/remediation.md Normal file
View File

@@ -0,0 +1,108 @@
# ConfigKit Inheritance Review Remediation
## Review Scope And Overall Result
Reviewed on 2026-10-04, comparing implementation commits `ec99d48` through
`6450994` with the baseline `9e74df0`, the ConfigKit feature roadmap, and its
eight-stage implementation plan.
The architecture matches the intended design. ConfigKit v0.5.0 owns source
composition and native sibling inheritance before decoding. Notarius retains
its typed defaults, presence-sensitive validation, declaration-relative path
normalization, detached runtime configuration, catalog-dependent resolution,
and private redacted inspection boundary. The production configuration adapter
needed no changes; the work adds the dependency update, integration tests,
documentation, and the derived development pipeline example.
No production regression or inadvertently strengthened requirement was found.
In particular, the implementation adds no inheritance whitelist, new schema
version, custom merging, path-containment rule, template exemption, or release
gate. The newly reserved `extends` key is the expressly adopted upstream
compatibility change.
Two follow-ups were identified: a path-test blind spot and an incomplete
documented command. Completion is recorded under each finding. This document
records remediation work only; it does not authorize
unrelated runtime refactoring.
## Finding 1 — Make The Path Fixture Distinguish Declaring Directories
Priority: P2. Category: verification gap.
Status: addressed on 2026-10-04. The fixture now uses distinct parent-, child-,
and overlay-local paths for the source-base assertions and retains a separate
`../` reference. `GOWORK=off go test ./internal/core/config` passed. No
production resolver change was needed.
Evidence: `internal/core/config/inheritance_test.go:338–436`,
`TestInheritedReferencePathsKeepDeclaringSource`.
The parent, child, and overlay files live in `root/parents`, `root/children`,
and `root/overlays`. Their resource paths all use `../fixtures/<name>`, and
the expected paths all point into `root/fixtures`. Resolving one of these
paths from any of the three directories yields the same result. Thus, choosing
the child's directory for an inherited step/nested reference, or choosing the
parent's directory for an overlay replacement, could still pass this test.
`TestInheritedInspectionRetainsOriginsRedactionAndOwnership` does distinguish
the parent directory for a simple inherited `./shared.txt` reference. It does
not close the blind spot for the nested, ordered-step, direct-child replacement,
and overlay cases in the larger path fixture.
Required change:
- Revise the existing path fixture to use directory-distinguishing relative
paths for these cases, such as `./shared.txt`, `./step.txt`, and a child-local
replacement. Assert inherited paths against the parent's directory, direct
replacements against the child's directory, and overlay-supplied paths
against the overlay's directory.
- Retain a separate legitimate `../` example to protect the existing host-path
behavior. Avoid making every path converge on one shared sibling directory.
- Keep this as a focused real-file configuration test. Change production
resolution only if the improved assertions expose an actual defect.
Acceptance: choosing a parent, child, or overlay directory incorrectly produces
a different absolute path and fails the relevant assertion. Nested inheritance,
inherited ordered-step references, child replacements, and overlay replacements
still pass with the actual resolver.
Validation: `GOWORK=off go test ./internal/core/config`.
## Finding 2 — Supply Input In The Development Pipeline Command
Priority: P3. Category: documentation accuracy.
Status: addressed on 2026-10-04. The example now supplies the pipeline ID,
configuration path, and transcript input path, and links to the CLI run
reference. The flags were checked against that reference, and
`GOWORK=off go test ./internal/cli` passed without a live model request.
Evidence: `docs/config.md:43–49`, in Maintained Examples.
The new example says to select the development pipeline with
`notarius run dnd-session-development --config /path/to/dnd-complete.config.yml`.
The run command requires `--input`. Executing the equivalent command against
the maintained configuration returns `notarius: run requires --input` with CLI
exit status 2, before pipeline execution.
Required change: include a transcript input placeholder in the invocation,
for example `--input /path/to/transcript.json`, and retain the link to the CLI
reference for the full invocation contract. No CLI behavior change or new test
framework is needed.
Acceptance: the documented command supplies both required run inputs—the
pipeline ID and input path—and uses the implemented flag names. Verify against
the CLI reference and existing parser coverage without a live model request.
## Checks Completed During Review
- `GOWORK=off go test ./...` passed.
- `GOWORK=off go vet ./...` passed.
- `GOWORK=off go build` of `./cmd/notarius` to a temporary destination passed.
- `GOWORK=off go mod tidy -diff` produced no changes.
- The selected ConfigKit module is v0.5.0; PromptKit remains v0.11.2.
- The implementation diff passed whitespace checks.
- The incomplete development invocation was reproduced without provider calls.
These checks describe the review baseline. Subsequent remediation and its
focused validation are recorded under each finding.

View File

@@ -162,3 +162,9 @@ pipelines:
module: appendorder
normalize:
module: dnd/enemy-events
# Development variant: disable PromptKit structural repairs while retaining
# the inherited steps and configured producer retry budgets.
dnd-session-development:
extends: dnd-session
structured_output_repair_attempts: 0

2
go.mod
View File

@@ -3,7 +3,7 @@ module gitea.maximumdirect.net/eric/notarius
go 1.25.5
require (
gitea.maximumdirect.net/eric/ConfigKit v0.4.2
gitea.maximumdirect.net/eric/ConfigKit v0.5.0
gitea.maximumdirect.net/eric/promptkit v0.11.2
github.com/santhosh-tekuri/jsonschema/v6 v6.0.2
gopkg.in/yaml.v3 v3.0.1 // indirect

4
go.sum
View File

@@ -1,5 +1,5 @@
gitea.maximumdirect.net/eric/ConfigKit v0.4.2 h1:ZMxmv8pOnLtAx2hFTf8Z+pHKUbR89xErkBhwjr4yK24=
gitea.maximumdirect.net/eric/ConfigKit v0.4.2/go.mod h1:tYWpsrbLyu05r8+1MapWp5fN6ZrX9I8hU2xEPedYmYk=
gitea.maximumdirect.net/eric/ConfigKit v0.5.0 h1:2KwG10+kApWM3avOoYA+p768xcbHUp7tQ2rqizBNpyg=
gitea.maximumdirect.net/eric/ConfigKit v0.5.0/go.mod h1:tYWpsrbLyu05r8+1MapWp5fN6ZrX9I8hU2xEPedYmYk=
gitea.maximumdirect.net/eric/promptkit v0.11.2 h1:2RqI+KtflvSh7vLQaTAQCmMaFQscEA7EPwPazyvXRoY=
gitea.maximumdirect.net/eric/promptkit v0.11.2/go.mod h1:onm/FBPUKQ8/NSPwTEk26Ut+dVX2iXcq8vEseB+g/0E=
gitea.maximumdirect.net/eric/promptkit-backend-openrouter/v2 v2.20260926.0 h1:wdab+R+lgMZxzYwhbAuCrW7zdSQ1cW9ovBR6UzLmZR4=

View File

@@ -40,6 +40,130 @@ func TestConfigKitReadOnlyCommandsLoadImportsAndOverlays(t *testing.T) {
}
}
func TestPipelineInheritanceCommandsSelectAndInspectDerivedPipelines(t *testing.T) {
t.Parallel()
path := writeCommandConfigContent(t, `version: 5
pipelines:
" parent ":
input:
module: seriatim
artifacts:
spells:
extract:
module: dnd/spells
structured_output_repair_attempts: 2
child:
extends: " parent "
structured_output_repair_attempts: 0
`)
options := commandContractOptions(t)
var stdout, stderr bytes.Buffer
if code := RunWithOptions([]string{"pipelines", "list", "--config", path}, &stdout, &stderr, options); code != 0 || stdout.String() != "child\nparent\n" || stderr.Len() != 0 {
t.Fatalf("list = %d, %q, %q", code, stdout.String(), stderr.String())
}
stdout.Reset()
stderr.Reset()
if code := RunWithOptions([]string{"config", "validate", "--config", path, "--pipeline", "child"}, &stdout, &stderr, options); code != 0 || !strings.Contains(stdout.String(), `is valid for pipeline "child"`) || stderr.Len() != 0 {
t.Fatalf("validate child = %d, %q, %q", code, stdout.String(), stderr.String())
}
stdout.Reset()
stderr.Reset()
if code := RunWithOptions([]string{"config", "effective", "--config", path}, &stdout, &stderr, options); code != 0 || !strings.Contains(stdout.String(), "module: seriatim") || !strings.Contains(stdout.String(), "structured_output_repair_attempts: 0") || strings.Contains(stdout.String(), "extends:") || stderr.Len() != 0 {
t.Fatalf("effective = %d, %q, %q", code, stdout.String(), stderr.String())
}
stdout.Reset()
stderr.Reset()
if code := RunWithOptions([]string{"config", "explain", "--config", path, "/pipelines/child/input/module"}, &stdout, &stderr, options); code != 0 || !strings.Contains(stdout.String(), `path: "/pipelines/child/input/module"`) || !strings.Contains(stdout.String(), "parent") || stderr.Len() != 0 {
t.Fatalf("explain inherited input = %d, %q, %q", code, stdout.String(), stderr.String())
}
}
func TestPipelineInheritanceLoadFailuresUseRuntimeDiagnostics(t *testing.T) {
t.Parallel()
for _, test := range []struct {
name string
content string
args []string
want string
}{
{
name: "missing parent",
content: `version: 5
pipelines:
child:
extends: absent
`,
args: []string{"config", "validate", "--pipeline", "child"},
want: "inheritance.missing",
},
{
name: "cycle",
content: `version: 5
pipelines:
first:
extends: second
second:
extends: first
`,
args: []string{"config", "validate", "--pipeline", "first"},
want: "cycle",
},
{
name: "invalid directive type",
content: `version: 5
pipelines:
child:
extends: 7
`,
args: []string{"config", "validate", "--pipeline", "child"},
want: "extends",
},
{
name: "unselected missing parent",
content: `version: 5
pipelines:
parent:
input:
module: seriatim
child:
extends: parent
unrelated:
extends: absent
`,
args: []string{"config", "validate", "--pipeline", "child"},
want: "inheritance.missing",
},
{
name: "exact sibling mismatch",
content: `version: 5
pipelines:
parent:
input:
module: seriatim
child:
extends: "parent "
`,
args: []string{"pipelines", "list"},
want: "inheritance.missing",
},
} {
t.Run(test.name, func(t *testing.T) {
t.Parallel()
path := writeCommandConfigContent(t, test.content)
args := append([]string{}, test.args...)
args = append(args, "--config", path)
var stdout, stderr bytes.Buffer
if code := RunWithOptions(args, &stdout, &stderr, commandContractOptions(t)); code != 1 || stdout.Len() != 0 || stderr.Len() == 0 || !strings.Contains(stderr.String(), test.want) {
t.Fatalf("code=%d stdout=%q stderr=%q", code, stdout.String(), stderr.String())
}
})
}
}
func TestRetiredConfigProfileFlagUsesNormalUnknownFlagBehavior(t *testing.T) {
t.Parallel()

View File

@@ -3,6 +3,7 @@ package cli
import (
"context"
"encoding/json"
"fmt"
"os"
"path/filepath"
"sort"
@@ -25,6 +26,7 @@ import (
func TestMaintainedExamplesLoadResolveAndList(t *testing.T) {
components := productionTestComponents(t)
completePipelines := make(map[string]pipeline.ResolvedPipeline)
for _, example := range maintainedExampleFiles(t) {
t.Run(example.name, func(t *testing.T) {
cfg := loadMaintainedExample(t, example.path)
@@ -58,9 +60,45 @@ func TestMaintainedExamplesLoadResolveAndList(t *testing.T) {
t.Fatalf("materialize maintained example references for %q: %v", pipelineID, err)
}
if example.name == "complete" {
if pipelineID == "dnd-session-development" {
parent, found := completePipelines["dnd-session"]
if !found {
t.Fatal("complete parent pipeline was not resolved before its development variant")
}
if strings.Join(exampleStepLaneIDs(materialized), "|") != strings.Join(exampleStepLaneIDs(parent), "|") {
t.Fatalf("development variant steps and lanes = %v, want parent composition %v", exampleStepLaneIDs(materialized), exampleStepLaneIDs(parent))
}
if got, want := strings.Join(generatedReferenceBindingDescriptions(materialized), "|"), strings.Join(generatedReferenceBindingDescriptions(parent), "|"); got != want {
t.Fatalf("development generated reference bindings = %q, want parent bindings %q", got, want)
}
eligibleBindings := 0
for _, step := range materialized.Steps {
for _, lane := range step.ArtifactLanes {
for _, binding := range []struct {
module pipeline.ModuleBinding
executionClass contracts.ExecutionClass
}{
{module: lane.Extract, executionClass: lane.ExtractExecutionClass},
{module: lane.Normalize, executionClass: lane.NormalizeExecutionClass},
} {
if binding.executionClass != "llm_backed" {
continue
}
eligibleBindings++
if got := binding.module.StructuredOutputRepairAttempts; got == nil || *got != 0 {
t.Fatalf("development %s repair attempts = %v, want explicit zero", binding.module.Module, got)
}
}
}
}
if eligibleBindings == 0 {
t.Fatal("development variant had no eligible LLM bindings for repair policy")
}
}
if got := exampleStepLaneIDs(materialized); strings.Join(got, "|") != "describe-session:item-registry,location-registry,npc-registry,scene-descriptions|extract-events:combat-turns,item-occurrences,location-occurrences,npc-occurrences,spells|track-enemies:enemy-events" {
t.Fatalf("complete example steps and lanes = %v, want the documented D&D extractor composition", got)
}
completePipelines[pipelineID] = materialized
locationLane := referenceContractLane(t, materialized, "location-registry")
if locationLane.ArtifactKind != dnd.LocationRegistryKind || locationLane.Extract.Module != locationextract.Key || locationLane.Extract.Retries != 2 || locationLane.Merge.Module != pipeline.DefaultMergeModule || locationLane.Normalize.Module != locationnormalize.Key || locationLane.Normalize.Retries != 2 {
t.Fatalf("location lane = %#v, want typed registry composition", locationLane)
@@ -164,12 +202,16 @@ func TestMaintainedExamplesValidateEffectiveProfilesOffline(t *testing.T) {
t.Setenv("OPENROUTER_API_KEY", "")
for _, example := range maintainedExampleFiles(t) {
t.Run(example.name, func(t *testing.T) {
var stdout, stderr strings.Builder
code := RunWithOptions([]string{
"config", "validate", "--config", example.path, "--pipeline", "dnd-session",
}, &stdout, &stderr, Options{})
if code != 0 || stderr.Len() != 0 || !strings.Contains(stdout.String(), `valid for pipeline "dnd-session"`) {
t.Fatalf("code=%d stdout=%q stderr=%q", code, stdout.String(), stderr.String())
for _, pipelineID := range example.pipelineIDs {
t.Run(pipelineID, func(t *testing.T) {
var stdout, stderr strings.Builder
code := RunWithOptions([]string{
"config", "validate", "--config", example.path, "--pipeline", pipelineID,
}, &stdout, &stderr, Options{})
if code != 0 || stderr.Len() != 0 || !strings.Contains(stdout.String(), `valid for pipeline "`+pipelineID+`"`) {
t.Fatalf("code=%d stdout=%q stderr=%q", code, stdout.String(), stderr.String())
}
})
}
})
}
@@ -188,6 +230,31 @@ func exampleStepLaneIDs(resolved pipeline.ResolvedPipeline) []string {
return result
}
func generatedReferenceBindingDescriptions(resolved pipeline.ResolvedPipeline) []string {
var descriptions []string
for _, step := range resolved.Steps {
for _, lane := range step.ArtifactLanes {
for _, target := range []struct {
name string
target pipeline.ResolvedReferenceTarget
}{
{name: "extract", target: lane.ExtractReferences},
{name: "merge", target: lane.MergeReferences},
{name: "normalize", target: lane.NormalizeReferences},
} {
for _, binding := range target.target.Bindings {
if binding.Artifact == nil {
continue
}
descriptions = append(descriptions, fmt.Sprintf("%s/%s/%s/%s=%s/%s", step.ID, lane.ID, target.name, binding.SlotName, binding.Artifact.Step, binding.Artifact.Lane))
}
}
}
}
sort.Strings(descriptions)
return descriptions
}
func TestMaintainedMinimalInvocationProducesJSONBundle(t *testing.T) {
outputRoot := filepath.Join(t.TempDir(), "output")
fake := &productionFakeLLMClient{}

View File

@@ -824,7 +824,7 @@ func maintainedExampleFiles(t *testing.T) []maintainedExample {
t.Helper()
return []maintainedExample{
{name: "minimal", path: repositoryPath("examples", "dnd-minimal.config.yml"), transcriptPath: repositoryPath("examples", "seriatim-minimal-transcript.json"), pipelineIDs: []string{"dnd-session"}},
{name: "complete", path: repositoryPath("examples", "dnd-complete.config.yml"), transcriptPath: repositoryPath("examples", "dnd-complete-transcript.json"), pipelineIDs: []string{"dnd-session"}},
{name: "complete", path: repositoryPath("examples", "dnd-complete.config.yml"), transcriptPath: repositoryPath("examples", "dnd-complete-transcript.json"), pipelineIDs: []string{"dnd-session", "dnd-session-development"}},
}
}

View File

@@ -0,0 +1,619 @@
package config
import (
"bytes"
"errors"
"os"
"path/filepath"
"strings"
"testing"
configkit "gitea.maximumdirect.net/eric/ConfigKit"
)
func TestPipelineInheritancePreservesCompositionAndReplacesSteps(t *testing.T) {
t.Parallel()
tests := []struct {
name string
childSteps string
wantStepIDs []string
}{
{
name: "inherits ordered steps",
wantStepIDs: []string{"prepare", "summarize"},
},
{
name: "replaces ordered steps",
childSteps: `
steps:
- id: replacement
artifacts:
summary:
extract: {module: extract}`,
wantStepIDs: []string{"replacement"},
},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
source := `version: 5
pipelines:
parent:
input: {module: input}
structured_output_repair_attempts: 2
steps:
- id: prepare
artifacts:
score:
extract: {module: extract, retries: 2}
- id: summarize
artifacts:
notes:
extract: {module: extract}
child:
extends: parent
llm_profile: local
structured_output_repair_attempts: 0` + test.childSteps + "\n"
loaded, err := loadConfigSource(configkit.Bytes("inheritance.yml", []byte(source)), LoadOptions{})
if err != nil {
t.Fatalf("loadConfigSource() error = %v", err)
}
parent := loaded.Config.Pipelines["parent"]
child := loaded.Config.Pipelines["child"]
if parent.Input.Module != "input" || child.Input.Module != "input" {
t.Fatalf("parent/child inputs = %#v / %#v", parent.Input, child.Input)
}
if child.LLMProfile != "local" ||
child.StructuredOutputRepairAttempts == nil || *child.StructuredOutputRepairAttempts != 0 {
t.Fatalf("child overrides = %#v", child)
}
if parent.StructuredOutputRepairAttempts == nil || *parent.StructuredOutputRepairAttempts != 2 {
t.Fatalf("parent settings changed: %#v", parent)
}
if len(child.Steps) != len(test.wantStepIDs) {
t.Fatalf("child steps = %#v, want %v", child.Steps, test.wantStepIDs)
}
for i, want := range test.wantStepIDs {
if child.Steps[i].ID != want {
t.Fatalf("child step %d = %q, want %q", i, child.Steps[i].ID, want)
}
}
if test.childSteps == "" && child.Steps[0].Artifacts["score"].Extract.Module != "extract" {
t.Fatalf("inherited first artifact = %#v", child.Steps[0].Artifacts["score"])
}
for _, id := range []string{"parent", "child"} {
resolved, err := loaded.Config.Resolve(ResolveInput{PipelineID: id, Catalog: effectiveCatalog(t)})
if err != nil {
t.Fatalf("Resolve(%q) error = %v", id, err)
}
if resolved.ResolvedPipeline.ID != id {
t.Fatalf("resolved pipeline ID = %q, want %q", resolved.ResolvedPipeline.ID, id)
}
wantResolvedSteps := len(test.wantStepIDs)
if id == "parent" {
wantResolvedSteps = 2
}
if len(resolved.ResolvedPipeline.Steps) != wantResolvedSteps {
t.Fatalf("resolved %q steps = %#v", id, resolved.ResolvedPipeline.Steps)
}
if id == "child" && resolved.ResolvedPipeline.Steps[0].ArtifactLanes[0].Extract.LLMProfile != "local" {
t.Fatalf("resolved child profile = %q", resolved.ResolvedPipeline.Steps[0].ArtifactLanes[0].Extract.LLMProfile)
}
}
})
}
}
func TestArtifactLaneInheritanceUsesSiblingMappingComposition(t *testing.T) {
t.Parallel()
loaded, err := loadConfigSource(configkit.Bytes("lanes.yml", []byte(`version: 5
pipelines:
session:
input: {module: input}
artifacts:
base:
extract: {module: extract, retries: 2}
derived:
extends: base
extract: {retries: 4}
`)), LoadOptions{})
if err != nil {
t.Fatalf("loadConfigSource() error = %v", err)
}
lanes := loaded.Config.Pipelines["session"].Artifacts
if lanes["base"].Extract.Module != "extract" || lanes["base"].Extract.Retries != 2 {
t.Fatalf("base lane = %#v", lanes["base"])
}
if lanes["derived"].Extract.Module != "extract" || lanes["derived"].Extract.Retries != 4 {
t.Fatalf("derived lane = %#v", lanes["derived"])
}
if _, err := loaded.Config.Resolve(ResolveInput{PipelineID: "session", Catalog: effectiveCatalog(t)}); err != nil {
t.Fatalf("Resolve() error = %v", err)
}
}
func TestInheritedPresenceOverridesRemainExplicit(t *testing.T) {
t.Parallel()
loaded, err := loadConfigSource(configkit.Bytes("presence.yml", []byte(`version: 5
pipelines:
parent:
input: {module: input}
structured_output_repair_attempts: 3
chunk:
module: generic
validators: []
artifacts:
score:
extract:
module: extract
validators: [{module: configured-a}]
child:
extends: parent
structured_output_repair_attempts: 0
artifacts:
score:
extract:
validators: []
`)), LoadOptions{})
if err != nil {
var loadError *configkit.LoadError
if errors.As(err, &loadError) {
t.Fatalf("loadConfigSource() diagnostics = %#v", loadError.Diagnostics())
}
t.Fatalf("loadConfigSource() error = %v", err)
}
parent := loaded.Config.Pipelines["parent"]
child := loaded.Config.Pipelines["child"]
if parent.Chunk.Module != "generic" || !parent.Chunk.Validators.Set || parent.Chunk.Validators.Validators == nil || len(parent.Chunk.Validators.Validators) != 0 {
t.Fatalf("parent empty chunk validators = %#v", parent.Chunk)
}
if !parent.Artifacts["score"].Extract.Validators.Set || len(parent.Artifacts["score"].Extract.Validators.Validators) != 1 ||
parent.Artifacts["score"].Extract.Validators.Validators[0].Module != "configured-a" {
t.Fatalf("parent extraction validators = %#v", parent.Artifacts["score"].Extract.Validators)
}
if parent.StructuredOutputRepairAttempts == nil || *parent.StructuredOutputRepairAttempts != 3 {
t.Fatalf("parent repair budget = %#v", parent.StructuredOutputRepairAttempts)
}
if child.StructuredOutputRepairAttempts == nil || *child.StructuredOutputRepairAttempts != 0 {
t.Fatalf("child zero repair budget = %#v", child.StructuredOutputRepairAttempts)
}
extract := child.Artifacts["score"].Extract
if !extract.Validators.Set || extract.Validators.Validators == nil || len(extract.Validators.Validators) != 0 {
t.Fatalf("child empty extraction validators = %#v", extract.Validators)
}
if !child.Chunk.Validators.Set || child.Chunk.Validators.Validators == nil || len(child.Chunk.Validators.Validators) != 0 {
t.Fatalf("inherited empty chunk validators = %#v", child.Chunk.Validators)
}
resolved, err := loaded.Config.Resolve(ResolveInput{PipelineID: "child", Catalog: effectiveCatalog(t)})
if err != nil {
t.Fatalf("Resolve(child) error = %v", err)
}
lanes := resolved.ResolvedPipeline.AllArtifactLanes()
if len(lanes) != 1 || lanes[0].Extract.StructuredOutputRepairAttempts == nil || *lanes[0].Extract.StructuredOutputRepairAttempts != 0 {
t.Fatalf("resolved child repair budget = %#v", lanes)
}
if !resolved.ResolvedPipeline.Chunk.Validators.Set || len(resolved.ResolvedPipeline.Chunk.Validators.Validators) != 0 ||
!lanes[0].Extract.Validators.Set || len(lanes[0].Extract.Validators.Validators) != 0 {
t.Fatalf("resolved empty validator overrides = chunk %#v, extraction %#v", resolved.ResolvedPipeline.Chunk.Validators, lanes[0].Extract.Validators)
}
}
func TestInheritedFalseOptionRetainsExplicitMetadata(t *testing.T) {
t.Parallel()
loaded, err := loadConfigSource(configkit.Bytes("false-option.yml", []byte(`version: 5
pipelines:
parent:
input: {module: input}
output:
module: json
options:
evidence_context: {enabled: false}
child:
extends: parent
`)), LoadOptions{})
if err != nil {
t.Fatalf("loadConfigSource() error = %v", err)
}
options := loaded.Config.Pipelines["child"].Output.Options["evidence_context"].(map[string]any)
if enabled, ok := options["enabled"].(bool); !ok || enabled {
t.Fatalf("inherited evidence_context.enabled = %#v", options["enabled"])
}
path := configkit.Keys("pipelines", "child", "output", "options", "evidence_context", "enabled")
info, ok := loaded.result.Metadata().Lookup(path)
if !ok || !info.Explicit || info.Null || len(info.Origin.Inheritance) == 0 {
t.Fatalf("inherited false metadata = %#v, found=%v", info, ok)
}
}
func TestInheritedAndDirectNullRepairBudgetsRemainInvalid(t *testing.T) {
t.Parallel()
for _, test := range []struct {
name string
source string
path string
}{
{
name: "child null",
source: `version: 5
pipelines:
parent: {input: {module: input}, structured_output_repair_attempts: 2}
child:
extends: parent
structured_output_repair_attempts: null
`,
path: "/pipelines/child/structured_output_repair_attempts",
},
{
name: "inherited null",
source: `version: 5
pipelines:
parent: {input: {module: input}, structured_output_repair_attempts: null}
child: {extends: parent}
`,
path: "/pipelines/parent/structured_output_repair_attempts",
},
} {
t.Run(test.name, func(t *testing.T) {
_, err := loadConfigSource(configkit.Bytes("null.yml", []byte(test.source)), LoadOptions{})
assertDocumentDiagnosticAt(t, err, test.path)
})
}
}
func TestOverlayAndOverridesComposeBeforeInheritance(t *testing.T) {
t.Parallel()
root := t.TempDir()
overlays := filepath.Join(root, "overlays")
if err := os.MkdirAll(overlays, 0o700); err != nil {
t.Fatal(err)
}
main := filepath.Join(root, "config.yml")
if err := os.WriteFile(main, []byte(`version: 5
pipelines:
parent:
input: {module: input}
structured_output_repair_attempts: 1
inherited:
extends: parent
overridden:
extends: parent
structured_output_repair_attempts: 0
cancelled:
extends: parent
input: {module: input}
`), 0o600); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(filepath.Join(overlays, "development.yml"), []byte(`pipelines:
parent:
structured_output_repair_attempts: 2
cancelled:
extends: null
`), 0o600); err != nil {
t.Fatal(err)
}
loaded, err := Load(main, LoadOptions{Overlay: "development"})
if err != nil {
t.Fatalf("Load() error = %v", err)
}
if got := *loaded.Config.Pipelines["inherited"].StructuredOutputRepairAttempts; got != 2 {
t.Fatalf("overlay-inherited repair budget = %d, want 2", got)
}
if got := *loaded.Config.Pipelines["overridden"].StructuredOutputRepairAttempts; got != 0 {
t.Fatalf("child replacement repair budget = %d, want 0", got)
}
if loaded.Config.Pipelines["cancelled"].StructuredOutputRepairAttempts != nil {
t.Fatalf("cancelled child inherited parent settings: %#v", loaded.Config.Pipelines["cancelled"])
}
withOverride, err := Load(main, LoadOptions{
Overlay: "development",
Overrides: []configkit.Override{{
Path: configkit.Keys("pipelines", "parent", "structured_output_repair_attempts"),
Value: 3,
Flag: "--repair-budget",
}},
})
if err != nil {
t.Fatalf("Load() with explicit override error = %v", err)
}
if got := *withOverride.Config.Pipelines["inherited"].StructuredOutputRepairAttempts; got != 3 {
t.Fatalf("override-inherited repair budget = %d, want 3", got)
}
}
func TestInheritedReferencePathsKeepDeclaringSource(t *testing.T) {
t.Parallel()
root := t.TempDir()
parentDirectory := filepath.Join(root, "parents")
childDirectory := filepath.Join(root, "children")
overlayDirectory := filepath.Join(root, "overlays")
fixtureDirectory := filepath.Join(root, "fixtures")
for _, directory := range []string{parentDirectory, childDirectory, overlayDirectory, fixtureDirectory} {
if err := os.MkdirAll(directory, 0o700); err != nil {
t.Fatal(err)
}
}
for _, fixture := range []struct{ directory, name string }{
{parentDirectory, "shared.txt"},
{parentDirectory, "parent.txt"},
{parentDirectory, "step.txt"},
{childDirectory, "child.txt"},
{overlayDirectory, "overlay.txt"},
{fixtureDirectory, "outside.txt"},
} {
if err := os.WriteFile(filepath.Join(fixture.directory, fixture.name), []byte("fixture\n"), 0o600); err != nil {
t.Fatal(err)
}
}
parent := filepath.Join(parentDirectory, "pipelines.yml")
if err := os.WriteFile(parent, []byte(`pipelines:
" parent ":
input: {module: input}
references:
" shared ": {path: ./shared.txt}
nested:
extends: " shared "
source: {path: ./parent.txt}
outside: {path: ../fixtures/outside.txt}
steps:
- id: first
references:
step-source: {path: ./step.txt}
artifacts:
summary:
extract: {module: extract}
`), 0o600); err != nil {
t.Fatal(err)
}
child := filepath.Join(childDirectory, "pipelines.yml")
if err := os.WriteFile(child, []byte(`pipelines:
child:
extends: " parent "
references:
source: {path: ./child.txt}
`), 0o600); err != nil {
t.Fatal(err)
}
main := filepath.Join(root, "notarius.yml")
if err := os.WriteFile(main, []byte(`version: 5
configkit:
imports:
- path: parents/pipelines.yml
- path: children/pipelines.yml
`), 0o600); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(filepath.Join(overlayDirectory, "development.yml"), []byte(`pipelines:
" parent ":
references:
" shared ": {path: ./overlay.txt}
`), 0o600); err != nil {
t.Fatal(err)
}
loaded, err := loadConfig(main, LoadOptions{})
if err != nil {
t.Fatalf("loadConfig() error = %v", err)
}
assertReferencePath := func(pipelineID, referenceID, want string) {
t.Helper()
if got := loaded.Config.Pipelines[pipelineID].References[referenceID].Path; got != want {
t.Errorf("%s reference %s path = %q, want %q", pipelineID, referenceID, got, want)
}
}
assertReferencePath("parent", "shared", filepath.Join(parentDirectory, "shared.txt"))
assertReferencePath("child", "shared", filepath.Join(parentDirectory, "shared.txt"))
assertReferencePath("child", "nested", filepath.Join(parentDirectory, "shared.txt"))
assertReferencePath("parent", "source", filepath.Join(parentDirectory, "parent.txt"))
assertReferencePath("child", "source", filepath.Join(childDirectory, "child.txt"))
// Keep ../ coverage separate from paths that distinguish the declaring file.
assertReferencePath("child", "outside", filepath.Join(fixtureDirectory, "outside.txt"))
if got := loaded.Config.Pipelines["child"].Steps[0].References["step-source"].Path; got != filepath.Join(parentDirectory, "step.txt") {
t.Errorf("inherited step reference path = %q", got)
}
overlaid, err := loadConfig(main, LoadOptions{Overlay: "development"})
if err != nil {
t.Fatalf("loadConfig() with overlay error = %v", err)
}
for _, pipelineID := range []string{"parent", "child"} {
for _, referenceID := range []string{"shared", "nested"} {
if got, want := overlaid.Config.Pipelines[pipelineID].References[referenceID].Path, filepath.Join(overlayDirectory, "overlay.txt"); got != want {
t.Errorf("overlaid %s reference %s path = %q, want %q", pipelineID, referenceID, got, want)
}
}
}
if got, want := overlaid.Config.Pipelines["child"].References["source"].Path, filepath.Join(childDirectory, "child.txt"); got != want {
t.Errorf("child replacement path = %q, want %q", got, want)
}
if got, want := overlaid.Config.Pipelines["child"].Steps[0].References["step-source"].Path, filepath.Join(parentDirectory, "step.txt"); got != want {
t.Errorf("overlaid inherited step reference path = %q, want %q", got, want)
}
}
func TestInheritedInspectionRetainsOriginsRedactionAndOwnership(t *testing.T) {
t.Parallel()
root := t.TempDir()
parentDirectory := filepath.Join(root, "parents")
childDirectory := filepath.Join(root, "children")
overlayDirectory := filepath.Join(root, "overlays")
for _, directory := range []string{parentDirectory, childDirectory, overlayDirectory} {
if err := os.MkdirAll(directory, 0o700); err != nil {
t.Fatal(err)
}
}
parentFile := filepath.Join(parentDirectory, "pipelines.yml")
if err := os.WriteFile(parentFile, []byte(`pipelines:
parent:
input: {module: input}
structured_output_repair_attempts: 2
references:
source: {path: ./shared.txt}
artifacts:
summary:
extract:
module: extract
options:
nested: {value: original}
credentials: {api_key: fake-parent-secret}
`), 0o600); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(filepath.Join(parentDirectory, "shared.txt"), []byte("source\n"), 0o600); err != nil {
t.Fatal(err)
}
childFile := filepath.Join(childDirectory, "pipelines.yml")
if err := os.WriteFile(childFile, []byte(`pipelines:
child:
extends: parent
structured_output_repair_attempts: 0
`), 0o600); err != nil {
t.Fatal(err)
}
main := filepath.Join(root, "notarius.yml")
if err := os.WriteFile(main, []byte(`version: 5
configkit:
imports:
- path: parents/pipelines.yml
- path: children/pipelines.yml
`), 0o600); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(filepath.Join(overlayDirectory, "development.yml"), []byte(`pipelines:
parent:
artifacts:
summary:
extract:
options:
credentials: {api_key: fake-overlay-secret}
`), 0o600); err != nil {
t.Fatal(err)
}
loaded, err := Load(main, LoadOptions{Overlay: "development"})
if err != nil {
t.Fatalf("Load() error = %v", err)
}
child := loaded.Config.Pipelines["child"]
wantPath := filepath.Join(parentDirectory, "shared.txt")
if got := child.References["source"].Path; got != wantPath {
t.Fatalf("runtime inherited path = %q, want %q", got, wantPath)
}
if child.StructuredOutputRepairAttempts == nil || *child.StructuredOutputRepairAttempts != 0 {
t.Fatalf("runtime inherited repair setting = %#v", child.StructuredOutputRepairAttempts)
}
var effective bytes.Buffer
if err := loaded.WriteEffective(&effective); err != nil {
t.Fatalf("WriteEffective() error = %v", err)
}
if !strings.Contains(effective.String(), wantPath) || strings.Contains(effective.String(), "extends:") ||
!strings.Contains(effective.String(), "structured_output_repair_attempts: 0") {
t.Fatalf("effective configuration is inconsistent with runtime conversion: %s", effective.String())
}
for _, secret := range []string{"fake-parent-secret", "fake-overlay-secret"} {
if strings.Contains(effective.String(), secret) {
t.Fatalf("effective configuration exposed %q", secret)
}
}
for _, test := range []struct {
name string
path configkit.Path
}{
{name: "repair setting", path: configkit.Keys("pipelines", "child", "structured_output_repair_attempts")},
{name: "normalized reference", path: configkit.Keys("pipelines", "child", "references", "source", "path")},
} {
t.Run(test.name, func(t *testing.T) {
explanation, ok := loaded.result.Explain(test.path)
if !ok || len(explanation.History) == 0 {
t.Fatalf("inherited explanation = %#v, found=%v", explanation, ok)
}
inheritanceFromParent := false
for _, change := range explanation.History {
if len(change.Origin.Inheritance) == 0 {
continue
}
for _, location := range change.Origin.Locations {
for _, source := range loaded.result.Sources() {
if source.ID == location.SourceID && source.Path == parentFile {
inheritanceFromParent = true
}
}
}
}
if !inheritanceFromParent {
t.Fatalf("inheritance history lacks parent source evidence: %#v", explanation.History)
}
var rendered bytes.Buffer
if err := loaded.WriteExplanation(&rendered, test.path.String()); err != nil {
t.Fatalf("WriteExplanation() error = %v", err)
}
if !strings.Contains(rendered.String(), filepath.ToSlash(filepath.Join("parents", "pipelines.yml"))) {
t.Fatalf("explanation lacks parent source evidence: %s", rendered.String())
}
if test.name == "normalized reference" {
foundNormalization := false
for _, change := range explanation.History {
if change.Origin.Operation == configkit.ChangeNormalization {
foundNormalization = true
}
}
if !foundNormalization || !strings.Contains(rendered.String(), wantPath) {
t.Fatalf("normalized path evidence = %#v; rendered %s", explanation.History, rendered.String())
}
}
})
}
secretPath := configkit.Keys("pipelines", "child", "artifacts", "summary", "extract", "options", "credentials", "api_key")
secretExplanation, ok := loaded.result.Explain(secretPath)
if !ok || !secretExplanation.Effective.Redacted {
t.Fatalf("inherited secret explanation = %#v, found=%v", secretExplanation, ok)
}
redactedHistory := false
for _, change := range secretExplanation.History {
redactedHistory = redactedHistory || change.Before.Redacted || change.After.Redacted
}
if !redactedHistory {
t.Fatalf("secret history was not redacted: %#v", secretExplanation.History)
}
var secretOutput bytes.Buffer
if err := loaded.WriteExplanation(&secretOutput, secretPath.String()); err != nil {
t.Fatalf("WriteExplanation(secret) error = %v", err)
}
if strings.Contains(secretOutput.String(), "fake-parent-secret") || strings.Contains(secretOutput.String(), "fake-overlay-secret") ||
!strings.Contains(secretOutput.String(), "[REDACTED]") {
t.Fatalf("secret explanation did not redact values and history: %s", secretOutput.String())
}
childOptions := loaded.Config.Pipelines["child"].Artifacts["summary"].Extract.Options
childOptions["nested"].(map[string]any)["value"] = "changed"
parentValue := loaded.Config.Pipelines["parent"].Artifacts["summary"].Extract.Options["nested"].(map[string]any)["value"]
if parentValue != "original" {
t.Fatalf("child runtime mutation changed parent value: %#v", parentValue)
}
var frozen bytes.Buffer
if err := loaded.WriteEffective(&frozen); err != nil {
t.Fatalf("WriteEffective() after runtime mutation error = %v", err)
}
if !strings.Contains(frozen.String(), "original") || strings.Contains(frozen.String(), "changed") {
t.Fatalf("runtime mutation changed frozen inspection: %s", frozen.String())
}
}