Compare commits
16 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| aae01a59ce | |||
| 71cf084e8b | |||
| e327a5eef7 | |||
| af62b3bacc | |||
| e1097cc7a5 | |||
| 3f226c91d0 | |||
| 64509947df | |||
| 29c7c165a5 | |||
| 4ef69314e1 | |||
| 1d1434b118 | |||
| f4779fede9 | |||
| 01c08ec8e4 | |||
| d7ea0fd32d | |||
| ec99d48619 | |||
| 9e74df0b6e | |||
| 6e5e277426 |
@@ -40,6 +40,21 @@ have no effect; `NOTARIUS_CONFIG` remains a source-discovery input.
|
|||||||
[LLM profile](../examples/llm-profiles/dnd-extraction.yml) and imports its
|
[LLM profile](../examples/llm-profiles/dnd-extraction.yml) and imports its
|
||||||
pipeline from [a fragment](../examples/pipelines/dnd-session.yml).
|
pipeline from [a fragment](../examples/pipelines/dnd-session.yml).
|
||||||
|
|
||||||
|
The complete fragment also defines `dnd-session-development`, which inherits
|
||||||
|
the complete pipeline and profile while setting
|
||||||
|
`structured_output_repair_attempts: 0`. Select it with:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
notarius run dnd-session-development \
|
||||||
|
--config /path/to/dnd-complete.config.yml \
|
||||||
|
--input /path/to/transcript.json
|
||||||
|
```
|
||||||
|
|
||||||
|
See the [CLI reference](cli.md#run) for the full invocation contract. This
|
||||||
|
variant disables PromptKit structural repairs without changing configured producer
|
||||||
|
retry budgets. When using another profile, operators can instead select
|
||||||
|
`dnd-session` and set its `llm_profile` to that profile.
|
||||||
|
|
||||||
Use these complete files as starting points rather than combining the
|
Use these complete files as starting points rather than combining the
|
||||||
illustrative fragments in this reference.
|
illustrative fragments in this reference.
|
||||||
|
|
||||||
@@ -80,10 +95,12 @@ umask, setgid inheritance, and default ACLs; see [Operations](operations.md#cach
|
|||||||
|
|
||||||
Every nonempty relative configured destination or reference path resolves from
|
Every nonempty relative configured destination or reference path resolves from
|
||||||
the canonical directory of the file that supplies its surviving declaration.
|
the canonical directory of the file that supplies its surviving declaration.
|
||||||
This includes imported fragments and selected configuration overlays. Built-in
|
This includes imported fragments, selected configuration overlays, and values
|
||||||
relative destinations use the selected main configuration file's directory;
|
copied through inheritance: the copied value keeps its original declaring
|
||||||
explicit `--output-dir` and `--debug-dir` values use the invocation working
|
file. A child-supplied replacement resolves from the child's declaring file.
|
||||||
directory. Absolute values remain unchanged.
|
Built-in relative destinations use the selected main configuration file's
|
||||||
|
directory; explicit `--output-dir` and `--debug-dir` values use the invocation
|
||||||
|
working directory. Absolute values remain unchanged.
|
||||||
|
|
||||||
The root file can use ConfigKit imports to compose disjoint definitions. Imports
|
The root file can use ConfigKit imports to compose disjoint definitions. Imports
|
||||||
and selected overlays remain under the configuration root; they do not merge or
|
and selected overlays remain under the configuration root; they do not merge or
|
||||||
@@ -91,6 +108,50 @@ append a pipeline's `steps` sequence. An overlay replaces sequences as defined
|
|||||||
by ConfigKit, so changing an individual ordered binding requires supplying the
|
by ConfigKit, so changing an individual ordered binding requires supplying the
|
||||||
whole replacement sequence.
|
whole replacement sequence.
|
||||||
|
|
||||||
|
## Sibling Inheritance
|
||||||
|
|
||||||
|
ConfigKit v0.5.0 supports native `extends` composition in mappings. A directive
|
||||||
|
names exactly one immediate sibling in the same mapping; it is not a path or a
|
||||||
|
cross-scope lookup. Imports, the selected overlay, and explicit overrides are
|
||||||
|
composed before inheritance, which runs before typed decoding and normal
|
||||||
|
Notarius validation. Every definition, including a parent and definitions not
|
||||||
|
selected for a run, remains subject to ordinary validation.
|
||||||
|
|
||||||
|
Child mappings recursively overlay parent mappings. A child sequence, scalar,
|
||||||
|
or null replaces the inherited value. This makes pipeline inheritance useful
|
||||||
|
for reusing a complete pipeline while replacing whole ordered lists:
|
||||||
|
|
||||||
|
~~~yaml
|
||||||
|
pipelines:
|
||||||
|
production:
|
||||||
|
input: {module: seriatim}
|
||||||
|
llm_profile: dnd-extraction
|
||||||
|
# Other complete pipeline settings and ordered steps...
|
||||||
|
development:
|
||||||
|
extends: production
|
||||||
|
structured_output_repair_attempts: 0
|
||||||
|
~~~
|
||||||
|
|
||||||
|
Pipeline names are siblings within `pipelines`; nested mappings such as named
|
||||||
|
artifact lanes also have their own sibling namespace. Step entries are a
|
||||||
|
sequence, so a step cannot extend another step. Replacing `steps` or a validator
|
||||||
|
chain supplies the complete replacement sequence; entries are not appended or
|
||||||
|
merged by identifier.
|
||||||
|
|
||||||
|
Inherited values count as explicit input for Notarius's existing
|
||||||
|
presence-sensitive rules: zero, false, and empty-list overrides stay explicit
|
||||||
|
instead of restoring a default. Null also replaces the inherited value, but
|
||||||
|
each field's ordinary type and validation rules still apply; null is not
|
||||||
|
universally valid. Omitting `extends` preserves a relationship supplied by a
|
||||||
|
lower-priority source, while `extends: null` cancels that relationship. It
|
||||||
|
does not independently clear other inherited fields.
|
||||||
|
|
||||||
|
`extends` is now a reserved key in every input mapping, including dynamic
|
||||||
|
module options and named maps. Existing ordinary data or entry names using that
|
||||||
|
key must be renamed along with any consumer that reads them. Configuration
|
||||||
|
overlays, pipeline inheritance, and PromptKit profile `base_profile`
|
||||||
|
inheritance are separate mechanisms.
|
||||||
|
|
||||||
## PromptKit Profiles
|
## PromptKit Profiles
|
||||||
|
|
||||||
The optional **promptkit** object selects one source of profile definitions and
|
The optional **promptkit** object selects one source of profile definitions and
|
||||||
|
|||||||
@@ -2,17 +2,18 @@
|
|||||||
|
|
||||||
Notarius uses the public root package
|
Notarius uses the public root package
|
||||||
[`gitea.maximumdirect.net/eric/ConfigKit`](https://gitea.maximumdirect.net/eric/ConfigKit)
|
[`gitea.maximumdirect.net/eric/ConfigKit`](https://gitea.maximumdirect.net/eric/ConfigKit)
|
||||||
at v0.4.2 for version 5 configuration acquisition, imports, overlays, explicit
|
at v0.5.0 for version 5 configuration acquisition, imports, overlays, explicit
|
||||||
overrides, diagnostics, and safe inspection rendering.
|
overrides, native sibling inheritance, diagnostics, and safe inspection
|
||||||
|
rendering. ConfigKit composes source precedence and inheritance before typed
|
||||||
|
decoding; Notarius does not perform a second inheritance merge.
|
||||||
|
|
||||||
Notarius owns the typed document, domain defaults and validation, runtime
|
Notarius owns the typed document, domain defaults and validation, runtime
|
||||||
conversion, catalog-dependent pipeline resolution, source-aware configured
|
conversion, catalog-dependent pipeline resolution, source-aware configured
|
||||||
paths, and command discovery. It uses only ConfigKit's public path, result,
|
paths, and command discovery. It uses only ConfigKit's public path, result,
|
||||||
diagnostic, and resolution-source APIs. Upstream documentation owns general
|
diagnostic, and resolution-source APIs. Upstream documentation owns general
|
||||||
ConfigKit semantics; [Configuration](../config.md) owns Notarius's file and
|
ConfigKit semantics; [Configuration](../config.md) owns Notarius's file and
|
||||||
operator contract.
|
operator contract. See the pinned upstream [v0.5.0 release notes](https://gitea.maximumdirect.net/eric/ConfigKit/src/commit/dc09959cca4e105f49ec316c6d4ba585bd2340bd/docs/releases/v0.5.0.md)
|
||||||
|
and [configuration contract](https://gitea.maximumdirect.net/eric/ConfigKit/src/commit/dc09959cca4e105f49ec316c6d4ba585bd2340bd/docs/config.md).
|
||||||
|
|
||||||
The v0.4.2 patch updates upstream release tooling and policy/licensing
|
See the canonical [ConfigKit attribution](../oss/attribution.md#configkit) for
|
||||||
documentation without changing configuration behavior or the public API. See
|
its license and notices.
|
||||||
the canonical [ConfigKit attribution](../oss/attribution.md#configkit) for its
|
|
||||||
license and notices.
|
|
||||||
|
|||||||
@@ -15,7 +15,7 @@ runs anything.
|
|||||||
|
|
||||||
| Boundary | Inputs | Outputs | Does not own |
|
| Boundary | Inputs | Outputs | Does not own |
|
||||||
| --- | --- | --- | --- |
|
| --- | --- | --- | --- |
|
||||||
| Loading | Selected file path, profile, overrides, and invocation directory | **LoadedConfig** with ConfigKit inspection metadata and a populated **Config** | Choosing the file path or reporting a command result. |
|
| Loading | Selected file path, overlay, overrides, and invocation directory | **LoadedConfig** with ConfigKit inspection metadata and a populated **Config** | Choosing the file path or reporting a command result. |
|
||||||
| Validation | **Config** | Structural configuration errors with pipeline, lane, or binding context | Module availability, capabilities, or construction. |
|
| Validation | **Config** | Structural configuration errors with pipeline, lane, or binding context | Module availability, capabilities, or construction. |
|
||||||
| Resolution | Valid **Config**, selected pipeline and lanes, runtime reference changes, LLM override, and module catalog | **EffectiveConfig** with a **ResolvedPipeline** | Materializing reference bytes, preparing modules, execution, or filesystem state. |
|
| Resolution | Valid **Config**, selected pipeline and lanes, runtime reference changes, LLM override, and module catalog | **EffectiveConfig** with a **ResolvedPipeline** | Materializing reference bytes, preparing modules, execution, or filesystem state. |
|
||||||
| Inspection | **LoadedConfig** and an optional JSON Pointer | Redacted effective YAML or explanation text | Pipeline selection, module preparation, or provider work. |
|
| Inspection | **LoadedConfig** and an optional JSON Pointer | Redacted effective YAML or explanation text | Pipeline selection, module preparation, or provider work. |
|
||||||
@@ -28,10 +28,21 @@ at their respective [CLI](cli.md), [pipeline](pipeline.md), and
|
|||||||
|
|
||||||
## Loading And Validation
|
## Loading And Validation
|
||||||
|
|
||||||
ConfigKit acquires the selected root, imports, optional selected overlay, and
|
ConfigKit composes the selected root, imports, optional selected overlay, and
|
||||||
explicit CLI overrides in precedence order. Its document definition supplies
|
explicit CLI overrides in precedence order, then expands native sibling
|
||||||
typed defaults, source-aware path normalization, and structural validation
|
inheritance before typed decoding. The document definition supplies typed
|
||||||
before conversion to the independently owned runtime **Config**. Only
|
defaults and structural validation; Notarius's recorded resolution changes
|
||||||
|
normalize configured paths after decode. Path normalization follows the source
|
||||||
|
of the surviving value, including a value copied through inheritance, rather
|
||||||
|
than the inheritance directive's location. Inherited input remains explicit
|
||||||
|
for presence-sensitive behavior, including zero, false, and empty values;
|
||||||
|
existing field validation still governs whether the resulting values are valid.
|
||||||
|
ConfigKit's inspection history retains inheritance transfers alongside later
|
||||||
|
normalization, and effective configuration renders the normalized values used
|
||||||
|
by runtime conversion. The private typed document consumes the directive, so
|
||||||
|
no upstream inheritance type or unresolved directive reaches runtime pipeline
|
||||||
|
or module contracts. See [Configuration](../config.md#sibling-inheritance)
|
||||||
|
for the operator syntax and field rules. Only
|
||||||
`NOTARIUS_CONFIG` participates in discovery; retired field-level environment
|
`NOTARIUS_CONFIG` participates in discovery; retired field-level environment
|
||||||
variables do not participate in loading. Version 5 decoding rejects scalar
|
variables do not participate in loading. Version 5 decoding rejects scalar
|
||||||
module bindings and scalar external references. ConfigKit retains redacted,
|
module bindings and scalar external references. ConfigKit retains redacted,
|
||||||
|
|||||||
@@ -18,7 +18,7 @@ License and notices: [CC BY-SA 4.0](licenses/CC-BY-SA-4.0.txt) |
|
|||||||
|
|
||||||
## ConfigKit
|
## ConfigKit
|
||||||
|
|
||||||
Selected module: `gitea.maximumdirect.net/eric/ConfigKit` at `v0.4.2`.
|
Selected module: `gitea.maximumdirect.net/eric/ConfigKit` at `v0.5.0`.
|
||||||
|
|
||||||
Copyright (C) 2026 Eric Rakestraw
|
Copyright (C) 2026 Eric Rakestraw
|
||||||
|
|
||||||
@@ -26,10 +26,10 @@ SPDX-License-Identifier: LGPL-3.0-or-later
|
|||||||
|
|
||||||
License and notices: [LGPL 3.0](licenses/LGPL-3.0.txt) |
|
License and notices: [LGPL 3.0](licenses/LGPL-3.0.txt) |
|
||||||
[GPL 3.0](licenses/GPL-3.0.txt) |
|
[GPL 3.0](licenses/GPL-3.0.txt) |
|
||||||
[COPYRIGHT](https://gitea.maximumdirect.net/eric/ConfigKit/src/commit/67c03690181c5af8dde6be325abac241dae6732f/COPYRIGHT) |
|
[COPYRIGHT](https://gitea.maximumdirect.net/eric/ConfigKit/src/commit/dc09959cca4e105f49ec316c6d4ba585bd2340bd/COPYRIGHT) |
|
||||||
[LICENSE](https://gitea.maximumdirect.net/eric/ConfigKit/src/commit/67c03690181c5af8dde6be325abac241dae6732f/LICENSE).
|
[LICENSE](https://gitea.maximumdirect.net/eric/ConfigKit/src/commit/dc09959cca4e105f49ec316c6d4ba585bd2340bd/LICENSE).
|
||||||
|
|
||||||
[Source](https://gitea.maximumdirect.net/eric/ConfigKit/src/commit/67c03690181c5af8dde6be325abac241dae6732f).
|
[Source](https://gitea.maximumdirect.net/eric/ConfigKit/src/commit/dc09959cca4e105f49ec316c6d4ba585bd2340bd).
|
||||||
|
|
||||||
## PromptKit
|
## PromptKit
|
||||||
|
|
||||||
|
|||||||
229
docs/roadmap/configkit.md
Normal file
229
docs/roadmap/configkit.md
Normal file
@@ -0,0 +1,229 @@
|
|||||||
|
# ConfigKit 0.5 And Configuration Inheritance
|
||||||
|
|
||||||
|
## Purpose
|
||||||
|
|
||||||
|
Adopt ConfigKit v0.5.0 and make its sibling inheritance useful and supported in
|
||||||
|
Notarius configuration, particularly for deriving development or testing
|
||||||
|
pipelines from a complete D&D pipeline without duplicating the pipeline body.
|
||||||
|
|
||||||
|
ConfigKit owns source composition and inheritance. Notarius continues to own
|
||||||
|
typed configuration, domain defaults and validation, pipeline resolution, and
|
||||||
|
execution. This work does not introduce a second configuration-merging engine.
|
||||||
|
|
||||||
|
The [implementation plan](implementation.md) owns the ordered work and
|
||||||
|
validation sequence. Release preparation and publication remain separate.
|
||||||
|
|
||||||
|
## Reviewed Baseline
|
||||||
|
|
||||||
|
Reviewed on 2026-10-03. Notarius pins ConfigKit v0.4.2 and uses its ordinary
|
||||||
|
`Load` API in `internal/core/config/load.go`. Configuration version 5 supports
|
||||||
|
imports, selected configuration overlays, explicit command overrides, recorded
|
||||||
|
normalization, and redacted effective/explain output. Runtime configuration is
|
||||||
|
detached from the ConfigKit inspection snapshot.
|
||||||
|
|
||||||
|
Named pipelines and artifact lanes are mappings; pipeline `steps` and validator
|
||||||
|
chains are sequences. The complete example imports
|
||||||
|
`examples/pipelines/dnd-session.yml`, while the existing development overlay
|
||||||
|
changes operational settings. Relative configured paths are resolved from the
|
||||||
|
file supplying their surviving declaration. The current path resolver uses
|
||||||
|
ConfigKit origin/source metadata and host filesystem paths.
|
||||||
|
|
||||||
|
Target ConfigKit v0.5.0 at full source commit
|
||||||
|
`dc09959cca4e105f49ec316c6d4ba585bd2340bd`. Consult the pinned
|
||||||
|
[release guide](https://gitea.maximumdirect.net/eric/ConfigKit/src/tag/v0.5.0/docs/releases/v0.5.0.md)
|
||||||
|
and [configuration contract](https://gitea.maximumdirect.net/eric/ConfigKit/src/tag/v0.5.0/docs/config.md).
|
||||||
|
The release adds inheritance, layered loading, filesystem-backed sources,
|
||||||
|
mapping sources, and passive resource contexts. Its Go requirement and YAML
|
||||||
|
dependency are unchanged. The upgrade has not yet been trialed against
|
||||||
|
Notarius's test suite at this review baseline.
|
||||||
|
|
||||||
|
## Target End State
|
||||||
|
|
||||||
|
### Native Sibling Inheritance
|
||||||
|
|
||||||
|
Operators can define a named pipeline with `extends: parent-pipeline` and
|
||||||
|
override only the settings that differ. The parent remains an ordinary named,
|
||||||
|
valid pipeline; no abstract-template flag, template registry, or special
|
||||||
|
validation exemption is introduced. Derived pipelines use the existing run,
|
||||||
|
listing, validation, effective-configuration, and explanation commands.
|
||||||
|
|
||||||
|
Use ConfigKit's native semantics without restricting inheritance to pipelines
|
||||||
|
or implementing a Notarius-specific subset:
|
||||||
|
|
||||||
|
- `extends` selects one exact key among the mapping's immediate siblings.
|
||||||
|
It is not a path, cross-scope lookup, or multiple-parent expression.
|
||||||
|
- Source precedence is assembled before inheritance, and inheritance runs
|
||||||
|
before typed decoding and application resolution. Inherited input is explicit
|
||||||
|
for presence-sensitive defaults, including zero, false, empty, and null.
|
||||||
|
- Child mappings recursively overlay parent mappings; sequences, scalars,
|
||||||
|
and null replace inherited values. Existing typed/domain validation still
|
||||||
|
determines whether the resulting value is valid.
|
||||||
|
- Omitting `extends` retains a lower-priority relationship; `extends: null`
|
||||||
|
cancels the relationship, not arbitrary inherited fields independently.
|
||||||
|
- Root and sequence-element mappings have no sibling namespace for their own
|
||||||
|
string relationship. Nested mappings may have one. In particular, individual
|
||||||
|
entries of the `steps` sequence cannot extend other step entries.
|
||||||
|
- ConfigKit rejects missing/non-mapping parents, invalid directive types, and
|
||||||
|
cycles, and resolves effective relationships even in unselected entries.
|
||||||
|
- Parent definitions receive normal typed decoding and Notarius validation.
|
||||||
|
Inheritance is not a way to hide invalid partial pipeline definitions.
|
||||||
|
- Inherited nested relationships resolve at their destination according to
|
||||||
|
ConfigKit's rules; do not pre-expand or reinterpret them in Notarius.
|
||||||
|
|
||||||
|
Names are matched by ConfigKit before Notarius's existing identifier trimming.
|
||||||
|
Document exact-key lookup rather than introducing pre-inheritance name rewriting.
|
||||||
|
|
||||||
|
The principal documented workflow is whole-pipeline reuse. Inheritance does not
|
||||||
|
append steps, merge sequences by `id`, or allow a child to override one list
|
||||||
|
element without supplying a replacement list. The same applies to validator
|
||||||
|
chains. Do not disguise this boundary through custom list-merging code.
|
||||||
|
|
||||||
|
### Paths, Provenance, And Runtime Boundaries
|
||||||
|
|
||||||
|
An inherited relative reference or resource path remains relative to its
|
||||||
|
original declaring file, not the child's file or the invocation directory.
|
||||||
|
Imported parents, overlays, and nested inheritance must retain that behavior.
|
||||||
|
Direct child declarations use their own source context. Built-in relative
|
||||||
|
destinations and CLI-supplied paths keep their existing bases.
|
||||||
|
|
||||||
|
Review `documentPathBase` and related original-path/normalization handling
|
||||||
|
against the new origin metadata. Make narrowly required corrections if needed;
|
||||||
|
do not assume that successful typed decoding proves correct path resolution.
|
||||||
|
Effective output must agree with the runtime paths, and explanations must retain
|
||||||
|
the upstream derivation evidence through recorded Notarius normalization.
|
||||||
|
|
||||||
|
Keep the ConfigKit boundary private: no upstream inheritance objects or
|
||||||
|
unresolved directives reach runtime pipeline/module types. ConfigKit expands
|
||||||
|
relationships once during loading. Notarius's existing binding-profile,
|
||||||
|
validation-policy, and module-default resolution still operate on the resulting
|
||||||
|
configuration; they are not replaced with generic sibling inheritance.
|
||||||
|
|
||||||
|
Configuration overlays, pipeline inheritance, and PromptKit `base_profile`
|
||||||
|
inheritance remain distinct mechanisms. The latter still belongs to PromptKit.
|
||||||
|
|
||||||
|
### Compatibility And Upgrade Policy
|
||||||
|
|
||||||
|
Keep Notarius configuration `version: 5`. The typed runtime shape is unchanged;
|
||||||
|
the new capability belongs to configuration composition. No new CLI flag or
|
||||||
|
inheritance opt-in setting is needed or available upstream.
|
||||||
|
|
||||||
|
The significant compatibility change is that `extends` becomes a reserved key
|
||||||
|
in every input mapping, including dynamic module-option maps and named maps.
|
||||||
|
It is consumed before typed decoding, so it can no longer be ordinary data or
|
||||||
|
an ordinary entry name there. Audit maintained configuration/fixtures for such
|
||||||
|
usage and document that affected operator configurations must rename the key
|
||||||
|
or identifier and any associated consumer field. Do not silently strip, alias,
|
||||||
|
or bypass the directive to preserve an incompatible interpretation.
|
||||||
|
|
||||||
|
Configurations without that collision should retain their existing meaning.
|
||||||
|
Preserve discovery, import rules, overlay selection, explicit CLI override
|
||||||
|
precedence, strict decoding, redaction, defaults, and validation. Existing
|
||||||
|
configuration-stage inheritance does not alter the fixed execution pipeline,
|
||||||
|
artifact contracts, warning policy, or checkpoint identity design.
|
||||||
|
|
||||||
|
The eventual Notarius release should be a new minor version, explaining both
|
||||||
|
the feature and the newly reserved input key. Release preparation/publication
|
||||||
|
is separate from this work; no exact release number is selected here.
|
||||||
|
|
||||||
|
## Required Work
|
||||||
|
|
||||||
|
### Dependency And Configuration Integration
|
||||||
|
|
||||||
|
Upgrade `go.mod` and `go.sum` to the exact target. Keep PromptKit and unrelated
|
||||||
|
dependencies unchanged unless dependency resolution demonstrably requires it.
|
||||||
|
Continue using ordinary `configkit.Load`, the existing typed defaults factory,
|
||||||
|
and source-aware resolution/validation callbacks.
|
||||||
|
|
||||||
|
Inspect the load, operational-document, path, conversion, inspection, and CLI
|
||||||
|
boundaries. Add only integration adjustments demonstrated necessary by the
|
||||||
|
upstream change. There should be no `Extends` field added to Notarius runtime
|
||||||
|
types and no homegrown inheritance algorithm.
|
||||||
|
|
||||||
|
Update ConfigKit's factual attribution version and COPYRIGHT/LICENSE/source
|
||||||
|
links to the target commit after inspecting its actual notices. Preserve the
|
||||||
|
selected project grant and reuse unchanged local license texts.
|
||||||
|
|
||||||
|
### Maintained Example
|
||||||
|
|
||||||
|
Extend the complete example's imported pipeline definitions with a small
|
||||||
|
`dnd-session-development` sibling deriving from `dnd-session`, demonstrating a
|
||||||
|
pipeline-level setting override such as a smaller structural-repair budget.
|
||||||
|
Use already supported values and existing profile sources so the example
|
||||||
|
validates without an invented model/profile dependency. Explain that
|
||||||
|
`llm_profile` is another useful override when the operator supplies that profile.
|
||||||
|
|
||||||
|
Retain the runnable parent, minimal example, and existing operational overlay.
|
||||||
|
Do not duplicate the complete steps list or add a third standalone configuration
|
||||||
|
merely to demonstrate inheritance. Update example-loading/listing expectations
|
||||||
|
for the additional pipeline, checking inherited behavior rather than snapshots
|
||||||
|
of incidental formatting or total line counts.
|
||||||
|
|
||||||
|
### Canonical Documentation
|
||||||
|
|
||||||
|
- `docs/config.md`: own syntax, composition order, exact sibling lookup,
|
||||||
|
replacement/null semantics, path bases, reserved-key migration, and limits
|
||||||
|
on step/validator-list composition. Link to the maintained example and pinned
|
||||||
|
upstream rules rather than duplicating the entire upstream manual.
|
||||||
|
- `docs/internal/configuration.md`: describe expansion before typed decoding,
|
||||||
|
presence and source-aware normalization, and the unchanged runtime boundary.
|
||||||
|
- `docs/integrations/pkg-configkit.md`: update the supported version, public
|
||||||
|
contract links, and integration scope.
|
||||||
|
- `docs/oss/attribution.md`: update the dependency's verified source/notice links.
|
||||||
|
- `docs/cli.md` and consumer guides: update only where navigation or examples
|
||||||
|
benefit from showing selection of the derived pipeline. Do not invent new
|
||||||
|
invocation syntax or duplicate the configuration reference.
|
||||||
|
|
||||||
|
The existing ConfigKit architecture decision remains applicable. No new ADR is
|
||||||
|
required solely for consuming the library's composition feature; do not rewrite
|
||||||
|
an accepted ADR or turn new private implementation details into invariants.
|
||||||
|
|
||||||
|
## Verification And Acceptance
|
||||||
|
|
||||||
|
Use focused Notarius boundary tests, not a duplicate of ConfigKit's full suite:
|
||||||
|
|
||||||
|
- Existing maintained configurations continue loading and resolving.
|
||||||
|
- A derived pipeline retains its own identifier and inherits the complete
|
||||||
|
parent pipeline while applying its explicit overrides.
|
||||||
|
- Imports and selected overlays participate in composition before inheritance;
|
||||||
|
explicit CLI overrides keep their documented precedence.
|
||||||
|
- Representative zero/false/null inputs exercise existing presence-sensitive
|
||||||
|
defaulting or validation correctly; null is not universally valid merely
|
||||||
|
because inheritance can represent it.
|
||||||
|
- Replacing a steps or validator sequence does not concatenate or merge it.
|
||||||
|
- References inherited from a parent in another file retain their declaring
|
||||||
|
directory, while child-supplied replacements use the child's directory.
|
||||||
|
- Effective and explanation output remain redacted and expose useful source
|
||||||
|
and inheritance information after application normalization.
|
||||||
|
- Representative invalid inheritance produces ConfigKit load diagnostics through
|
||||||
|
the normal Notarius CLI error path, without a parallel validation engine.
|
||||||
|
- Maintained examples and their imported fragments validate offline without
|
||||||
|
credentials or provider calls.
|
||||||
|
|
||||||
|
Configuration and CLI boundary coverage should protect document/path/operational,
|
||||||
|
effective/explanation, and maintained-example behavior. Framework-resolution
|
||||||
|
coverage is appropriate where a distinct handoff risk requires it. Acceptance
|
||||||
|
includes the offline default suite, vet, and build with `GOWORK=off`, dependency
|
||||||
|
tidiness, and documentation diff/link checks appropriate to the changes. The
|
||||||
|
implementation plan owns their sequencing; these checks are not additional
|
||||||
|
permanent release gates.
|
||||||
|
|
||||||
|
## Deferred Capabilities And Non-Goals
|
||||||
|
|
||||||
|
- Do not adopt `LoadLayered` or add a new underlay precedence tier now. Existing
|
||||||
|
imports, overlays, and typed defaults remain the loading model. Bundled
|
||||||
|
pipeline presets or site-wide default layers need a separate operator-facing
|
||||||
|
design if selected later.
|
||||||
|
- Do not move Go defaults into embedded YAML or mapping sources solely because
|
||||||
|
v0.5.0 provides those APIs.
|
||||||
|
- Do not generalize references to arbitrary `fs.FS` resources in this work.
|
||||||
|
`SourceContext` may be used if it materially helps the required path behavior,
|
||||||
|
but its filesystem access rules must not silently replace Notarius's existing
|
||||||
|
host-path contract or introduce new path restrictions.
|
||||||
|
- No abstract templates, multiple inheritance, list-element composition,
|
||||||
|
new template schema, new pipeline scheduler, or extra configuration knobs.
|
||||||
|
- No opt-in gate or Notarius-only prohibition on other valid native inheritance
|
||||||
|
locations; use upstream semantics and existing domain validation.
|
||||||
|
|
||||||
|
There are no significant open design decisions at this baseline. Implementation
|
||||||
|
should verify the new metadata behavior and correct concrete integration gaps
|
||||||
|
without broadening the agreed feature scope.
|
||||||
528
docs/roadmap/implementation.md
Normal file
528
docs/roadmap/implementation.md
Normal file
@@ -0,0 +1,528 @@
|
|||||||
|
# ConfigKit Inheritance Implementation Plan
|
||||||
|
|
||||||
|
## Purpose And Scope
|
||||||
|
|
||||||
|
Implement [ConfigKit 0.5 And Configuration Inheritance](configkit.md). That
|
||||||
|
feature roadmap owns the intended behavior and policy choices. This document
|
||||||
|
owns the ordered implementation work. Each numbered stage is one prompt for a
|
||||||
|
Luna coding agent; complete its acceptance criteria before starting the next.
|
||||||
|
|
||||||
|
The starting baseline pins ConfigKit v0.4.2 and PromptKit v0.11.2, uses ordinary
|
||||||
|
`configkit.Load`, and accepts Notarius configuration version 5. Upgrade only
|
||||||
|
ConfigKit to v0.5.0, source commit
|
||||||
|
`dc09959cca4e105f49ec316c6d4ba585bd2340bd`. The release tag is annotated: use
|
||||||
|
`git rev-parse 'v0.5.0^{commit}'` when verifying the source commit in a checkout.
|
||||||
|
|
||||||
|
The implementation adds supported native configuration inheritance and a
|
||||||
|
maintained example. It does not require changing pipeline execution, artifact
|
||||||
|
schemas, or release tooling. Release preparation and publication remain a
|
||||||
|
separate task; this feature belongs in a future minor release.
|
||||||
|
|
||||||
|
## Working Instructions For Every Stage
|
||||||
|
|
||||||
|
- Read `docs/development.md`, `docs/policy/agents.md`, and the feature roadmap.
|
||||||
|
Use the development guide to select additional applicable policies, including
|
||||||
|
architecture, testing, documentation, and OSS for the changes being made.
|
||||||
|
- Inspect the named code and tests before editing. Prefer the repository's
|
||||||
|
code knowledge graph for code discovery. Inspect the pinned upstream release
|
||||||
|
and public API when an integration detail needs confirmation; do not assume
|
||||||
|
a sibling checkout's current branch equals the released version.
|
||||||
|
- Keep ordinary `configkit.Load`, typed defaults, and the existing private
|
||||||
|
configuration boundary. ConfigKit handles `extends` before decoding; add no
|
||||||
|
`Extends` field, second merge pass, application template registry, or opt-in
|
||||||
|
flag. Accept valid native inheritance locations under existing domain rules.
|
||||||
|
- Make production changes only where a concrete integration failure requires
|
||||||
|
them. A stage may consist of focused tests and documentation when the
|
||||||
|
existing adapter already behaves correctly. Fix a regression rather than
|
||||||
|
weakening an established expectation to accommodate the upgrade.
|
||||||
|
- Test Notarius's boundary with the real ConfigKit dependency and small,
|
||||||
|
deterministic fixtures. Reuse existing helpers and coverage; do not replicate
|
||||||
|
ConfigKit's complete test matrix, add diagnostic snapshots, or assert
|
||||||
|
incidental output formatting. Keep the default suite offline and independent
|
||||||
|
of credentials and model calls.
|
||||||
|
- Use `GOWORK=off` for Go commands so a sibling checkout cannot substitute for
|
||||||
|
the pinned modules. Update canonical documentation as each behavior lands.
|
||||||
|
Review the diff, run the stage's checks, and commit the coherent result under
|
||||||
|
the agent policy. Record any unresolved failure in the handoff rather than
|
||||||
|
marking that stage complete. Do not push or publish a release as part of this
|
||||||
|
plan.
|
||||||
|
|
||||||
|
## Stage 1 — Upgrade The Dependency And Its Notices
|
||||||
|
|
||||||
|
> Completion: complete.
|
||||||
|
|
||||||
|
Goal: establish the released dependency baseline without redesigning the
|
||||||
|
configuration adapter.
|
||||||
|
|
||||||
|
Inspect:
|
||||||
|
|
||||||
|
- `go.mod`, `go.sum`, and `internal/core/config/load.go`.
|
||||||
|
- `docs/integrations/pkg-configkit.md` and the ConfigKit entry in
|
||||||
|
`docs/oss/attribution.md`.
|
||||||
|
- ConfigKit's pinned `docs/releases/v0.5.0.md`, `go.mod`, `COPYRIGHT`, and
|
||||||
|
`LICENSE`; consult the sibling checkout with `git show v0.5.0:<path>`.
|
||||||
|
|
||||||
|
Work:
|
||||||
|
|
||||||
|
1. Run `GOWORK=off go get gitea.maximumdirect.net/eric/ConfigKit@v0.5.0`, then
|
||||||
|
`GOWORK=off go mod tidy`. Review the selected dependency graph. Retain
|
||||||
|
PromptKit v0.11.2 and unrelated pins unless module resolution requires a
|
||||||
|
change; explain any such change in the handoff.
|
||||||
|
2. Search maintained configuration, fixtures, YAML tags, and documentation for
|
||||||
|
ordinary uses of the key or entry name `extends`. For example, use
|
||||||
|
`rg -n '\bextends\b' examples internal docs assets`. Classify any matches:
|
||||||
|
new inheritance instructions are valid; ordinary field/key uses collide
|
||||||
|
with the new reserved directive. Rename actual maintained collisions and
|
||||||
|
their consumers together, without an alias or bypass. Do not rename the
|
||||||
|
unrelated PromptKit `base_profile` setting.
|
||||||
|
3. Run focused existing tests. Address compile/API compatibility failures with
|
||||||
|
the smallest necessary adapter changes. Native inheritance needs no
|
||||||
|
registration or load option.
|
||||||
|
4. Update the integration document's pin and brief description to v0.5.0;
|
||||||
|
replace the v0.4.2 patch-only explanation with the new composition boundary.
|
||||||
|
Link to pinned upstream release/configuration documentation.
|
||||||
|
5. Update only ConfigKit's attribution version and exact source/notice links
|
||||||
|
to the target commit, preserving the verified copyright and
|
||||||
|
`LGPL-3.0-or-later` expression. Reuse the existing LGPL/GPL texts when
|
||||||
|
unchanged. Do not refresh unrelated component attribution or policies.
|
||||||
|
|
||||||
|
Acceptance criteria:
|
||||||
|
|
||||||
|
- The declared and selected ConfigKit module is v0.5.0, without a workspace or
|
||||||
|
local `replace` dependency.
|
||||||
|
- Existing configuration and CLI tests pass; maintained ordinary `extends`
|
||||||
|
collisions, if any, have been resolved explicitly.
|
||||||
|
- Integration and attribution documents agree with the actual dependency and
|
||||||
|
imported source commit. The adapter still uses ordinary `Load`.
|
||||||
|
|
||||||
|
Validation:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
GOWORK=off go list -m gitea.maximumdirect.net/eric/ConfigKit
|
||||||
|
GOWORK=off go test ./internal/core/config ./internal/cli
|
||||||
|
git diff --check
|
||||||
|
```
|
||||||
|
|
||||||
|
Prompt size: one dependency upgrade, its notices, and demonstrated compatibility
|
||||||
|
adjustments. Leave new inheritance coverage to the following stages.
|
||||||
|
|
||||||
|
## Stage 2 — Prove Pipeline Composition And Runtime Conversion
|
||||||
|
|
||||||
|
> Completion: complete.
|
||||||
|
|
||||||
|
Goal: establish whole-pipeline reuse through the real configuration boundary.
|
||||||
|
|
||||||
|
Inspect `load.go`, `document.go`, `loaded_config.go`, `pipeline_document.go`,
|
||||||
|
`document_test.go`, `pipeline_document_test.go`, and
|
||||||
|
`effective_config_contract_test.go` under `internal/core/config`. Reuse the
|
||||||
|
existing test catalog for runtime resolution.
|
||||||
|
|
||||||
|
Work:
|
||||||
|
|
||||||
|
1. Add focused loading coverage in `internal/core/config/inheritance_test.go`.
|
||||||
|
Define a valid parent pipeline and a child with `extends: parent` plus a
|
||||||
|
pipeline-level override. Assert the inherited input, bindings, and ordered
|
||||||
|
steps survive conversion, while the child retains its own pipeline ID and
|
||||||
|
the parent keeps its own settings.
|
||||||
|
Keep the fixture small rather than copying the complete D&D example.
|
||||||
|
2. Resolve the resulting child with the existing catalog. Assert its selected
|
||||||
|
ID and overridden setting reach the resolved pipeline. Use the catalog's
|
||||||
|
registered module keys and avoid generated-reference requirements in this
|
||||||
|
small fixture; the full generated-reference handoff is covered in Stage 7.
|
||||||
|
Reuse existing resolution tests for unrelated defaults and digests.
|
||||||
|
3. Add one replacement case: the child supplies a complete different `steps`
|
||||||
|
list. Assert replacement and ordering, with no parent steps appended or
|
||||||
|
merged by ID. Both resulting pipelines must satisfy ordinary validation.
|
||||||
|
4. Add one native nested-mapping case using sibling artifact lanes in an
|
||||||
|
`artifacts` mapping: a child lane inherits a valid parent lane and overrides
|
||||||
|
a binding value. This proves that Notarius does not limit inheritance to
|
||||||
|
pipeline mappings. Other upstream-supported mapping shapes need no custom
|
||||||
|
permission mechanism.
|
||||||
|
5. In `docs/config.md`, add the canonical inheritance explanation: exact
|
||||||
|
immediate-sibling lookup, one parent, source composition before expansion,
|
||||||
|
recursive mapping combination, sequence/scalar/null replacement, normal
|
||||||
|
validation of parent and child, and all effective relationships being
|
||||||
|
resolved. State that step-sequence entries cannot extend each other.
|
||||||
|
Include a small illustrative pipeline snippet and the reserved `extends`
|
||||||
|
migration note. Distinguish configuration overlays and PromptKit profiles.
|
||||||
|
|
||||||
|
Acceptance criteria:
|
||||||
|
|
||||||
|
- Loading and resolving a child yields an ordinary independently identified
|
||||||
|
pipeline with the expected inherited composition and explicit override.
|
||||||
|
- Replacement lists retain only the child list, in its configured order.
|
||||||
|
- Valid nested artifact-lane inheritance works through native ConfigKit
|
||||||
|
behavior without extra runtime types or application merging.
|
||||||
|
- Configuration version remains 5 and the user reference describes the
|
||||||
|
implemented syntax and reserved-key compatibility impact.
|
||||||
|
|
||||||
|
Validation:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
GOWORK=off go test ./internal/core/config
|
||||||
|
git diff --check
|
||||||
|
```
|
||||||
|
|
||||||
|
Prompt size: small composition fixtures, runtime handoff, and one canonical
|
||||||
|
syntax section. Presence-sensitive cases belong to Stage 3.
|
||||||
|
|
||||||
|
## Stage 3 — Preserve Presence And Existing Domain Validation
|
||||||
|
|
||||||
|
> Completion: complete.
|
||||||
|
|
||||||
|
Goal: ensure inherited input remains explicit through Notarius normalization,
|
||||||
|
defaulting, and validator-chain conversion.
|
||||||
|
|
||||||
|
Inspect `operational_document.go`, `pipeline_document.go`,
|
||||||
|
`document_normalizer.go`, `loaded_config.go`, and their existing presence tests.
|
||||||
|
Extend `inheritance_test.go` rather than adding a parallel fixture framework.
|
||||||
|
|
||||||
|
Work:
|
||||||
|
|
||||||
|
1. Cover inherited `structured_output_repair_attempts: 0`: it remains a
|
||||||
|
populated zero override after loading and resolution, rather than reverting
|
||||||
|
to a pipeline or prompt default. Also cover a child explicitly overriding a
|
||||||
|
nonzero parent budget with zero.
|
||||||
|
2. Cover an inherited supported `validators: []` on a chunk or extraction
|
||||||
|
binding. Its runtime override retains `Set=true` and an empty chain. Cover
|
||||||
|
a child replacing a nonempty parent chain with `[]`; do not concatenate it
|
||||||
|
or reinstall default validators because the child list is empty.
|
||||||
|
3. Cover an inherited false module-option value, such as
|
||||||
|
`output.options.evidence_context.enabled: false`, remaining false in runtime
|
||||||
|
configuration. Inspect the decoded document's input metadata to establish
|
||||||
|
explicit presence where needed, rather than relying on truthiness.
|
||||||
|
4. Cover inherited or child-supplied null for a field already rejected when
|
||||||
|
explicit, such as `structured_output_repair_attempts`. Require the existing
|
||||||
|
contextual load failure. Inheritance's support for null does not make that
|
||||||
|
field nullable. Reuse existing direct-input null coverage.
|
||||||
|
5. Use a small overlay fixture to demonstrate the sequencing: an overlay that
|
||||||
|
changes the parent before expansion changes the child's inherited value;
|
||||||
|
a child's own explicit replacement wins over the parent's value. Verify
|
||||||
|
`extends: null` cancellation using a child with its own valid input binding,
|
||||||
|
and verify omission of the directive in an overlay preserves the base
|
||||||
|
relationship. An explicit override record for the parent value must be
|
||||||
|
reflected in an inheriting child as well.
|
||||||
|
6. If any integration defect appears, correct Notarius's existing presence or
|
||||||
|
conversion handling, using ConfigKit's metadata. Add no application
|
||||||
|
inheritance evaluator. Refine `docs/config.md` to explain explicit
|
||||||
|
zero/false/empty/null and cancellation without promising universal validity.
|
||||||
|
|
||||||
|
Acceptance criteria:
|
||||||
|
|
||||||
|
- Inherited and direct replacements retain their existing explicit-value
|
||||||
|
semantics, including zero repair budgets and empty validator overrides.
|
||||||
|
- Existing null restrictions are neither relaxed nor expanded.
|
||||||
|
- Overlay/explicit-override composition precedes inheritance, and cancellation
|
||||||
|
behaves as described by the pinned upstream contract.
|
||||||
|
|
||||||
|
Validation:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
GOWORK=off go test ./internal/core/config
|
||||||
|
git diff --check
|
||||||
|
```
|
||||||
|
|
||||||
|
Prompt size: presence and precedence at one package boundary. This stage does
|
||||||
|
not change defaults or add CLI flags for arbitrary configuration overrides.
|
||||||
|
|
||||||
|
## Stage 4 — Verify Declaration-Relative Paths Through Inheritance
|
||||||
|
|
||||||
|
> Completion: complete.
|
||||||
|
|
||||||
|
Goal: keep inherited resource paths anchored to the original declaration.
|
||||||
|
|
||||||
|
Inspect `document_paths.go`, `document_paths_test.go`, the original-path map in
|
||||||
|
`pipeline_document.go`, and sequence mutation recording in
|
||||||
|
`document_normalizer.go`. Consult ConfigKit's public `InputInfo`, `Origin`, and
|
||||||
|
`Source` contracts at v0.5.0.
|
||||||
|
|
||||||
|
Work:
|
||||||
|
|
||||||
|
1. Extend real-file tests using `t.TempDir()`: import parent and child pipeline
|
||||||
|
definitions from different subdirectories beneath one configuration root.
|
||||||
|
Give the parent pipeline a relative external reference and one reference
|
||||||
|
inside an inherited ordered step. Assert both inherited paths resolve from
|
||||||
|
the parent's file; a direct child reference replacement resolves from the
|
||||||
|
child's file. Reference files may be minimal public fixture data.
|
||||||
|
2. Add an overlay replacement case where the parent reference is declared by
|
||||||
|
the overlay and subsequently inherited. Its base must be the overlay's
|
||||||
|
directory. A nested reference-mapping inheritance case must preserve its
|
||||||
|
original contributor when copied into a derived pipeline.
|
||||||
|
3. Combine one inherited path case with whitespace-normalized pipeline or
|
||||||
|
reference keys. The existing `originalPath` mechanism must still find the
|
||||||
|
input's origin. ConfigKit exact sibling matching happens before Notarius
|
||||||
|
trims names; do not pre-normalize the inheritance lookup.
|
||||||
|
4. Include a legitimate `../` reference, retaining the current host-path
|
||||||
|
behavior. Reuse existing coverage for built-in destination bases, explicit
|
||||||
|
CLI destination bases, absolute paths, and empty cache roots unless the
|
||||||
|
upgrade reveals a distinct regression.
|
||||||
|
5. Fix `documentPathBase` or related origin handling only if these tests expose
|
||||||
|
a failure. Query the surviving value's contributor, not the `extends`
|
||||||
|
control location or an arbitrary sibling. Keep the existing host-path
|
||||||
|
resolution model; do not replace it with resource opening through
|
||||||
|
`ResourceContext.FS`, which could introduce containment restrictions.
|
||||||
|
6. Clarify inherited declaring-file behavior in `docs/config.md`. Update the
|
||||||
|
loading/path explanation in `docs/internal/configuration.md` to reflect
|
||||||
|
expansion before decoding and source-aware recorded normalization.
|
||||||
|
|
||||||
|
Acceptance criteria:
|
||||||
|
|
||||||
|
- Imported, nested, and overlaid inherited paths use the file supplying the
|
||||||
|
surviving value; direct child replacements use their own declaring file.
|
||||||
|
- Normalized identifiers and inherited ordered-step references retain correct
|
||||||
|
bases without unrecorded sequence mutation failures.
|
||||||
|
- Existing path access and destination policies are preserved. No new path
|
||||||
|
containment, existence, or resource-format restriction is introduced.
|
||||||
|
|
||||||
|
Validation:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
GOWORK=off go test ./internal/core/config
|
||||||
|
git diff --check
|
||||||
|
```
|
||||||
|
|
||||||
|
Prompt size: one path-resolution seam and focused real-file regressions.
|
||||||
|
Inspection rendering is checked separately in Stage 5.
|
||||||
|
|
||||||
|
## Stage 5 — Preserve Inspection, Derivation, And Ownership
|
||||||
|
|
||||||
|
> Completion: complete.
|
||||||
|
|
||||||
|
Goal: prove inherited configuration remains correctly inspectable after
|
||||||
|
Notarius's recorded normalization and runtime conversion.
|
||||||
|
|
||||||
|
Inspect `loaded_config.go`, `loaded_config_test.go`, `document_normalizer.go`,
|
||||||
|
`redaction_test.go`, and the new inherited path fixtures. Review upstream
|
||||||
|
metadata/history APIs rather than inventing a new provenance format.
|
||||||
|
|
||||||
|
Work:
|
||||||
|
|
||||||
|
1. Through `LoadedConfig.WriteEffective`, check a derived pipeline's expanded
|
||||||
|
values and resolved reference paths agree with its runtime configuration.
|
||||||
|
Verify consumed `extends` directives are absent from the effective typed
|
||||||
|
configuration. Assert selected values/paths, not the full YAML document.
|
||||||
|
2. Through `WriteExplanation`, inspect one inherited scalar and one inherited
|
||||||
|
path that Notarius normalizes. Assert useful parent/source and inheritance
|
||||||
|
evidence remains available alongside the final normalization result.
|
||||||
|
Use structured upstream metadata within the configuration package for
|
||||||
|
precise provenance checks, and minimal stable fragments for rendered text.
|
||||||
|
Do not require a new Notarius renderer or exact diagnostic wording.
|
||||||
|
3. Add a synthetic credential-shaped option to a parent and inherit it into a
|
||||||
|
child. Verify effective and explanation views redact its value, including
|
||||||
|
inherited/displaced history where present. Use only fake secret strings and
|
||||||
|
the existing sensitivity predicate; add no new credential rules.
|
||||||
|
4. Extend the existing ownership test: mutating a child's runtime nested map
|
||||||
|
must not mutate the parent's runtime map or the frozen inspection snapshot.
|
||||||
|
Retain the `RuntimeConfig()` detached-copy behavior without a new cloning
|
||||||
|
mechanism unless the test demonstrates a conversion defect.
|
||||||
|
5. Update `docs/internal/configuration.md` to describe inherited explicit input,
|
||||||
|
retained derivation, normalization, and the detached runtime boundary. Repair
|
||||||
|
any affected stale loading terminology, such as calling a configuration
|
||||||
|
overlay a profile. Link to `docs/config.md` for syntax and field rules.
|
||||||
|
|
||||||
|
Acceptance criteria:
|
||||||
|
|
||||||
|
- Effective paths agree with runtime paths; explanations retain useful source
|
||||||
|
and transfer evidence after normalization.
|
||||||
|
- Existing redaction applies to inherited values and history.
|
||||||
|
- Parent, child, returned copies, and inspection snapshots have the established
|
||||||
|
ownership behavior. Upstream inspection types stay private to configuration.
|
||||||
|
|
||||||
|
Validation:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
GOWORK=off go test ./internal/core/config
|
||||||
|
git diff --check
|
||||||
|
```
|
||||||
|
|
||||||
|
Prompt size: configuration inspection and ownership, with focused tests and its
|
||||||
|
internal documentation.
|
||||||
|
|
||||||
|
## Stage 6 — Exercise CLI Selection And Failure Reporting
|
||||||
|
|
||||||
|
> Completion: complete.
|
||||||
|
|
||||||
|
Goal: make derived pipelines first-class through the existing commands and
|
||||||
|
preserve the normal load-error path for invalid inheritance.
|
||||||
|
|
||||||
|
Inspect `internal/cli/configkit_commands_test.go`, `command_contract_test.go`,
|
||||||
|
and the configuration command/load helpers. Use the existing injected catalog
|
||||||
|
and fake collaborators; do not construct providers or make LLM calls.
|
||||||
|
|
||||||
|
Work:
|
||||||
|
|
||||||
|
1. Add a small valid parent/child fixture. Exercise `pipelines list` and
|
||||||
|
`config validate --pipeline child`; assert the child can be selected by its
|
||||||
|
own ID and listing remains sorted. Reuse existing command rendering tests
|
||||||
|
rather than duplicating every output mode.
|
||||||
|
2. Exercise `config effective` and `config explain` for that child to prove CLI
|
||||||
|
wiring reaches the safe upstream inspection views. Assert successful status
|
||||||
|
and relevant inherited evidence, not a whole-output snapshot.
|
||||||
|
3. Add representative invalid load cases for a missing parent, a cycle, and an
|
||||||
|
invalid directive type. Keep one invalid relationship in an unselected
|
||||||
|
pipeline to prove selecting a different valid pipeline cannot hide it.
|
||||||
|
Assert runtime-error status, no success output, and meaningful safe
|
||||||
|
configuration diagnostics. Reuse package-level diagnostics for more detailed
|
||||||
|
path checks; ConfigKit owns the full malformed-directive matrix.
|
||||||
|
4. Check exact-key behavior with a whitespace-bearing parent name: the matching
|
||||||
|
declaration works and existing identifier normalization follows expansion.
|
||||||
|
A mismatch should produce the upstream missing-parent failure. No automatic
|
||||||
|
trimming, aliases, or application-specific inheritance restrictions are
|
||||||
|
added.
|
||||||
|
5. Change production CLI code only if these tests expose a wiring defect.
|
||||||
|
`docs/cli.md` may gain a short link to configuration inheritance where useful,
|
||||||
|
but commands and flags remain the same.
|
||||||
|
|
||||||
|
Acceptance criteria:
|
||||||
|
|
||||||
|
- Derived pipelines use the ordinary selection, listing, validation, and
|
||||||
|
inspection commands.
|
||||||
|
- Invalid relationships reach the existing load-error reporting path, even
|
||||||
|
when their entries are not selected for execution.
|
||||||
|
- Read-only command tests require neither credentials nor provider work.
|
||||||
|
|
||||||
|
Validation:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
GOWORK=off go test ./internal/cli ./internal/core/config
|
||||||
|
git diff --check
|
||||||
|
```
|
||||||
|
|
||||||
|
Prompt size: command-boundary behavior and representative error propagation.
|
||||||
|
|
||||||
|
## Stage 7 — Publish And Test The Development Pipeline Example
|
||||||
|
|
||||||
|
> Completion: complete.
|
||||||
|
|
||||||
|
Goal: give operators one complete, runnable demonstration without duplicating
|
||||||
|
the full pipeline.
|
||||||
|
|
||||||
|
Inspect `examples/dnd-complete.config.yml`,
|
||||||
|
`examples/pipelines/dnd-session.yml`, `examples/overlays/development.yml`,
|
||||||
|
`internal/cli/example_contract_test.go`, and `maintainedExampleFiles` in
|
||||||
|
`internal/cli/production_contract_test.go`.
|
||||||
|
|
||||||
|
Work:
|
||||||
|
|
||||||
|
1. Add this sibling to `examples/pipelines/dnd-session.yml`:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
dnd-session-development:
|
||||||
|
extends: dnd-session
|
||||||
|
structured_output_repair_attempts: 0
|
||||||
|
```
|
||||||
|
|
||||||
|
Keep it beneath `pipelines`, beside the complete parent. Explain with a short
|
||||||
|
comment that this development variant disables PromptKit structural repairs
|
||||||
|
while inheriting the steps and configured producer-stage retry budgets.
|
||||||
|
It uses the parent's existing `dnd-extraction` profile.
|
||||||
|
2. Update the complete example's expected pipeline IDs to include the child in
|
||||||
|
sorted order. Keep the minimal example and existing operational overlay.
|
||||||
|
Do not add a third standalone configuration or duplicate the parent's steps.
|
||||||
|
3. Extend maintained-example coverage to load, resolve, and materialize external
|
||||||
|
references for both complete-example pipelines. Compare the child to the
|
||||||
|
parent for step/lane composition and generated-reference bindings, and assert
|
||||||
|
the selected zero structural-repair setting reaches eligible resolved
|
||||||
|
bindings. Do not assert equal pipeline digests: their IDs/settings differ.
|
||||||
|
4. Update `TestMaintainedExamplesValidateEffectiveProfilesOffline` to validate
|
||||||
|
every expected pipeline in each maintained example, including the new child,
|
||||||
|
with credentials absent. Retain the existing two-root-example inventory.
|
||||||
|
5. Update `docs/config.md`'s maintained-example guidance to show selection of the
|
||||||
|
variant and explain that operators may instead override `llm_profile` when
|
||||||
|
supplying another profile. Link to the inherited fragment. Add only a short
|
||||||
|
configuration link in `docs/consumers/dnd-pipeline.md` if useful; keep its
|
||||||
|
complete parent workflow and artifact instructions intact.
|
||||||
|
|
||||||
|
Acceptance criteria:
|
||||||
|
|
||||||
|
- The existing complete root imports both valid pipelines, and the derived
|
||||||
|
pipeline contains only the relationship and its intentional override.
|
||||||
|
- Both variants resolve and validate offline using the maintained profile,
|
||||||
|
external references, generated bindings, and full module catalog.
|
||||||
|
- Maintained examples and navigation accurately explain the variant without
|
||||||
|
confusing it with the operational overlay or PromptKit profile inheritance.
|
||||||
|
|
||||||
|
Validation:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
GOWORK=off go test ./internal/cli -run 'TestMaintained'
|
||||||
|
GOWORK=off go test ./internal/core/config ./internal/cli
|
||||||
|
git diff --check
|
||||||
|
```
|
||||||
|
|
||||||
|
Prompt size: one example fragment, its existing test inventory, and focused
|
||||||
|
operator guidance. No full live extraction run is needed.
|
||||||
|
|
||||||
|
## Stage 8 — Consolidate Documentation And Verify The Complete Upgrade
|
||||||
|
|
||||||
|
> Completion: complete.
|
||||||
|
|
||||||
|
Goal: finish with an internally consistent, validated upgrade ready for review.
|
||||||
|
|
||||||
|
Inspect all changed files, `docs/config.md`,
|
||||||
|
`docs/internal/configuration.md`, `docs/integrations/pkg-configkit.md`,
|
||||||
|
`docs/oss/attribution.md`, and affected CLI/consumer navigation. Compare against
|
||||||
|
the feature roadmap's acceptance criteria.
|
||||||
|
|
||||||
|
Work:
|
||||||
|
|
||||||
|
1. Confirm canonical ownership: configuration defines syntax/precedence/path
|
||||||
|
rules; internals explain the adapter; integration documentation identifies
|
||||||
|
the supported upstream contract; attribution records the imported version
|
||||||
|
and notices. Remove duplicate definitions introduced during earlier stages.
|
||||||
|
2. Check the documentation covers the reserved key in every input mapping,
|
||||||
|
exact sibling naming, normal validation of all definitions, replacement of
|
||||||
|
step/validator sequences, source-relative inherited paths, explicit values,
|
||||||
|
and cancellation. Do not describe deferred `LoadLayered`, filesystem source,
|
||||||
|
mapping source, or underlay workflows as available Notarius features.
|
||||||
|
3. Search for stale ConfigKit v0.4.2/current-pin claims in active documentation
|
||||||
|
and dependency files, ordinary reserved-key collisions, and confusing profile
|
||||||
|
terminology. Leave historical releases and archived roadmaps intact. Verify
|
||||||
|
relative document links locally and pinned external links where accessible;
|
||||||
|
no new documentation-checking tool or recurring network gate is required.
|
||||||
|
4. Run the full offline default suite, vet, and native build. Build to a
|
||||||
|
`mktemp -d` destination so no binary is left in the repository. Run
|
||||||
|
`GOWORK=off go mod tidy -diff`; an unexpected diff must be resolved and
|
||||||
|
reviewed rather than accepted silently.
|
||||||
|
5. Review the final implementation for unintended restrictions or guarantees:
|
||||||
|
no new schema version, inheritance whitelist, abstract-template exemption,
|
||||||
|
custom list merge, host-path restriction, configuration source tier,
|
||||||
|
additional CLI knobs, or upstream types leaking into runtime contracts.
|
||||||
|
6. Report the implemented behavior, successful checks, and any concrete gaps.
|
||||||
|
Keep roadmap retirement and release publication for a later request.
|
||||||
|
|
||||||
|
Acceptance criteria:
|
||||||
|
|
||||||
|
- Every required outcome in `configkit.md` is implemented and covered at the
|
||||||
|
appropriate boundary, with no unexplained scope expansion.
|
||||||
|
- Maintained examples and canonical documents agree with the selected module
|
||||||
|
and observed loading behavior. Dependency metadata is tidy.
|
||||||
|
- Full offline tests, vet, build, and relevant diff/link checks pass. Any
|
||||||
|
unavailable optional external-link verification is reported accurately.
|
||||||
|
|
||||||
|
Validation:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
GOWORK=off go test ./...
|
||||||
|
GOWORK=off go vet ./...
|
||||||
|
build_dir=$(mktemp -d)
|
||||||
|
GOWORK=off go build -o "$build_dir/notarius" ./cmd/notarius
|
||||||
|
GOWORK=off go mod tidy -diff
|
||||||
|
git diff --check
|
||||||
|
git status --short
|
||||||
|
```
|
||||||
|
|
||||||
|
These are development checks for this upgrade, not additional permanent release
|
||||||
|
gates. Do not repeat successful checks for unchanged content solely to satisfy
|
||||||
|
another handoff step; rerun the affected checks after any correction.
|
||||||
|
|
||||||
|
Prompt size: final review, small documentation consolidation, and aggregate
|
||||||
|
verification. A newly discovered substantial defect must be reported with its
|
||||||
|
evidence and required follow-up rather than folded into an unrelated refactor.
|
||||||
|
|
||||||
|
## Open Questions
|
||||||
|
|
||||||
|
None at this baseline. The dependency target, native inheritance semantics,
|
||||||
|
path policy, example override, documentation owners, and deferred features are
|
||||||
|
specified. Whether an adapter needs a small correction is determined by the
|
||||||
|
integration tests above, rather than a new product-policy decision.
|
||||||
409
docs/roadmap/promptkit.md
Normal file
409
docs/roadmap/promptkit.md
Normal file
@@ -0,0 +1,409 @@
|
|||||||
|
# PromptKit 0.12 Configuration Migration
|
||||||
|
|
||||||
|
## Purpose
|
||||||
|
|
||||||
|
Adopt PromptKit v0.12.0 after the completed ConfigKit upgrade, replacing the
|
||||||
|
removed configuration APIs and asset formats with PromptKit's ConfigKit-backed
|
||||||
|
configuration boundary. Make the new composition, inheritance, presence, and
|
||||||
|
captured-configuration capabilities useful without introducing a second loader
|
||||||
|
or changing Notarius's domain extraction and pipeline acceptance policies.
|
||||||
|
|
||||||
|
This is a feature roadmap, not an execution sequence. The operator-source and
|
||||||
|
native-overlay decisions were accepted on 2026-10-04 and are incorporated below.
|
||||||
|
A separate implementation plan will define the execution sequence. Release
|
||||||
|
preparation and publication are separate work.
|
||||||
|
|
||||||
|
## Reviewed Baseline
|
||||||
|
|
||||||
|
Reviewed on 2026-10-04 against Notarius commit
|
||||||
|
`af62b3b`, with ConfigKit v0.5.0 and PromptKit v0.11.2 selected in `go.mod`.
|
||||||
|
The remote PromptKit v0.12.0 tag resolves to full source commit
|
||||||
|
`4b8169da286917fde7875425bb32b799b12c6af3`, matching the sibling checkout.
|
||||||
|
Primary upstream references are the
|
||||||
|
[release guide](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.12.0/docs/releases/v0.12.0.md),
|
||||||
|
[consumer migration guide](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.12.0/docs/consumers/pkg-promptkit.md#migrate-from-the-previous-configuration-api),
|
||||||
|
and [format reference](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.12.0/docs/formats.md).
|
||||||
|
These were reviewed in the local checkout; the remote tag was verified directly.
|
||||||
|
|
||||||
|
The current integration has these migration-sensitive boundaries:
|
||||||
|
|
||||||
|
- `internal/framework/llm/asset_registry.go` flattens module-owned prompt,
|
||||||
|
schema, and fallback-profile filesystems and constructs `WithPromptFS`,
|
||||||
|
`WithSchemaFS`, and `WithFallbackProfileFS` options.
|
||||||
|
- `promptkit_profiles.go` shares profile-source construction between the
|
||||||
|
execution client and credential-free explicit-profile inspection. It uses
|
||||||
|
`Config.ProfileDir`, `WithProfileFile`, and the backend registration option.
|
||||||
|
The inspection engine supplies `PromptDir: "."` without preparing a prompt.
|
||||||
|
- `promptkit_client.go` constructs the engine and uses prepared execution,
|
||||||
|
semantic correction messages, structural repair overrides, observations,
|
||||||
|
direct sessions, reasoning overrides, and provider-neutral error adaptation.
|
||||||
|
- `promptkit_profile_fingerprint.go` combines a pinned catalog marker with
|
||||||
|
reread operator profile files and fallback asset bytes. The optional local
|
||||||
|
endpoint contributes a separate identity; local concurrency is excluded.
|
||||||
|
- Embedded prompt definitions throughout `assets/dnd/` and
|
||||||
|
`assets/generic/normalize/deduplication/` use top-level `id` and `version`.
|
||||||
|
`assets/dnd/profiles/dnd-extraction.yaml` and the maintained deployment file
|
||||||
|
`examples/llm-profiles/dnd-extraction.yml` use top-level `id`.
|
||||||
|
- The public profile contract says a matching operator profile completely
|
||||||
|
replaces a fallback or catalog profile. It documents `base_profile`
|
||||||
|
inheritance. These claims do not match v0.12.0's native semantics.
|
||||||
|
|
||||||
|
The current focused LLM, CLI, and module-integration tests pass with
|
||||||
|
`GOWORK=off`. This establishes the baseline only: no trial dependency upgrade,
|
||||||
|
live inference, or production model-quality evaluation has been performed.
|
||||||
|
|
||||||
|
## Upstream Changes Relevant To Notarius
|
||||||
|
|
||||||
|
PromptKit v0.12.0 is an incompatible configuration/API release:
|
||||||
|
|
||||||
|
- `Config.Sources` accepts ordered ConfigKit underlays and an optional main
|
||||||
|
input. Legacy prompt/profile directory fields and configuration-source
|
||||||
|
options are removed. Schema and injected-runtime options remain available.
|
||||||
|
- Documents use keyed `backends`, `profiles`, and `prompts` mappings, with a
|
||||||
|
second version mapping beneath each prompt ID. Embedded `id`, `version`, and
|
||||||
|
`base_profile` fields are invalid; profile inheritance uses `extends`.
|
||||||
|
- Files within one source group compose strictly, not by filename precedence.
|
||||||
|
Later groups overlay earlier groups recursively. Explicit null, zero,
|
||||||
|
false, and empty values can differ from omission; `{}` does not clear a map.
|
||||||
|
- Engine construction validates every final effective definition, including
|
||||||
|
unused definitions, and captures message content and static templates.
|
||||||
|
Ordinary execution and inspection do not reload them. Schemas, request
|
||||||
|
artifacts, credentials, and provider operations retain their runtime timing.
|
||||||
|
- `InspectConfiguration` exposes captured presence, sources, diagnostics,
|
||||||
|
explanations, and redacted effective output. These reports are not
|
||||||
|
automatically safe to publish: ordinary prompt/provider content remains data.
|
||||||
|
- Maintained catalogs move to `/v3`, both at `v3.20260930.0`.
|
||||||
|
PromptKit and Notarius already share the required ConfigKit v0.5.0 and Go
|
||||||
|
1.25.5 baseline; no additional Go-version upgrade is required by this release.
|
||||||
|
|
||||||
|
## Target End State
|
||||||
|
|
||||||
|
### One Native Configuration Boundary
|
||||||
|
|
||||||
|
PromptKit owns the effective backend/profile/prompt configuration and all
|
||||||
|
upstream composition and inheritance. Notarius selects its sources and adapts
|
||||||
|
the resulting engine to the existing provider-neutral completion contract.
|
||||||
|
ConfigKit and PromptKit types remain inside the LLM adapter's explicit
|
||||||
|
integration boundary, not in domain modules or pipeline contracts.
|
||||||
|
|
||||||
|
Retain Notarius configuration `version: 5`, `promptkit.profile_file`,
|
||||||
|
`promptkit.profile_dir`, their mutual exclusion and declaring-file-relative
|
||||||
|
path handling, `transient_retries`, and the optional conventional local backend.
|
||||||
|
Do not add replacement Notarius configuration fields merely because the
|
||||||
|
upstream constructor has changed. Document the changed external file format.
|
||||||
|
|
||||||
|
Use one shared source-construction path for execution and profile preflight.
|
||||||
|
The source precedence is:
|
||||||
|
|
||||||
|
1. PromptKit's maintained catalog layer, supplied by PromptKit itself.
|
||||||
|
2. Registered application prompt and fallback-profile assets.
|
||||||
|
3. The explicitly selected operator file or directory.
|
||||||
|
4. A programmatic `local` backend contribution when Notarius's
|
||||||
|
`promptkit.local_backend` is configured, so those explicit Notarius values
|
||||||
|
win over conflicting file declarations of those fields.
|
||||||
|
|
||||||
|
The existing selected file/directory accepts native combined `profiles`,
|
||||||
|
`backends`, and `prompts` documents. Enable explicit ConfigKit imports on these
|
||||||
|
operator inputs. Describe the familiar `profile_file` and `profile_dir` fields
|
||||||
|
as PromptKit configuration sources whose principal workflow is profile
|
||||||
|
selection; do not add a section whitelist or a custom merge language.
|
||||||
|
Domain modules remain responsible for interpreting and validating model output,
|
||||||
|
even when an operator supplies a prompt overlay.
|
||||||
|
|
||||||
|
Inputs within each group use native strict composition. Directory discovery is
|
||||||
|
recursive over YAML files and has no hidden per-file priority. No unselected
|
||||||
|
directory is scanned. Construct filesystem-backed inputs with usable resource
|
||||||
|
contexts so referenced message files retain their declaring-source base.
|
||||||
|
Do not combine host profile files with embedded content under a fabricated
|
||||||
|
common filesystem root.
|
||||||
|
|
||||||
|
Document import layouts that do not accidentally select the same definitions
|
||||||
|
through both directory discovery and imports: a file-root import layout is
|
||||||
|
often the simplest choice. Use native import/resource rules rather than
|
||||||
|
inventing deduplication, filename override order, or broader host access.
|
||||||
|
|
||||||
|
Use `FilesystemInputs` for the existing flattened embedded views,
|
||||||
|
`DirectoryInputs` or an individual ConfigKit file input for the operator
|
||||||
|
source, and `BackendSource` with the presence-aware local-backend helper for
|
||||||
|
programmatic backend configuration. Keep schemas on the existing schema
|
||||||
|
filesystem boundary. There is no need to deserialize catalog assets, register
|
||||||
|
catalog backends individually, or recreate PromptKit inheritance.
|
||||||
|
|
||||||
|
Profile preflight does not need dummy prompt directories or synthetic prompts.
|
||||||
|
Both construction paths include the same catalog, application prompt/fallback,
|
||||||
|
operator, and local-backend layers. This matters for partial operator prompt
|
||||||
|
overlays: inspecting a layer without its application base would validate a
|
||||||
|
different effective definition. Call `InspectProfile` on the constructed engine
|
||||||
|
without preparing a synthetic request. Construction still captures and
|
||||||
|
validates the effective prompt definitions; preflight does not resolve
|
||||||
|
credential values, prepare request artifacts, or contact providers.
|
||||||
|
|
||||||
|
### Keyed Assets Without Prompt Behavior Changes
|
||||||
|
|
||||||
|
Every maintained prompt declaration becomes a keyed document of this shape:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
prompts:
|
||||||
|
dnd.spells:
|
||||||
|
v1:
|
||||||
|
# Existing definition body, without embedded id/version fields.
|
||||||
|
```
|
||||||
|
|
||||||
|
Profiles become keyed entries beneath `profiles`. Preserve literal prompt IDs,
|
||||||
|
the `v1` version, logical profile IDs, selected response schemas, content paths,
|
||||||
|
message order, roles, shared fragment bytes, cache-control metadata, and
|
||||||
|
configured structural repair budgets. Do not conflate this migration with
|
||||||
|
prompt rewriting, schema changes, or durable artifact version changes.
|
||||||
|
|
||||||
|
The application-owned `dnd-extraction` fallback continues to explicitly select
|
||||||
|
`openrouter`, `openai/gpt-5.6-luna`, timeout 240, and service tier `flex`, with
|
||||||
|
reasoning effort unset. Do not replace that choice with an upstream catalog
|
||||||
|
alias or adopt unrelated model defaults. Migrate the maintained operator
|
||||||
|
example and relevant inline Go/YAML test fixtures as well as production assets.
|
||||||
|
|
||||||
|
Module ownership, ordered asset manifests, scoped mounting, and selected-asset
|
||||||
|
hashing remain intact. Check virtual filenames and source-relative
|
||||||
|
`content_file` resolution after wrapping definitions; do not move business
|
||||||
|
logic into the root `assets` package.
|
||||||
|
|
||||||
|
### Presence, Inheritance, And Operator Migration
|
||||||
|
|
||||||
|
The adopted policy is native recursive overlay, not a compatibility shim
|
||||||
|
for complete-definition replacement. Operators can override only the fields
|
||||||
|
they want. Omitted fields retain lower-layer values, and `extends` uses native
|
||||||
|
literal sibling-key lookup. Changing a backend or model does not erase
|
||||||
|
explicit inherited settings; backend-derived defaults are applied to the
|
||||||
|
final backend after inheritance.
|
||||||
|
|
||||||
|
Document intentional clearing with `null`, including reasoning/service tier,
|
||||||
|
sampling controls, credential lookup, and provider maps where supported.
|
||||||
|
Distinguish null from zero/false/blank and from an empty mapping. For example,
|
||||||
|
switching an operator's `dnd-extraction` overlay to a local model may also need
|
||||||
|
`service_tier: null` to remove the fallback's explicit `flex` setting. Do not
|
||||||
|
automatically scrub settings based on model-name heuristics.
|
||||||
|
|
||||||
|
Translate `base_profile` to `extends` in maintained examples and explain that
|
||||||
|
operators must migrate their external files. Do not retain legacy aliases or
|
||||||
|
an alternate parser. Preserve exact PromptKit keys; do not introduce a new
|
||||||
|
identifier normalization pass. Update tests that assumed whole replacement
|
||||||
|
or selection-time validation deliberately, rather than treating those
|
||||||
|
intentional upstream changes as regressions.
|
||||||
|
|
||||||
|
Native eager validation applies to final effective values, not independently
|
||||||
|
to each superseded contribution. Do not add a source-history audit or raw-layer
|
||||||
|
validation that prevents legitimate overrides from repairing lower layers.
|
||||||
|
|
||||||
|
### Native Operator Capabilities
|
||||||
|
|
||||||
|
Support the capabilities of the unified format directly, without new Notarius
|
||||||
|
configuration knobs or model-specific adaptation:
|
||||||
|
|
||||||
|
- Custom or multiple backend definitions, including endpoint, credential
|
||||||
|
environment name, provider parameters, concurrency, and queue capacity.
|
||||||
|
Operators may also overlay maintained backend settings, for example changing
|
||||||
|
OpenRouter's admission limit. Backend IDs are configuration keys, not a
|
||||||
|
closed registration list. Notarius's global scheduler still applies.
|
||||||
|
- Profile inheritance, partial overrides, and recursive provider-parameter
|
||||||
|
composition. Explicit null suppression and zero/false values follow native
|
||||||
|
field rules. Native sibling inheritance is not limited by a Notarius
|
||||||
|
profiles-only whitelist; prompt/backend mappings follow upstream rules too.
|
||||||
|
- File-configured `api_key_required`, with PromptKit's exact credential
|
||||||
|
precedence and runtime checks. In particular, true suppresses a backend's
|
||||||
|
implicit environment source; an explicitly configured profile environment
|
||||||
|
source remains usable. This is an operator choice, not a new mandatory
|
||||||
|
credential policy or a reason to require credentials during preflight.
|
||||||
|
- Deployment prompt overlays and declaring-source-relative message files,
|
||||||
|
including imported content. Prompt version selection remains caller-owned;
|
||||||
|
this upgrade does not infer a new module version from file order or add
|
||||||
|
automatic latest-version selection.
|
||||||
|
|
||||||
|
The existing `local_backend` object remains a convenient highest-priority
|
||||||
|
contribution for that conventional backend. Other backend settings remain
|
||||||
|
available through the unified operator source; do not copy every upstream field
|
||||||
|
into a parallel Notarius backend schema.
|
||||||
|
|
||||||
|
Keep the two maintained top-level configuration examples. Use the complete
|
||||||
|
example's existing PromptKit deployment file and small canonical documentation
|
||||||
|
snippets to demonstrate native inheritance/partial overrides, custom backend
|
||||||
|
configuration, imports, null clearing, and prompt editing without adding a
|
||||||
|
large unrelated example suite. Not every capability needs its own runnable
|
||||||
|
pipeline.
|
||||||
|
|
||||||
|
### Captured Execution And Checkpoint Identity
|
||||||
|
|
||||||
|
One engine represents one captured configuration for its lifetime. Editing
|
||||||
|
operator definitions or message files requires constructing a new engine;
|
||||||
|
there is no watcher or automatic hot reload in this scope. Preserve the normal
|
||||||
|
CLI lifecycle without promising that independently constructed preflight and
|
||||||
|
execution engines are one transactional filesystem snapshot.
|
||||||
|
|
||||||
|
Compute and retain checkpoint identity from the configuration actually
|
||||||
|
captured by the execution engine. Do not identify an old captured engine by
|
||||||
|
files reread after it was constructed. Identity must account for effective
|
||||||
|
profile/backend semantics and the pinned PromptKit/catalog revisions. It must
|
||||||
|
also account for operator prompt definitions and their
|
||||||
|
captured prompt content, not just `content_file` path strings.
|
||||||
|
|
||||||
|
Use public captured inspection and prompt-definition equality values where
|
||||||
|
appropriate; do not reach into PromptKit internals. `InspectPrompt.PromptHash`
|
||||||
|
includes captured content and is an opaque equality value. Inspection reports
|
||||||
|
alone do not embed referenced message-file bodies. Preserve application
|
||||||
|
module/schema fingerprints, actual-used profile provenance, and the existing
|
||||||
|
exclusion of concurrency-only local settings from semantic checkpoint identity.
|
||||||
|
A conservative identity across effective profiles is acceptable; per-selected-
|
||||||
|
profile optimization is not a prerequisite for this upgrade.
|
||||||
|
|
||||||
|
Raw sources, configuration history, prompt bodies, endpoints, and credential
|
||||||
|
values must not be added to ordinary manifests or checkpoint metadata. The
|
||||||
|
new inspection API supports both private captured identity and the explicitly
|
||||||
|
requested operator reports below. Do not automatically dump reports into run
|
||||||
|
output or debug bundles. Hash-only identity is distinct from publishing an
|
||||||
|
inspection report, and redaction is not a substitute for covering execution
|
||||||
|
semantics.
|
||||||
|
|
||||||
|
### Operator-Facing PromptKit Inspection
|
||||||
|
|
||||||
|
Provide explicit CLI operations for three complementary views:
|
||||||
|
|
||||||
|
- **Effective configuration:** write the captured, ConfigKit-redacted PromptKit
|
||||||
|
configuration, including effective keyed backends, profiles, and prompts.
|
||||||
|
Use `Engine.InspectConfiguration().WriteEffective` rather than reconstructing
|
||||||
|
YAML from Notarius's profile declarations or runtime manifests.
|
||||||
|
- **Setting explanation:** explain one canonical JSON Pointer in that captured
|
||||||
|
configuration, including source history, layer overrides, and inheritance.
|
||||||
|
Use the captured inspection's explanation API. Literal IDs containing `/`
|
||||||
|
or `~` use ordinary JSON Pointer escaping, not a dot-path interpretation.
|
||||||
|
- **Resolved profile:** inspect one explicit profile ID with `InspectProfile`
|
||||||
|
and present its backend-completed execution settings, including target,
|
||||||
|
effective reasoning/provider controls, credential environment name and
|
||||||
|
requirement, and selected backend identity. Endpoint-only profiles remain
|
||||||
|
identifiable as such. Do not conflate configured profile fields with values
|
||||||
|
derived from backend defaults.
|
||||||
|
|
||||||
|
Use the same selected Notarius configuration/overlay, registered application
|
||||||
|
assets, operator sources, and local-backend contribution as execution and
|
||||||
|
preflight. A report describes the engine captured for that invocation, not a
|
||||||
|
live view of a separate running process. Constructing the engine performs
|
||||||
|
native definition/content validation, but these commands do not need transcript
|
||||||
|
input, execute pipeline stages, prepare requests, resolve credential values,
|
||||||
|
admit provider capacity, or contact a model. Required-credential profiles remain
|
||||||
|
inspectable when the named environment variable is unset.
|
||||||
|
|
||||||
|
Keep existing Notarius `config effective` and `config explain` behavior intact;
|
||||||
|
the PromptKit view must be an explicit selection, not a replacement or an
|
||||||
|
automatic second document appended to their output. Define the concrete command
|
||||||
|
syntax in the implementation plan and maintain it in `docs/cli.md` when it
|
||||||
|
lands. Standard output contains the selected report only; ordinary diagnostics
|
||||||
|
and failures follow the existing CLI conventions. Unknown paths/profiles and
|
||||||
|
invalid pointers must be distinguishable without leaking raw configuration
|
||||||
|
through error messages.
|
||||||
|
|
||||||
|
Reports are operator-requested application data. Preserve upstream credential
|
||||||
|
redaction and Notarius's existing secret-handling boundary; include credential
|
||||||
|
variable names, never their values. Configuration explanations can contain
|
||||||
|
overridden values and inline prompt/provider content, and effective YAML is not
|
||||||
|
a transcript-free or generally sanitized report. Document these properties and
|
||||||
|
operator-owned output handling without adding permission enforcement, new
|
||||||
|
confirmation gates, automatic report persistence, or blanket content removal.
|
||||||
|
|
||||||
|
Effective configuration, backend-completed profile settings, and prepared
|
||||||
|
request details are different views. These commands do not render a prompt
|
||||||
|
with transcript inputs or return captured message-file bodies merely because
|
||||||
|
they appear in the configuration as `content_file` references. Keep complete
|
||||||
|
request/response inspection on the existing explicitly requested debug path.
|
||||||
|
|
||||||
|
### Existing Execution Policies Remain Unchanged
|
||||||
|
|
||||||
|
Preserve prepared execution, the exact latest-response semantic correction
|
||||||
|
pair, prompt-prefix caching, direct sessions, reasoning overrides, cancellation,
|
||||||
|
the shared scheduler, local/backend admission, observer accounting, error
|
||||||
|
classification, and credential redaction. Structural repairs, transient HTTP
|
||||||
|
retries, outer producer attempts, and validator execution budgets remain
|
||||||
|
separate. The existing one-repair maintained prompt default remains unchanged.
|
||||||
|
|
||||||
|
The migration does not change extraction/normalization semantics, warning
|
||||||
|
classification, artifact contracts, subprocess receipts, or reusable-state
|
||||||
|
eligibility. Construction errors may occur earlier under native eager
|
||||||
|
validation; adapt them through the existing safe adapter/CLI error boundary,
|
||||||
|
not by exposing arbitrary upstream configuration content.
|
||||||
|
|
||||||
|
## Required Work
|
||||||
|
|
||||||
|
- Update `go.mod` and `go.sum` to PromptKit v0.12.0 and its selected `/v3`
|
||||||
|
catalogs. Keep ConfigKit v0.5.0 and unrelated dependencies unless module
|
||||||
|
resolution requires a change. Remove obsolete `/v2` requirements after tidy.
|
||||||
|
- Replace removed constructor/source/backend APIs in the LLM adapter and
|
||||||
|
shared profile-preflight construction; retain schema and injected-client
|
||||||
|
options. Remove obsolete adapter helpers rather than layering a parallel
|
||||||
|
configuration system beside them.
|
||||||
|
- Migrate every registered prompt and fallback profile, the maintained
|
||||||
|
deployment profile, and affected test fixtures to keyed definitions.
|
||||||
|
- Adopt the unified source policy, explicit operator imports, presence/null
|
||||||
|
behavior, native operator capabilities, and captured
|
||||||
|
lifetime; replace live-file checkpoint fingerprinting with captured identity.
|
||||||
|
- Add explicit effective-configuration, setting-explanation, and resolved-
|
||||||
|
profile CLI inspection using the shared captured-engine construction and
|
||||||
|
public PromptKit inspection APIs. Keep upstream inspection values within
|
||||||
|
the adapter boundary and preserve ordinary Notarius configuration reports.
|
||||||
|
- Update behavioral tests at the adapter, CLI preflight, asset registry, and
|
||||||
|
representative assembled-module boundaries. Reuse real ConfigKit/PromptKit
|
||||||
|
loading and offline injected clients or local HTTP fixtures.
|
||||||
|
- Update canonical documentation when implementation lands:
|
||||||
|
`docs/config.md` owns selected sources, precedence, profile syntax and
|
||||||
|
migration; `docs/operations.md` owns deployment and reload guidance;
|
||||||
|
`docs/internal/llm.md` owns construction, capture, and checkpoint mechanics;
|
||||||
|
`docs/integrations/pkg-promptkit.md` owns the supported upstream boundary and
|
||||||
|
pinned links; `docs/cli.md` owns the new inspection operations and output/error
|
||||||
|
behavior. Add a short troubleshooting workflow in `docs/operations.md` that
|
||||||
|
links to those references. Revise consumer examples only where affected.
|
||||||
|
- Update `docs/oss/attribution.md` for PromptKit and the two `/v3` catalogs,
|
||||||
|
checking their actual notices and exact source commits. PromptKit remains
|
||||||
|
LGPL-3.0-or-later; the catalogs remain MIT at the reviewed revisions. Reuse
|
||||||
|
unchanged standard texts and preserve the project's own license grant.
|
||||||
|
|
||||||
|
Native layered overrides change an existing documented operator contract.
|
||||||
|
Add a concise Notarius ADR recording the accepted native composition and
|
||||||
|
unified-source ownership/compatibility decision rather than duplicating the
|
||||||
|
upstream manual. Existing historical ADRs need not be rewritten.
|
||||||
|
|
||||||
|
## Verification And Acceptance
|
||||||
|
|
||||||
|
Acceptance is behavioral, not an inventory of private helper names:
|
||||||
|
|
||||||
|
- All registered D&D and generic prompts construct and prepare with migrated
|
||||||
|
assets, resolving shared/module content in the correct source context.
|
||||||
|
Request identities, message content/order, schemas, and cache controls retain
|
||||||
|
their intended behavior. Do not add exact shared-prefix-length detector tests.
|
||||||
|
- Operator overrides, inheritance, explicit zero/null, duplicate definitions
|
||||||
|
within a group, local backend precedence, endpoint-only profiles, and invalid
|
||||||
|
unused final definitions follow the native configuration policy. Cover
|
||||||
|
representative imported combined definitions, custom/backend catalog
|
||||||
|
overlays, partial prompt overlays with their embedded base, and optional
|
||||||
|
versus explicitly required credentials. Credential-free preflight and
|
||||||
|
execution resolve the same source ordering and application definitions.
|
||||||
|
- Engine/file mutation tests demonstrate captured settings and content;
|
||||||
|
fingerprints stay tied to that engine and change when a new engine captures
|
||||||
|
different execution-relevant settings or operator prompt content.
|
||||||
|
- CLI inspection reports effective values, setting origins, and backend-
|
||||||
|
completed profiles from the same source policy without transcript input,
|
||||||
|
credentials, or provider calls. Cover representative inheritance/null
|
||||||
|
overrides, escaped literal IDs, required-credential inspection with an unset
|
||||||
|
variable, and invalid/unknown selectors. Verify redaction and unchanged
|
||||||
|
ordinary Notarius reports without pinning incidental report formatting.
|
||||||
|
- Structural repair, semantic feedback, sessions/reasoning, capacity failure,
|
||||||
|
transient errors, safe diagnostics, and observations retain existing
|
||||||
|
provider-neutral behavior using deterministic boundary tests.
|
||||||
|
- Maintained minimal, complete, and inherited development pipelines validate
|
||||||
|
without credentials or provider calls. External legacy profile files receive
|
||||||
|
clear migration guidance, not silent reinterpretation.
|
||||||
|
|
||||||
|
Use focused LLM, CLI, and module-integration tests during implementation, then
|
||||||
|
`GOWORK=off go test ./...`, `GOWORK=off go vet ./...`, a temporary native build,
|
||||||
|
and `GOWORK=off go mod tidy -diff` for this shared-boundary migration. Verify
|
||||||
|
selected module versions, attribution links, maintained examples, and stale
|
||||||
|
API/format claims. These are migration checks, not new permanent release gates.
|
||||||
|
Live provider testing is optional follow-up evidence, not required acceptance.
|
||||||
|
|
||||||
|
This work warrants a minor Notarius release because the accepted external
|
||||||
|
profile format and override behavior change. Keep Notarius configuration at
|
||||||
|
version 5; select the actual release number during separate release preparation.
|
||||||
108
docs/roadmap/remediation.md
Normal file
108
docs/roadmap/remediation.md
Normal file
@@ -0,0 +1,108 @@
|
|||||||
|
# ConfigKit Inheritance Review Remediation
|
||||||
|
|
||||||
|
## Review Scope And Overall Result
|
||||||
|
|
||||||
|
Reviewed on 2026-10-04, comparing implementation commits `ec99d48` through
|
||||||
|
`6450994` with the baseline `9e74df0`, the ConfigKit feature roadmap, and its
|
||||||
|
eight-stage implementation plan.
|
||||||
|
|
||||||
|
The architecture matches the intended design. ConfigKit v0.5.0 owns source
|
||||||
|
composition and native sibling inheritance before decoding. Notarius retains
|
||||||
|
its typed defaults, presence-sensitive validation, declaration-relative path
|
||||||
|
normalization, detached runtime configuration, catalog-dependent resolution,
|
||||||
|
and private redacted inspection boundary. The production configuration adapter
|
||||||
|
needed no changes; the work adds the dependency update, integration tests,
|
||||||
|
documentation, and the derived development pipeline example.
|
||||||
|
|
||||||
|
No production regression or inadvertently strengthened requirement was found.
|
||||||
|
In particular, the implementation adds no inheritance whitelist, new schema
|
||||||
|
version, custom merging, path-containment rule, template exemption, or release
|
||||||
|
gate. The newly reserved `extends` key is the expressly adopted upstream
|
||||||
|
compatibility change.
|
||||||
|
|
||||||
|
Two follow-ups were identified: a path-test blind spot and an incomplete
|
||||||
|
documented command. Completion is recorded under each finding. This document
|
||||||
|
records remediation work only; it does not authorize
|
||||||
|
unrelated runtime refactoring.
|
||||||
|
|
||||||
|
## Finding 1 — Make The Path Fixture Distinguish Declaring Directories
|
||||||
|
|
||||||
|
Priority: P2. Category: verification gap.
|
||||||
|
|
||||||
|
Status: addressed on 2026-10-04. The fixture now uses distinct parent-, child-,
|
||||||
|
and overlay-local paths for the source-base assertions and retains a separate
|
||||||
|
`../` reference. `GOWORK=off go test ./internal/core/config` passed. No
|
||||||
|
production resolver change was needed.
|
||||||
|
|
||||||
|
Evidence: `internal/core/config/inheritance_test.go:338–436`,
|
||||||
|
`TestInheritedReferencePathsKeepDeclaringSource`.
|
||||||
|
|
||||||
|
The parent, child, and overlay files live in `root/parents`, `root/children`,
|
||||||
|
and `root/overlays`. Their resource paths all use `../fixtures/<name>`, and
|
||||||
|
the expected paths all point into `root/fixtures`. Resolving one of these
|
||||||
|
paths from any of the three directories yields the same result. Thus, choosing
|
||||||
|
the child's directory for an inherited step/nested reference, or choosing the
|
||||||
|
parent's directory for an overlay replacement, could still pass this test.
|
||||||
|
|
||||||
|
`TestInheritedInspectionRetainsOriginsRedactionAndOwnership` does distinguish
|
||||||
|
the parent directory for a simple inherited `./shared.txt` reference. It does
|
||||||
|
not close the blind spot for the nested, ordered-step, direct-child replacement,
|
||||||
|
and overlay cases in the larger path fixture.
|
||||||
|
|
||||||
|
Required change:
|
||||||
|
|
||||||
|
- Revise the existing path fixture to use directory-distinguishing relative
|
||||||
|
paths for these cases, such as `./shared.txt`, `./step.txt`, and a child-local
|
||||||
|
replacement. Assert inherited paths against the parent's directory, direct
|
||||||
|
replacements against the child's directory, and overlay-supplied paths
|
||||||
|
against the overlay's directory.
|
||||||
|
- Retain a separate legitimate `../` example to protect the existing host-path
|
||||||
|
behavior. Avoid making every path converge on one shared sibling directory.
|
||||||
|
- Keep this as a focused real-file configuration test. Change production
|
||||||
|
resolution only if the improved assertions expose an actual defect.
|
||||||
|
|
||||||
|
Acceptance: choosing a parent, child, or overlay directory incorrectly produces
|
||||||
|
a different absolute path and fails the relevant assertion. Nested inheritance,
|
||||||
|
inherited ordered-step references, child replacements, and overlay replacements
|
||||||
|
still pass with the actual resolver.
|
||||||
|
|
||||||
|
Validation: `GOWORK=off go test ./internal/core/config`.
|
||||||
|
|
||||||
|
## Finding 2 — Supply Input In The Development Pipeline Command
|
||||||
|
|
||||||
|
Priority: P3. Category: documentation accuracy.
|
||||||
|
|
||||||
|
Status: addressed on 2026-10-04. The example now supplies the pipeline ID,
|
||||||
|
configuration path, and transcript input path, and links to the CLI run
|
||||||
|
reference. The flags were checked against that reference, and
|
||||||
|
`GOWORK=off go test ./internal/cli` passed without a live model request.
|
||||||
|
|
||||||
|
Evidence: `docs/config.md:43–49`, in Maintained Examples.
|
||||||
|
|
||||||
|
The new example says to select the development pipeline with
|
||||||
|
`notarius run dnd-session-development --config /path/to/dnd-complete.config.yml`.
|
||||||
|
The run command requires `--input`. Executing the equivalent command against
|
||||||
|
the maintained configuration returns `notarius: run requires --input` with CLI
|
||||||
|
exit status 2, before pipeline execution.
|
||||||
|
|
||||||
|
Required change: include a transcript input placeholder in the invocation,
|
||||||
|
for example `--input /path/to/transcript.json`, and retain the link to the CLI
|
||||||
|
reference for the full invocation contract. No CLI behavior change or new test
|
||||||
|
framework is needed.
|
||||||
|
|
||||||
|
Acceptance: the documented command supplies both required run inputs—the
|
||||||
|
pipeline ID and input path—and uses the implemented flag names. Verify against
|
||||||
|
the CLI reference and existing parser coverage without a live model request.
|
||||||
|
|
||||||
|
## Checks Completed During Review
|
||||||
|
|
||||||
|
- `GOWORK=off go test ./...` passed.
|
||||||
|
- `GOWORK=off go vet ./...` passed.
|
||||||
|
- `GOWORK=off go build` of `./cmd/notarius` to a temporary destination passed.
|
||||||
|
- `GOWORK=off go mod tidy -diff` produced no changes.
|
||||||
|
- The selected ConfigKit module is v0.5.0; PromptKit remains v0.11.2.
|
||||||
|
- The implementation diff passed whitespace checks.
|
||||||
|
- The incomplete development invocation was reproduced without provider calls.
|
||||||
|
|
||||||
|
These checks describe the review baseline. Subsequent remediation and its
|
||||||
|
focused validation are recorded under each finding.
|
||||||
@@ -162,3 +162,9 @@ pipelines:
|
|||||||
module: appendorder
|
module: appendorder
|
||||||
normalize:
|
normalize:
|
||||||
module: dnd/enemy-events
|
module: dnd/enemy-events
|
||||||
|
|
||||||
|
# Development variant: disable PromptKit structural repairs while retaining
|
||||||
|
# the inherited steps and configured producer retry budgets.
|
||||||
|
dnd-session-development:
|
||||||
|
extends: dnd-session
|
||||||
|
structured_output_repair_attempts: 0
|
||||||
|
|||||||
2
go.mod
2
go.mod
@@ -3,7 +3,7 @@ module gitea.maximumdirect.net/eric/notarius
|
|||||||
go 1.25.5
|
go 1.25.5
|
||||||
|
|
||||||
require (
|
require (
|
||||||
gitea.maximumdirect.net/eric/ConfigKit v0.4.2
|
gitea.maximumdirect.net/eric/ConfigKit v0.5.0
|
||||||
gitea.maximumdirect.net/eric/promptkit v0.11.2
|
gitea.maximumdirect.net/eric/promptkit v0.11.2
|
||||||
github.com/santhosh-tekuri/jsonschema/v6 v6.0.2
|
github.com/santhosh-tekuri/jsonschema/v6 v6.0.2
|
||||||
gopkg.in/yaml.v3 v3.0.1 // indirect
|
gopkg.in/yaml.v3 v3.0.1 // indirect
|
||||||
|
|||||||
4
go.sum
4
go.sum
@@ -1,5 +1,5 @@
|
|||||||
gitea.maximumdirect.net/eric/ConfigKit v0.4.2 h1:ZMxmv8pOnLtAx2hFTf8Z+pHKUbR89xErkBhwjr4yK24=
|
gitea.maximumdirect.net/eric/ConfigKit v0.5.0 h1:2KwG10+kApWM3avOoYA+p768xcbHUp7tQ2rqizBNpyg=
|
||||||
gitea.maximumdirect.net/eric/ConfigKit v0.4.2/go.mod h1:tYWpsrbLyu05r8+1MapWp5fN6ZrX9I8hU2xEPedYmYk=
|
gitea.maximumdirect.net/eric/ConfigKit v0.5.0/go.mod h1:tYWpsrbLyu05r8+1MapWp5fN6ZrX9I8hU2xEPedYmYk=
|
||||||
gitea.maximumdirect.net/eric/promptkit v0.11.2 h1:2RqI+KtflvSh7vLQaTAQCmMaFQscEA7EPwPazyvXRoY=
|
gitea.maximumdirect.net/eric/promptkit v0.11.2 h1:2RqI+KtflvSh7vLQaTAQCmMaFQscEA7EPwPazyvXRoY=
|
||||||
gitea.maximumdirect.net/eric/promptkit v0.11.2/go.mod h1:onm/FBPUKQ8/NSPwTEk26Ut+dVX2iXcq8vEseB+g/0E=
|
gitea.maximumdirect.net/eric/promptkit v0.11.2/go.mod h1:onm/FBPUKQ8/NSPwTEk26Ut+dVX2iXcq8vEseB+g/0E=
|
||||||
gitea.maximumdirect.net/eric/promptkit-backend-openrouter/v2 v2.20260926.0 h1:wdab+R+lgMZxzYwhbAuCrW7zdSQ1cW9ovBR6UzLmZR4=
|
gitea.maximumdirect.net/eric/promptkit-backend-openrouter/v2 v2.20260926.0 h1:wdab+R+lgMZxzYwhbAuCrW7zdSQ1cW9ovBR6UzLmZR4=
|
||||||
|
|||||||
@@ -40,6 +40,130 @@ func TestConfigKitReadOnlyCommandsLoadImportsAndOverlays(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func TestPipelineInheritanceCommandsSelectAndInspectDerivedPipelines(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
path := writeCommandConfigContent(t, `version: 5
|
||||||
|
pipelines:
|
||||||
|
" parent ":
|
||||||
|
input:
|
||||||
|
module: seriatim
|
||||||
|
artifacts:
|
||||||
|
spells:
|
||||||
|
extract:
|
||||||
|
module: dnd/spells
|
||||||
|
structured_output_repair_attempts: 2
|
||||||
|
child:
|
||||||
|
extends: " parent "
|
||||||
|
structured_output_repair_attempts: 0
|
||||||
|
`)
|
||||||
|
options := commandContractOptions(t)
|
||||||
|
|
||||||
|
var stdout, stderr bytes.Buffer
|
||||||
|
if code := RunWithOptions([]string{"pipelines", "list", "--config", path}, &stdout, &stderr, options); code != 0 || stdout.String() != "child\nparent\n" || stderr.Len() != 0 {
|
||||||
|
t.Fatalf("list = %d, %q, %q", code, stdout.String(), stderr.String())
|
||||||
|
}
|
||||||
|
|
||||||
|
stdout.Reset()
|
||||||
|
stderr.Reset()
|
||||||
|
if code := RunWithOptions([]string{"config", "validate", "--config", path, "--pipeline", "child"}, &stdout, &stderr, options); code != 0 || !strings.Contains(stdout.String(), `is valid for pipeline "child"`) || stderr.Len() != 0 {
|
||||||
|
t.Fatalf("validate child = %d, %q, %q", code, stdout.String(), stderr.String())
|
||||||
|
}
|
||||||
|
|
||||||
|
stdout.Reset()
|
||||||
|
stderr.Reset()
|
||||||
|
if code := RunWithOptions([]string{"config", "effective", "--config", path}, &stdout, &stderr, options); code != 0 || !strings.Contains(stdout.String(), "module: seriatim") || !strings.Contains(stdout.String(), "structured_output_repair_attempts: 0") || strings.Contains(stdout.String(), "extends:") || stderr.Len() != 0 {
|
||||||
|
t.Fatalf("effective = %d, %q, %q", code, stdout.String(), stderr.String())
|
||||||
|
}
|
||||||
|
|
||||||
|
stdout.Reset()
|
||||||
|
stderr.Reset()
|
||||||
|
if code := RunWithOptions([]string{"config", "explain", "--config", path, "/pipelines/child/input/module"}, &stdout, &stderr, options); code != 0 || !strings.Contains(stdout.String(), `path: "/pipelines/child/input/module"`) || !strings.Contains(stdout.String(), "parent") || stderr.Len() != 0 {
|
||||||
|
t.Fatalf("explain inherited input = %d, %q, %q", code, stdout.String(), stderr.String())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestPipelineInheritanceLoadFailuresUseRuntimeDiagnostics(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
for _, test := range []struct {
|
||||||
|
name string
|
||||||
|
content string
|
||||||
|
args []string
|
||||||
|
want string
|
||||||
|
}{
|
||||||
|
{
|
||||||
|
name: "missing parent",
|
||||||
|
content: `version: 5
|
||||||
|
pipelines:
|
||||||
|
child:
|
||||||
|
extends: absent
|
||||||
|
`,
|
||||||
|
args: []string{"config", "validate", "--pipeline", "child"},
|
||||||
|
want: "inheritance.missing",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "cycle",
|
||||||
|
content: `version: 5
|
||||||
|
pipelines:
|
||||||
|
first:
|
||||||
|
extends: second
|
||||||
|
second:
|
||||||
|
extends: first
|
||||||
|
`,
|
||||||
|
args: []string{"config", "validate", "--pipeline", "first"},
|
||||||
|
want: "cycle",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "invalid directive type",
|
||||||
|
content: `version: 5
|
||||||
|
pipelines:
|
||||||
|
child:
|
||||||
|
extends: 7
|
||||||
|
`,
|
||||||
|
args: []string{"config", "validate", "--pipeline", "child"},
|
||||||
|
want: "extends",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "unselected missing parent",
|
||||||
|
content: `version: 5
|
||||||
|
pipelines:
|
||||||
|
parent:
|
||||||
|
input:
|
||||||
|
module: seriatim
|
||||||
|
child:
|
||||||
|
extends: parent
|
||||||
|
unrelated:
|
||||||
|
extends: absent
|
||||||
|
`,
|
||||||
|
args: []string{"config", "validate", "--pipeline", "child"},
|
||||||
|
want: "inheritance.missing",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "exact sibling mismatch",
|
||||||
|
content: `version: 5
|
||||||
|
pipelines:
|
||||||
|
parent:
|
||||||
|
input:
|
||||||
|
module: seriatim
|
||||||
|
child:
|
||||||
|
extends: "parent "
|
||||||
|
`,
|
||||||
|
args: []string{"pipelines", "list"},
|
||||||
|
want: "inheritance.missing",
|
||||||
|
},
|
||||||
|
} {
|
||||||
|
t.Run(test.name, func(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
path := writeCommandConfigContent(t, test.content)
|
||||||
|
args := append([]string{}, test.args...)
|
||||||
|
args = append(args, "--config", path)
|
||||||
|
var stdout, stderr bytes.Buffer
|
||||||
|
if code := RunWithOptions(args, &stdout, &stderr, commandContractOptions(t)); code != 1 || stdout.Len() != 0 || stderr.Len() == 0 || !strings.Contains(stderr.String(), test.want) {
|
||||||
|
t.Fatalf("code=%d stdout=%q stderr=%q", code, stdout.String(), stderr.String())
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
func TestRetiredConfigProfileFlagUsesNormalUnknownFlagBehavior(t *testing.T) {
|
func TestRetiredConfigProfileFlagUsesNormalUnknownFlagBehavior(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
|
|||||||
@@ -3,6 +3,7 @@ package cli
|
|||||||
import (
|
import (
|
||||||
"context"
|
"context"
|
||||||
"encoding/json"
|
"encoding/json"
|
||||||
|
"fmt"
|
||||||
"os"
|
"os"
|
||||||
"path/filepath"
|
"path/filepath"
|
||||||
"sort"
|
"sort"
|
||||||
@@ -25,6 +26,7 @@ import (
|
|||||||
|
|
||||||
func TestMaintainedExamplesLoadResolveAndList(t *testing.T) {
|
func TestMaintainedExamplesLoadResolveAndList(t *testing.T) {
|
||||||
components := productionTestComponents(t)
|
components := productionTestComponents(t)
|
||||||
|
completePipelines := make(map[string]pipeline.ResolvedPipeline)
|
||||||
for _, example := range maintainedExampleFiles(t) {
|
for _, example := range maintainedExampleFiles(t) {
|
||||||
t.Run(example.name, func(t *testing.T) {
|
t.Run(example.name, func(t *testing.T) {
|
||||||
cfg := loadMaintainedExample(t, example.path)
|
cfg := loadMaintainedExample(t, example.path)
|
||||||
@@ -58,9 +60,45 @@ func TestMaintainedExamplesLoadResolveAndList(t *testing.T) {
|
|||||||
t.Fatalf("materialize maintained example references for %q: %v", pipelineID, err)
|
t.Fatalf("materialize maintained example references for %q: %v", pipelineID, err)
|
||||||
}
|
}
|
||||||
if example.name == "complete" {
|
if example.name == "complete" {
|
||||||
|
if pipelineID == "dnd-session-development" {
|
||||||
|
parent, found := completePipelines["dnd-session"]
|
||||||
|
if !found {
|
||||||
|
t.Fatal("complete parent pipeline was not resolved before its development variant")
|
||||||
|
}
|
||||||
|
if strings.Join(exampleStepLaneIDs(materialized), "|") != strings.Join(exampleStepLaneIDs(parent), "|") {
|
||||||
|
t.Fatalf("development variant steps and lanes = %v, want parent composition %v", exampleStepLaneIDs(materialized), exampleStepLaneIDs(parent))
|
||||||
|
}
|
||||||
|
if got, want := strings.Join(generatedReferenceBindingDescriptions(materialized), "|"), strings.Join(generatedReferenceBindingDescriptions(parent), "|"); got != want {
|
||||||
|
t.Fatalf("development generated reference bindings = %q, want parent bindings %q", got, want)
|
||||||
|
}
|
||||||
|
eligibleBindings := 0
|
||||||
|
for _, step := range materialized.Steps {
|
||||||
|
for _, lane := range step.ArtifactLanes {
|
||||||
|
for _, binding := range []struct {
|
||||||
|
module pipeline.ModuleBinding
|
||||||
|
executionClass contracts.ExecutionClass
|
||||||
|
}{
|
||||||
|
{module: lane.Extract, executionClass: lane.ExtractExecutionClass},
|
||||||
|
{module: lane.Normalize, executionClass: lane.NormalizeExecutionClass},
|
||||||
|
} {
|
||||||
|
if binding.executionClass != "llm_backed" {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
eligibleBindings++
|
||||||
|
if got := binding.module.StructuredOutputRepairAttempts; got == nil || *got != 0 {
|
||||||
|
t.Fatalf("development %s repair attempts = %v, want explicit zero", binding.module.Module, got)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if eligibleBindings == 0 {
|
||||||
|
t.Fatal("development variant had no eligible LLM bindings for repair policy")
|
||||||
|
}
|
||||||
|
}
|
||||||
if got := exampleStepLaneIDs(materialized); strings.Join(got, "|") != "describe-session:item-registry,location-registry,npc-registry,scene-descriptions|extract-events:combat-turns,item-occurrences,location-occurrences,npc-occurrences,spells|track-enemies:enemy-events" {
|
if got := exampleStepLaneIDs(materialized); strings.Join(got, "|") != "describe-session:item-registry,location-registry,npc-registry,scene-descriptions|extract-events:combat-turns,item-occurrences,location-occurrences,npc-occurrences,spells|track-enemies:enemy-events" {
|
||||||
t.Fatalf("complete example steps and lanes = %v, want the documented D&D extractor composition", got)
|
t.Fatalf("complete example steps and lanes = %v, want the documented D&D extractor composition", got)
|
||||||
}
|
}
|
||||||
|
completePipelines[pipelineID] = materialized
|
||||||
locationLane := referenceContractLane(t, materialized, "location-registry")
|
locationLane := referenceContractLane(t, materialized, "location-registry")
|
||||||
if locationLane.ArtifactKind != dnd.LocationRegistryKind || locationLane.Extract.Module != locationextract.Key || locationLane.Extract.Retries != 2 || locationLane.Merge.Module != pipeline.DefaultMergeModule || locationLane.Normalize.Module != locationnormalize.Key || locationLane.Normalize.Retries != 2 {
|
if locationLane.ArtifactKind != dnd.LocationRegistryKind || locationLane.Extract.Module != locationextract.Key || locationLane.Extract.Retries != 2 || locationLane.Merge.Module != pipeline.DefaultMergeModule || locationLane.Normalize.Module != locationnormalize.Key || locationLane.Normalize.Retries != 2 {
|
||||||
t.Fatalf("location lane = %#v, want typed registry composition", locationLane)
|
t.Fatalf("location lane = %#v, want typed registry composition", locationLane)
|
||||||
@@ -164,15 +202,19 @@ func TestMaintainedExamplesValidateEffectiveProfilesOffline(t *testing.T) {
|
|||||||
t.Setenv("OPENROUTER_API_KEY", "")
|
t.Setenv("OPENROUTER_API_KEY", "")
|
||||||
for _, example := range maintainedExampleFiles(t) {
|
for _, example := range maintainedExampleFiles(t) {
|
||||||
t.Run(example.name, func(t *testing.T) {
|
t.Run(example.name, func(t *testing.T) {
|
||||||
|
for _, pipelineID := range example.pipelineIDs {
|
||||||
|
t.Run(pipelineID, func(t *testing.T) {
|
||||||
var stdout, stderr strings.Builder
|
var stdout, stderr strings.Builder
|
||||||
code := RunWithOptions([]string{
|
code := RunWithOptions([]string{
|
||||||
"config", "validate", "--config", example.path, "--pipeline", "dnd-session",
|
"config", "validate", "--config", example.path, "--pipeline", pipelineID,
|
||||||
}, &stdout, &stderr, Options{})
|
}, &stdout, &stderr, Options{})
|
||||||
if code != 0 || stderr.Len() != 0 || !strings.Contains(stdout.String(), `valid for pipeline "dnd-session"`) {
|
if code != 0 || stderr.Len() != 0 || !strings.Contains(stdout.String(), `valid for pipeline "`+pipelineID+`"`) {
|
||||||
t.Fatalf("code=%d stdout=%q stderr=%q", code, stdout.String(), stderr.String())
|
t.Fatalf("code=%d stdout=%q stderr=%q", code, stdout.String(), stderr.String())
|
||||||
}
|
}
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func exampleStepLaneIDs(resolved pipeline.ResolvedPipeline) []string {
|
func exampleStepLaneIDs(resolved pipeline.ResolvedPipeline) []string {
|
||||||
@@ -188,6 +230,31 @@ func exampleStepLaneIDs(resolved pipeline.ResolvedPipeline) []string {
|
|||||||
return result
|
return result
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func generatedReferenceBindingDescriptions(resolved pipeline.ResolvedPipeline) []string {
|
||||||
|
var descriptions []string
|
||||||
|
for _, step := range resolved.Steps {
|
||||||
|
for _, lane := range step.ArtifactLanes {
|
||||||
|
for _, target := range []struct {
|
||||||
|
name string
|
||||||
|
target pipeline.ResolvedReferenceTarget
|
||||||
|
}{
|
||||||
|
{name: "extract", target: lane.ExtractReferences},
|
||||||
|
{name: "merge", target: lane.MergeReferences},
|
||||||
|
{name: "normalize", target: lane.NormalizeReferences},
|
||||||
|
} {
|
||||||
|
for _, binding := range target.target.Bindings {
|
||||||
|
if binding.Artifact == nil {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
descriptions = append(descriptions, fmt.Sprintf("%s/%s/%s/%s=%s/%s", step.ID, lane.ID, target.name, binding.SlotName, binding.Artifact.Step, binding.Artifact.Lane))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
sort.Strings(descriptions)
|
||||||
|
return descriptions
|
||||||
|
}
|
||||||
|
|
||||||
func TestMaintainedMinimalInvocationProducesJSONBundle(t *testing.T) {
|
func TestMaintainedMinimalInvocationProducesJSONBundle(t *testing.T) {
|
||||||
outputRoot := filepath.Join(t.TempDir(), "output")
|
outputRoot := filepath.Join(t.TempDir(), "output")
|
||||||
fake := &productionFakeLLMClient{}
|
fake := &productionFakeLLMClient{}
|
||||||
|
|||||||
@@ -824,7 +824,7 @@ func maintainedExampleFiles(t *testing.T) []maintainedExample {
|
|||||||
t.Helper()
|
t.Helper()
|
||||||
return []maintainedExample{
|
return []maintainedExample{
|
||||||
{name: "minimal", path: repositoryPath("examples", "dnd-minimal.config.yml"), transcriptPath: repositoryPath("examples", "seriatim-minimal-transcript.json"), pipelineIDs: []string{"dnd-session"}},
|
{name: "minimal", path: repositoryPath("examples", "dnd-minimal.config.yml"), transcriptPath: repositoryPath("examples", "seriatim-minimal-transcript.json"), pipelineIDs: []string{"dnd-session"}},
|
||||||
{name: "complete", path: repositoryPath("examples", "dnd-complete.config.yml"), transcriptPath: repositoryPath("examples", "dnd-complete-transcript.json"), pipelineIDs: []string{"dnd-session"}},
|
{name: "complete", path: repositoryPath("examples", "dnd-complete.config.yml"), transcriptPath: repositoryPath("examples", "dnd-complete-transcript.json"), pipelineIDs: []string{"dnd-session", "dnd-session-development"}},
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
619
internal/core/config/inheritance_test.go
Normal file
619
internal/core/config/inheritance_test.go
Normal file
@@ -0,0 +1,619 @@
|
|||||||
|
package config
|
||||||
|
|
||||||
|
import (
|
||||||
|
"bytes"
|
||||||
|
"errors"
|
||||||
|
"os"
|
||||||
|
"path/filepath"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
configkit "gitea.maximumdirect.net/eric/ConfigKit"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestPipelineInheritancePreservesCompositionAndReplacesSteps(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
tests := []struct {
|
||||||
|
name string
|
||||||
|
childSteps string
|
||||||
|
wantStepIDs []string
|
||||||
|
}{
|
||||||
|
{
|
||||||
|
name: "inherits ordered steps",
|
||||||
|
wantStepIDs: []string{"prepare", "summarize"},
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "replaces ordered steps",
|
||||||
|
childSteps: `
|
||||||
|
steps:
|
||||||
|
- id: replacement
|
||||||
|
artifacts:
|
||||||
|
summary:
|
||||||
|
extract: {module: extract}`,
|
||||||
|
wantStepIDs: []string{"replacement"},
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
for _, test := range tests {
|
||||||
|
t.Run(test.name, func(t *testing.T) {
|
||||||
|
source := `version: 5
|
||||||
|
pipelines:
|
||||||
|
parent:
|
||||||
|
input: {module: input}
|
||||||
|
structured_output_repair_attempts: 2
|
||||||
|
steps:
|
||||||
|
- id: prepare
|
||||||
|
artifacts:
|
||||||
|
score:
|
||||||
|
extract: {module: extract, retries: 2}
|
||||||
|
- id: summarize
|
||||||
|
artifacts:
|
||||||
|
notes:
|
||||||
|
extract: {module: extract}
|
||||||
|
child:
|
||||||
|
extends: parent
|
||||||
|
llm_profile: local
|
||||||
|
structured_output_repair_attempts: 0` + test.childSteps + "\n"
|
||||||
|
loaded, err := loadConfigSource(configkit.Bytes("inheritance.yml", []byte(source)), LoadOptions{})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("loadConfigSource() error = %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
parent := loaded.Config.Pipelines["parent"]
|
||||||
|
child := loaded.Config.Pipelines["child"]
|
||||||
|
if parent.Input.Module != "input" || child.Input.Module != "input" {
|
||||||
|
t.Fatalf("parent/child inputs = %#v / %#v", parent.Input, child.Input)
|
||||||
|
}
|
||||||
|
if child.LLMProfile != "local" ||
|
||||||
|
child.StructuredOutputRepairAttempts == nil || *child.StructuredOutputRepairAttempts != 0 {
|
||||||
|
t.Fatalf("child overrides = %#v", child)
|
||||||
|
}
|
||||||
|
if parent.StructuredOutputRepairAttempts == nil || *parent.StructuredOutputRepairAttempts != 2 {
|
||||||
|
t.Fatalf("parent settings changed: %#v", parent)
|
||||||
|
}
|
||||||
|
if len(child.Steps) != len(test.wantStepIDs) {
|
||||||
|
t.Fatalf("child steps = %#v, want %v", child.Steps, test.wantStepIDs)
|
||||||
|
}
|
||||||
|
for i, want := range test.wantStepIDs {
|
||||||
|
if child.Steps[i].ID != want {
|
||||||
|
t.Fatalf("child step %d = %q, want %q", i, child.Steps[i].ID, want)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if test.childSteps == "" && child.Steps[0].Artifacts["score"].Extract.Module != "extract" {
|
||||||
|
t.Fatalf("inherited first artifact = %#v", child.Steps[0].Artifacts["score"])
|
||||||
|
}
|
||||||
|
|
||||||
|
for _, id := range []string{"parent", "child"} {
|
||||||
|
resolved, err := loaded.Config.Resolve(ResolveInput{PipelineID: id, Catalog: effectiveCatalog(t)})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("Resolve(%q) error = %v", id, err)
|
||||||
|
}
|
||||||
|
if resolved.ResolvedPipeline.ID != id {
|
||||||
|
t.Fatalf("resolved pipeline ID = %q, want %q", resolved.ResolvedPipeline.ID, id)
|
||||||
|
}
|
||||||
|
wantResolvedSteps := len(test.wantStepIDs)
|
||||||
|
if id == "parent" {
|
||||||
|
wantResolvedSteps = 2
|
||||||
|
}
|
||||||
|
if len(resolved.ResolvedPipeline.Steps) != wantResolvedSteps {
|
||||||
|
t.Fatalf("resolved %q steps = %#v", id, resolved.ResolvedPipeline.Steps)
|
||||||
|
}
|
||||||
|
if id == "child" && resolved.ResolvedPipeline.Steps[0].ArtifactLanes[0].Extract.LLMProfile != "local" {
|
||||||
|
t.Fatalf("resolved child profile = %q", resolved.ResolvedPipeline.Steps[0].ArtifactLanes[0].Extract.LLMProfile)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestArtifactLaneInheritanceUsesSiblingMappingComposition(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
loaded, err := loadConfigSource(configkit.Bytes("lanes.yml", []byte(`version: 5
|
||||||
|
pipelines:
|
||||||
|
session:
|
||||||
|
input: {module: input}
|
||||||
|
artifacts:
|
||||||
|
base:
|
||||||
|
extract: {module: extract, retries: 2}
|
||||||
|
derived:
|
||||||
|
extends: base
|
||||||
|
extract: {retries: 4}
|
||||||
|
`)), LoadOptions{})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("loadConfigSource() error = %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
lanes := loaded.Config.Pipelines["session"].Artifacts
|
||||||
|
if lanes["base"].Extract.Module != "extract" || lanes["base"].Extract.Retries != 2 {
|
||||||
|
t.Fatalf("base lane = %#v", lanes["base"])
|
||||||
|
}
|
||||||
|
if lanes["derived"].Extract.Module != "extract" || lanes["derived"].Extract.Retries != 4 {
|
||||||
|
t.Fatalf("derived lane = %#v", lanes["derived"])
|
||||||
|
}
|
||||||
|
if _, err := loaded.Config.Resolve(ResolveInput{PipelineID: "session", Catalog: effectiveCatalog(t)}); err != nil {
|
||||||
|
t.Fatalf("Resolve() error = %v", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestInheritedPresenceOverridesRemainExplicit(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
loaded, err := loadConfigSource(configkit.Bytes("presence.yml", []byte(`version: 5
|
||||||
|
pipelines:
|
||||||
|
parent:
|
||||||
|
input: {module: input}
|
||||||
|
structured_output_repair_attempts: 3
|
||||||
|
chunk:
|
||||||
|
module: generic
|
||||||
|
validators: []
|
||||||
|
artifacts:
|
||||||
|
score:
|
||||||
|
extract:
|
||||||
|
module: extract
|
||||||
|
validators: [{module: configured-a}]
|
||||||
|
child:
|
||||||
|
extends: parent
|
||||||
|
structured_output_repair_attempts: 0
|
||||||
|
artifacts:
|
||||||
|
score:
|
||||||
|
extract:
|
||||||
|
validators: []
|
||||||
|
`)), LoadOptions{})
|
||||||
|
if err != nil {
|
||||||
|
var loadError *configkit.LoadError
|
||||||
|
if errors.As(err, &loadError) {
|
||||||
|
t.Fatalf("loadConfigSource() diagnostics = %#v", loadError.Diagnostics())
|
||||||
|
}
|
||||||
|
t.Fatalf("loadConfigSource() error = %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
parent := loaded.Config.Pipelines["parent"]
|
||||||
|
child := loaded.Config.Pipelines["child"]
|
||||||
|
if parent.Chunk.Module != "generic" || !parent.Chunk.Validators.Set || parent.Chunk.Validators.Validators == nil || len(parent.Chunk.Validators.Validators) != 0 {
|
||||||
|
t.Fatalf("parent empty chunk validators = %#v", parent.Chunk)
|
||||||
|
}
|
||||||
|
if !parent.Artifacts["score"].Extract.Validators.Set || len(parent.Artifacts["score"].Extract.Validators.Validators) != 1 ||
|
||||||
|
parent.Artifacts["score"].Extract.Validators.Validators[0].Module != "configured-a" {
|
||||||
|
t.Fatalf("parent extraction validators = %#v", parent.Artifacts["score"].Extract.Validators)
|
||||||
|
}
|
||||||
|
if parent.StructuredOutputRepairAttempts == nil || *parent.StructuredOutputRepairAttempts != 3 {
|
||||||
|
t.Fatalf("parent repair budget = %#v", parent.StructuredOutputRepairAttempts)
|
||||||
|
}
|
||||||
|
if child.StructuredOutputRepairAttempts == nil || *child.StructuredOutputRepairAttempts != 0 {
|
||||||
|
t.Fatalf("child zero repair budget = %#v", child.StructuredOutputRepairAttempts)
|
||||||
|
}
|
||||||
|
extract := child.Artifacts["score"].Extract
|
||||||
|
if !extract.Validators.Set || extract.Validators.Validators == nil || len(extract.Validators.Validators) != 0 {
|
||||||
|
t.Fatalf("child empty extraction validators = %#v", extract.Validators)
|
||||||
|
}
|
||||||
|
if !child.Chunk.Validators.Set || child.Chunk.Validators.Validators == nil || len(child.Chunk.Validators.Validators) != 0 {
|
||||||
|
t.Fatalf("inherited empty chunk validators = %#v", child.Chunk.Validators)
|
||||||
|
}
|
||||||
|
|
||||||
|
resolved, err := loaded.Config.Resolve(ResolveInput{PipelineID: "child", Catalog: effectiveCatalog(t)})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("Resolve(child) error = %v", err)
|
||||||
|
}
|
||||||
|
lanes := resolved.ResolvedPipeline.AllArtifactLanes()
|
||||||
|
if len(lanes) != 1 || lanes[0].Extract.StructuredOutputRepairAttempts == nil || *lanes[0].Extract.StructuredOutputRepairAttempts != 0 {
|
||||||
|
t.Fatalf("resolved child repair budget = %#v", lanes)
|
||||||
|
}
|
||||||
|
if !resolved.ResolvedPipeline.Chunk.Validators.Set || len(resolved.ResolvedPipeline.Chunk.Validators.Validators) != 0 ||
|
||||||
|
!lanes[0].Extract.Validators.Set || len(lanes[0].Extract.Validators.Validators) != 0 {
|
||||||
|
t.Fatalf("resolved empty validator overrides = chunk %#v, extraction %#v", resolved.ResolvedPipeline.Chunk.Validators, lanes[0].Extract.Validators)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestInheritedFalseOptionRetainsExplicitMetadata(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
loaded, err := loadConfigSource(configkit.Bytes("false-option.yml", []byte(`version: 5
|
||||||
|
pipelines:
|
||||||
|
parent:
|
||||||
|
input: {module: input}
|
||||||
|
output:
|
||||||
|
module: json
|
||||||
|
options:
|
||||||
|
evidence_context: {enabled: false}
|
||||||
|
child:
|
||||||
|
extends: parent
|
||||||
|
`)), LoadOptions{})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("loadConfigSource() error = %v", err)
|
||||||
|
}
|
||||||
|
options := loaded.Config.Pipelines["child"].Output.Options["evidence_context"].(map[string]any)
|
||||||
|
if enabled, ok := options["enabled"].(bool); !ok || enabled {
|
||||||
|
t.Fatalf("inherited evidence_context.enabled = %#v", options["enabled"])
|
||||||
|
}
|
||||||
|
path := configkit.Keys("pipelines", "child", "output", "options", "evidence_context", "enabled")
|
||||||
|
info, ok := loaded.result.Metadata().Lookup(path)
|
||||||
|
if !ok || !info.Explicit || info.Null || len(info.Origin.Inheritance) == 0 {
|
||||||
|
t.Fatalf("inherited false metadata = %#v, found=%v", info, ok)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestInheritedAndDirectNullRepairBudgetsRemainInvalid(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
for _, test := range []struct {
|
||||||
|
name string
|
||||||
|
source string
|
||||||
|
path string
|
||||||
|
}{
|
||||||
|
{
|
||||||
|
name: "child null",
|
||||||
|
source: `version: 5
|
||||||
|
pipelines:
|
||||||
|
parent: {input: {module: input}, structured_output_repair_attempts: 2}
|
||||||
|
child:
|
||||||
|
extends: parent
|
||||||
|
structured_output_repair_attempts: null
|
||||||
|
`,
|
||||||
|
path: "/pipelines/child/structured_output_repair_attempts",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "inherited null",
|
||||||
|
source: `version: 5
|
||||||
|
pipelines:
|
||||||
|
parent: {input: {module: input}, structured_output_repair_attempts: null}
|
||||||
|
child: {extends: parent}
|
||||||
|
`,
|
||||||
|
path: "/pipelines/parent/structured_output_repair_attempts",
|
||||||
|
},
|
||||||
|
} {
|
||||||
|
t.Run(test.name, func(t *testing.T) {
|
||||||
|
_, err := loadConfigSource(configkit.Bytes("null.yml", []byte(test.source)), LoadOptions{})
|
||||||
|
assertDocumentDiagnosticAt(t, err, test.path)
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestOverlayAndOverridesComposeBeforeInheritance(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
root := t.TempDir()
|
||||||
|
overlays := filepath.Join(root, "overlays")
|
||||||
|
if err := os.MkdirAll(overlays, 0o700); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
main := filepath.Join(root, "config.yml")
|
||||||
|
if err := os.WriteFile(main, []byte(`version: 5
|
||||||
|
pipelines:
|
||||||
|
parent:
|
||||||
|
input: {module: input}
|
||||||
|
structured_output_repair_attempts: 1
|
||||||
|
inherited:
|
||||||
|
extends: parent
|
||||||
|
overridden:
|
||||||
|
extends: parent
|
||||||
|
structured_output_repair_attempts: 0
|
||||||
|
cancelled:
|
||||||
|
extends: parent
|
||||||
|
input: {module: input}
|
||||||
|
`), 0o600); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err := os.WriteFile(filepath.Join(overlays, "development.yml"), []byte(`pipelines:
|
||||||
|
parent:
|
||||||
|
structured_output_repair_attempts: 2
|
||||||
|
cancelled:
|
||||||
|
extends: null
|
||||||
|
`), 0o600); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
|
||||||
|
loaded, err := Load(main, LoadOptions{Overlay: "development"})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("Load() error = %v", err)
|
||||||
|
}
|
||||||
|
if got := *loaded.Config.Pipelines["inherited"].StructuredOutputRepairAttempts; got != 2 {
|
||||||
|
t.Fatalf("overlay-inherited repair budget = %d, want 2", got)
|
||||||
|
}
|
||||||
|
if got := *loaded.Config.Pipelines["overridden"].StructuredOutputRepairAttempts; got != 0 {
|
||||||
|
t.Fatalf("child replacement repair budget = %d, want 0", got)
|
||||||
|
}
|
||||||
|
if loaded.Config.Pipelines["cancelled"].StructuredOutputRepairAttempts != nil {
|
||||||
|
t.Fatalf("cancelled child inherited parent settings: %#v", loaded.Config.Pipelines["cancelled"])
|
||||||
|
}
|
||||||
|
|
||||||
|
withOverride, err := Load(main, LoadOptions{
|
||||||
|
Overlay: "development",
|
||||||
|
Overrides: []configkit.Override{{
|
||||||
|
Path: configkit.Keys("pipelines", "parent", "structured_output_repair_attempts"),
|
||||||
|
Value: 3,
|
||||||
|
Flag: "--repair-budget",
|
||||||
|
}},
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("Load() with explicit override error = %v", err)
|
||||||
|
}
|
||||||
|
if got := *withOverride.Config.Pipelines["inherited"].StructuredOutputRepairAttempts; got != 3 {
|
||||||
|
t.Fatalf("override-inherited repair budget = %d, want 3", got)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestInheritedReferencePathsKeepDeclaringSource(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
root := t.TempDir()
|
||||||
|
parentDirectory := filepath.Join(root, "parents")
|
||||||
|
childDirectory := filepath.Join(root, "children")
|
||||||
|
overlayDirectory := filepath.Join(root, "overlays")
|
||||||
|
fixtureDirectory := filepath.Join(root, "fixtures")
|
||||||
|
for _, directory := range []string{parentDirectory, childDirectory, overlayDirectory, fixtureDirectory} {
|
||||||
|
if err := os.MkdirAll(directory, 0o700); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
for _, fixture := range []struct{ directory, name string }{
|
||||||
|
{parentDirectory, "shared.txt"},
|
||||||
|
{parentDirectory, "parent.txt"},
|
||||||
|
{parentDirectory, "step.txt"},
|
||||||
|
{childDirectory, "child.txt"},
|
||||||
|
{overlayDirectory, "overlay.txt"},
|
||||||
|
{fixtureDirectory, "outside.txt"},
|
||||||
|
} {
|
||||||
|
if err := os.WriteFile(filepath.Join(fixture.directory, fixture.name), []byte("fixture\n"), 0o600); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
parent := filepath.Join(parentDirectory, "pipelines.yml")
|
||||||
|
if err := os.WriteFile(parent, []byte(`pipelines:
|
||||||
|
" parent ":
|
||||||
|
input: {module: input}
|
||||||
|
references:
|
||||||
|
" shared ": {path: ./shared.txt}
|
||||||
|
nested:
|
||||||
|
extends: " shared "
|
||||||
|
source: {path: ./parent.txt}
|
||||||
|
outside: {path: ../fixtures/outside.txt}
|
||||||
|
steps:
|
||||||
|
- id: first
|
||||||
|
references:
|
||||||
|
step-source: {path: ./step.txt}
|
||||||
|
artifacts:
|
||||||
|
summary:
|
||||||
|
extract: {module: extract}
|
||||||
|
`), 0o600); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
child := filepath.Join(childDirectory, "pipelines.yml")
|
||||||
|
if err := os.WriteFile(child, []byte(`pipelines:
|
||||||
|
child:
|
||||||
|
extends: " parent "
|
||||||
|
references:
|
||||||
|
source: {path: ./child.txt}
|
||||||
|
`), 0o600); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
main := filepath.Join(root, "notarius.yml")
|
||||||
|
if err := os.WriteFile(main, []byte(`version: 5
|
||||||
|
configkit:
|
||||||
|
imports:
|
||||||
|
- path: parents/pipelines.yml
|
||||||
|
- path: children/pipelines.yml
|
||||||
|
`), 0o600); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err := os.WriteFile(filepath.Join(overlayDirectory, "development.yml"), []byte(`pipelines:
|
||||||
|
" parent ":
|
||||||
|
references:
|
||||||
|
" shared ": {path: ./overlay.txt}
|
||||||
|
`), 0o600); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
|
||||||
|
loaded, err := loadConfig(main, LoadOptions{})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("loadConfig() error = %v", err)
|
||||||
|
}
|
||||||
|
assertReferencePath := func(pipelineID, referenceID, want string) {
|
||||||
|
t.Helper()
|
||||||
|
if got := loaded.Config.Pipelines[pipelineID].References[referenceID].Path; got != want {
|
||||||
|
t.Errorf("%s reference %s path = %q, want %q", pipelineID, referenceID, got, want)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
assertReferencePath("parent", "shared", filepath.Join(parentDirectory, "shared.txt"))
|
||||||
|
assertReferencePath("child", "shared", filepath.Join(parentDirectory, "shared.txt"))
|
||||||
|
assertReferencePath("child", "nested", filepath.Join(parentDirectory, "shared.txt"))
|
||||||
|
assertReferencePath("parent", "source", filepath.Join(parentDirectory, "parent.txt"))
|
||||||
|
assertReferencePath("child", "source", filepath.Join(childDirectory, "child.txt"))
|
||||||
|
// Keep ../ coverage separate from paths that distinguish the declaring file.
|
||||||
|
assertReferencePath("child", "outside", filepath.Join(fixtureDirectory, "outside.txt"))
|
||||||
|
if got := loaded.Config.Pipelines["child"].Steps[0].References["step-source"].Path; got != filepath.Join(parentDirectory, "step.txt") {
|
||||||
|
t.Errorf("inherited step reference path = %q", got)
|
||||||
|
}
|
||||||
|
|
||||||
|
overlaid, err := loadConfig(main, LoadOptions{Overlay: "development"})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("loadConfig() with overlay error = %v", err)
|
||||||
|
}
|
||||||
|
for _, pipelineID := range []string{"parent", "child"} {
|
||||||
|
for _, referenceID := range []string{"shared", "nested"} {
|
||||||
|
if got, want := overlaid.Config.Pipelines[pipelineID].References[referenceID].Path, filepath.Join(overlayDirectory, "overlay.txt"); got != want {
|
||||||
|
t.Errorf("overlaid %s reference %s path = %q, want %q", pipelineID, referenceID, got, want)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if got, want := overlaid.Config.Pipelines["child"].References["source"].Path, filepath.Join(childDirectory, "child.txt"); got != want {
|
||||||
|
t.Errorf("child replacement path = %q, want %q", got, want)
|
||||||
|
}
|
||||||
|
if got, want := overlaid.Config.Pipelines["child"].Steps[0].References["step-source"].Path, filepath.Join(parentDirectory, "step.txt"); got != want {
|
||||||
|
t.Errorf("overlaid inherited step reference path = %q, want %q", got, want)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestInheritedInspectionRetainsOriginsRedactionAndOwnership(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
root := t.TempDir()
|
||||||
|
parentDirectory := filepath.Join(root, "parents")
|
||||||
|
childDirectory := filepath.Join(root, "children")
|
||||||
|
overlayDirectory := filepath.Join(root, "overlays")
|
||||||
|
for _, directory := range []string{parentDirectory, childDirectory, overlayDirectory} {
|
||||||
|
if err := os.MkdirAll(directory, 0o700); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
parentFile := filepath.Join(parentDirectory, "pipelines.yml")
|
||||||
|
if err := os.WriteFile(parentFile, []byte(`pipelines:
|
||||||
|
parent:
|
||||||
|
input: {module: input}
|
||||||
|
structured_output_repair_attempts: 2
|
||||||
|
references:
|
||||||
|
source: {path: ./shared.txt}
|
||||||
|
artifacts:
|
||||||
|
summary:
|
||||||
|
extract:
|
||||||
|
module: extract
|
||||||
|
options:
|
||||||
|
nested: {value: original}
|
||||||
|
credentials: {api_key: fake-parent-secret}
|
||||||
|
`), 0o600); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err := os.WriteFile(filepath.Join(parentDirectory, "shared.txt"), []byte("source\n"), 0o600); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
childFile := filepath.Join(childDirectory, "pipelines.yml")
|
||||||
|
if err := os.WriteFile(childFile, []byte(`pipelines:
|
||||||
|
child:
|
||||||
|
extends: parent
|
||||||
|
structured_output_repair_attempts: 0
|
||||||
|
`), 0o600); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
main := filepath.Join(root, "notarius.yml")
|
||||||
|
if err := os.WriteFile(main, []byte(`version: 5
|
||||||
|
configkit:
|
||||||
|
imports:
|
||||||
|
- path: parents/pipelines.yml
|
||||||
|
- path: children/pipelines.yml
|
||||||
|
`), 0o600); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err := os.WriteFile(filepath.Join(overlayDirectory, "development.yml"), []byte(`pipelines:
|
||||||
|
parent:
|
||||||
|
artifacts:
|
||||||
|
summary:
|
||||||
|
extract:
|
||||||
|
options:
|
||||||
|
credentials: {api_key: fake-overlay-secret}
|
||||||
|
`), 0o600); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
|
||||||
|
loaded, err := Load(main, LoadOptions{Overlay: "development"})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("Load() error = %v", err)
|
||||||
|
}
|
||||||
|
child := loaded.Config.Pipelines["child"]
|
||||||
|
wantPath := filepath.Join(parentDirectory, "shared.txt")
|
||||||
|
if got := child.References["source"].Path; got != wantPath {
|
||||||
|
t.Fatalf("runtime inherited path = %q, want %q", got, wantPath)
|
||||||
|
}
|
||||||
|
if child.StructuredOutputRepairAttempts == nil || *child.StructuredOutputRepairAttempts != 0 {
|
||||||
|
t.Fatalf("runtime inherited repair setting = %#v", child.StructuredOutputRepairAttempts)
|
||||||
|
}
|
||||||
|
|
||||||
|
var effective bytes.Buffer
|
||||||
|
if err := loaded.WriteEffective(&effective); err != nil {
|
||||||
|
t.Fatalf("WriteEffective() error = %v", err)
|
||||||
|
}
|
||||||
|
if !strings.Contains(effective.String(), wantPath) || strings.Contains(effective.String(), "extends:") ||
|
||||||
|
!strings.Contains(effective.String(), "structured_output_repair_attempts: 0") {
|
||||||
|
t.Fatalf("effective configuration is inconsistent with runtime conversion: %s", effective.String())
|
||||||
|
}
|
||||||
|
for _, secret := range []string{"fake-parent-secret", "fake-overlay-secret"} {
|
||||||
|
if strings.Contains(effective.String(), secret) {
|
||||||
|
t.Fatalf("effective configuration exposed %q", secret)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
for _, test := range []struct {
|
||||||
|
name string
|
||||||
|
path configkit.Path
|
||||||
|
}{
|
||||||
|
{name: "repair setting", path: configkit.Keys("pipelines", "child", "structured_output_repair_attempts")},
|
||||||
|
{name: "normalized reference", path: configkit.Keys("pipelines", "child", "references", "source", "path")},
|
||||||
|
} {
|
||||||
|
t.Run(test.name, func(t *testing.T) {
|
||||||
|
explanation, ok := loaded.result.Explain(test.path)
|
||||||
|
if !ok || len(explanation.History) == 0 {
|
||||||
|
t.Fatalf("inherited explanation = %#v, found=%v", explanation, ok)
|
||||||
|
}
|
||||||
|
inheritanceFromParent := false
|
||||||
|
for _, change := range explanation.History {
|
||||||
|
if len(change.Origin.Inheritance) == 0 {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
for _, location := range change.Origin.Locations {
|
||||||
|
for _, source := range loaded.result.Sources() {
|
||||||
|
if source.ID == location.SourceID && source.Path == parentFile {
|
||||||
|
inheritanceFromParent = true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if !inheritanceFromParent {
|
||||||
|
t.Fatalf("inheritance history lacks parent source evidence: %#v", explanation.History)
|
||||||
|
}
|
||||||
|
var rendered bytes.Buffer
|
||||||
|
if err := loaded.WriteExplanation(&rendered, test.path.String()); err != nil {
|
||||||
|
t.Fatalf("WriteExplanation() error = %v", err)
|
||||||
|
}
|
||||||
|
if !strings.Contains(rendered.String(), filepath.ToSlash(filepath.Join("parents", "pipelines.yml"))) {
|
||||||
|
t.Fatalf("explanation lacks parent source evidence: %s", rendered.String())
|
||||||
|
}
|
||||||
|
if test.name == "normalized reference" {
|
||||||
|
foundNormalization := false
|
||||||
|
for _, change := range explanation.History {
|
||||||
|
if change.Origin.Operation == configkit.ChangeNormalization {
|
||||||
|
foundNormalization = true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if !foundNormalization || !strings.Contains(rendered.String(), wantPath) {
|
||||||
|
t.Fatalf("normalized path evidence = %#v; rendered %s", explanation.History, rendered.String())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
secretPath := configkit.Keys("pipelines", "child", "artifacts", "summary", "extract", "options", "credentials", "api_key")
|
||||||
|
secretExplanation, ok := loaded.result.Explain(secretPath)
|
||||||
|
if !ok || !secretExplanation.Effective.Redacted {
|
||||||
|
t.Fatalf("inherited secret explanation = %#v, found=%v", secretExplanation, ok)
|
||||||
|
}
|
||||||
|
redactedHistory := false
|
||||||
|
for _, change := range secretExplanation.History {
|
||||||
|
redactedHistory = redactedHistory || change.Before.Redacted || change.After.Redacted
|
||||||
|
}
|
||||||
|
if !redactedHistory {
|
||||||
|
t.Fatalf("secret history was not redacted: %#v", secretExplanation.History)
|
||||||
|
}
|
||||||
|
var secretOutput bytes.Buffer
|
||||||
|
if err := loaded.WriteExplanation(&secretOutput, secretPath.String()); err != nil {
|
||||||
|
t.Fatalf("WriteExplanation(secret) error = %v", err)
|
||||||
|
}
|
||||||
|
if strings.Contains(secretOutput.String(), "fake-parent-secret") || strings.Contains(secretOutput.String(), "fake-overlay-secret") ||
|
||||||
|
!strings.Contains(secretOutput.String(), "[REDACTED]") {
|
||||||
|
t.Fatalf("secret explanation did not redact values and history: %s", secretOutput.String())
|
||||||
|
}
|
||||||
|
|
||||||
|
childOptions := loaded.Config.Pipelines["child"].Artifacts["summary"].Extract.Options
|
||||||
|
childOptions["nested"].(map[string]any)["value"] = "changed"
|
||||||
|
parentValue := loaded.Config.Pipelines["parent"].Artifacts["summary"].Extract.Options["nested"].(map[string]any)["value"]
|
||||||
|
if parentValue != "original" {
|
||||||
|
t.Fatalf("child runtime mutation changed parent value: %#v", parentValue)
|
||||||
|
}
|
||||||
|
var frozen bytes.Buffer
|
||||||
|
if err := loaded.WriteEffective(&frozen); err != nil {
|
||||||
|
t.Fatalf("WriteEffective() after runtime mutation error = %v", err)
|
||||||
|
}
|
||||||
|
if !strings.Contains(frozen.String(), "original") || strings.Contains(frozen.String(), "changed") {
|
||||||
|
t.Fatalf("runtime mutation changed frozen inspection: %s", frozen.String())
|
||||||
|
}
|
||||||
|
}
|
||||||
Reference in New Issue
Block a user