eric/notarius
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Add ConfigKit version 5 document boundary

Browse Source
This commit is contained in:
Eric Rakestraw
2026-09-13 16:18:30 +00:00
parent c403a38b50
commit b822e7f510
7 changed files with 398 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

54
docs/adr/0018-adopt-configkit.md Normal file
View File

@@ -0,0 +1,54 @@
# ADR-0018: Adopt ConfigKit for configuration loading
**Status:** Accepted
**Date:** 2026-09-13
## Context
Notarius's configuration boundary currently owns strict YAML parsing, input
presence bookkeeping, defaults, normalization, environment overlays, and
configuration inspection. Those general-purpose responsibilities have grown
alongside application-specific pipeline and module semantics.
ConfigKit v0.3.0 provides strict typed YAML loading, imports, configuration
profiles, explicit CLI overrides, recorded defaults and normalization, safe
inspection rendering, and released source-origin APIs. The application needs
those source records to resolve filesystem paths according to the file that
supplied a surviving declaration.
## Decision
Notarius will adopt the root public `gitea.maximumdirect.net/eric/ConfigKit`
package at v0.3.0 for its version 5 configuration contract. ConfigKit owns
source composition, strict decoding, input presence, metadata, and safe
effective/explanation rendering. Notarius retains its typed document schema,
field semantics, runtime conversion, catalog-dependent pipeline resolution,
PromptKit profile handling, and physical run preparation.
Version 5 is a deliberate breaking file-contract change. It uses canonical
object module bindings and reference sources, and enables ConfigKit imports
and configuration profiles. The selected configuration file's canonical
directory is the base for its declared application paths; ConfigKit's released
`Resolution.InputInfo` and `Resolution.Source` APIs provide their origins.
`NOTARIUS_CONFIG` remains file discovery, while field-level operational
environment overlays are retired. Explicit configuration-valued CLI flags are
represented as ConfigKit overrides; invocation actions remain outside that
boundary.
## Alternatives considered
- Continue extending the custom YAML loader. This preserves duplicated
general-purpose parsing, presence, composition, and inspection machinery.
- Retain version 4 alongside version 5. This would leave a supported dual
configuration boundary and delay removal of the machinery being replaced.
- Wrap ConfigKit in a compatibility pre-parser. This weakens ConfigKit's
strict contract and recreates the application-owned parser complexity.
## Consequences
The implementation migrates configuration behavior in bounded increments, but
the released application will have one ConfigKit-backed loader and one version
5 contract. Operator documentation and maintained examples must move with the
cutover. ConfigKit metadata is retained only at the configuration publication
boundary; its types do not become module or runtime pipeline interfaces.

2
docs/roadmap/implementation.md
View File

@@ -148,6 +148,8 @@ semantics for arbitrary dynamic module options.
## Stage 1 — Add ConfigKit And The Version 5 Document Types
**Status:** Completed 2026-09-13.
**Goal:** Establish the new file contract without switching production commands.
**Work:**

2
go.mod
View File

@@ -3,6 +3,7 @@ module gitea.maximumdirect.net/eric/notarius
go 1.25.5
require (
gitea.maximumdirect.net/eric/ConfigKit v0.3.0
gitea.maximumdirect.net/eric/promptkit v0.9.0
github.com/santhosh-tekuri/jsonschema/v6 v6.0.2
gopkg.in/yaml.v3 v3.0.1
@@ -13,4 +14,5 @@ require golang.org/x/text v0.40.0
require (
gitea.maximumdirect.net/eric/promptkit-backend-openrouter v1.0.0 // indirect
gitea.maximumdirect.net/eric/promptkit-backend-rakestrawhome v1.0.0 // indirect
go.yaml.in/yaml/v4 v4.0.0-rc.6 // indirect
)

4
go.sum
View File

@@ -1,3 +1,5 @@
gitea.maximumdirect.net/eric/ConfigKit v0.3.0 h1:LgXEUjR0KsgwGBznOWa8FJTQ+iElctHtI1via5vKdu4=
gitea.maximumdirect.net/eric/ConfigKit v0.3.0/go.mod h1:tYWpsrbLyu05r8+1MapWp5fN6ZrX9I8hU2xEPedYmYk=
gitea.maximumdirect.net/eric/promptkit v0.9.0 h1:IpvDRC8L6xRxQ9hpuyKOmMc5b6MeLTKYyx+h1YAjy08=
gitea.maximumdirect.net/eric/promptkit v0.9.0/go.mod h1:oMJ/WUJImUtwJ5e+6MAGECPYAErAkOaKel0G+3T/b4E=
gitea.maximumdirect.net/eric/promptkit-backend-openrouter v1.0.0 h1:lc062euk2qseO//D762i3JaFyulDNML3eQQX7DkYTho=
@@ -8,6 +10,8 @@ github.com/dlclark/regexp2 v1.11.0 h1:G/nrcoOa7ZXlpoa/91N3X7mM3r8eIlMBBJZvsz/mxK
github.com/dlclark/regexp2 v1.11.0/go.mod h1:DHkYz0B9wPfa6wondMfaivmHpzrQ3v9q8cnmRbL6yW8=
github.com/santhosh-tekuri/jsonschema/v6 v6.0.2 h1:KRzFb2m7YtdldCEkzs6KqmJw4nqEVZGK7IN2kJkjTuQ=
github.com/santhosh-tekuri/jsonschema/v6 v6.0.2/go.mod h1:JXeL+ps8p7/KNMjDQk3TCwPpBy0wYklyWTfbkIzdIFU=
go.yaml.in/yaml/v4 v4.0.0-rc.6 h1:1h7H1ohdUh93/FyE4YaDa1Zh64K6VVbjF4K6WUxMtH4=
go.yaml.in/yaml/v4 v4.0.0-rc.6/go.mod h1:aZqd9kCMsGL7AuUv/m/PvWLdg5sjJsZ4oHDEnfPPfY0=
golang.org/x/text v0.40.0 h1:Ub2Z6/xjgF1WrYQz2nuITOEegKFtiIy+rieRJ5lHZKs=
golang.org/x/text v0.40.0/go.mod h1:hpnzDAfGV753zIKo+wk3u1bVKCGPbrnF7+7LBF/UHVY=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405 h1:yhCVgyC4o1eVCa2tZl7eS0r+SDo693bJlVdllGtEeKM=

122
internal/core/config/document.go Normal file
View File

@@ -0,0 +1,122 @@
package config
import "gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
// Document is the version 5 configuration shape decoded by ConfigKit. It is
// deliberately separate from Config so YAML presence and composition concerns
// do not reach runtime pipeline types.
type Document struct {
Version string `yaml:"version"`
PromptKit *PromptKitDocument `yaml:"promptkit,omitempty"`
Pipelines map[string]PipelineDocument `yaml:"pipelines,omitempty"`
Concurrency *ConcurrencyDocument `yaml:"concurrency,omitempty"`
Output *OutputDocument `yaml:"output,omitempty"`
Cache *CacheDocument `yaml:"cache,omitempty"`
Debug *DebugDocument `yaml:"debug,omitempty"`
}
type PromptKitDocument struct {
ProfileDir *string `yaml:"profile_dir,omitempty"`
ProfileFile *string `yaml:"profile_file,omitempty"`
LocalBackend *PromptKitLocalBackendDocument `yaml:"local_backend,omitempty"`
}
type PromptKitLocalBackendDocument struct {
Endpoint *string `yaml:"endpoint,omitempty"`
ConcurrencyLimit *int `yaml:"concurrency_limit,omitempty"`
}
type PipelineDocument struct {
LLMProfile *string `yaml:"llm_profile,omitempty"`
StructuredOutputRepairAttempts *int `yaml:"structured_output_repair_attempts,omitempty"`
ValidationPolicy *ValidationPolicyDocument `yaml:"validation_policy,omitempty"`
Input BindingDocument `yaml:"input"`
Chunk *BindingDocument `yaml:"chunk,omitempty"`
Artifacts map[string]ArtifactLaneDocument `yaml:"artifacts,omitempty"`
Steps []PipelineStepDocument `yaml:"steps,omitempty"`
Output *BindingDocument `yaml:"output,omitempty"`
References map[string]ReferenceDocument `yaml:"references,omitempty"`
}
type PipelineStepDocument struct {
ID string `yaml:"id"`
Artifacts map[string]ArtifactLaneDocument `yaml:"artifacts"`
References map[string]ReferenceDocument `yaml:"references,omitempty"`
}
type ArtifactLaneDocument struct {
Extract BindingDocument `yaml:"extract"`
Merge *BindingDocument `yaml:"merge,omitempty"`
Normalize *BindingDocument `yaml:"normalize,omitempty"`
Validators []ValidatorDocument `yaml:"validators,omitempty"`
References map[string]ReferenceDocument `yaml:"references,omitempty"`
}
// BindingDocument represents a producer binding. ValidatorDocument is a
// deliberately finite subset so nested validator-only fields are rejected by
// ConfigKit's strict decoder.
type BindingDocument struct {
Module string `yaml:"module"`
LLMProfile *string `yaml:"llm_profile,omitempty"`
StructuredOutputRepairAttempts *int `yaml:"structured_output_repair_attempts,omitempty"`
ValidationPolicy *ValidationPolicyDocument `yaml:"validation_policy,omitempty"`
Retries int `yaml:"retries,omitempty"`
Options map[string]any `yaml:"options,omitempty"`
References map[string]ReferenceDocument `yaml:"references,omitempty"`
Validators []ValidatorDocument `yaml:"validators,omitempty"`
}
type ValidatorDocument struct {
Module string `yaml:"module"`
LLMProfile *string `yaml:"llm_profile,omitempty"`
StructuredOutputRepairAttempts *int `yaml:"structured_output_repair_attempts,omitempty"`
Retries int `yaml:"retries,omitempty"`
Options map[string]any `yaml:"options,omitempty"`
}
// ReferenceDocument is the explicit version 5 union. Later document
// validation establishes which locations permit generated references and
// requires exactly one populated branch.
type ReferenceDocument struct {
Path *string `yaml:"path,omitempty"`
Artifact *ArtifactReferenceDocument `yaml:"artifact,omitempty"`
}
type ArtifactReferenceDocument struct {
Step string `yaml:"step"`
Lane string `yaml:"lane"`
}
type ValidationPolicyDocument struct {
ProducerStructuralFailure *pipeline.ProducerStructuralFailureAction `yaml:"producer_structural_failure,omitempty"`
SemanticRejection *pipeline.SemanticRejectionAction `yaml:"semantic_rejection,omitempty"`
ValidatorFailure *pipeline.ValidatorFailureAction `yaml:"validator_failure,omitempty"`
}
type ConcurrencyDocument struct {
TotalLLM *int `yaml:"total_llm,omitempty"`
StageWorkers map[string]int `yaml:"stage_workers,omitempty"`
}
type OutputDocument struct {
Directory *string `yaml:"directory,omitempty"`
}
type CacheDocument struct {
ChunkPlans *ChunkPlanCacheDocument `yaml:"chunk_plans,omitempty"`
Checkpoints *CheckpointCacheDocument `yaml:"checkpoints,omitempty"`
}
type ChunkPlanCacheDocument struct {
Directory *string `yaml:"directory,omitempty"`
Mode *string `yaml:"mode,omitempty"`
}
type CheckpointCacheDocument struct {
Enabled *bool `yaml:"enabled,omitempty"`
Directory *string `yaml:"directory,omitempty"`
}
type DebugDocument struct {
Directory *string `yaml:"directory,omitempty"`
}

154
internal/core/config/document_test.go Normal file
View File

@@ -0,0 +1,154 @@
package config
import (
"errors"
"strings"
"testing"
configkit "gitea.maximumdirect.net/eric/ConfigKit"
)
func TestDocumentLoadAcceptsVersion5AndCanonicalShapes(t *testing.T) {
t.Parallel()
for _, source := range []string{
`version: 5
pipelines:
session:
input:
module: seriatim
references:
party:
path: ./party.txt
steps:
- id: describe
artifacts:
spells:
extract:
module: dnd/spells
validators:
- module: generic/valid_json
options:
strict: true
- id: use-artifact
references:
spells:
artifact:
step: describe
lane: spells
artifacts:
locations:
extract:
module: dnd/locations
`,
`version: "5"
pipelines:
session:
input:
module: seriatim
`,
} {
loaded, err := loadDocumentSource(configkit.Bytes("document.yml", []byte(source)), LoadOptions{})
if err != nil {
t.Fatalf("loadDocumentSource() error = %v", err)
}
profile, ok := loaded.Config.Pipelines["session"]
if !ok || profile.Input.Module != "seriatim" {
t.Fatalf("pipeline = %#v", profile)
}
}
}
func TestDocumentLoadRejectsUnsupportedVersionsBeforeDocumentDecoding(t *testing.T) {
t.Parallel()
tests := []struct {
name string
source string
code configkit.Code
}{
{name: "missing", source: "unknown: value\n", code: configkit.CodeVersionError},
{name: "null", source: "version: null\nunknown: value\n", code: configkit.CodeVersionError},
{name: "version four", source: "version: 4\nunknown: value\n", code: configkit.CodeVersionError},
{name: "unsupported", source: "version: 6\nunknown: value\n", code: configkit.CodeVersionError},
{name: "noninteger", source: "version: 5.0\n", code: configkit.CodeVersionType},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
_, err := loadDocumentSource(configkit.Bytes("document.yml", []byte(test.source)), LoadOptions{})
var loadError *configkit.LoadError
if !errors.As(err, &loadError) {
t.Fatalf("error = %v, want ConfigKit load error", err)
}
diagnostics := loadError.Diagnostics()
if len(diagnostics) != 1 || diagnostics[0].Code != test.code || diagnostics[0].Path.String() != "/version" {
t.Fatalf("diagnostics = %#v", diagnostics)
}
if strings.Contains(diagnostics[0].Message, "unknown") {
t.Fatalf("document decoding ran before version validation: %#v", diagnostics)
}
})
}
}
func TestDocumentLoadRejectsShorthandAndUnknownFields(t *testing.T) {
t.Parallel()
tests := []struct {
name string
source string
code configkit.Code
path string
}{
{
name: "binding shorthand",
source: "version: 5\npipelines:\n session:\n input: seriatim\n",
code: configkit.CodeDecodeType,
path: "/pipelines/session/input",
},
{
name: "validator nested producer field",
source: "version: 5\npipelines:\n session:\n input:\n module: seriatim\n validators:\n - module: generic/valid_json\n validators: []\n",
code: configkit.CodeDecodeUnknownField,
path: "/pipelines/session/input/validators/0/validators",
},
{
name: "unknown field",
source: "version: 5\npipelines:\n session:\n input:\n module: seriatim\n unknown: true\n",
code: configkit.CodeDecodeUnknownField,
path: "/pipelines/session/input/unknown",
},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
_, err := loadDocumentSource(configkit.Bytes("document.yml", []byte(test.source)), LoadOptions{})
var loadError *configkit.LoadError
if !errors.As(err, &loadError) {
t.Fatalf("error = %v, want ConfigKit load error", err)
}
diagnostics := loadError.Diagnostics()
if len(diagnostics) != 1 || diagnostics[0].Code != test.code || diagnostics[0].Path.String() != test.path {
t.Fatalf("diagnostics = %#v", diagnostics)
}
})
}
}
func TestDocumentDefinitionReturnsIndependentDefaults(t *testing.T) {
t.Parallel()
first, err := loadDocumentSource(configkit.Bytes("first.yml", []byte("version: 5\n")), LoadOptions{})
if err != nil {
t.Fatal(err)
}
first.Config.Pipelines["changed"] = PipelineDocument{}
first.Config.Concurrency.StageWorkers["extract"] = 3
second, err := loadDocumentSource(configkit.Bytes("second.yml", []byte("version: 5\n")), LoadOptions{})
if err != nil {
t.Fatal(err)
}
if len(second.Config.Pipelines) != 0 || len(second.Config.Concurrency.StageWorkers) != 0 {
t.Fatalf("defaults aliased across loads: %#v", second.Config)
}
}

60
internal/core/config/load.go Normal file
View File

@@ -0,0 +1,60 @@
package config
import (
configkit "gitea.maximumdirect.net/eric/ConfigKit"
)
// LoadOptions contains configuration selections supplied by an application
// boundary. It is intentionally narrow: configuration discovery and CLI flag
// parsing remain outside this package.
type LoadOptions struct {
Profile string
Overrides []configkit.Override
}
// loadDocument is the private ConfigKit load boundary under construction.
// Production commands continue to use the existing version 4 loader until the
// runtime conversion and command wiring are complete.
func loadDocument(path string, options LoadOptions) (*configkit.Result[Document], error) {
return loadDocumentSource(configkit.File(path), options)
}
func loadDocumentSource(source configkit.Source, options LoadOptions) (*configkit.Result[Document], error) {
return configkit.Load(source, configkit.Options[Document]{
Definition: documentDefinition(),
Profile: options.Profile,
Overrides: options.Overrides,
})
}
func documentDefinition() configkit.Definition[Document] {
return configkit.Definition[Document]{
Defaults: func() Document {
return Document{
Pipelines: map[string]PipelineDocument{},
Concurrency: &ConcurrencyDocument{
StageWorkers: map[string]int{},
},
}
},
Version: &configkit.VersionRule{
Path: configkit.Keys("version"),
Check: validateDocumentVersion,
},
}
}
func validateDocumentVersion(version configkit.Version, validation *configkit.Validation) error {
if !version.Present || version.Null {
validation.Error(configkit.Keys("version"), "configuration version is required")
return nil
}
if version.Text == "4" {
validation.Error(configkit.Keys("version"), "configuration version 4 is no longer supported; migrate to version 5")
return nil
}
if version.Text != "5" {
validation.Error(configkit.Keys("version"), "unsupported configuration version")
}
return nil
}