diff --git a/README.md b/README.md index 7f66627..068df6d 100644 --- a/README.md +++ b/README.md @@ -31,6 +31,8 @@ func ReadConfig(path string) (Config, error) { ``` The [configuration contract](docs/config.md) describes implemented behavior. The -[initial target design](docs/roadmap/initial.md) describes planned behavior. +[consumer adoption guide](docs/consumers/README.md) provides maintained Narratio-, +Notarius-, WeatherReporter-, and Seriatim-shaped examples and migration guidance. +The [initial target design](docs/roadmap/initial.md) describes planned behavior. The [implementation plan](docs/roadmap/implementation.md) owns the pending stages. See the [development guide](docs/development.md) for repository orientation. diff --git a/docs/config.md b/docs/config.md index b457511..c7a571a 100644 --- a/docs/config.md +++ b/docs/config.md @@ -585,6 +585,24 @@ introduce typed secret references. Descriptors are reused by typed decoding, projection, and cloning so those operations cannot acquire different field bindings. +### Canonical module options + +Applications with a known module catalog represent each configured item as an object +with an explicit module identifier and an options mapping. Ordinary typed structs, +including a small union of optional option fields, use the same strict binding above; +application validation decides which fields and combinations apply to each module. +String shorthand, a library module registry, and custom structural decoder hooks are +not supported alternatives. + +When a runtime-selected catalog genuinely requires dynamic options, +`map[string]any` retains ConfigKit's canonical dynamic scalars. In particular, +integers are `int64`; application validation owns allowed keys, value constraints, +range checks, and conversion to runtime `int` or catalog objects. ConfigKit does not +provide generic subtree conversion, and consumers must not add a YAML marshal/decode +round trip to exercise supported behavior. The +[consumer adoption guide](consumers/README.md) contains compiled typed and dynamic +examples. + ## Defaults and typed decoding The private decoder starts from an independently owned clone of the application's diff --git a/docs/consumers/README.md b/docs/consumers/README.md new file mode 100644 index 0000000..4e72e65 --- /dev/null +++ b/docs/consumers/README.md @@ -0,0 +1,98 @@ +# Consumer Adoption Guide + +ConfigKit removes parser setup, strict decoding, presence bookkeeping, import and +profile traversal, precedence, provenance, secret lookup, and safe inspection from +applications. Consumers still own their configuration structs, defaults, semantic +validation, schema-version policy, CLI parsing, runtime matching and conversion, +resource checks, remote acquisition, and service initialization. + +The maintained [`examples/consumers`](../../examples/consumers) package demonstrates +four sanitized shapes using only ConfigKit and the Go standard library: + +| Shape | Demonstrated fit | Application code deliberately retained | +| --- | --- | --- | +| Narratio-like | Fresh defaults, explicit partial imports, artifact maps, recorded inherited defaults, normalization, and source ownership | Artifact meaning and the separate `model_profile` domain field | +| Notarius-like | Canonical object bindings, a string schema version, strict dynamic options, and `int64` dynamic integers | Catalog selection, allowed option keys, range-checked conversion to runtime `int` | +| WeatherReporter-like | Defaults-only loading, report maps, canonical module objects, typed option unions, explicit CLI records, and typed secret startup | Report/module compatibility, rejection of explicitly empty domain lists, CLI parsing, and startup ordering | +| Seriatim-like | Independent strict ordered speaker and autocorrect rule files | First-match and ordered replacement algorithms; no central application configuration file | + +The YAML under [`testdata`](../../examples/consumers/testdata) is loaded by the +executable examples and ordinary tests. It contains no private application data, +credentials, network calls, or dependency on a sibling checkout. + +## Happy path + +An application defines ordinary types and passes one explicit source. Optional +features appear only when the application needs them: + +```go +result, err := configkit.Load[Config](configkit.File(path), configkit.Options[Config]{ + Definition: rules, + Imports: useDeclaredImports, + Profile: selectedConfigProfile, + Overrides: explicitlySetFlags, +}) +if err != nil { + return err +} +return start(result.Config) +``` + +Simple consumers use neither imports nor profiles. A complex consumer can opt into +both without redefining their syntax. ConfigKit profiles are configuration overlays; +they do not select or replace an application's LLM, model, pipeline, report, or +execution profile concepts. + +Known module catalogs should use canonical object entries with a module identifier +and typed options. A small union struct is sufficient when the available option +fields are known; validation selects which fields apply to each module. Runtime- +selected catalogs may retain `map[string]any`, whose integers are `int64`; applications +validate keys and ranges and convert only at their runtime boundary. Neither shape +needs YAML reserialization, a parser hook, a ConfigKit registry, or a generic subtree +conversion API. + +## Intentional migration changes + +The examples preserve domain meaning while adopting one configuration convention: + +- Explicit empty strings, lists, zeroes, false values, and nulls remain intentional. + ConfigKit accepts an empty list structurally; WeatherReporter-shaped validation + rejects an explicitly or programmatically supplied empty module/template list as a + domain rule. Loaded and constructed values use the same validator. +- Canonical module and object bindings replace string shorthand. Unknown fields and + incompatible option shapes fail at their semantic paths. +- Schema versions are strings. Consumers with integer versions migrate their files + instead of enabling another scalar convention. +- Integers inside dynamic options are portable `int64` values. Runtime `int` + conversion is explicit and range checked. +- Strict YAML intentionally rejects duplicate keys, aliases, merge keys, unsupported + tags/scalars, trailing documents, and unknown typed fields. Diagnostics expose + stable classifications and safe paths, not copied parser text. +- General environment-variable configuration overlays retire. Application CLI + parsers translate only explicitly set configuration flags into `Override` records. + Separate invocation state remains outside configuration. +- Credential fields migrate to environment-name `Secret` references. Applications + explicitly call `LoadSecretsDir` before `Load` when they choose file bootstrap; + blocked empty variables remain blocked, and ordinary `Load` never changes the + process environment. + +These examples are migration evidence, not completed changes to Narratio, Notarius, +WeatherReporter, or Seriatim. Updating those repositories, their deployed YAML, CLI +wiring, runtime catalogs, and operational documentation remains separate downstream +work. + +## Maintained roadmap examples + +Every nontrivial API sketch from the feature roadmap has a compiled example or a +focused behavioral owner: + +| Roadmap surface | Maintained owner | +| --- | --- | +| `Load`, `NoSource`, validation, versions, resolution, `SecretEnv`, overrides, and explanation | Executable examples in [`example_test.go`](../../example_test.go) | +| Source/import/profile pipeline rules and artifact defaults | Narratio-shaped executable example and fixture | +| Canonical module objects and dynamic option handling | WeatherReporter- and Notarius-shaped examples and migration tests | +| `LoadSecretsDir`, bootstrap reporting, typed lookup, CLI explanation | `Example_weatherStartup` | +| Results, metadata, diagnostics, effective output, and writer failures | Focused root-package load, provenance, rendering, and explanation tests | + +The package-level import-boundary test covers the consumer examples too: supported +usage names no YAML dependency, parser type, parser option, or parser error chain. diff --git a/docs/development.md b/docs/development.md index 9a257e2..a6062af 100644 --- a/docs/development.md +++ b/docs/development.md @@ -25,7 +25,7 @@ redacted effective rendering and historical value explanation. The | Documentation changes | [Documentation policy](policy/documentation.md) and the owning document | | Architecture decisions or future work | [Documentation policy](policy/documentation.md), [initial target design](roadmap/initial.md), and relevant ADRs listed below | | Code and test changes | [Architecture policy](policy/architecture.md), [testing policy](policy/testing.md), and owning contracts/tests as they are introduced | -| Examples or copyable assets | The owning implemented contract and related maintained examples, once available | +| Examples or copyable assets | The owning implemented contract and [consumer adoption guide](consumers/README.md) | | Release preparation | [Release policy](policy/release.md) and [release-readiness gaps](roadmap/release-readiness.md); the operational runbook will be added before release | Accepted decisions: diff --git a/docs/internal/extraction.md b/docs/internal/extraction.md index 5fc2d58..ed433e7 100644 --- a/docs/internal/extraction.md +++ b/docs/internal/extraction.md @@ -11,6 +11,8 @@ govern every adaptation. | --- | --- | --- | --- | | Narratio | `b68435128be1e24b94fc7ec259d6e9093669fd99` | Clean `main` tracking `origin/main` | Primary parsing, composition, import, profile, presence, defaults, resolution, and inspection source | | WeatherReporter | `5d306a202471e10fe98923a1df27d0b0fce5c666` | Clean `main` tracking `origin/main` | Secrets bootstrap and simple-consumer/defaults/options evidence | +| Notarius | `77ae132e5609916da40dcd4fcc749c000a63cbac` | Clean `main` tracking `origin/main` | Canonical binding, string-version, and dynamic-option migration evidence | +| Seriatim | `0fc92f364316c874d366b00cdc884a33e1900b25` | Clean `main` tracking `origin/main` | Independent ordered speaker/autocorrect rule evidence | These revisions were inspected on 2026-09-06. Paths below are relative to each application repository. Record later source revisions and adaptations here when @@ -369,10 +371,42 @@ Assigned to consumer-shaped examples and migration checks: semantics, registries, rendering, and module option validation remain application responsibilities. +Consumer-example implementation record (2026-09-06): at Narratio revision +`b68435128be1e24b94fc7ec259d6e9093669fd99`, the maintained consumer cases adapt +`TestMaintainedExtractionExamplesPreservePublishedContracts`, +`TestConfigValidateAndShowPipelineOnlyAreSideEffectFree`, and the stable source- +ownership tests into a sanitized partial-import/artifact fixture. ConfigKit replaces +application composition envelopes and command output with `_configkit.imports`, +typed defaults, recorded resolution, metadata, and explanation. Artifact meaning, +model/pipeline profiles, commands, and generated-domain attribution remain in the +example application layer; no production function was copied. + +At WeatherReporter revision `5d306a202471e10fe98923a1df27d0b0fce5c666`, +`TestDefaults`, explicit-empty-list cases, loaded/constructed consistency cases, and +nonmutating module-option normalization were adapted into the WeatherReporter-shaped +fixtures and tests in [`examples/consumers`](../../examples/consumers). ConfigKit's +detached defaults replace source mutation, canonical module objects replace string +shorthand, ordinary validation retains empty-list and option compatibility rules, +and explicit `Override` records replace external configuration-field overlays. The +startup example composes the separately implemented bootstrap report, typed secret +lookup, and explanation APIs without copying WeatherReporter's registries or +rendering/runtime modules. + +Notarius revision `77ae132e5609916da40dcd4fcc749c000a63cbac` supplied +binding/version/dynamic-option migration evidence only. The local example uses an +ordinary canonical object, a string `VersionRule`, ConfigKit's `int64` dynamic +integer convention, and application range checking; it does not copy a catalog or +environment overlay. Seriatim revision +`0fc92f364316c874d366b00cdc884a33e1900b25` supplied ordered rule-file evidence. +The local examples load speaker and autocorrect files independently and keep their +matching/replacement algorithms outside ConfigKit. Neither source is a build or test +dependency. + ## Application-only material Do not extract application commands, filesystem discovery, logging, runtime initialization, remote clients, LLM/provider setup, audio/transcript fixtures, configuration digests, report rendering, campaign/session algorithms, or consumer -resource validation into ConfigKit. Later consumer examples may use small synthetic -types and fixtures to prove that these responsibilities remain straightforward. +resource validation into ConfigKit. Maintained consumer examples use only small +synthetic types and fixtures to prove that these responsibilities remain +straightforward. diff --git a/docs/internal/overview.md b/docs/internal/overview.md index cf00336..ba2498d 100644 --- a/docs/internal/overview.md +++ b/docs/internal/overview.md @@ -42,13 +42,17 @@ module and package documentation boundary. Its implemented public primitives are - frozen-snapshot redaction and canonical effective output in `render.go`; - bounded immutable input applications, safe historical replay, detached explanation records, and deterministic human rendering in `explain.go`, with the - only upstream dumper integration isolated in `internal/yaml/write.go`. + only upstream dumper integration isolated in `internal/yaml/write.go`; and +- a separate maintained `examples/consumers` package with four sanitized consumer + shapes, fixtures, domain validators, and runtime-boundary functions. The [configuration contract](../config.md) owns implemented input, schema, decoding, loading lifecycle, path imports, composition, profile/CLI overlay, version checking, recorded resolution, typed secret references/bootstrap, semantic validation, metadata, secret classification, redacted effective rendering, historical explanation, and ownership behavior. +The [consumer adoption guide](../consumers/README.md) owns current application-fit +and migration guidance. The source-extraction inventory records the application foundations assigned to future implementation work: diff --git a/docs/roadmap/implementation.md b/docs/roadmap/implementation.md index 4100dc3..7b9c5ab 100644 --- a/docs/roadmap/implementation.md +++ b/docs/roadmap/implementation.md @@ -1437,7 +1437,56 @@ Acceptance: ## Stage 20 — Prove consumer ergonomics and migration contracts -Status: pending. +Status: complete. + +Completion evidence (2026-09-06): + +- Added the maintained [`examples/consumers`](../../examples/consumers) package and + sanitized local fixtures for four application shapes. The Narratio-like example + uses fresh defaults, partial imports, artifacts, recorded inherited defaults, and + an application model profile distinct from an optional ConfigKit profile. The + Notarius-like example uses canonical object bindings, a string version rule, + dynamic options, and range-checked `int64` conversion at the runtime boundary. +- Added WeatherReporter-like defaults-only and file loads, report maps, canonical + module objects, typed option unions, and shared loaded/constructed validation. + Application validation—not the ConfigKit schema—rejects explicit empty module and + path-template lists and null required module options. Recorded normalization acts + on detached values without mutating caller-owned constructed options. Added + independent ordered Seriatim-like speaker and autocorrect rule loaders while + retaining matching/replacement algorithms in the example package. +- Added executable examples for all four shapes and a WeatherReporter-shaped startup + flow. The startup explicitly renders `SecretsReport`, demonstrates an existing + empty variable blocking its file and a separate successful disposable lookup, + translates only an explicitly set `--units` flag into an `Override`, and renders + its explanation. It verifies ordinary `Load` preserves the process environment + and never uses a real credential. +- Generalized the assigned Narratio inspection/default/artifact outcomes and + WeatherReporter defaults, explicit-empty, loaded/constructed consistency, and + option-normalization cases. [`docs/internal/extraction.md`](../internal/extraction.md) + records the rechecked clean revisions, adapted cases, intentional differences, + and no-copy boundaries for Narratio `b68435128be1e24b94fc7ec259d6e9093669fd99`, + WeatherReporter `5d306a202471e10fe98923a1df27d0b0fce5c666`, Notarius + `77ae132e5609916da40dcd4fcc749c000a63cbac`, and Seriatim + `0fc92f364316c874d366b00cdc884a33e1900b25`. No sibling checkout is a build or test + dependency. +- Added negative migration fixtures for integer versions, string module shorthand, + explicit empty lists, explicit null required options, and strict unknown fields. + They fail with stable `version.type`, `decode.type`, `validate.error`, and + `decode.unknown_field` classifications at semantic paths without parser text. + Dynamic options bind directly without YAML reserialization or a generic subtree + conversion API. +- Added the [consumer adoption guide](../consumers/README.md), current canonical + module-options contract, README navigation, and component inventory. The guide + maps every feature-roadmap API sketch to an executable example or focused test, + describes removed plumbing versus retained domain code, and lists explicit + empty/null, object, version, integer, strict-YAML, environment-overlay, CLI, and + secret-reference migrations without claiming downstream repositories are migrated. +- Confirmed the example package directly imports only ConfigKit and the standard + library, then reviewed the exported surface and existing AST import-boundary test + for aliases, generic constraints, callbacks, record fields, and safe error chains. + Verified `go test ./...`, `go test -race ./...`, `go vet ./...`, `go build ./...`, + `go mod tidy -diff`, and `git diff --check` with Go 1.26.7 on linux/amd64. No + deviations were required; broader hardening remains later work. Read: feature consumer-fit table, module options, external values, and both criteria. diff --git a/docs/roadmap/initial.md b/docs/roadmap/initial.md index a76b144..7c19023 100644 --- a/docs/roadmap/initial.md +++ b/docs/roadmap/initial.md @@ -989,95 +989,10 @@ public records, replay ordering, redaction, rendering, and allocation bounds. ### Module-specific options -Use a canonical object entry with an explicit module identifier and options -mapping. Shorthand strings and custom structural decoders are not required to -express the domain information. For a known application module catalog, model -the available option fields with ordinary typed structs; application validation -selects the meaningful fields and rejects incompatible combinations. - -Illustrative YAML for an application that supports two modules: - -```yaml -modules: - - module: chunker - options: - window_units: 12 - - module: discussion - options: - sections: [short_term] -``` - -The corresponding typed configuration and validator need no second parser: - -```go -type ModuleOptions struct { - WindowUnits *int `yaml:"window_units"` - Sections []string `yaml:"sections"` -} - -type ModuleSpec struct { - Module string `yaml:"module"` - Options ModuleOptions `yaml:"options"` -} - -type ModuleConfig struct { - Modules []ModuleSpec `yaml:"modules"` -} - -func validateModules(cfg *ModuleConfig, v *configkit.Validation) { - for index, spec := range cfg.Modules { - path := configkit.Keys("modules").Index(index) - switch spec.Module { - case "chunker": - if spec.Options.WindowUnits == nil || *spec.Options.WindowUnits < 1 { - v.Error(path.Key("options").Key("window_units"), "must be a positive integer") - } - if spec.Options.Sections != nil { - v.Error(path.Key("options").Key("sections"), "not supported by this module") - } - case "discussion": - if spec.Options.WindowUnits != nil { - v.Error(path.Key("options").Key("window_units"), "not supported by this module") - } - if len(spec.Options.Sections) == 0 { - v.Error(path.Key("options").Key("sections"), "must contain at least one section") - } - default: - v.Error(path.Key("module"), "unsupported module") - } - } -} - -func ReadModuleConfig(path string) (*configkit.Result[ModuleConfig], error) { - return configkit.Load(configkit.File(path), configkit.Options[ModuleConfig]{ - Definition: configkit.Definition[ModuleConfig]{Validate: validateModules}, - }) -} -``` - -The pointer and nil-slice states above express optional domain values; this -application treats explicit null as unselected for those optional alternatives. -ConfigKit rejects unknown fields and invalid scalar types. The application owns -which supported options apply to a module and their domain constraints. Larger -catalogs may use nested typed option objects rather than one flat options struct. -Neither shape requires a library registry, parser hook, or configurable merge rule. - -For an application whose option schema is genuinely selected at runtime, -`map[string]any` remains available. ConfigKit owns parsing, paths, and the single -dynamic scalar representation; application validation owns the selected schema, -including allowed keys and value constraints. Conversion into runtime module -arguments is application code. Dynamic integers remain checked `int64`; existing -`int` assertions must be adapted with range checks when converting to `int`. - -The initial API does not add a generic subtree-conversion helper. Prefer typed -configuration where option shapes are known, and direct domain validation where -they are dynamic. Do not introduce a second YAML marshal/decode round trip as the -blessed pattern. Reconsider a narrow conversion helper only if a concrete dynamic -catalog demonstrates substantial duplicated type-binding infrastructure that -ordinary models cannot remove. Such a helper would have to retain strictness and -source attribution; that is a separate justified extension, not an escape hatch -for legacy shorthand. Configuration inspection describes the validated input -model, not the application's separately constructed runtime module objects. +Canonical typed and dynamic module options are implemented through the ordinary +schema. The [current module-options contract](../config.md#canonical-module-options) +and [consumer adoption guide](../consumers/README.md) own the maintained object +forms, validation boundary, and executable examples. ### Supported types and projection diff --git a/examples/consumers/consumers.go b/examples/consumers/consumers.go new file mode 100644 index 0000000..3d5c28b --- /dev/null +++ b/examples/consumers/consumers.go @@ -0,0 +1,331 @@ +// Package consumers contains small, maintained application shapes that exercise +// ConfigKit without making consumer domain policy part of the library API. +package consumers + +import ( + "math" + "strconv" + "strings" + "time" + + configkit "gitea.maximumdirect.net/eric/ConfigKit" +) + +// Artifact is a Narratio-shaped generated output declaration. +type Artifact struct { + Output string `yaml:"output"` + Timeout time.Duration `yaml:"timeout"` +} + +// NarratioConfig demonstrates application defaults, partial imports, and +// application-owned model-profile and artifact semantics. +type NarratioConfig struct { + PollInterval time.Duration `yaml:"poll_interval"` + Timeout time.Duration `yaml:"timeout"` + ModelProfile string `yaml:"model_profile"` + Artifacts map[string]Artifact `yaml:"artifacts"` +} + +// LoadNarratio loads the maintained import fixture with Narratio-shaped rules. +func LoadNarratio(path string) (*configkit.Result[NarratioConfig], error) { + return LoadNarratioProfile(path, "") +} + +// LoadNarratioProfile selects a ConfigKit configuration overlay independently +// from the application's ModelProfile field. +func LoadNarratioProfile(path, profile string) (*configkit.Result[NarratioConfig], error) { + return configkit.Load(configkit.File(path), configkit.Options[NarratioConfig]{ + Imports: true, Profile: profile, + Definition: configkit.Definition[NarratioConfig]{ + Defaults: func() NarratioConfig { return NarratioConfig{PollInterval: time.Minute} }, + Resolve: resolveNarratio, + Validate: validateNarratio, + }, + }) +} + +func resolveNarratio(configuration *NarratioConfig, resolution *configkit.Resolution) error { + configkit.Default(resolution, configkit.Keys("timeout"), &configuration.Timeout, + configuration.PollInterval/10, "derived from poll interval", configkit.Keys("poll_interval")) + for name, artifact := range configuration.Artifacts { + path := configkit.Keys("artifacts", name) + configkit.Default(resolution, path.Key("timeout"), &artifact.Timeout, + configuration.Timeout, "inherited pipeline timeout", configkit.Keys("timeout")) + configkit.Normalize(resolution, path.Key("output"), &artifact.Output, + strings.TrimSpace(artifact.Output), "trimmed artifact output") + configuration.Artifacts[name] = artifact + } + return nil +} + +func validateNarratio(configuration *NarratioConfig, validation *configkit.Validation) { + if configuration.PollInterval <= 0 { + validation.Error(configkit.Keys("poll_interval"), "must be positive") + } + if configuration.Timeout <= 0 { + validation.Error(configkit.Keys("timeout"), "must be positive") + } + for name, artifact := range configuration.Artifacts { + path := configkit.Keys("artifacts", name) + if artifact.Output == "" { + validation.Error(path.Key("output"), "must be nonempty") + } + if artifact.Timeout <= 0 { + validation.Error(path.Key("timeout"), "must be positive") + } + } +} + +// ObjectBinding is a Notarius-shaped canonical object binding. Options remain +// dynamic because the application catalog selects their schema at runtime. +type ObjectBinding struct { + Object string `yaml:"object"` + Options map[string]any `yaml:"options"` +} + +// NotariusConfig demonstrates string versions and dynamic catalog options. +type NotariusConfig struct { + Version string `yaml:"version"` + Bindings []ObjectBinding `yaml:"bindings"` +} + +// LoadNotarius loads a Notarius-shaped configuration. +func LoadNotarius(path string) (*configkit.Result[NotariusConfig], error) { + return configkit.Load(configkit.File(path), configkit.Options[NotariusConfig]{ + Definition: configkit.Definition[NotariusConfig]{ + Version: &configkit.VersionRule{Path: configkit.Keys("version"), Check: requireVersion("v1")}, + Validate: validateNotarius, + }, + }) +} + +func requireVersion(expected string) func(configkit.Version, *configkit.Validation) error { + return func(version configkit.Version, validation *configkit.Validation) error { + if !version.Present || version.Null || version.Text != expected { + validation.Error(configkit.Keys("version"), "unsupported configuration version") + } + return nil + } +} + +func validateNotarius(configuration *NotariusConfig, validation *configkit.Validation) { + for index, binding := range configuration.Bindings { + path := configkit.Keys("bindings").Index(index) + if binding.Object != "summary" && binding.Object != "registry" { + validation.Error(path.Key("object"), "unsupported object binding") + } + for key, value := range binding.Options { + optionPath := path.Key("options").Key(key) + switch key { + case "window_units": + units, ok := value.(int64) + if !ok || units < 1 { + validation.Error(optionPath, "must be a positive integer") + } + case "format": + if _, ok := value.(string); !ok { + validation.Error(optionPath, "must be a string") + } + default: + validation.Error(optionPath, "unsupported object option") + } + } + } +} + +// NotariusWindow converts ConfigKit's portable dynamic integer only at the +// application runtime boundary. +func NotariusWindow(configuration NotariusConfig, binding int) (int, bool) { + if binding < 0 || binding >= len(configuration.Bindings) { + return 0, false + } + value, ok := configuration.Bindings[binding].Options["window_units"].(int64) + if !ok || value < 1 || (strconv.IntSize == 32 && value > math.MaxInt32) { + return 0, false + } + return int(value), true +} + +// ModuleOptions is a WeatherReporter-shaped union of known module option fields. +type ModuleOptions struct { + WindowUnits *int `yaml:"window_units"` + Sections []string `yaml:"sections"` +} + +// ModuleSpec uses the canonical object form instead of a string shorthand. +type ModuleSpec struct { + Module string `yaml:"module"` + Options ModuleOptions `yaml:"options"` +} + +// ReportConfig keeps report-specific lists application-owned. +type ReportConfig struct { + Modules []ModuleSpec `yaml:"modules"` + PathTemplates []string `yaml:"path_templates"` +} + +// WeatherConfig demonstrates simple defaults, reports, module options, and an +// optional typed secret reference. +type WeatherConfig struct { + Units string `yaml:"units"` + Reports map[string]ReportConfig `yaml:"reports"` + Token *configkit.Secret `yaml:"token"` +} + +// LoadWeather loads a WeatherReporter-shaped source and explicit CLI records. +func LoadWeather(source configkit.Source, overrides ...configkit.Override) (*configkit.Result[WeatherConfig], error) { + return configkit.Load(source, configkit.Options[WeatherConfig]{ + Overrides: overrides, + Definition: weatherDefinition(func() WeatherConfig { + return WeatherConfig{Units: "metric"} + }), + }) +} + +// LoadConstructedWeather runs the same rules over a detached constructed value. +func LoadConstructedWeather(configuration WeatherConfig) (*configkit.Result[WeatherConfig], error) { + return configkit.Load(configkit.NoSource(), configkit.Options[WeatherConfig]{ + Definition: weatherDefinition(func() WeatherConfig { return configuration }), + }) +} + +func weatherDefinition(defaults func() WeatherConfig) configkit.Definition[WeatherConfig] { + return configkit.Definition[WeatherConfig]{ + Defaults: defaults, + Resolve: func(configuration *WeatherConfig, resolution *configkit.Resolution) error { + for reportName, report := range configuration.Reports { + var modules []ModuleSpec + if report.Modules != nil { + modules = append(make([]ModuleSpec, 0, len(report.Modules)), report.Modules...) + } + for index, module := range modules { + if module.Options.Sections != nil { + sections := append([]string(nil), module.Options.Sections...) + for position := range sections { + sections[position] = strings.TrimSpace(sections[position]) + } + module.Options.Sections = sections + } + modules[index] = module + } + configkit.Normalize(resolution, configkit.Keys("reports", reportName, "modules"), + &report.Modules, modules, "normalized module options") + configuration.Reports[reportName] = report + } + return nil + }, + Validate: validateWeather, + } +} + +func validateWeather(configuration *WeatherConfig, validation *configkit.Validation) { + if configuration.Units != "metric" && configuration.Units != "us" { + validation.Error(configkit.Keys("units"), "must be metric or us") + } + for reportName, report := range configuration.Reports { + path := configkit.Keys("reports", reportName) + if report.Modules != nil && len(report.Modules) == 0 { + validation.Error(path.Key("modules"), "must contain at least one module") + } + if report.PathTemplates != nil && len(report.PathTemplates) == 0 { + validation.Error(path.Key("path_templates"), "must contain at least one template") + } + for index, module := range report.Modules { + modulePath := path.Key("modules").Index(index) + switch module.Module { + case "chunker": + if module.Options.WindowUnits == nil || *module.Options.WindowUnits < 1 { + validation.Error(modulePath.Key("options").Key("window_units"), "must be a positive integer") + } + if module.Options.Sections != nil { + validation.Error(modulePath.Key("options").Key("sections"), "not supported by this module") + } + case "discussion": + if module.Options.WindowUnits != nil { + validation.Error(modulePath.Key("options").Key("window_units"), "not supported by this module") + } + if len(module.Options.Sections) == 0 { + validation.Error(modulePath.Key("options").Key("sections"), "must contain at least one section") + } + default: + validation.Error(modulePath.Key("module"), "unsupported module") + } + } + } +} + +// SpeakerRule and SpeakerRules preserve declaration order for application matching. +type SpeakerRule struct { + Prefix string `yaml:"prefix"` + Name string `yaml:"name"` +} + +type SpeakerRules struct { + Rules []SpeakerRule `yaml:"rules"` +} + +// AutocorrectRule and AutocorrectRules preserve replacement order. +type AutocorrectRule struct { + Find string `yaml:"find"` + Replace string `yaml:"replace"` +} + +type AutocorrectRules struct { + Rules []AutocorrectRule `yaml:"rules"` +} + +// LoadSpeakerRules loads one independent Seriatim-shaped rule file. +func LoadSpeakerRules(path string) (*configkit.Result[SpeakerRules], error) { + return configkit.Load(configkit.File(path), configkit.Options[SpeakerRules]{ + Definition: configkit.Definition[SpeakerRules]{Validate: validateSpeakerRules}, + }) +} + +// LoadAutocorrectRules loads another independent ordered rule file. +func LoadAutocorrectRules(path string) (*configkit.Result[AutocorrectRules], error) { + return configkit.Load(configkit.File(path), configkit.Options[AutocorrectRules]{ + Definition: configkit.Definition[AutocorrectRules]{Validate: validateAutocorrectRules}, + }) +} + +func validateSpeakerRules(configuration *SpeakerRules, validation *configkit.Validation) { + seen := make(map[string]int) + for index, rule := range configuration.Rules { + path := configkit.Keys("rules").Index(index) + if rule.Prefix == "" { + validation.Error(path.Key("prefix"), "must be nonempty") + } + if previous, ok := seen[rule.Prefix]; ok { + validation.Error(path.Key("prefix"), "prefix must be unique", + configkit.Keys("rules").Index(previous).Key("prefix")) + } + seen[rule.Prefix] = index + } +} + +func validateAutocorrectRules(configuration *AutocorrectRules, validation *configkit.Validation) { + for index, rule := range configuration.Rules { + path := configkit.Keys("rules").Index(index) + if rule.Find == "" { + validation.Error(path.Key("find"), "must be nonempty") + } + } +} + +// SpeakerName is application runtime matching, not configuration infrastructure. +func SpeakerName(rules SpeakerRules, input string) (string, bool) { + for _, rule := range rules.Rules { + if strings.HasPrefix(input, rule.Prefix) { + return rule.Name, true + } + } + return "", false +} + +// ApplyAutocorrect is application runtime replacement in declared order. +func ApplyAutocorrect(rules AutocorrectRules, input string) string { + for _, rule := range rules.Rules { + input = strings.ReplaceAll(input, rule.Find, rule.Replace) + } + return input +} diff --git a/examples/consumers/consumers_test.go b/examples/consumers/consumers_test.go new file mode 100644 index 0000000..255d913 --- /dev/null +++ b/examples/consumers/consumers_test.go @@ -0,0 +1,252 @@ +package consumers_test + +import ( + "bytes" + "errors" + "fmt" + "os" + "path/filepath" + "reflect" + "runtime" + "strings" + "testing" + + configkit "gitea.maximumdirect.net/eric/ConfigKit" + "gitea.maximumdirect.net/eric/ConfigKit/examples/consumers" +) + +func Example_narratio() { + result, err := consumers.LoadNarratio(fixture("narratio", "pipeline.yml")) + artifact := result.Config.Artifacts["transcript"] + fmt.Println(result.Config.ModelProfile, artifact.Output, result.Config.Timeout, artifact.Timeout, err) + // Output: concise transcript.txt 6s 6s +} + +func Example_notarius() { + result, err := consumers.LoadNotarius(fixture("notarius", "config.yml")) + window, ok := consumers.NotariusWindow(result.Config, 0) + fmt.Println(result.Config.Version, result.Config.Bindings[0].Object, window, ok, err) + // Output: v1 summary 12 true +} + +func Example_weatherReporter() { + defaults, defaultsErr := consumers.LoadWeather(configkit.NoSource()) + result, err := consumers.LoadWeather(configkit.File(fixture("weather", "config.yml"))) + discussion := result.Config.Reports["daily"].Modules[1] + fmt.Println(defaults.Config.Units, result.Config.Units, discussion.Options.Sections[0], defaultsErr, err) + // Output: metric us short_term +} + +func Example_seriatim() { + speakers, speakerErr := consumers.LoadSpeakerRules(fixture("seriatim", "speakers.yml")) + autocorrect, autocorrectErr := consumers.LoadAutocorrectRules(fixture("seriatim", "autocorrect.yml")) + name, found := consumers.SpeakerName(speakers.Config, "Guest: teh event occured") + text := consumers.ApplyAutocorrect(autocorrect.Config, "teh event occured") + fmt.Println(name, found, text, speakerErr, autocorrectErr) + // Output: visitor true the event occurred +} + +func Example_weatherStartup() { + const blockedName = "CONFIGKIT_WEATHER_EXAMPLE_BLOCKED" + const tokenName = "CONFIGKIT_WEATHER_EXAMPLE_TOKEN" + restoreBlocked := preserveEnvironment(blockedName) + restoreToken := preserveEnvironment(tokenName) + defer restoreBlocked() + defer restoreToken() + _ = os.Setenv(blockedName, "") + _ = os.Unsetenv(tokenName) + + directory, _ := os.MkdirTemp("", "configkit-weather-example-") + defer os.RemoveAll(directory) + _ = os.WriteFile(filepath.Join(directory, blockedName), []byte("unused-example-value\n"), 0o600) + _ = os.WriteFile(filepath.Join(directory, tokenName), []byte("disposable-example-credential\n"), 0o600) + report, bootstrapErr := configkit.LoadSecretsDir(directory) + var reportOutput bytes.Buffer + _, reportWriteErr := report.WriteTo(&reportOutput) + + result, loadErr := consumers.LoadWeather(configkit.Bytes("weather.yml", []byte("units: metric\ntoken: "+tokenName+"\n")), + configkit.Override{Path: configkit.Keys("units"), Value: "us", Flag: "--units"}) + var explanation bytes.Buffer + explainErr := result.WriteExplanation(&explanation, configkit.Keys("units")) + blocked, loaded := secretStatus(report, blockedName), secretStatus(report, tokenName) + value, present := os.LookupEnv(tokenName) + fmt.Printf("blocked=%s loaded=%s report=%t reference=%s explained=%t unchanged=%t errors=%t\n", + blocked, loaded, reportOutput.Len() > 0, result.Config.Token.EnvName(), + strings.Contains(explanation.String(), "--units"), present && value == "disposable-example-credential", + bootstrapErr != nil || reportWriteErr != nil || loadErr != nil || explainErr != nil) + // Output: blocked=skipped_existing loaded=loaded report=true reference=CONFIGKIT_WEATHER_EXAMPLE_TOKEN explained=true unchanged=true errors=false +} + +func TestMaintainedConsumerFixturesPreserveDomainOutcomes(t *testing.T) { + narratio, err := consumers.LoadNarratio(fixture("narratio", "pipeline.yml")) + if err != nil || narratio.Config.Artifacts["transcript"].Output != "transcript.txt" || + narratio.Config.ModelProfile != "concise" || len(narratio.Sources()) != 2 { + t.Fatalf("Narratio shape = %#v, %v", narratio, err) + } + if origin, ok := narratio.Metadata().Lookup(configkit.Keys("artifacts", "transcript", "output")); !ok || origin.Origin.Layer != configkit.LayerResolution { + t.Fatalf("artifact origin = %#v, %v", origin, ok) + } + profiled, err := consumers.LoadNarratioProfile(fixture("narratio", "pipeline.yml"), "testing") + if err != nil || profiled.Config.PollInterval.String() != "2m0s" || + profiled.Config.ModelProfile != "concise" || len(profiled.Sources()) != 3 { + t.Fatalf("configuration/application profiles = %#v, %v", profiled, err) + } + + notarius, err := consumers.LoadNotarius(fixture("notarius", "config.yml")) + window, converted := consumers.NotariusWindow(notarius.Config, 0) + if err != nil || !converted || window != 12 { + t.Fatalf("Notarius shape = %#v, %v, %v", notarius, converted, err) + } + if _, ok := notarius.Config.Bindings[0].Options["window_units"].(int64); !ok { + t.Fatalf("dynamic integer type = %T", notarius.Config.Bindings[0].Options["window_units"]) + } + + weather, err := consumers.LoadWeather(configkit.File(fixture("weather", "config.yml"))) + if err != nil || weather.Config.Reports["daily"].Modules[1].Options.Sections[0] != "short_term" { + t.Fatalf("WeatherReporter shape = %#v, %v", weather, err) + } + + speakers, err := consumers.LoadSpeakerRules(fixture("seriatim", "speakers.yml")) + if err != nil { + t.Fatal(err) + } + name, ok := consumers.SpeakerName(speakers.Config, "Host: introduction") + if !ok || name != "narrator" || speakers.Config.Rules[0].Prefix != "Host:" { + t.Fatalf("ordered speaker rules = %#v, %q, %v", speakers.Config.Rules, name, ok) + } +} + +func TestMigrationFixturesFailWithStableCodesAndPaths(t *testing.T) { + tests := []struct { + name string + load func() error + code configkit.Code + path string + }{ + { + name: "integer schema version", + load: func() error { _, err := consumers.LoadNotarius(fixture("notarius", "integer-version.yml")); return err }, + code: configkit.CodeVersionType, path: "/version", + }, + { + name: "module shorthand", + load: func() error { + _, err := consumers.LoadWeather(configkit.File(fixture("weather", "shorthand-module.yml"))) + return err + }, + code: configkit.CodeDecodeType, path: "/reports/daily/modules/0", + }, + { + name: "explicit empty modules", + load: func() error { + _, err := consumers.LoadWeather(configkit.File(fixture("weather", "empty-modules.yml"))) + return err + }, + code: configkit.CodeValidateError, path: "/reports/daily/modules", + }, + { + name: "explicit empty path templates", + load: func() error { + _, err := consumers.LoadWeather(configkit.File(fixture("weather", "empty-paths.yml"))) + return err + }, + code: configkit.CodeValidateError, path: "/reports/daily/path_templates", + }, + { + name: "explicit null required option", + load: func() error { + _, err := consumers.LoadWeather(configkit.File(fixture("weather", "null-window.yml"))) + return err + }, + code: configkit.CodeValidateError, path: "/reports/daily/modules/0/options/window_units", + }, + { + name: "strict unknown rule field", + load: func() error { + _, err := consumers.LoadSpeakerRules(fixture("seriatim", "unknown-field.yml")) + return err + }, + code: configkit.CodeDecodeUnknownField, path: "/rules/0/color", + }, + } + for _, test := range tests { + t.Run(test.name, func(t *testing.T) { + var loadError *configkit.LoadError + if err := test.load(); !errors.As(err, &loadError) { + t.Fatalf("error = %T %v", err, err) + } + diagnostics := loadError.Diagnostics() + if len(diagnostics) == 0 || diagnostics[0].Code != test.code || diagnostics[0].Path.String() != test.path { + t.Fatalf("diagnostics = %#v", diagnostics) + } + if strings.Contains(diagnostics[0].Message, "wrong-before-profile") { + t.Fatalf("diagnostic exposed raw parser input: %#v", diagnostics[0]) + } + }) + } +} + +func TestWeatherLoadedAndConstructedRulesAgreeWithoutMutation(t *testing.T) { + loaded, err := consumers.LoadWeather(configkit.File(fixture("weather", "config.yml"))) + if err != nil { + t.Fatal(err) + } + reconstructed, err := consumers.LoadConstructedWeather(loaded.Config) + if err != nil || !reflect.DeepEqual(reconstructed.Config, loaded.Config) { + t.Fatalf("loaded/constructed outcomes differ: %#v, %v", reconstructed, err) + } + constructed := consumers.WeatherConfig{Units: loaded.Config.Units, Reports: map[string]consumers.ReportConfig{}} + constructed.Reports["daily"] = consumers.ReportConfig{ + Modules: []consumers.ModuleSpec{{Module: "discussion", Options: consumers.ModuleOptions{Sections: []string{" short_term "}}}}, + PathTemplates: []string{"daily/{date}.md"}, + } + wantInput := constructed.Reports["daily"].Modules[0].Options.Sections[0] + result, err := consumers.LoadConstructedWeather(constructed) + if err != nil { + t.Fatal(err) + } + if got := constructed.Reports["daily"].Modules[0].Options.Sections[0]; got != wantInput { + t.Fatalf("constructed options mutated from %q to %q", wantInput, got) + } + if result.Config.Reports["daily"].Modules[0].Options.Sections[0] != "short_term" { + t.Fatalf("normalized result = %#v", result.Config.Reports["daily"]) + } + + empty := consumers.WeatherConfig{Units: "metric", Reports: map[string]consumers.ReportConfig{ + "daily": {Modules: []consumers.ModuleSpec{}}, + }} + _, err = consumers.LoadConstructedWeather(empty) + var loadError *configkit.LoadError + if !errors.As(err, &loadError) || loadError.Diagnostics()[0].Code != configkit.CodeValidateError { + t.Fatalf("constructed empty modules error = %v", err) + } + if !reflect.DeepEqual(empty.Reports["daily"].Modules, []consumers.ModuleSpec{}) { + t.Fatal("constructed empty module input was mutated") + } +} + +func fixture(parts ...string) string { + _, file, _, _ := runtime.Caller(0) + values := append([]string{filepath.Dir(file), "testdata"}, parts...) + return filepath.Join(values...) +} + +func preserveEnvironment(name string) func() { + value, present := os.LookupEnv(name) + return func() { + if present { + _ = os.Setenv(name, value) + } else { + _ = os.Unsetenv(name) + } + } +} + +func secretStatus(report *configkit.SecretsReport, name string) configkit.SecretFileStatus { + for _, entry := range report.Entries() { + if entry.Name == name { + return entry.Status + } + } + return "missing" +} diff --git a/examples/consumers/testdata/narratio/artifacts.yml b/examples/consumers/testdata/narratio/artifacts.yml new file mode 100644 index 0000000..0f7df0c --- /dev/null +++ b/examples/consumers/testdata/narratio/artifacts.yml @@ -0,0 +1,3 @@ +artifacts: + transcript: + output: ' transcript.txt ' diff --git a/examples/consumers/testdata/narratio/pipeline.yml b/examples/consumers/testdata/narratio/pipeline.yml new file mode 100644 index 0000000..26f2c0b --- /dev/null +++ b/examples/consumers/testdata/narratio/pipeline.yml @@ -0,0 +1,5 @@ +_configkit: + imports: + - path: artifacts.yml +poll_interval: 1m +model_profile: concise diff --git a/examples/consumers/testdata/narratio/profiles/testing.yml b/examples/consumers/testdata/narratio/profiles/testing.yml new file mode 100644 index 0000000..a2e53dc --- /dev/null +++ b/examples/consumers/testdata/narratio/profiles/testing.yml @@ -0,0 +1 @@ +poll_interval: 2m diff --git a/examples/consumers/testdata/notarius/config.yml b/examples/consumers/testdata/notarius/config.yml new file mode 100644 index 0000000..ec3f86a --- /dev/null +++ b/examples/consumers/testdata/notarius/config.yml @@ -0,0 +1,6 @@ +version: v1 +bindings: + - object: summary + options: + window_units: 12 + format: markdown diff --git a/examples/consumers/testdata/notarius/integer-version.yml b/examples/consumers/testdata/notarius/integer-version.yml new file mode 100644 index 0000000..a0ce75a --- /dev/null +++ b/examples/consumers/testdata/notarius/integer-version.yml @@ -0,0 +1,2 @@ +version: 1 +bindings: [] diff --git a/examples/consumers/testdata/seriatim/autocorrect.yml b/examples/consumers/testdata/seriatim/autocorrect.yml new file mode 100644 index 0000000..e53a7be --- /dev/null +++ b/examples/consumers/testdata/seriatim/autocorrect.yml @@ -0,0 +1,5 @@ +rules: + - find: teh + replace: the + - find: occured + replace: occurred diff --git a/examples/consumers/testdata/seriatim/speakers.yml b/examples/consumers/testdata/seriatim/speakers.yml new file mode 100644 index 0000000..70dceeb --- /dev/null +++ b/examples/consumers/testdata/seriatim/speakers.yml @@ -0,0 +1,5 @@ +rules: + - prefix: 'Host:' + name: narrator + - prefix: 'Guest:' + name: visitor diff --git a/examples/consumers/testdata/seriatim/unknown-field.yml b/examples/consumers/testdata/seriatim/unknown-field.yml new file mode 100644 index 0000000..16c61f6 --- /dev/null +++ b/examples/consumers/testdata/seriatim/unknown-field.yml @@ -0,0 +1,4 @@ +rules: + - prefix: 'Host:' + name: narrator + color: blue diff --git a/examples/consumers/testdata/weather/config.yml b/examples/consumers/testdata/weather/config.yml new file mode 100644 index 0000000..b26528e --- /dev/null +++ b/examples/consumers/testdata/weather/config.yml @@ -0,0 +1,11 @@ +units: us +reports: + daily: + modules: + - module: chunker + options: + window_units: 12 + - module: discussion + options: + sections: [' short_term '] + path_templates: ['daily/{date}.md'] diff --git a/examples/consumers/testdata/weather/empty-modules.yml b/examples/consumers/testdata/weather/empty-modules.yml new file mode 100644 index 0000000..3fe879e --- /dev/null +++ b/examples/consumers/testdata/weather/empty-modules.yml @@ -0,0 +1,3 @@ +reports: + daily: + modules: [] diff --git a/examples/consumers/testdata/weather/empty-paths.yml b/examples/consumers/testdata/weather/empty-paths.yml new file mode 100644 index 0000000..785f9c7 --- /dev/null +++ b/examples/consumers/testdata/weather/empty-paths.yml @@ -0,0 +1,3 @@ +reports: + daily: + path_templates: [] diff --git a/examples/consumers/testdata/weather/null-window.yml b/examples/consumers/testdata/weather/null-window.yml new file mode 100644 index 0000000..14c993c --- /dev/null +++ b/examples/consumers/testdata/weather/null-window.yml @@ -0,0 +1,6 @@ +reports: + daily: + modules: + - module: chunker + options: + window_units: null diff --git a/examples/consumers/testdata/weather/shorthand-module.yml b/examples/consumers/testdata/weather/shorthand-module.yml new file mode 100644 index 0000000..37794f8 --- /dev/null +++ b/examples/consumers/testdata/weather/shorthand-module.yml @@ -0,0 +1,3 @@ +reports: + daily: + modules: [chunker]