Adopt ConfigKit for pipeline loading
This commit is contained in:
@@ -49,8 +49,8 @@ sources` use the same `--config`, `--campaign`, `--campaign-file`, and
|
||||
`--profile` selection rules as pipeline commands, but do not select, discover,
|
||||
or load a session. They do not read credential values or create runtime state.
|
||||
`narratio config diff <left-profile> <right-profile>` uses the same pipeline and
|
||||
campaign selectors, resolves each named profile independently from one parsed
|
||||
root source set, and does not accept a separate `--profile` flag.
|
||||
campaign selectors, resolves each named profile with an independent ConfigKit
|
||||
load, and does not accept a separate `--profile` flag.
|
||||
|
||||
Campaign selection is optional only when the resolved pipeline has no
|
||||
`scriptorium.artifact_families`. When families are declared, Narratio selects a
|
||||
@@ -59,7 +59,7 @@ then parses the campaign-owned party and expands concrete artifacts and any
|
||||
family publish rules before validation. `config validate` prints the resulting
|
||||
root, profile, and effective digest. `config show` emits the normalized
|
||||
effective pipeline YAML, with defaults and concrete expansion included but
|
||||
composition and family declarations omitted. `config sources` prints a stable
|
||||
ConfigKit directives and family declarations omitted. `config sources` prints a stable
|
||||
source projection instead of effective values: root/import/profile/default
|
||||
ownership plus campaign/party and generated-family records. Canonical derived
|
||||
players trace to the party. The [CLI reference](cli.md#config-validate-config-show-config-sources-and-config-diff)
|
||||
@@ -77,8 +77,9 @@ difference.
|
||||
|
||||
Use the [production/testing bundle](../examples/production-testing/pipeline.yml)
|
||||
as the complete copyable migration reference. Split stable pipeline settings
|
||||
into explicit additive imports, place production/testing differences in one
|
||||
selected overlay, and retain a production `default_profile`. Convert campaign
|
||||
into explicit additive imports and place production/testing differences in the
|
||||
fixed `profiles/production.yml` and `profiles/testing.yml` overlays. Convert
|
||||
campaign
|
||||
rosters to [canonical party input](integrations/party.md), then express
|
||||
character work as families. Inspect the result
|
||||
with `config validate`, `config show`, and `config sources`; use `config diff`
|
||||
@@ -96,16 +97,15 @@ remote state with an unsafe legacy identity must be migrated before use.
|
||||
|
||||
## Validation and Merge Rules
|
||||
|
||||
- YAML decode is strict (`KnownFields(true)`) and accepts exactly one document:
|
||||
unknown fields or trailing documents fail load.
|
||||
- ConfigKit performs strict typed YAML decoding and accepts exactly one document;
|
||||
unknown fields, duplicate keys, and trailing documents fail load.
|
||||
- A pipeline file may explicitly import additive YAML fragments through the
|
||||
root-only `composition.imports` list. Imported files contribute fields to one
|
||||
root-only `configkit.imports` list. Imported files contribute fields to one
|
||||
logical pipeline document; they do not override fields supplied by the root
|
||||
or another import.
|
||||
- A root pipeline may declare named profiles. Exactly one profile is selected
|
||||
by an option-aware caller or by `composition.default_profile`; a caller's
|
||||
explicit selection takes precedence. Declaring profiles without either form
|
||||
of selection is an error.
|
||||
- An omitted profile, an empty profile option, and the exact name `default`
|
||||
select the base configuration. Any other name selects
|
||||
`profiles/<name>.yml` relative to the pipeline root.
|
||||
- Configured timeout and retry-delay durations must be positive. An omitted
|
||||
artifact timeout continues to inherit its configured Scriptorium timeout.
|
||||
- Session files must be concrete; unresolved `{{ ... }}` placeholders fail load.
|
||||
@@ -123,22 +123,16 @@ remote state with an unsafe legacy identity must be migrated before use.
|
||||
- local (`audio_dir` or `audio_files`), or
|
||||
- S3 (`audio_s3.prefix`).
|
||||
|
||||
### Pipeline composition
|
||||
### Pipeline imports and profiles
|
||||
|
||||
Large pipeline configurations may be split into explicitly named fragments and
|
||||
may declare one overlay per selectable profile:
|
||||
may use fixed-path profile overlays:
|
||||
|
||||
```yaml
|
||||
composition:
|
||||
configkit:
|
||||
imports:
|
||||
- config/storage.yml
|
||||
- config/integrations.yaml
|
||||
default_profile: production
|
||||
profiles:
|
||||
production:
|
||||
overlay: profiles/production.yml
|
||||
testing:
|
||||
overlay: profiles/testing.yml
|
||||
- path: config/storage.yml
|
||||
- path: config/integrations.yaml
|
||||
|
||||
campaigns:
|
||||
root: /usr/local/share/narratio/campaigns
|
||||
@@ -152,36 +146,29 @@ Imports are resolved relative to the directory containing the root pipeline
|
||||
file and are loaded in declaration order. Narratio does not scan directories or
|
||||
infer fragments. Each import must be a confined regular `.yml` or `.yaml` file:
|
||||
absolute paths, traversal, symlinks, directories, duplicate files, and an
|
||||
import of the root pipeline itself are rejected. Only the root pipeline may
|
||||
contain `composition`; nested composition is rejected.
|
||||
import of the root pipeline itself are rejected. Imported files cannot declare
|
||||
another `configkit` control section.
|
||||
|
||||
Composition is additive. A map may be extended by multiple files when every
|
||||
Base merging is additive. A map may be extended by multiple files when every
|
||||
leaf is distinct, but a scalar, list, or map/list/scalar kind cannot be claimed
|
||||
more than once, even when the repeated values are identical. Conflict errors
|
||||
name the full field path and every source that claimed it. The assembled YAML is
|
||||
then decoded against the normal strict pipeline schema and defaults are applied
|
||||
once.
|
||||
name the full field path and every source that claimed it. Narratio additionally
|
||||
rejects an artifact or artifact-family key declared by more than one base file,
|
||||
even when those files contribute disjoint children.
|
||||
|
||||
Profile names are case-sensitive, non-empty, trimmed, and cannot contain
|
||||
control characters. If `profiles` is present, it must contain at least one
|
||||
entry and every entry must contain only an `overlay` path. An explicit profile
|
||||
selection overrides `default_profile`; unknown and explicitly empty selections
|
||||
fail. Narratio never selects the first profile implicitly and does not read a
|
||||
profile selection from the environment.
|
||||
Profile names are case-sensitive and use ConfigKit's safe fixed-path naming
|
||||
rules. Only the selected overlay is opened; a malformed unselected overlay does
|
||||
not affect a base or different-profile load. Maps merge recursively, overlay
|
||||
scalars replace base scalars, and overlay lists replace base lists completely.
|
||||
Explicit `false`, zero, empty strings, empty containers, and null values remain
|
||||
explicit inputs and are not refilled by defaults. Profiles cannot inherit,
|
||||
stack, import files, or declare another ConfigKit control section.
|
||||
|
||||
Every declared overlay is resolved relative to the root pipeline directory and
|
||||
must satisfy the same confined regular-YAML-file rules as an import. Narratio
|
||||
parses every declared overlay even when it is not selected, then applies only
|
||||
the selected one. Maps merge recursively, overlay scalars replace base scalars,
|
||||
and overlay lists replace base lists completely. Explicit `false`, zero, empty
|
||||
lists, and empty maps remain meaningful. YAML null cannot delete a value, and
|
||||
kind changes are rejected. Profiles cannot inherit from or stack with other
|
||||
profiles, and overlays cannot import files or declare profiles.
|
||||
|
||||
After composition, Narratio strictly decodes the result, applies centralized
|
||||
defaults once, resolves ordinary paths, and computes a deterministic effective
|
||||
During ConfigKit loading, Narratio applies fresh typed defaults, recorded
|
||||
entry-level defaults and normalization, read-only file-local validation, and
|
||||
ordinary path resolution before computing a deterministic effective
|
||||
configuration digest. The digest represents the normalized, secret-free
|
||||
runtime pipeline mapping; it excludes composition declarations, source
|
||||
runtime pipeline mapping; it excludes ConfigKit directives, source
|
||||
provenance, profile identity, and raw environment secret values. Equivalent
|
||||
effective mappings therefore have the same digest regardless of how fields are
|
||||
split among the root and imports.
|
||||
@@ -274,9 +261,7 @@ Rules:
|
||||
|
||||
| Field | Type | Required | Default / Rule |
|
||||
| --- | --- | --- | --- |
|
||||
| `composition.imports[]` | list of strings | No | explicit additive pipeline fragments relative to the root pipeline directory; `.yml` or `.yaml` regular files only |
|
||||
| `composition.default_profile` | string | Conditional | selected when profiles exist and no caller explicitly selects one; must name a declared profile |
|
||||
| `composition.profiles.<name>.overlay` | string | Conditional | required for every declared profile; one confined `.yml` or `.yaml` overlay relative to the root pipeline directory |
|
||||
| `configkit.imports[].path` | string | No | explicit additive pipeline fragment relative to the root pipeline directory; `.yml` or `.yaml` regular files only |
|
||||
| `pipeline.workspace.root` | string | No | `/var/lib/narratio` |
|
||||
| `pipeline.workspace.cleanup_after_publish` | bool | No | `false`; selects the published run's workspace target for final cleanup and requires effective `pipeline.publish.enabled=true` and `pipeline.publish.upload_run=true` |
|
||||
| `pipeline.campaigns.root` | string | No | `/usr/local/share/narratio/campaigns` |
|
||||
|
||||
Reference in New Issue
Block a user