Plan the ConfigKit upgrade and test isolation
This commit is contained in:
149
docs/roadmap/configkit-v0.3.0.md
Normal file
149
docs/roadmap/configkit-v0.3.0.md
Normal file
@@ -0,0 +1,149 @@
|
||||
# Adopt ConfigKit v0.3.0
|
||||
|
||||
Status: Accepted — implementation pending
|
||||
|
||||
Execution order and stage acceptance checks belong to the
|
||||
[implementation plan](implementation.md). This roadmap owns scope, policy, and
|
||||
the target end state.
|
||||
|
||||
## Goal
|
||||
|
||||
Upgrade Narratio from ConfigKit v0.1.2 to the released v0.3.0 while preserving
|
||||
Narratio's configuration contracts. Keep this a bounded dependency-maintenance
|
||||
change, not a new configuration feature or a second configuration refactor.
|
||||
|
||||
The upgrade also includes correcting the test-isolation problems discovered
|
||||
during evaluation so that validation does not depend on whether the developer
|
||||
has installed Narratio configuration on the host.
|
||||
|
||||
## Background and Evaluation
|
||||
|
||||
ConfigKit v0.3.0 adds callback-scoped `Resolution.InputInfo` and
|
||||
`Resolution.Source` queries and separates its public API from its private engine.
|
||||
The public imports, signatures, secret handling, and existing metadata APIs,
|
||||
including `Metadata.HistoricalPaths`, remain supported. Its minimum Go version
|
||||
remains 1.25; Narratio's maintained Go 1.26 baseline needs no change. The private
|
||||
YAML dependency remains `go.yaml.in/yaml/v4 v4.0.0-rc.6`.
|
||||
|
||||
The intervening v0.2.0 release allows decimal-integer schema-version input to
|
||||
reach an application's version checker as text. It also rejects signed sequence
|
||||
index tokens such as `+0` and `-0`; literal mapping keys are unaffected.
|
||||
Narratio constructs sequence paths using ConfigKit's index API and does not need
|
||||
a user-facing path migration.
|
||||
|
||||
ConfigKit now treats macOS as best-effort. This does not require changing
|
||||
Narratio's Linux amd64/arm64 support or its own best-effort macOS arm64
|
||||
cross-build gate.
|
||||
|
||||
The 2026-09-16 evaluation used the released dependency through a temporary module
|
||||
file, without changing Narratio's tracked files:
|
||||
|
||||
- Build, vet, and Linux arm64/macOS arm64 cross-builds passed.
|
||||
- The ordinary suite exposed one upgrade-related test expectation mismatch:
|
||||
integer party schema versions still fail, but with an application-level
|
||||
unsupported-version diagnostic instead of `version.type`.
|
||||
- Configuration race tests passed with that expectation adjusted in a temporary
|
||||
test overlay.
|
||||
- Three CLI cases failed on both v0.1.2 and v0.3.0 because they assumed no default
|
||||
pipeline configuration was installed. These are pre-existing isolation gaps,
|
||||
not demonstrated ConfigKit regressions. Other packages passed the ordinary
|
||||
suite; the overall suite was not green.
|
||||
|
||||
Upstream references are the tagged `docs/releases/v0.2.0.md` and
|
||||
`docs/releases/v0.3.0.md` in the ConfigKit repository. Narratio's current owners
|
||||
remain the [configuration reference](../config.md),
|
||||
[configuration internals](../internal/configuration.md), and
|
||||
[architecture policy](../policy/architecture.md).
|
||||
|
||||
## Required Changes
|
||||
|
||||
### Released dependency
|
||||
|
||||
Pin `gitea.maximumdirect.net/eric/ConfigKit v0.3.0` in `go.mod`, refresh
|
||||
`go.sum`, and retain a tidy module graph. Do not introduce local replacements,
|
||||
a sibling-checkout requirement, vendoring, or workspace-only dependency wiring.
|
||||
Production code must use only ConfigKit's public root package.
|
||||
|
||||
### Party-version regression coverage
|
||||
|
||||
Update `TestParsePartyCanonicalRejectsInvalidDocuments` in
|
||||
`internal/config/party_test.go` so that integer `schema_version: 1` is tested as
|
||||
an unsupported version rather than a ConfigKit type error. Preserve the required
|
||||
`narratio.party.v1` identifier: the upgrade must not make integer party versions
|
||||
valid or introduce coercion for ordinary string fields.
|
||||
|
||||
Keep assertions focused on rejection and the unsupported-version meaning, not
|
||||
exact diagnostic prose. Retain meaningful coverage for malformed version input
|
||||
and strict ordinary-field decoding without duplicating ConfigKit's parser suite.
|
||||
|
||||
### Host-independent CLI tests
|
||||
|
||||
Remove installed-configuration assumptions from these failing cases:
|
||||
|
||||
- `TestExecuteAnalyzeMissingConfigUsesRunStageLoadingPath` and
|
||||
`TestExecutePublishMissingConfigUsesRunStageLoadingPath` in
|
||||
`internal/app/analyze_artifacts_commands_test.go`;
|
||||
- the `run missing config uses defaults` case of
|
||||
`TestExecuteMissingRequiredFlags` in `internal/app/commands_test.go`.
|
||||
|
||||
Use the existing command dependency/discovery boundaries where possible, or a
|
||||
narrow invocation-local seam if needed, to exercise missing-config behavior
|
||||
against controlled temporary locations. Preserve coverage that these commands
|
||||
route through ordinary configuration discovery. Inspect adjacent missing-config
|
||||
tests for the same assumption and repair affected cases within this boundary.
|
||||
|
||||
Tests must neither read operator credentials/configuration nor create, rename,
|
||||
delete, or require absence of files in real system configuration locations.
|
||||
Avoid process-global search-path mutation and do not change production search
|
||||
order merely to satisfy tests. Follow the
|
||||
[testing policy](../policy/testing.md) when selecting the narrowest durable seam.
|
||||
|
||||
### Documentation
|
||||
|
||||
Update current documentation only where adoption changes an actual claim.
|
||||
Record the dependency upgrade and diagnostic adjustment in the applicable
|
||||
unreleased release note; do not rewrite already-published historical notes.
|
||||
No operator configuration migration is expected. If no unreleased note exists,
|
||||
retain the summary in the implementation completion record until a separate
|
||||
release task selects a version; published notes remain immutable.
|
||||
|
||||
## Preserved Behavior and Non-Goals
|
||||
|
||||
- Pipeline paths remain relative to the root pipeline file, not the importing
|
||||
file that declares a value. Do not adopt the new resolver provenance APIs
|
||||
merely because they are available.
|
||||
- Imports, profile selection, explicit-null/default semantics, unique base
|
||||
artifact ownership, party expansion, dependency inference, and safe
|
||||
show/diff/source/catalog projections retain their existing behavior.
|
||||
- Resolve only effective credential references. Profiles can replace or remove
|
||||
unused base credentials, and hidden duplicate definitions remain rejected
|
||||
through historical metadata without independently loading the base.
|
||||
- Secret bootstrap, redaction, runtime adapter delivery, and resume evidence
|
||||
remain unchanged. The separate
|
||||
[Notarius credential-forwarding proposal](notarius-credential-forwarding.md)
|
||||
is not implemented by this upgrade.
|
||||
- No new schema versions, CLI flags, path policy, supported platforms, Go
|
||||
baseline change, or upstream modifications are required.
|
||||
- No performance improvement is claimed. The primary benefit is maintaining
|
||||
compatibility with the current upstream library and its explicit public API.
|
||||
|
||||
## Validation and Target End State
|
||||
|
||||
The released dependency works without consumer production-code migration except
|
||||
any narrowly justified testability wiring. Existing valid configurations keep
|
||||
their meaning, and integer party versions remain invalid with a truthful
|
||||
unsupported-version diagnostic. Missing-config tests pass regardless of host
|
||||
installation state.
|
||||
|
||||
Acceptance requires the full validation set in the
|
||||
[development guide](../development.md): ordinary and race tests, vet, build,
|
||||
documentation links, and maintained examples. Verify module tidiness and all
|
||||
three Narratio cross-build targets. Recheck profile credential replacement/null
|
||||
removal, hidden duplicate ownership, provenance, and credential-safe inspection
|
||||
through existing regression coverage; add tests only for uncovered meaningful
|
||||
risks. Validation must not rely on skipped failures or temporary test overlays.
|
||||
|
||||
Before publication, run the normal
|
||||
[release-candidate procedure](../release.md). Cross-compilation is not native
|
||||
macOS runtime evidence. Tagging, pushing, and release publication are separate
|
||||
actions, not part of implementing this roadmap.
|
||||
@@ -1,522 +1,201 @@
|
||||
# ConfigKit Adoption Implementation Plan
|
||||
|
||||
Status: Complete
|
||||
|
||||
Implement the [ConfigKit adoption roadmap](configkit-adoption.md) in the order
|
||||
below. The target is Narratio v2.0.0: ConfigKit v0.1.1, the reduced platform
|
||||
matrix, and the artifact-authoring workflow described there. Each stage is a
|
||||
bounded implementation assignment for a gpt-5.6-terra coding agent. Mark a stage
|
||||
complete only after its changes and acceptance checks succeed; record a short
|
||||
completion and verification note beside its status.
|
||||
|
||||
## Instructions for every stage
|
||||
|
||||
- Read `docs/development.md` and all three policies under `docs/policy/`.
|
||||
Read the relevant roadmap section and the stage-specific sources listed below.
|
||||
Paths identify starting points, not an exhaustive edit allowlist; inspect callers
|
||||
and tests before changing a package boundary.
|
||||
- Use the tagged ConfigKit public API. The local `../ConfigKit` checkout is useful
|
||||
reference material, but verify its version before relying on it. The module
|
||||
cache for v0.1.1 is an alternative. Do not require a sibling checkout, a local
|
||||
`replace`, a Go workspace, an upstream patch, or an unpinned revision to build.
|
||||
- Keep Narratio domain policy in `internal/config`, startup and selectors in
|
||||
`internal/app`, and credential delivery in the owning adapters. Do not build
|
||||
a general configuration framework around ConfigKit.
|
||||
- Keep each stage buildable and run focused behavioral tests for its changed
|
||||
boundary. Temporary migration helpers may exist between stages only where
|
||||
named here; they must not introduce runtime format auto-detection or fallback.
|
||||
- Update each canonical current-behavior document when its behavior changes.
|
||||
The roadmap remains the owner of unimplemented work. Preserve historical
|
||||
release notes; their descriptions of old platforms and formats are history.
|
||||
- Tests use real ConfigKit and temporary files, with fake external adapters or
|
||||
storage where needed. Use synthetic credentials and isolate environment
|
||||
mutation; such tests must not run in parallel. Retain domain and security
|
||||
coverage, remove superseded mechanism tests, and avoid exact diagnostic prose
|
||||
or private callback choreography assertions.
|
||||
- Do not tag or publish v2.0.0 as part of this plan. Release preparation belongs
|
||||
here; release authorization and tag publication are a later task. The existing
|
||||
release completion boundary remains successful upstream tag push, with no CI
|
||||
polling or waiting.
|
||||
|
||||
## Decisions that apply across stages
|
||||
|
||||
The roadmap is authoritative for product scope. These implementation choices
|
||||
make that scope concrete:
|
||||
|
||||
- Enable ConfigKit imports and profile selection for pipeline files only.
|
||||
Campaign, session, and party are single-source definitions with imports and
|
||||
profiles disabled. Preserve their domain schemas and source-relative inputs.
|
||||
Omitted or empty profile and the exact name `default` select the base; other
|
||||
names use ConfigKit's fixed `profiles/<name>.yml` convention.
|
||||
- Defaults apply to absent fields. Explicit empty/zero/false/null values follow
|
||||
ConfigKit decoding and then domain validation; do not refill invalid explicit
|
||||
values with legacy defaults. Optional null stage sections remain absent;
|
||||
required sections fail domain validation rather than being resurrected.
|
||||
- Keep ConfigKit metadata/result snapshots separate from runtime-only Narratio
|
||||
state. ConfigKit discards ignored and unexported fields when detaching values;
|
||||
attach root paths, retained party bytes, family catalogs, and digest metadata
|
||||
after loading. No validator may mutate bound configuration.
|
||||
- Use one Narratio-owned safe effective projection for the final expanded
|
||||
pipeline. It contains configuration values and credential reference names,
|
||||
never resolved credential bytes. It feeds show, semantic diff, digest, and
|
||||
catalog data. ConfigKit metadata remains the authority for input provenance;
|
||||
Narratio adds only domain-generated attribution.
|
||||
- Rename `storage.s3.access_key_id_env` to `access_key_id`,
|
||||
`storage.s3.secret_access_key_env` to `secret_access_key`, and
|
||||
`audita.llm_api_key_env` to `llm_api_key`. Each is an optional `*configkit.Secret`.
|
||||
An explicit reference always resolves during loading, even if the associated
|
||||
stage will not execute. Require both S3 fields together or neither. With neither,
|
||||
retain the AWS SDK default credential chain; do not synthesize mandatory secret
|
||||
references that would break role-based authentication or local-only inspection.
|
||||
With no Audita reference, preserve its existing downstream credential discovery.
|
||||
Explicit references replace Narratio's old per-adapter environment lookup.
|
||||
- Keep absent S3 credentials out of defaults; migration examples explicitly name
|
||||
the existing AWS variables when static credentials are intended. A non-nil
|
||||
missing/empty reference fails instead of silently falling back. This preserves
|
||||
provider-managed credentials while making explicit configuration trustworthy.
|
||||
- `config artifacts` uses the existing configuration command selectors and text
|
||||
output convention. No new JSON mode, artifact creation wizard, inferred filename
|
||||
identity, or profile catalog is added.
|
||||
- Preserve the roadmap's unique base artifact definition rule explicitly:
|
||||
ConfigKit rejects conflicting leaves but can compose disjoint children of the
|
||||
same mapping. Use its public source metadata to reject an ordinary artifact key
|
||||
declared by multiple base files, even if their fields are disjoint. Apply the
|
||||
same rule to family keys. A selected profile may override or extend a definition.
|
||||
This is a narrow Narratio declaration rule, not a different merge algorithm.
|
||||
- Relax only the obsolete configuration/secrets filesystem restrictions identified
|
||||
by the roadmap. Do not remove session restore, cleanup, publication, workspace
|
||||
confinement, or ordinary shared-workspace mode guarantees as collateral work.
|
||||
|
||||
## Stage 1 — Reduce the platform and release matrix
|
||||
|
||||
Status: Complete — Reduced CI and release output to Linux amd64/arm64 and macOS
|
||||
arm64, removed Windows-only runtime implementations and accommodations, and
|
||||
verified focused tests, all three cross-builds, exact asset names, and embedded
|
||||
version reporting.
|
||||
|
||||
Read: roadmap “Platform Contract”; `scripts/build-release-assets.sh`,
|
||||
`scripts/check-release-candidate.sh`, `.woodpecker/`, `internal/releasecheck`,
|
||||
`docs/release.md`, and platform-specific files in application, file operations,
|
||||
and subprocess packages.
|
||||
|
||||
1. Make Linux amd64, Linux arm64, and macOS arm64 the only maintained build and
|
||||
release targets. Remove Windows suffix/packaging branches and macOS amd64
|
||||
targets from scripts, workflows, tests, and current documentation.
|
||||
2. Delete Windows-only implementations and tests; simplify paired build-tag
|
||||
files where the remaining implementations can be shared. Preserve portable
|
||||
code when retaining it adds no special support machinery. Preserve functioning
|
||||
Unix subprocess cancellation, locking, and file operations on both supported OSes.
|
||||
3. Retain the local Linux verification gates and cross-build gates for all three
|
||||
published assets. Do not introduce a native macOS runner requirement or claim
|
||||
that cross-compilation verifies runtime behavior.
|
||||
4. Update architecture/operations platform statements and contributor/release
|
||||
guidance. Keep existing release protections and the no-CI-wait boundary intact.
|
||||
|
||||
Acceptance: focused platform/subprocess/fileops/releasecheck tests pass; build all
|
||||
three targets with `CGO_ENABLED=0`; verify the asset builder produces exactly the
|
||||
three expected versioned binaries and the host binary reports the supplied version.
|
||||
Existing release tests still prove guarded publication. No current support promise
|
||||
or asset list retains Windows or macOS amd64.
|
||||
|
||||
## Stage 2 — Separate configuration normalization from validation
|
||||
|
||||
Status: Complete — Added explicit deterministic pipeline normalization before
|
||||
validation, made Notarius validation read-only on accepted and rejected inputs,
|
||||
documented the boundary, and verified config race tests plus app/stage consumers.
|
||||
|
||||
Read: `internal/config/load.go`, `defaults.go`, `normalize.go`, `validate*.go`,
|
||||
`artifact_families.go`, and ConfigKit's “Recorded application resolution”,
|
||||
“Semantic validation”, and “Defaults and typed decoding” contracts.
|
||||
|
||||
1. Separate value-producing normalization from read-only validation, including
|
||||
Notarius reference/output normalization currently performed by
|
||||
`validateNotarius`. Audit all validators for map, slice, pointer, and scalar
|
||||
mutations, including changes that would occur only on error paths.
|
||||
2. Keep reusable domain normalization ordinary package code. Validation checks the
|
||||
already normalized candidate and must work for both loaded and programmatically
|
||||
constructed values without silently fixing them.
|
||||
3. Inventory defaults in `load.go`: static defaults, per-map-entry/list-element
|
||||
defaults, and inherited or presence-sensitive defaults. Retain named constants
|
||||
in `defaults.go`. This inventory should guide code and concise comments, not
|
||||
become a permanent parallel defaults document.
|
||||
4. Keep the current loader calling normalization explicitly before validation
|
||||
during this preparatory stage. The ConfigKit resolver replaces that loading
|
||||
choreography in Stage 4. Preserve family expansion as an explicit domain step.
|
||||
|
||||
Acceptance: focused configuration tests prove normalization is deterministic and
|
||||
validation does not mutate accepted or rejected inputs. Cover representative
|
||||
Notarius normalization/collision handling and existing artifact/publish rules.
|
||||
Do not add a separate mutation test for every trivial validator.
|
||||
|
||||
## Stage 3 — Replace composition-dependent effective projection
|
||||
|
||||
Status: Complete — Introduced one typed safe projection for effective YAML,
|
||||
semantic values, source paths, and the versioned v3 digest; removed composition
|
||||
parsing from those consumers and verified ordering, expansion, relocation,
|
||||
nil/empty, exact-integer, CLI diff, and semantic-resume behavior.
|
||||
|
||||
Read: `internal/config/effective_pipeline.go`, `effective_digest.go`,
|
||||
`pipeline_composition.go`, `internal/app/config_commands.go`, the config-diff
|
||||
implementation, and `docs/internal/configuration.md`.
|
||||
|
||||
1. Introduce one package-owned, schema-specific safe projection of the effective
|
||||
pipeline. Use explicit typed projections with stable configuration field names;
|
||||
preserve null versus empty containers and exact integers. Ordinary nested
|
||||
configuration types may be reused where they contain no runtime state.
|
||||
Credential fields must be projected explicitly as reference names when Stage 7
|
||||
changes their types. Do not introduce a generic reflection schema engine.
|
||||
2. Render this projection as deterministic YAML using the existing YAML encoder;
|
||||
encoding output is not a replacement YAML loader. Produce semantic records
|
||||
directly from the safe projection using standard JSON encoding/decoding with
|
||||
number preservation if a plain value tree is useful. Do not reparse YAML or
|
||||
reconstruct `compositionNode` trees to compare or hash effective values.
|
||||
3. Make show, diff values, and digest use that common projection. Mapping order is
|
||||
immaterial; sequences remain atomic for diff; nil and empty remain distinct.
|
||||
Exclude family declarations once expanded, all runtime metadata, profile names,
|
||||
source locations, and raw credentials. Preserve logical Notarius path handling
|
||||
so moving an equivalent bundle does not change its digest.
|
||||
4. Version the changed digest encoding explicitly using the package's digest
|
||||
version constant. Digest/profile metadata remains provenance, never a blanket
|
||||
resume cache key. Keep stage/artifact semantic evidence behavior intact.
|
||||
5. Decouple enumeration of effective field paths from the legacy composition
|
||||
renderer. Input ownership can still come from the current loader until Stage 4.
|
||||
|
||||
Acceptance: show/diff/digest tests cover equivalent monolithic/split values,
|
||||
different map insertion order, null/empty distinction, family-expanded outputs,
|
||||
and bundle relocation. A meaningful artifact change changes the digest; formatting
|
||||
and profile naming alone do not. Existing semantic resume tests pass. No effective
|
||||
projection consumer needs the custom composition parser/renderer.
|
||||
|
||||
## Stage 4 — Cut pipeline loading over to ConfigKit
|
||||
|
||||
Status: Complete — Replaced pipeline composition with ConfigKit v0.1.1 imports,
|
||||
fixed-path profile selection, fresh typed defaults, recorded resolution, source
|
||||
metadata adaptation, and duplicate base artifact/family enforcement; removed the
|
||||
legacy parser and verified all tests plus the configuration race suite.
|
||||
|
||||
Read: ConfigKit v0.1.1 `docs/config.md`, `docs/consumers/README.md`, and its
|
||||
Narratio-shaped compiled example; Narratio `internal/config/load.go`,
|
||||
`composition.go`, `pipeline_composition.go`, and profile/composition tests.
|
||||
|
||||
1. Pin `gitea.maximumdirect.net/eric/ConfigKit v0.1.1` in `go.mod`/`go.sum`.
|
||||
Implement a small pipeline definition using `Load`, `File`, enabled imports,
|
||||
`Options.Profile`, defaults, recorded resolution, and read-only validation.
|
||||
2. Migrate static defaults to fresh default values. Apply map-entry, list-element,
|
||||
and inherited defaults with `Resolution.Default`/`IsExplicit`; use recorded
|
||||
normalization for transformations from Stage 2. Do not call old post-decode
|
||||
zero-filling defaults. Remove `NormalizeConfig.UnmarshalYAML` and its private
|
||||
presence bit. Include publication list-entry defaults, optional section
|
||||
allocation, and trim's Scriptorium requirement in this conversion.
|
||||
3. Perform file-local semantic checks in the definition. Defer checks needing
|
||||
canonical party expansion or campaign/session context to the existing explicit
|
||||
resolution boundary; do not reject references to future concrete family members
|
||||
merely because the campaign has not loaded yet. Run full checks after expansion.
|
||||
4. Replace pipeline composition and profile loaders. `configkit.imports` is the
|
||||
only directive; no `composition` or `_configkit` alias is accepted. Pass profile
|
||||
selection directly under the agreed convention; unselected overlays are not read.
|
||||
`LoadPipelineProfilePair` may remain a thin pair of independent loads, with no
|
||||
shared parse cache or paired source-snapshot promise.
|
||||
5. Adapt ConfigKit sources and metadata to Narratio's existing reporting records.
|
||||
Translate paths structurally using public path APIs, not error-message parsing.
|
||||
Preserve multiple contributing locations for composite/profile-owned mappings
|
||||
and distinguish defaults/resolution from authored values.
|
||||
Check artifact/family map-entry ownership for duplicate base declarations using
|
||||
those locations; ignore synthetic defaults and allow the selected overlay.
|
||||
6. Attach runtime metadata only after load succeeds. Preserve root-relative runtime
|
||||
path interpretation and capture logical paths before making them absolute.
|
||||
Retain family declarations and original ConfigKit metadata for later expansion.
|
||||
7. Replace generic mechanism tests with a small integration set proving actual
|
||||
Narratio wiring. Convert fixtures affected by this stage. Delete
|
||||
`composition.go`, obsolete pipeline traversal/overlay/presence machinery, and
|
||||
their redundant tests once the callers are migrated.
|
||||
|
||||
Acceptance: a real root with imports and a testing overlay loads; duplicate claims
|
||||
fail with useful source attribution, including two base files contributing disjoint
|
||||
fields to one artifact or family key; base/default selection works; malformed
|
||||
unselected profiles do not fail base loading. Omitted defaults and representative
|
||||
explicit false/zero/empty/null inputs obey the new semantics. Invalid domain values
|
||||
still fail. Commands use the new pipeline loader with no format fallback. Update
|
||||
`docs/config.md` and configuration internals for the implemented boundary.
|
||||
|
||||
## Stage 5 — Migrate campaign, session, and canonical party loading
|
||||
|
||||
Status: Complete — Migrated campaign, session, and canonical-party documents to
|
||||
independent ConfigKit definitions, preserved bounded session inspection, exact
|
||||
party bytes and metadata-derived authored order, and retained identity, source,
|
||||
family-expansion, and validation behavior; verified all tests and focused races.
|
||||
|
||||
Read: `internal/config/load.go`, `party.go`, `party_resolution.go`,
|
||||
`artifact_families.go`, session-template tests, `internal/app/config_loader.go`,
|
||||
and ConfigKit source/schema/version contracts.
|
||||
|
||||
1. Use separate ConfigKit definitions for campaign, session, and party. Replace
|
||||
`decodeStrictYAML*` and the custom party YAML-node parser. Model the party input
|
||||
as the existing string schema version and character-keyed typed map, with the
|
||||
existing `alias` list, ordered class list, and optional positive class level.
|
||||
2. Keep party domain checks: valid IDs, required display names and classes,
|
||||
trimmed/control-free names, case-insensitive name/alias ambiguity, and duplicate
|
||||
class checks. Validate the required version through `VersionRule`. Keep an
|
||||
exact independent copy of authored bytes outside ConfigKit's schema object.
|
||||
3. Preserve authored character order using successful metadata source coordinates
|
||||
for the single party input when constructing `CanonicalParty.Characters`.
|
||||
Preserve class/alias sequence order naturally. Family expansion and players
|
||||
projection continue using their existing deterministic ID sorting. Do not keep
|
||||
a second YAML parse just to recover mapping order.
|
||||
4. Preserve the narrow session placeholder rejection before typed load, expected
|
||||
session/previous-session identity checks, campaign selection, stable-input
|
||||
precedence, and local versus remotely downloaded session provenance.
|
||||
Retain the initially loaded pipeline/campaign during remote session acquisition.
|
||||
5. Where bytes must be retained or inspected before `ConfigKit.Bytes`, bound the
|
||||
initial read too, using the existing fileops bounded-read helper and a named
|
||||
package constant of 8 MiB matching the pinned source limit. Otherwise prefer
|
||||
`ConfigKit.File`. Do not read an unbounded file and rely on later rejection.
|
||||
6. Complete post-party family and publish-rule expansion, then full cross-config
|
||||
validation and digest refresh. Keep generated origins outside the library
|
||||
snapshot. Remove unused decoder plumbing and obsolete party parsing tests.
|
||||
|
||||
Acceptance: retained authored party bytes are identical; domain errors, aliases,
|
||||
class order, family origins, and derived players are correct. Local and remote
|
||||
session loads preserve identity, input path bases, and the original pipeline.
|
||||
Unknown fields and disabled directives fail through ConfigKit. Focused config,
|
||||
prepare/extract, and application loading tests pass.
|
||||
|
||||
## Stage 6 — Move secrets bootstrap before all configuration loads
|
||||
|
||||
Status: Complete — Added invocation-scoped ConfigKit secrets bootstrap before
|
||||
all pipeline loads and adapter construction, exposed consistent explicit and
|
||||
conventional directory selection across the CLI, removed the retired pipeline
|
||||
schema and custom loader, and verified focused, full, and race test suites.
|
||||
|
||||
Read: roadmap “Credential and Secrets Convention”; ConfigKit's bootstrap
|
||||
contract; `internal/app/secrets_env*.go`, `config_loader.go`, `config_commands.go`,
|
||||
`object_store.go`, command dependencies, and command argument owners.
|
||||
|
||||
1. Add a shared `--secrets-dir` selector to every CLI path that loads a pipeline,
|
||||
including configuration inspection/diff, execution aliases, planning, publish,
|
||||
and session/storage operations. Route all through one command-owned bootstrap
|
||||
context. Help/version and commands that do not load configuration do not bootstrap.
|
||||
2. Discover the root pipeline first. An explicit relative directory is based on
|
||||
caller CWD; otherwise choose `secrets/` beside the resolved root pipeline.
|
||||
An explicitly empty flag is invalid. Missing conventional directories are
|
||||
skipped; missing explicit directories and unreadable/invalid existing directories
|
||||
fail. Use ConfigKit's selected-directory symlink behavior without custom rejection.
|
||||
3. Call `LoadSecretsDir` at most once per command before any ConfigKit load or
|
||||
adapter creation, including both loads of config diff and remote session lookup.
|
||||
Use invocation state, not a package-global `sync.Once`. A new command gets its
|
||||
own bootstrap; library config loaders themselves never mutate the environment.
|
||||
4. Present safe bootstrap warnings/failures through stderr or the existing safe
|
||||
diagnostic boundary, never into YAML show output. Preserve the partial report
|
||||
on bootstrap failure. Do not add rollback or reinterpret ConfigKit file rules.
|
||||
5. Remove late bootstrap calls in object-store and runner composition, the custom
|
||||
directory reader and permission platform files, `pipeline.secrets`, its defaults
|
||||
and validators, and their obsolete tests. Convert affected fixtures immediately.
|
||||
|
||||
Acceptance: test absent conventional directory, explicit override from another
|
||||
CWD, invalid explicit directory, already-set empty/nonempty environment values,
|
||||
and bootstrap failure before adapter construction. A diff invocation and a remote
|
||||
session acquisition each bootstrap once. Config inspection requires no session,
|
||||
workspace writes, manifests, or runtime clients. Update CLI, config, operations,
|
||||
architecture, and troubleshooting to the new bootstrap/permission contract.
|
||||
|
||||
## Stage 7 — Resolve typed credentials and protect all output boundaries
|
||||
|
||||
Status: Complete — Migrated S3 and Audita credentials to optional ConfigKit
|
||||
typed references, passed resolved values directly to runtime adapters, preserved
|
||||
the AWS default provider chain when unconfigured, and verified reference-only
|
||||
inspection, provenance, generated configuration, manifest, and diagnostic
|
||||
boundaries with focused, full, and race test suites.
|
||||
|
||||
Read: ConfigKit “Typed secret references” and rendering contracts;
|
||||
`internal/config/config.go`, `defaults.go`, `validate_adapters.go`, the safe
|
||||
projection from Stage 3, `internal/adapters/storage/s3_backend.go`,
|
||||
`internal/adapters/audita/subprocess.go`, and their composition callers.
|
||||
|
||||
1. Implement the three renamed optional typed fields and S3 pair policy above.
|
||||
Reject old `_env` schema keys; remove their default-name and lookup helpers.
|
||||
Convert maintained fixtures with synthetic environment setup where references
|
||||
are present. Do not add credentials to unrelated local pipeline fixtures.
|
||||
2. Pass resolved S3 values to the SDK adapter's explicit static provider only when
|
||||
the pair is configured. With neither, preserve its default provider chain.
|
||||
Pass Audita the resolved value and safe reference name; deliver the value only
|
||||
through the child environment's existing `AUDITA_LLM_API_KEY` override.
|
||||
Do not re-read the environment for an explicitly resolved reference.
|
||||
3. Adapt every serialization, digest, source/diff output, diagnostic, generated
|
||||
invocation config, manifest, and provenance caller to safe reference names.
|
||||
Never YAML-marshal ConfigKit's opaque secret directly or call `Value()` in an
|
||||
inspection path. Secret changes alone must not alter effective digests or
|
||||
invalidate artifact evidence; reference-name changes may change provenance.
|
||||
4. Keep ConfigKit errors structured and safe. Domain validation messages must
|
||||
avoid raw credential values; do not unwrap private parser errors or interpolate
|
||||
whole trusted configuration structs into error messages.
|
||||
5. Preserve downstream tool ownership for Notarius/Scriptorium credentials that
|
||||
Narratio does not configure. Do not add a parallel credential schema for them.
|
||||
|
||||
Acceptance: use distinctive synthetic secrets to prove no leakage in show, sources,
|
||||
diff, catalog-ready projections, success/failure diagnostics, manifests, or generated
|
||||
Audita configuration. Verify the child receives the credential and an environment
|
||||
change after loading does not change explicit delivery. Test S3 paired references,
|
||||
partial-pair rejection, missing effective references, and nil SDK fallback offline.
|
||||
Update adapter and credential documentation and remove obsolete lookup tests.
|
||||
|
||||
## Stage 8 — Infer and normalize artifact dependencies once
|
||||
|
||||
Status: Complete — Finalized one normalized artifact graph from explicit ordering
|
||||
edges and configured-artifact inputs (including rewritten family members), retained
|
||||
edge origins for diagnostics, and verified focused/full tests plus the full race
|
||||
suite and an assembled input-only analyze workflow.
|
||||
|
||||
Read: `internal/config/validate_scriptorium.go`, `artifact_families.go`,
|
||||
`internal/artifactpolicy`, `internal/artifacts`, analyze planning/execution, and
|
||||
their artifact dependency and semantic-evidence tests.
|
||||
|
||||
1. At the final configuration resolution boundary, union explicit `depends_on`
|
||||
with configured-artifact input references. Use the existing source-ID parser;
|
||||
do not infer edges from filenames, arbitrary strings, or non-artifact inputs.
|
||||
2. Infer after family member inputs become concrete `narratio.artifact.<key>`
|
||||
sources. Normalize, deduplicate, and sort the effective dependency list once.
|
||||
Keep explicit ordering-only dependencies valid. Repeated finalization must not
|
||||
accumulate edges or alter ordering. Store source-edge attribution separately
|
||||
from the effective graph for diagnostics and generated provenance.
|
||||
3. Apply existing existence, self-reference, enabled-dependency, output, and cycle
|
||||
validation to the union. Remove the error requiring an input to be repeated in
|
||||
`depends_on`. Retain diagnostics pointing to the explicit list entry or the
|
||||
input that contributed an invalid edge, including family origin where relevant.
|
||||
4. Ensure execution, targeted analysis, planning, prerequisite closure, reuse,
|
||||
forced regeneration, and semantic evidence consume this same concrete graph.
|
||||
No adapter or command may infer a second graph. Refresh final projection/digest
|
||||
after inference; equivalent explicit and inferred declarations compare equally.
|
||||
|
||||
Acceptance: cover input-only dependency ordering, duplicate explicit/inferred edges,
|
||||
ordering-only edges, missing/self/cyclic references, and rewritten family inputs.
|
||||
Use an assembled fake-artifact workflow to prove upstream output is supplied to a
|
||||
dependent artifact and failure/resume retains successful predecessors. Existing
|
||||
force/selection and invalidation tests remain valid. Update config and analyze
|
||||
internals with the implemented inference rule.
|
||||
|
||||
## Stage 9 — Add the effective artifact catalog command
|
||||
|
||||
Status: Complete — Added the read-only `config artifacts` catalog over finalized
|
||||
configuration, including effective dependencies and definition/family/party
|
||||
provenance; documented it and verified focused, full, and race test suites.
|
||||
|
||||
Read: `internal/app/config_commands.go`, config diff/source rendering, the safe
|
||||
projection and family-origin catalogs, and `docs/cli.md`.
|
||||
|
||||
1. Add `narratio config artifacts` to dispatch and help. Reuse ordinary pipeline,
|
||||
profile, campaign, campaign-file, and secrets-directory parsing/resolution.
|
||||
Resolve a campaign when families require it or when explicitly selected; a
|
||||
pipeline with only ordinary artifacts can be inspected without a campaign.
|
||||
2. Render one stable record per effective concrete artifact in key order with
|
||||
key, `narratio.artifact.<key>` source ID, defining source location(s), enabled
|
||||
state, output path, and sorted effective dependencies. Family members additionally
|
||||
identify family key, character ID, defining family source, and party source.
|
||||
3. Derive identity from the map key. A profile may contribute to an existing
|
||||
artifact, so report contributing definition locations instead of inventing
|
||||
a single filename owner. Use a deterministic escaped tabular text rendering;
|
||||
include a header and no records for an empty catalog. Propagate writer errors.
|
||||
4. Use the same finalized state as show/sources/diff and runtime configuration.
|
||||
No session download, adapter initialization, manifest access, or workspace
|
||||
materialization is allowed. Secret bootstrap remains the agreed startup exception.
|
||||
|
||||
Acceptance: CLI tests cover ordinary and generated artifacts, disabled entries,
|
||||
profile-added/overridden artifacts, empty catalog, no-campaign ordinary inspection,
|
||||
required-campaign failure for families, stable ordering, and no runtime side effects.
|
||||
Check identity stays the same after moving an artifact file without changing its key.
|
||||
Document the command in its canonical CLI owner and link to config layout examples.
|
||||
|
||||
## Stage 10 — Finish the operator layout and v2 migration documentation
|
||||
|
||||
Status: Complete
|
||||
|
||||
Completion: Reworked the maintained bundle around an unprofiled production base,
|
||||
one-entry artifact and family imports, and a testing overlay; updated the example
|
||||
harness and inspection workflows; consolidated the v2 configuration guidance; and
|
||||
added the v2.0.0 operator migration checklist. Focused documentation/example tests,
|
||||
the offline inspection walkthrough, and the full normal/race suites pass.
|
||||
|
||||
Read: roadmap “Artifact Definition Ergonomics” and “Compatibility and Migration”;
|
||||
`examples/README.md`, existing example bundles and their loader test,
|
||||
`docs/config.md`, `docs/operations.md`, `docs/release.md`, and internal configuration
|
||||
and artifact documentation.
|
||||
|
||||
1. Convert maintained pipeline examples to a production base with explicit
|
||||
`configkit.imports`, `artifacts/<key>.yml`, `artifact-families/<key>.yml`, and
|
||||
`profiles/testing.yml`. Each recommended artifact/family file defines exactly
|
||||
one explicit map entry. Fold old production overlays into the base.
|
||||
2. Include an input-only dependency example and at least two character-family
|
||||
examples sharing a campaign-owned party roster. Keep supported inline artifacts
|
||||
legal; file layout is guidance, not a filename-based validation requirement.
|
||||
3. Update the example validation harness to load root configurations and selected
|
||||
testing profiles, not attempt to load each imported fragment as a full pipeline.
|
||||
Use synthetic environment setup for explicit references. No example includes
|
||||
real credentials or depends on local tools/cloud services to validate.
|
||||
4. Consolidate canonical documentation updates from earlier stages. Remove current
|
||||
references to old composition/profile catalogs, default-profile configuration,
|
||||
secrets.env_dir, credential `_env` fields, credential-free inspection guarantees,
|
||||
paired diff snapshots, and removed platform support. Explain explicit-value and
|
||||
null changes without promising reloadable inspection output.
|
||||
5. Add `docs/releases/v2.0.0.md` using the release-note structure, with a concrete
|
||||
migration checklist linking to the canonical config/CLI docs and examples.
|
||||
Include static S3-reference migration versus SDK credential discovery, new
|
||||
inspection prerequisites, artifact input dependency inference, and platform
|
||||
removals. Do not claim publication or edit older immutable release notes.
|
||||
|
||||
Acceptance: `go test ./internal/doccheck` and
|
||||
`go test ./internal/config -run '^TestExamplesLoadAndValidate$'` pass. Walk through
|
||||
the new operator examples with config validate/show/sources/diff/artifacts offline.
|
||||
Their identities, family members, origins, and testing overrides agree.
|
||||
|
||||
## Stage 11 — Verify the complete migration and remove remaining scaffolding
|
||||
|
||||
Status: Complete
|
||||
|
||||
Completion: Audited the accepted roadmap and the full change set from `1558c9c`;
|
||||
no legacy configuration loader, profile/composition traversal, presence store,
|
||||
custom secrets loader, or production YAML decoder remains. The migration removes a
|
||||
net 1,235 production and 843 test Go lines; remaining configuration code owns domain
|
||||
defaults, normalization, validation, expansion, provenance, and safe projections.
|
||||
Go 1.26.7 module hygiene, tests, race tests, vet, build, documentation, examples,
|
||||
and the v2.0.0 release-candidate checker pass. Linux amd64 was tested natively;
|
||||
Linux arm64 and macOS arm64 release assets were cross-built, with native macOS
|
||||
arm64 validation remaining best effort and outside the release gate.
|
||||
|
||||
Read: the whole feature roadmap, completed stage notes, and the current test and
|
||||
release procedures. Review the diff from the commit introducing this plan.
|
||||
|
||||
1. Audit every roadmap requirement against current code and executable evidence.
|
||||
Remove temporary migration adapters and any residual generic YAML parser,
|
||||
composer, profile traversal, explicit-presence store, old secrets loader,
|
||||
decoder hooks, or unused tests. `gopkg.in/yaml.v3` may remain for output encoding
|
||||
and unrelated external formats; it must not supply a second configuration loader.
|
||||
2. Confirm result-owned snapshots versus final expanded configuration are handled
|
||||
consistently; every inspection/runtime path sees normalized inferred dependencies
|
||||
and generated publish rules. Recheck secret safety and semantic resume boundaries.
|
||||
3. Inspect current platform conditions and asset lists. Retain historical notes but
|
||||
no active Windows/macOS amd64 obligation. Compare owned production/test code
|
||||
before and after; report removals and any remaining complexity with reasons.
|
||||
Do not chase a LOC quota by removing useful domain checks or regressions.
|
||||
4. Run the repository gates with the maintained Go 1.26 toolchain:
|
||||
|
||||
```sh
|
||||
go test ./...
|
||||
go test -race ./...
|
||||
go vet ./...
|
||||
go build ./...
|
||||
go test ./internal/doccheck
|
||||
go test ./internal/config -run '^TestExamplesLoadAndValidate$'
|
||||
```
|
||||
|
||||
5. Run `scripts/check-release-candidate.sh v2.0.0` after the release note and all
|
||||
candidate changes exist. This also verifies module hygiene and all three
|
||||
release assets with `GOWORK=off`. Resolve failures; do not bypass checks or
|
||||
publish a tag. Record native versus cross-build evidence accurately.
|
||||
|
||||
Acceptance: all gates pass and every roadmap outcome has direct evidence. No
|
||||
dependency on the sibling ConfigKit checkout remains. Record completion and any
|
||||
best-effort platform limitations, then hand off for a separate release decision.
|
||||
# ConfigKit v0.3.0 Implementation Plan
|
||||
|
||||
Status: Not started
|
||||
|
||||
Implement the [feature roadmap](configkit-v0.3.0.md) in sequential stage order.
|
||||
Each stage is a bounded assignment for a gpt-5.6-terra coding agent. Mark it
|
||||
complete only after its acceptance checks pass, and record the commands and
|
||||
results beside its status. This replaces the completed original ConfigKit
|
||||
migration plan; that history remains in Git.
|
||||
|
||||
## Working rules and fixed decisions
|
||||
|
||||
- Read `docs/development.md`, all documents under `docs/policy/`, the feature
|
||||
roadmap, and the stage-specific sources before editing.
|
||||
- Scope is the ConfigKit upgrade and the identified configuration-discovery test
|
||||
isolation gaps. Do not implement the separate Notarius credential-forwarding
|
||||
proposal, remove other roadmaps, or refactor unrelated command infrastructure.
|
||||
- Keep production configuration semantics unchanged: root-relative pipeline
|
||||
paths, current schema identifiers, imports/profiles, presence-aware defaults,
|
||||
historical ownership checks, effective-only secret resolution, safe inspection,
|
||||
and domain expansion. Do not adopt `Resolution.InputInfo` or
|
||||
`Resolution.Source` without a separate feature requirement.
|
||||
- Use the released ConfigKit v0.3.0 root API, not internal packages or sibling
|
||||
checkout state. No replace directive, workspace dependency, vendoring, parser
|
||||
fallback, or additional base load.
|
||||
- Preserve the Go 1.26 baseline and Linux amd64/arm64 plus best-effort macOS arm64
|
||||
targets. Do not change CI toolchains or upstream repositories in this work.
|
||||
- Tests use real configuration/discovery logic and temporary files. Synthetic
|
||||
credentials only; no real system configuration, cloud services, or adapter
|
||||
executables. Do not mutate global search paths in the tests being repaired.
|
||||
- Keep each stage buildable. Diagnose unexpected failures rather than skipping
|
||||
tests, weakening invariants, or changing snapshots indiscriminately.
|
||||
- Update canonical documentation only for implemented changes. Published release
|
||||
notes, including v2.0.0, are immutable. No tag, push, release, or CI polling is
|
||||
authorized by this plan.
|
||||
|
||||
## Stage 1 — Isolate command configuration discovery
|
||||
|
||||
Status: Not started
|
||||
|
||||
Purpose: Make missing-config command tests deterministic before changing the
|
||||
dependency, while exercising the real discovery logic.
|
||||
|
||||
Read:
|
||||
`internal/app/command_dependencies.go`, `commands.go`, `config_loader.go`,
|
||||
`pipeline_config_path.go`, `pipeline_config_path_test.go`,
|
||||
`analyze_artifacts_commands_test.go`, `commands_test.go`,
|
||||
`operator_helpers_test.go`; configuration discovery in `docs/cli.md` and
|
||||
`docs/config.md`.
|
||||
|
||||
Implementation:
|
||||
|
||||
1. Add a private `resolvePipelineConfigPath func(string) (string, error)` field
|
||||
to `commandDependencies`. Normalize a nil value to the existing
|
||||
`resolvePipelineConfigPath` function. Keep that function's production default
|
||||
candidates and explicit-path behavior unchanged.
|
||||
2. In `loadPipelineConfigWithDependencies`, call the normalized dependency
|
||||
instead of the free function. Discovery must still precede secret bootstrap
|
||||
and typed configuration loading. Preserve the selected path and all existing
|
||||
error propagation. Verify all command-loading paths continue through this
|
||||
shared owner; do not add per-command discovery implementations.
|
||||
3. Repair the analyze and publish missing-config tests and the
|
||||
`run missing config uses defaults` case in `TestExecuteMissingRequiredFlags`.
|
||||
Call `executeWithDependencies` with a discovery closure that invokes
|
||||
`resolvePipelineConfigPathWithCandidates` on an invocation-owned list of
|
||||
missing paths under `t.TempDir()`. Use fresh dependencies per invocation.
|
||||
A small shared test helper is appropriate for this repeated fixture.
|
||||
Do not merely inject a prewritten discovery error or add an explicit
|
||||
`--config` argument: these tests must still exercise default discovery.
|
||||
4. Convert `TestExecuteSessionInitMissingDefaultConfigReportsSearchedPaths`
|
||||
to the same seam, removing its global search-path mutation. Change
|
||||
`TestResolvePipelineConfigPathWithCandidatesMissing` to temporary missing
|
||||
paths rather than assumed-nonexistent absolute host paths.
|
||||
Inspect adjacent missing-config tests and apply the same treatment where
|
||||
needed; do not turn this into a general rewrite of unrelated successful
|
||||
command fixtures.
|
||||
5. Preserve the existing resolver tests for explicit precedence and first
|
||||
existing candidate. Add one representative command-boundary case using a
|
||||
temporary existing default pipeline and the real resolver to prove that
|
||||
successful discovery reaches the configured loader with that selected path.
|
||||
Stop at a controlled loader boundary; do not initialize runtime services.
|
||||
In a representative missing-path case, guard the loader/bootstrap/executor
|
||||
boundaries so accidental continuation after discovery failure fails the test.
|
||||
6. Assert nonzero exit status, command context, a short missing-config diagnostic,
|
||||
and the controlled searched path. Avoid complete prose snapshots. Keep all
|
||||
test fixtures outside real system configuration and secrets directories.
|
||||
|
||||
Acceptance:
|
||||
|
||||
- `go test ./internal/app -run 'TestResolvePipelineConfigPath|TestExecute.*Missing|TestExecuteSessionInitMissingDefaultConfig'`
|
||||
- `go test -race ./internal/app`
|
||||
- `go test ./...` on the existing ConfigKit v0.1.2 dependency.
|
||||
- The repaired tests own their candidate paths and do not depend on installed
|
||||
configuration being absent. Do not create/remove an actual system config to
|
||||
demonstrate this. Ordinary `Execute` still uses the production defaults.
|
||||
|
||||
## Stage 2 — Adopt the released dependency and version diagnostics
|
||||
|
||||
Status: Not started
|
||||
|
||||
Purpose: Upgrade without altering the set of valid Narratio configurations.
|
||||
|
||||
Read: `go.mod`, `go.sum`, `internal/config/party.go`,
|
||||
`party_test.go`, `pipeline_configkit.go`, `pipeline_configkit_test.go`,
|
||||
`document_configkit.go`, and ConfigKit's tagged v0.2.0/v0.3.0 release notes
|
||||
and public API documentation. The downloaded module is authoritative; a sibling
|
||||
checkout is reference material only.
|
||||
|
||||
Implementation:
|
||||
|
||||
1. Run `GOWORK=off go get gitea.maximumdirect.net/eric/ConfigKit@v0.3.0`
|
||||
and `GOWORK=off go mod tidy`. Review the dependency diff; this task does not
|
||||
authorize unrelated module or Go-version upgrades.
|
||||
2. Change the integer-version case in
|
||||
`TestParsePartyCanonicalRejectsInvalidDocuments` from a `version.type`
|
||||
expectation to unsupported-version rejection. Rename it to identify an
|
||||
unsupported integer version. Preserve `PartySchemaVersion =
|
||||
"narratio.party.v1"` and the existing application checker.
|
||||
3. Retain successful canonical-party coverage and ordinary-string strictness.
|
||||
Add one boolean schema-version case if no equivalent malformed-kind coverage
|
||||
exists, so the suite distinguishes an unsupported identifier from an invalid
|
||||
version kind. Do not duplicate ConfigKit's numeric grammar matrix.
|
||||
4. Run existing profile credential replacement/null-removal, hidden duplicate
|
||||
ordinary/family definition, safe inspection, provenance, digest, bootstrap,
|
||||
and maintained-example tests unchanged. In particular, keep the
|
||||
`Metadata.HistoricalPaths()` check and the single effective pipeline load.
|
||||
No production migration is expected beyond Stage 1's dependency seam.
|
||||
5. Investigate any additional behavior difference against the roadmap before
|
||||
changing production policy. Do not introduce private-engine imports or restore
|
||||
old parsing to preserve incidental error wording.
|
||||
|
||||
Acceptance:
|
||||
|
||||
- `GOWORK=off go mod tidy -diff` and `GOWORK=off go mod verify`.
|
||||
- `GOWORK=off go test ./internal/config ./internal/app`.
|
||||
- `GOWORK=off go test -race ./internal/config ./internal/app`.
|
||||
- Valid configurations and effective secret handling remain unchanged; integer
|
||||
party versions still fail. No temporary module file, test overlay, local
|
||||
replacement, or skipped failing case is needed.
|
||||
|
||||
## Stage 3 — Document and verify the completed adoption
|
||||
|
||||
Status: Not started
|
||||
|
||||
Purpose: Establish a clean, reproducible handoff for later release preparation.
|
||||
|
||||
Read: `docs/development.md`, `docs/internal/configuration.md`,
|
||||
`docs/internal/overview.md`, `docs/config.md`, `docs/release.md`,
|
||||
`scripts/build-release-assets.sh`, and `scripts/check-release-candidate.sh`.
|
||||
|
||||
Implementation:
|
||||
|
||||
1. Update internal command-dependency documentation to include the new
|
||||
invocation-local discovery seam where the dependency inventory is described.
|
||||
Update other current documentation only if a claim is actually affected.
|
||||
The unchanged pipeline path policy and party identifier need no migration
|
||||
instructions. Do not rewrite historical version pins in completed audit records.
|
||||
2. Record in this plan's completion notes the released dependency, diagnostic
|
||||
adjustment, test-isolation repair, and absence of operator migration.
|
||||
If a genuinely unreleased release note already exists, update it. There is
|
||||
currently no such note: v2.0.0 is already tagged. Do not invent a release
|
||||
version or modify v2.0.0; transfer this summary into the new note when a
|
||||
separate release task chooses its version.
|
||||
3. Run all checks below against the normal module files, with no overlays or
|
||||
skipped tests. Review the final diff for scope, formatting, whitespace,
|
||||
credentials, and accidental dependency changes. Record results and limitations.
|
||||
4. Mark the implementation stages complete and update the feature roadmap's
|
||||
status only after acceptance succeeds. Do not delete roadmap/audit records
|
||||
or implement the adjacent Notarius roadmap.
|
||||
|
||||
Required checks:
|
||||
|
||||
```sh
|
||||
GOWORK=off go mod tidy -diff
|
||||
GOWORK=off go mod verify
|
||||
GOWORK=off go test -count=1 ./...
|
||||
GOWORK=off go test -race -count=1 ./...
|
||||
GOWORK=off go vet ./...
|
||||
GOWORK=off go build ./...
|
||||
GOWORK=off go test ./internal/doccheck
|
||||
GOWORK=off go test ./internal/config -run '^TestExamplesLoadAndValidate$'
|
||||
git diff --check
|
||||
```
|
||||
|
||||
Also check formatting with `gofmt -l` on changed Go files. Use
|
||||
`scripts/build-release-assets.sh v0.0.0-test OUTPUT_DIR`, where
|
||||
`OUTPUT_DIR` is an empty absolute directory created with `mktemp -d`, to
|
||||
exercise the maintained Linux amd64/arm64 and macOS arm64 build path without
|
||||
choosing or publishing a release. Report macOS as cross-build evidence only.
|
||||
|
||||
Acceptance: All checks pass without access to an installed Narratio configuration
|
||||
or real credentials. The implementation no longer needs evaluation-only
|
||||
workarounds. The later release task must create its note, commit/push its candidate,
|
||||
and run the normal release-candidate checker; this plan does not claim those
|
||||
publication steps are complete.
|
||||
|
||||
## Open Questions
|
||||
|
||||
None required before implementation. The agreed roadmap and the explicit choices
|
||||
above resolve the migration decisions. Upstream credential-free inspection and
|
||||
native macOS arm64 CI remain deferred, not prerequisites for this plan.
|
||||
None. Dependency version, configuration behavior, discovery seam, test scope,
|
||||
and validation gates are decided. Release version selection is deliberately
|
||||
outside this implementation's scope and does not block the work.
|
||||
|
||||
157
docs/roadmap/notarius-credential-forwarding.md
Normal file
157
docs/roadmap/notarius-credential-forwarding.md
Normal file
@@ -0,0 +1,157 @@
|
||||
# Typed Notarius Credential Forwarding
|
||||
|
||||
Status: proposed
|
||||
|
||||
## Summary
|
||||
|
||||
Allow an operator to configure one typed credential reference for the Notarius
|
||||
stage. Narratio resolves that reference through its existing ConfigKit secrets
|
||||
bootstrap and supplies the credential to the Notarius subprocess under the
|
||||
referenced environment-variable name.
|
||||
|
||||
This request defines only the operator-facing configuration and invocation
|
||||
contract. It does not prescribe an internal implementation.
|
||||
|
||||
## Problem
|
||||
|
||||
An enabled Notarius pipeline may select a PromptKit profile that requires an
|
||||
API key, such as a profile whose `api_key_env` is `OPENROUTER_API_KEY`.
|
||||
Narratio already supports environment and secrets-directory bootstrap for typed
|
||||
credentials, but the Notarius configuration section has no credential field.
|
||||
|
||||
Consequently, an operator cannot declare the Notarius credential dependency in
|
||||
Narratio configuration. Supplying the variable to Narratio's environment or
|
||||
conventional secrets directory is not sufficient under the current public
|
||||
contract: the Notarius invocation receives Narratio's minimal child
|
||||
environment, and no selected Notarius credential is forwarded.
|
||||
|
||||
This leaves an otherwise valid configured extraction stage unable to
|
||||
authenticate to a provider without an out-of-band invocation mechanism.
|
||||
|
||||
## Required Operator-Facing API
|
||||
|
||||
Add one optional typed secret reference at:
|
||||
|
||||
```text
|
||||
pipeline.notarius.llm_api_key
|
||||
```
|
||||
|
||||
Example:
|
||||
|
||||
```yaml
|
||||
notarius:
|
||||
enabled: true
|
||||
config_path: /usr/local/etc/notarius/config.yml
|
||||
pipeline_id: dnd-session
|
||||
llm_api_key: OPENROUTER_API_KEY
|
||||
outputs:
|
||||
spells:
|
||||
lane_id: spells
|
||||
media_type: application/json
|
||||
schema_id: notarius.dnd.spells
|
||||
schema_version: v1
|
||||
module_key: dnd/spells
|
||||
```
|
||||
|
||||
`llm_api_key` contains an environment-variable name, never a raw credential.
|
||||
Its value uses the same typed-secret and ConfigKit bootstrap semantics as other
|
||||
Narratio credential references:
|
||||
|
||||
- when the field is absent, Narratio forwards no Notarius credential and
|
||||
preserves the current behavior;
|
||||
- when the field is present, its value must be a valid typed secret reference
|
||||
that resolves to a nonempty credential during configuration loading;
|
||||
- an existing process environment value takes precedence over a value loaded
|
||||
from the selected secrets directory;
|
||||
- `--secrets-dir` and conventional sibling `secrets/` discovery retain their
|
||||
existing meanings; and
|
||||
- a selected profile may replace or explicitly remove the field under ordinary
|
||||
ConfigKit overlay semantics.
|
||||
|
||||
For a configured reference such as `OPENROUTER_API_KEY`, the Notarius child
|
||||
process receives the resolved credential under exactly that environment name.
|
||||
This lets the operator coordinate the Narratio field with the `api_key_env`
|
||||
selected by the Notarius PromptKit profile without introducing
|
||||
provider-specific configuration in Narratio.
|
||||
|
||||
No new CLI flag is required. The existing Notarius argument vector, pipeline
|
||||
configuration path, working directory, reference bindings, receipt, and output
|
||||
contracts remain unchanged.
|
||||
|
||||
## Inspection and Diagnostic Contract
|
||||
|
||||
- `narratio config validate` resolves the configured secret reference and
|
||||
fails before stage execution when it is missing or empty.
|
||||
- `narratio config show` may expose the non-secret reference name, such as
|
||||
`OPENROUTER_API_KEY`, but never the resolved credential value.
|
||||
- `narratio config sources` and `narratio config diff` retain their existing
|
||||
secret-safe behavior.
|
||||
- Configuration digests, manifests, generated configuration, command
|
||||
arguments, persisted diagnostics, receipts, and other operator-visible
|
||||
output must not contain the resolved credential value.
|
||||
- Errors may identify the field or unresolved environment-variable name but
|
||||
must not include credential bytes.
|
||||
|
||||
## Compatibility
|
||||
|
||||
The field is optional and defaults to absent. Existing configurations retain
|
||||
their current behavior without migration.
|
||||
|
||||
The credential name belongs to deployment configuration rather than to a
|
||||
specific provider. Narratio does not infer a name from a Notarius pipeline,
|
||||
profile ID, backend, model, or configuration file. Operators must select a name
|
||||
that agrees with the Notarius PromptKit profile they deploy.
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- A pipeline with Notarius enabled and no `notarius.llm_api_key` continues to
|
||||
load and invoke Notarius with no forwarded provider credential.
|
||||
- A pipeline may set `notarius.llm_api_key` to a valid environment-variable
|
||||
reference.
|
||||
- The reference resolves from an existing process environment value or from
|
||||
Narratio's existing explicit or conventional secrets-directory bootstrap.
|
||||
- A missing or empty configured credential fails configuration loading before
|
||||
Notarius execution.
|
||||
- A successful Notarius invocation receives the resolved value under exactly
|
||||
the configured reference name.
|
||||
- Unrelated credentials present in Narratio's environment or secrets directory
|
||||
are not made available to Notarius merely because this field is configured.
|
||||
- Base and named-profile configuration follow ordinary ConfigKit replacement,
|
||||
explicit-null, source-reporting, diff, and secret-resolution semantics.
|
||||
- Effective inspection exposes at most the safe reference name and never the
|
||||
resolved value.
|
||||
- No raw credential appears in command arguments, generated configuration,
|
||||
manifests, receipts, persisted diagnostics, error messages, or test output.
|
||||
- The existing Notarius CLI arguments and result-validation contract remain
|
||||
unchanged.
|
||||
- Focused tests use synthetic credentials and a controlled Notarius substitute;
|
||||
they do not contact a model provider or require a real API key.
|
||||
- Configuration, CLI, integration, example, troubleshooting, and release
|
||||
documentation are updated only where their existing canonical ownership
|
||||
requires it when the feature is implemented.
|
||||
|
||||
## Non-goals
|
||||
|
||||
- Defining how Narratio implements credential resolution or subprocess
|
||||
construction internally.
|
||||
- Adding an arbitrary environment-variable map to the Notarius stage.
|
||||
- Forwarding every variable loaded from the secrets directory.
|
||||
- Inferring provider, backend, profile, model, or credential names from
|
||||
Notarius configuration.
|
||||
- Managing secret files, credential values, rotation, encryption, or an
|
||||
external secret store.
|
||||
- Changing Notarius configuration syntax, PromptKit profile syntax, CLI flags,
|
||||
receipt schemas, pipeline definitions, or output bundles.
|
||||
- Supporting more than one Notarius provider credential in this feature.
|
||||
- Validating a credential with its provider during configuration loading.
|
||||
- Adding credentials to command-line arguments, ordinary YAML values, or
|
||||
generated invocation files.
|
||||
|
||||
## Consumer Dependency
|
||||
|
||||
The D&D Ansible pipeline requires this operator-facing contract before it can
|
||||
provide complete suite-managed credential delivery. Its intended deployment
|
||||
places files named for configured secret references in Narratio's conventional
|
||||
`secrets/` directory. That deployment can select and provision the value only
|
||||
after Narratio provides a typed field that forwards the selected Notarius
|
||||
credential under the same reference name.
|
||||
Reference in New Issue
Block a user