Files
narratio/docs/roadmap/implementation.md

1251 lines
66 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Pipeline Configuration Ergonomics Implementation Plan
## Purpose And Status
This document is the executable implementation plan for
[`pipeline-configuration-ergonomics.md`](pipeline-configuration-ergonomics.md).
That feature roadmap owns the accepted user intent, policy choices, scope, and
target end state. This plan translates the roadmap into bounded stages suitable
for one `gpt-5.6-terra` implementation prompt apiece.
All stages are pending and must be implemented in numeric order. Each stage is
intended to leave the repository correct, documented for the behavior that is
then usable, and independently reviewable. Later stages may rely on the tested
contracts established by earlier stages, but must not silently redesign them.
## Settled Implementation Decisions
The following choices make this plan decision-complete:
- Effective pipeline resolution has one order: parse the root and declared
sources with presence information; additively merge root fields and imports;
apply exactly one selected profile overlay when profiles exist; strict-decode
the assembled mapping; apply centralized defaults; resolve ordinary paths
relative to the root pipeline; resolve the selected campaign and its party;
expand party-driven artifacts, dependencies, member sources, and publish
rules; run ordinary concrete validation; and render/digest the normalized
effective pipeline. Defaults must not be applied to individual fragments.
- `composition.imports` is additive and has no winner. A duplicate final path
in two base sources is an error even if its values are equal. The selected
profile overlay is the only overwrite layer. Lists are atomic in the base and
replace completely in an overlay; maps merge recursively; incompatible YAML
kinds and YAML null-as-deletion are rejected.
- Profile names are case-sensitive, must be non-empty, trimmed, and free of
control characters. If profiles are declared, exactly one must be selected by
an explicitly supplied CLI value or `default_profile`; there is no implicit
first profile. A configuration with imports but no profiles remains valid.
- Pipeline load options retain whether `--profile` was explicitly present so
an explicitly empty value is distinguishable from omission and rejected.
`LoadPipeline` remains a compatibility entry point for callers that do not
make an explicit selection; the option-aware loader is authoritative.
- The normalized effective digest covers the secret-free, defaulted, expanded
runtime pipeline mapping after composition. It excludes composition
declarations and runtime-only provenance. It uses logical configuration
values rather than machine-specific serialization accidents. Party-derived
concrete values and publish rules therefore affect the digest. The digest is
provenance only and is never a blanket resume key.
- Source ownership is tracked by complete YAML-style field path. Base leaves
retain their declaring root/import source, overlaid leaves belong to the
selected overlay, centralized defaults are marked as defaults, and generated
fields identify both the family declaration and canonical party as sources.
Source paths may appear in diagnostics and inspection output; raw secret
values may not.
- Semantic resume evidence uses a versioned, typed stage contract. The runner
compares a stage's current semantic fingerprint before reusing terminal
state, then invokes any existing stage-specific resume validator. Missing or
mismatched evidence makes the stage non-resumable and uses the existing fixed
transitive invalidation relation. Each stage hashes only Narratio-observable,
result-affecting values. Timeouts, concurrency, workspace and diagnostic
paths, executable paths, profile names, and raw credentials are excluded.
- Existing analyze artifact fingerprints remain the granular authority for the
`analyze` stage. Do not add an aggregate analyze fingerprint that would stale
every configured artifact after one member or model changes. Instead, audit
and extend the existing per-artifact fingerprint inputs where necessary.
- The canonical party source is campaign-owned. In canonical
`narratio.party.v1` mode, a separate campaign or session `players_file` and a
session-level `party_file` override are rejected. The isolated legacy mode
preserves the current effective party/players override behavior, treats the
unversioned party payload as opaque reference material, and cannot drive
families. Compatibility code must be grouped and commented for later
removal.
- A party document containing `schema_version` is an attempted canonical
document: an unsupported or malformed version is an error, not a legacy
fallback. Canonical documents are strict single-document YAML. Unversioned
legacy documents are not decoded into the canonical schema.
- The canonical players projection is emitted with
`schema_version: narratio.players.v1`, one entry per character sorted by
stable character ID, and aliases in declaration order. An absent or empty
alias list is omitted from a projection entry. `prepare` copies the canonical
party bytes unchanged and generates players bytes deterministically.
- `scriptorium.artifact_families` is a resolution-only declaration. Every
family expands for every canonical party character, including a disabled
family so explicit selection can use existing disabled-artifact semantics.
Runtime stages and adapters receive only the resulting ordinary concrete
artifacts plus bounded origin metadata.
- `{character_id}` must occur exactly once in each configured family output
path pattern and each non-empty family publish destination pattern. No other
brace token is accepted. Substitution is followed by the existing relative,
confined output/destination validation; arbitrary interpolation is not
introduced.
- Ordinary family `depends_on` entries name shared concrete artifacts.
Corresponding family members use `member_dependencies`. A
`narratio.member_artifact.<family>` input is legal only in a family, must name
one of its `member_dependencies`, and is rewritten to the concrete
`narratio.artifact.<family>_<character_id>` source before ordinary validation.
- Family selection is normalized to concrete keys before the existing
effective-artifact resolver. Selecting a family selects all members;
selecting a generated key selects only that member; mixed selections are
sorted and deduplicated. Typed optional family/character origin fields are
retained in planning, listing, run/analyze state, and reconciliation without
creating dynamic pipeline stages.
- An enabled family publish declaration requires an existing top-level publish
configuration and expands to ordinary concrete output rules. Omitting its
destination pattern derives each destination from the generated artifact's
output path. Top-level publish enablement retains its existing meaning.
- `config validate`, `show`, and `sources` accept the existing `--config`,
`--campaign`, and `--campaign-file` selection mechanisms plus `--profile`.
They do not discover a session or create runtime state. Campaign selection is
optional only when full resolution does not need party data. `config diff`
takes exactly two positional profile names and the same pipeline/campaign
flags, but no separate `--profile` flag.
- `config show` emits deterministic secret-free effective YAML.
`config sources` emits deterministic path/role/source records.
`config diff` emits sorted semantic records of the form added, removed, or
changed at a normalized field path, with values represented in deterministic
compact form; it does not diff raw source text.
## Instructions For Every Stage
Before changing code in each stage:
1. Read `docs/development.md`, all documents in `docs/policy/`, this plan, and
the portions of the feature roadmap relevant to that stage.
2. Follow `docs/development.md` to the current canonical user, integration, and
internal documents for every subsystem the stage changes.
3. Inspect the current implementation and its focused tests before editing.
Prefer the codebase knowledge graph for discovery, then read the exact owning
files and symbols.
4. Confirm the worktree state and preserve unrelated user changes.
During each stage:
- Keep changes within that stage's scope and the accepted roadmap. Do not add a
general configuration language, workflow DAG, arbitrary templating, nested
imports, profile inheritance, or runtime family loop.
- Preserve strict YAML, centralized defaults, fixed application-owned stage
sequencing/invalidation, manifest authority, root-confined filesystem work,
indirect secret handling, and adapter ownership of private configuration.
- Use the one production loading/resolution path created by this plan. Tests
may call narrower package owners, but commands may not reimplement merging,
profile selection, party loading, or family expansion.
- Follow the testing policy: protect behavior at its narrowest stable owner,
use real filesystem behavior in `t.TempDir()`, fake only external or
nondeterministic boundaries, keep default tests offline, and avoid repeating
every parser case at CLI or assembled-workflow layers.
- Update each canonical current-behavior document in the same stage that makes
the behavior usable. Until then, leave unimplemented behavior only in the
roadmap and this plan.
- Preserve backward-compatible reads of existing manifests and monolithic
configurations. Emit only the new canonical representation after a new
writer is introduced.
- Run focused tests while iterating, format changed Go files, and finish with
`go test ./...`. Also run the applicable static, race, build, documentation,
and maintained-example checks listed in `docs/development.md`; the final stage
must run the complete repository validation set.
At the end of each stage, leave a cohesive change that can be reviewed and
committed independently. Do not begin a later stage while an earlier stage has
failing tests, stale documentation, or unmet exit criteria.
## Stage 1 — Presence-Aware Composition Engine
**Status: Completed**
### Goal
Introduce the internal YAML representation and deterministic merge primitives
needed by imports and profiles without changing the public pipeline schema.
### Required Work
1. In `internal/config`, add a composition document model built from
`yaml.Node` or an equivalently presence-aware representation. Retain mapping
versus sequence versus scalar kind, explicit zero values, complete logical
field paths, source file identity, and declaration order for diagnostics.
2. Parse exactly one top-level mapping per source. Reject duplicate YAML keys,
aliases that produce ambiguous ownership, trailing documents, and malformed
YAML with source-qualified errors. Keep the existing strict decoder as the
final schema authority; this layer owns document mechanics and merge
presence, not a parallel field schema.
3. Implement separate operations for:
- additive base merge, which recursively joins disjoint maps and rejects any
duplicate final value/list/keyed entry; and
- overlay merge, which recursively merges maps, replaces same-kind scalar or
list values, and rejects incompatible kinds or deletion/null semantics.
4. Make conflicts report the complete configuration path and all claiming
sources. Preserve declared source order only for diagnostics, never as base
precedence.
5. Add deterministic traversal/rendering suitable for normalized YAML,
canonical digest input, semantic field comparison, and source reports. Map
keys must be sorted for effective output even when source declaration order
is retained separately.
6. Keep these APIs internal to the configuration owner. Do not accept a
`composition` field in `PipelineConfig` yet.
### Tests And Exit Criteria
- Package-level table tests cover explicit `false`, zero, empty maps/lists,
disjoint recursive maps, every duplicate class, list atomicity/replacement,
kind conflicts, null deletion attempts, duplicate YAML keys, non-mapping
documents, and trailing documents.
- Diagnostics identify the full field path and both or all relevant sources.
- Deterministic rendering and digest input are stable across map insertion and
source traversal order where semantics are equal.
- Existing `LoadPipeline` behavior and all monolithic examples remain
unchanged.
## Stage 2 — Explicit Additive Imports
**Status: Completed**
### Goal
Make root-owned `composition.imports` usable while preserving monolithic
configuration behavior and one root-relative path base.
### Required Work
1. Add the root-only composition envelope and ordered `imports` field. Strip
the envelope before strict `PipelineConfig` decoding. Root ordinary fields
participate as one additive base source alongside imported partial files.
2. Resolve import paths relative to the root pipeline directory. Accept only
non-empty relative `.yml` or `.yaml` paths that remain beneath that directory
and resolve through `Lstat` to regular files. Reject absolute paths,
traversal, symlinks/non-regular files, missing files, duplicate imports, and
an import of the root itself.
3. Parse every imported document through Stage 1. Reject `composition` in any
imported document, thereby preventing recursion, nested imports, profiles,
and cycles.
4. Additively merge root ordinary fields and imports, strict-decode the result,
apply existing defaults once, and resolve every ordinary relative pipeline
path against the root pipeline directory regardless of the declaring file.
5. Retain root/import paths and base field ownership on runtime-only resolution
metadata without exposing them as YAML fields or secrets.
6. Route `LoadPipeline` through the composed loader so monolithic and imported
roots cannot drift. Do not add an automatic `conf.d` scan.
7. Update `docs/config.md` and focused internal configuration documentation for
the implemented import schema, additive conflict contract, confinement, and
stable root-relative path semantics.
### Tests And Exit Criteria
- Tests cover monolithic compatibility; root-plus-import fields; disjoint
contributions beneath `scriptorium.artifacts`; duplicate root/import and
import/import leaves; identical duplicate values; lists; type conflicts;
missing/absolute/traversing/symlink/non-regular/unsupported-extension paths;
repeated imports; imported composition; root self-import; source-aware
errors; and deterministic source ordering.
- Moving the same relative path field between root and import resolves to the
same absolute runtime value.
- `TestExamplesLoadAndValidate` and existing strict loader tests pass without
fixture-wide changes.
## Stage 3 — Versioned Stage Semantic Resume Framework
**Status: Completed**
### Goal
Create one typed runner/manifest mechanism for result-affecting stage
configuration before selectable profiles can change those values.
### Required Work
1. Add an optional typed semantic-configuration fingerprint record to session
`StageRecord` and invocation `RunStageRecord`, containing a positive schema
version and lowercase SHA-256 digest. Use `omitempty` and preserve reads of
existing manifests without the field.
2. Add a narrow optional stage interface that returns the current versioned
semantic fingerprint from resolved `stage.Env` configuration. Provide one
helper that hashes deterministic JSON from stage-owned typed structs; reject
invalid versions and never hash raw secret values, arbitrary `map[string]any`
iteration, or the complete effective configuration digest.
3. In both execution and read-only planning, compare current evidence whenever
a terminal stage record would otherwise be reused. Missing legacy evidence
or a version/digest mismatch makes the stage non-resumable with a bounded,
actionable reason and follows the existing stale/dependent invalidation
path.
4. If the semantic comparison succeeds, continue to call the existing
stage-specific `ResumeValidator`; both checks are required. Forced execution
retains its current behavior and does not need a comparison to decide to
run.
5. Compute the fingerprint immediately before the stage decision, persist the
same value only with a successful or intentional skipped result, and copy it
to the run-stage record. Never promote fingerprint evidence from a failed or
interrupted execution.
6. Ensure a run-manifest skip records the reused semantic fingerprint for
invocation provenance without rewriting the authoritative session stage
result.
7. Update focused manifest, runner, plan, and resume internals. Do not claim
stage coverage until Stages 46 add it.
### Tests And Exit Criteria
- Manifest round-trip tests prove old records remain readable and new records
preserve version/digest in session and run state.
- Runner and plan tests cover matching, missing, mismatched, malformed, forced,
skipped, failed, and existing-resume-validator combinations.
- A mismatch marks the stage and only its fixed transitive dependents stale;
read-only plan predicts the same decision without mutation.
- Fingerprint tests use typed values and prove deterministic hashes without
asserting implementation-private serialization beyond the versioned
contract.
## Stage 4 — Prepare, Transcribe, And Merge Semantic Contracts
**Status: Completed**
### Goal
Protect reuse of the pipeline's input preparation and initial transcript work
with stage-specific semantic configuration evidence.
### Required Work
1. Implement Stage 3 fingerprint providers for `prepare`, `transcribe`, and
`merge`, each with an independently versioned private payload type.
2. The `prepare` payload must include resolved campaign/session input selection
semantics, local versus S3 audio selection, previous-session identity and
selected previous-artifact requirements, and any option that changes
prepared canonical bytes or names. Exclude absolute source/workspace paths
when their logical values and bytes are equivalent, cache/spool placement,
transfer tuning, and the complete profile/digest. Continue to rely on
prepared-input checksums and current source validation for content identity;
do not hash large audio a second time solely for configuration evidence.
3. The `transcribe` payload must include Narratio-visible recognition language
and service/model identity exposed by its configured adapter contract. It
must exclude retry, concurrency, timeout, credential, and diagnostic values.
4. The `merge` payload must include Seriatim operation/output schema and every
configured merge transformation that can change canonical transcript bytes.
Exclude executable, timeout, report, and retention settings.
5. Keep payload construction in the owning stage or a narrowly shared stage
helper; do not add a reflection-based whole-config hasher.
6. Update the focused prepare, transcribe, merge, operations, and resume
documentation for the evidence that is now observable and the external
private inputs that still require force.
### Tests And Exit Criteria
- Each stage has table-driven tests proving a representative semantic change
changes its fingerprint and representative operational/path-only changes do
not.
- Runner tests prove a changed prepare value stales its fixed descendants, a
changed transcribe value reuses prepare but stales transcribe descendants,
and a changed merge value reuses prepare/transcribe.
- Existing successful records without semantic evidence receive a safe one-time
rerun when the corresponding stage is selected.
- No test invokes live storage or transcription services.
## Stage 5 — Polish, Normalize, Trim, And Render Semantic Contracts
**Status: Completed**
### Goal
Make transcript refinement and rendering safe across profile-selected model and
prompt changes without invalidating unaffected upstream transcript work.
### Required Work
1. Implement independently versioned semantic payloads for `polish`,
`normalize`, `trim`, and `render`.
2. The `polish` payload must include every Narratio-visible Audita value that
can change canonical output, including model, validation model, module set,
transcript description, output schema, and explicitly selected external
configuration identity. Include a configured service endpoint when it can
select different semantics. Exclude executable path, timeout, concurrency,
report/debug paths, work retention, and credential environment names.
3. The `normalize` payload must include the Seriatim normalization operation,
schema, and transformation settings, while excluding operational runner
settings.
4. The `trim` payload must include its enablement, prompt/profile identifiers,
canonical input/output identity, Scriptorium variables and render policy
that affect canonical output, and relevant Seriatim transformation settings.
Exclude diagnostic render output, timeout, and executable path.
5. The `render` payload must include final format, title, timestamp/segment ID/
metadata inclusion, output identity, and any other canonical render choice.
6. Document that Narratio cannot observe private prompt/module/model/config file
contents behind a stable identifier; changes to those contents still require
force.
### Tests And Exit Criteria
- A production/testing Audita model change makes `polish` non-resumable while
leaving `prepare`, `transcribe`, and `merge` reusable.
- Each stage's semantic and operational exclusions are covered at its package
boundary, with one runner-level invalidation test per distinct dependency
branch rather than duplicated exhaustive cases.
- Render-only changes do not stale `extract`; trim changes stale both render and
extract through the existing fixed invalidation relation.
- Current stage, operations, and manifest documentation accurately describes
the implemented behavior.
## Stage 6 — Extract, Analyze, Publish, And Notify Resume Audit
**Status: Completed**
### Goal
Complete semantic resume coverage for downstream artifact work without
weakening extract validation or analyze's per-artifact granularity.
### Required Work
1. Give `extract` a versioned semantic payload containing enablement, Notarius
pipeline identity, declared output contracts, reference-slot/source mapping,
and canonical output identities. Combine it with the existing reference and
output resume validator; do not duplicate checksum logic in the payload.
Exclude binary, timeout, working directory, and private Notarius config file
contents.
2. Audit the existing analyze artifact fingerprint schema. Ensure it includes
all Narratio-visible effective prompt/profile identifiers, static and
generated variables, dependencies, input source identities/content
evidence, render-debug policy that affects canonical output, and output
identity. Bump its schema version only if its payload changes. Keep
reconciliation and partial selection artifact-granular.
3. Do not implement the Stage 3 aggregate fingerprint interface for `analyze`.
Adapt plan/runner integration only as necessary so its existing
`ResumeValidator` remains the authority and later family members participate
as ordinary artifacts.
4. Give `publish` semantic evidence for enabled behavior, normalized concrete
source/destination/required rules, upload-run policy, static lock policy,
and remote destination identity (backend, bucket, region/endpoint identity,
and root prefix). Exclude credentials, timeout/retry tuning, local workspace,
and run IDs. Retain immediate lock revalidation and commit safety.
5. Give `notify` semantic evidence for its configured delivery mode and any
Narratio-visible message-shaping option. Never hash credentials or remote
response data.
6. Publish a concise implemented coverage table in the focused resume/manifest
internals, linking to stage and integration owners rather than duplicating
their schemas.
### Tests And Exit Criteria
- Extract tests prove semantic changes and reference/output corruption are
independently non-resumable and operational changes are reusable.
- Analyze tests prove a model, variable, dependency, or input change affects
only the relevant artifact and its artifact dependencies, not every analysis
record or upstream transcript stage.
- Publish tests prove a destination/rule/target change reruns publication while
credential names and operational tuning do not; existing lock and commit
tests remain authoritative for destructive behavior.
- All canonical stages now have explicit semantic reuse coverage or the
documented analyze artifact-level equivalent before named profiles become
selectable.
## Stage 7 — Named Profile Composition And Effective Digest
**Status: Completed**
### Goal
Add strict single-profile overlay resolution and complete effective
configuration provenance within `internal/config`.
### Required Work
1. Extend the root-only composition envelope with `default_profile` and a
profile map whose only field is `overlay`. Reject unknown fields, empty
profile names, invalid defaults, nested composition, imports in overlays,
profile inheritance, and any attempt to stack profiles.
2. Add option-aware pipeline loading that distinguishes an explicitly supplied
profile from omission. Apply the selection rules in Settled Decisions and
return errors before stage or adapter composition.
3. Resolve every declared overlay path under the same confinement and regular
YAML-file rules as imports. Structurally parse every declared overlay to
catch missing files, malformed YAML, duplicate keys, trailing documents, or
forbidden composition; apply only the selected overlay to the base.
4. Use Stage 1 overlay semantics. Preserve explicit `false`, zero, empty-list,
and keyed-map additions; replace lists completely; reject kind changes and
null/deletion syntax.
5. Strict-decode and default once after overlay. Retain selected profile name
and source (`default` or `cli`), ordered root/import/overlay sources, and
leaf ownership as runtime-only provenance.
6. Generate a deterministic effective digest from the normalized secret-free
runtime pipeline mapping. Provide one recomputation hook for later
party/family expansion rather than inventing a second digest.
7. Keep `LoadPipeline` as the omission wrapper and update `LoadWithSession*`
option types so profile presence can flow through without API duplication.
8. Update `docs/config.md` and internal configuration documentation for the
implemented schema, exact selection/overlay rules, digest meaning, and
absence of inheritance, stacking, deletion, or environment selection. Do
not document `--profile` until Stage 9 exposes it.
### Tests And Exit Criteria
- Tests cover default and explicit selection; explicit-over-default; omitted
selection with profiles; profile-free imports; explicit profile against no
profiles; unknown/empty profiles; invalid defaults; malformed/unselected
overlay sources; map recursion; false/zero overrides; list replacement;
keyed artifact addition/disablement; kind/null conflicts; and nested
composition.
- Equal normalized results produce equal digests regardless of source split;
a semantic value change changes the digest; no raw secret value is loaded or
represented.
- Monolithic and import-only configurations keep their current behavior and
source-relative paths.
## Stage 8 — One Production Configuration Loading Path
**Status: Completed**
### Goal
Refactor application configuration loading so a composed pipeline/profile is
loaded once and carried unchanged through campaign, local/remote session, and
command-specific resolution before CLI profile selection is exposed.
### Required Work
1. Refactor `internal/app/config_loader.go` around one loaded pipeline/campaign
context that retains the option-aware pipeline result and provenance. The
subsequent session resolver must consume that loaded value rather than call
`LoadPipeline` or reread the root path.
2. Provide one configuration-package resolution entry point that can combine
an already loaded pipeline with campaign and optional session data. Keep
compatibility wrappers thin and route them through that owner.
3. Move pipeline and campaign discovery, explicit `--campaign-file` handling,
registry selection, and mutual-exclusion rules into shared application
helpers used by runtime and later inspection commands. Do not silently pick
a campaign when no existing command rule authorizes it.
4. Route local sessions, remote session loading, restore, plan, run, status,
helper commands, single-stage commands, session init/validate, locks,
artifact listing, session cleanup, and `clean --all` through the shared
pipeline load where applicable. Preserve their existing behavior and
mutation boundaries.
5. Ensure remote-session download changes only the session source and cannot
discard the already selected pipeline/profile. Closing temporary session
resources must not invalidate retained configuration/provenance.
6. Remove or make private any alternate production loader that could bypass
import/profile selection. Keep narrow pure test helpers only when they call
the same configuration package APIs.
7. Update focused application configuration-loader internals. This is a
behavior-preserving refactor; do not add public flags or user documentation.
### Tests And Exit Criteria
- Existing command, discovery, campaign registry, remote session, restore,
clean, and session-init tests pass unchanged except where fixtures must call
the new shared API.
- Add a counting/in-memory loader seam or equivalent behavioral evidence proving
one command invocation does not reread/reselect the root pipeline while
resolving its session.
- A test mutation of the pipeline file between base and session resolution
cannot create a mixed invocation; the initially loaded value is retained.
- No user-visible command behavior or runtime state format changes in this
stage.
## Stage 9 — Profile CLI Plumbing, Reporting, And Manifest Provenance
**Status: Completed**
### Goal
Expose profile selection consistently through every configuration-consuming
command and retain bounded invocation provenance without using profile identity
as a cache key.
### Required Work
1. Add an explicit-presence `--profile <name>` flag to the shared configuration
flags. Reject duplicates and explicitly empty values under the same
structural parsing standard as other singleton options.
2. Thread the profile selection through the Stage 8 loader for `run`, `session
plan`, `regenerate-artifacts`, `run-stage`, `analyze`, `publish`, status,
restore, session init/validate, artifacts, locks, cleanup, and every other
command that loads a pipeline. Convenience commands and aliases must not own
an independent profile rule.
3. Add typed optional selected-profile name/source and effective-config digest
fields to session and run manifests. Preserve old manifest reads. The run
manifest records the exact invocation; the session manifest records the most
recently resolved invocation provenance without changing stage reuse
decisions.
4. Apply current provenance before the first persistent run mutation and copy
it into terminal run state. A failed run still retains which effective
configuration was attempted. Read-only plan must report it without writing
any manifest.
5. Include concise profile (or `none`) and digest reporting in run/plan/status
output. When status also shows persisted provenance, label current resolved
versus last persisted values so a profile switch is not ambiguous.
6. Verify secret loading remains after configuration resolution and that raw
environment/file secret values never enter the digest, manifests, logs, or
output.
7. Update `docs/cli.md`, `docs/operations.md`, troubleshooting where useful,
and focused manifest/command internals for selection, reporting, resume
implications, and the force requirement for private external-tool changes.
### Tests And Exit Criteria
- Representative shared parser tests and command tests cover omitted/default,
explicit, unknown, duplicate, and explicitly empty profile values, including
the `regenerate-artifacts` alias path.
- Local, remote-session, restore, plan, and one helper command prove they retain
the same selected profile through final resolution; do not repeat the same
assertion for every wrapper.
- Manifest round trips preserve new provenance and old fixtures remain valid.
- An assembled runner test switches only the Audita model: prepare through
merge remain reusable, polish and its fixed dependents become stale, and the
profile name itself causes no unrelated invalidation.
## Stage 10 — Canonical Party Domain And Players Projection
**Status: Completed**
### Goal
Implement the strict versioned party contract and deterministic derived players
document as a pure configuration/domain boundary before wiring it into campaign
resolution.
### Required Work
1. Add typed canonical party structures for `narratio.party.v1`, keyed
characters, nested player/character values, optional singular `alias` list,
and required class entries with optional pointer levels. Retain stable
character IDs and declaration order where the public contract requires it.
2. Parse canonical mode as strict single-document YAML with known fields only.
A top-level `schema_version` selects canonical parsing; a wrong value,
malformed value, or otherwise malformed canonical document is an error.
Classify a document with no version as legacy without decoding it into the
canonical structures.
3. Validate non-empty characters; the existing configured-artifact key grammar
for IDs; exact trimmed, non-control display strings; required player and
character names; optional non-empty aliases; global case-insensitive
ambiguity across all character primary names and aliases; required non-empty
classes; case-insensitive duplicate classes per character; and positive
levels when present. Player display names may repeat.
4. Derive class summaries by preserving declared class order and joining
entries as `<name>` or `<name> <level>` with ` / `. Derive alias summaries by
joining declared aliases with `, `. Use Unicode-aware case-insensitive
comparison for ambiguity while retaining original spelling in outputs.
5. Produce the exact `narratio.players.v1` projection from canonical data, one
entry per character sorted by stable ID, with player name, character ID,
character name, and an optional alias list in declared order. Serialize it
deterministically with one trailing newline.
6. Return raw canonical bytes separately from normalized domain values so
prepare can later copy `party.yml` unchanged. Do not expose a general
campaign metadata extension map.
7. Keep legacy classification in a small, clearly named compatibility file or
boundary with a removal comment. It must not grow canonical transformation
behavior.
### Tests And Exit Criteria
- Table tests cover every schema rule, alias/primary collisions across and
within characters, Unicode case folding, repeated player names, invalid IDs,
whitespace/control characters, duplicate classes, absent/zero/negative
levels, unknown fields, trailing documents, unsupported versions, and
unversioned legacy classification.
- Multiclass and alias summaries preserve declaration order.
- Projection tests prove stable-ID sorting, repeated-player behavior, optional
alias omission, deterministic bytes, and no class leakage into the
players-only contract.
- Tests target exported/package domain behavior rather than each private YAML
walk helper.
## Stage 11 — Campaign Party Resolution And Isolated Legacy Mode
**Status: Completed**
### Goal
Make combined pipeline/campaign resolution own canonical party loading and
enforce one unambiguous canonical versus legacy input mode.
### Required Work
1. Resolve the campaign-owned `inputs.party_file` relative to the selected
`campaign.yml`, require a regular readable file, load/classify it through
Stage 10, and retain mode, canonical domain data/raw bytes, source path, and
source identity on runtime-only resolved configuration.
2. Split campaign validation into syntax/path-independent validation and final
combined input-mode validation. Do not require `players_file` before party
mode is known.
3. In canonical mode, reject a campaign or session `players_file` and reject a
session `party_file` override. Create a virtual resolved players input whose
source is the canonical party projection; it has no external source path.
4. In legacy mode, require the existing effective `players_file`, preserve
current campaign/session stable-input override semantics for both opaque
party and players files, and retain the exact prepared-input behavior. Add
explicit comments and names marking this compatibility surface for removal
after migration.
5. Make `Resolve`, `LoadWithSessionOptions`, plan/run loading, and any
pipeline-plus-campaign resolution entry point share this logic. A caller
cannot obtain a fully resolved canonical configuration while bypassing party
validation.
6. Add typed party/campaign provenance to the resolution metadata for later
family and `config sources` use. Do not place raw party contents in
manifests or logs.
7. Update `docs/config.md` and add/extend the canonical party integration
document under `docs/integrations/`. Describe legacy mode only as a bounded
migration path and link rather than duplicate the full schema elsewhere.
### Tests And Exit Criteria
- Combined-resolution tests cover canonical success, canonical separate
players rejection from campaign and session, canonical session party
override rejection, legacy success, missing legacy players, campaign/session
legacy overrides, missing/non-regular party files, and source-relative paths.
- The same canonical and legacy decision is observed through direct config
resolution, local command loading, and remote session loading with only one
representative application-level test.
- Existing legacy fixtures continue to load; new canonical fixtures do not
carry a separate players file.
- No party-driven family behavior is added yet.
## Stage 12 — Canonical Party And Derived Players Preparation
**Status: Completed**
### Goal
Materialize one canonical campaign roster and its deterministic players
projection through the existing prepared-input contracts consumed by Notarius
and Scriptorium.
### Required Work
1. Refactor `prepare` input materialization by party mode. In canonical mode,
copy the validated campaign party source bytes unchanged to
`inputs/party.yml` and atomically write Stage 10's projection to
`inputs/players.yml`. In legacy mode, keep the current two-file copy path in
the isolated compatibility owner.
2. Record separate `party` and `players` input records with their own content
checksums. Mark the players record with a stable source identifier such as
`derived_from_party`; do not pretend it came from a user `players_file`.
Retain party source/config provenance without recording its full content.
3. Use existing confined atomic/copy-if-changed filesystem helpers and current
group-readable permission policy. Remove an obsolete prior players output
safely when switching modes only through the normal prepared overwrite path.
4. Preserve `narratio.input.party` and `narratio.input.players` as the two
runtime source IDs. Ensure extract reference composition passes the prepared
canonical party unchanged to Notarius's `party` slot and the derived
projection to its `players` slot; do not add adapter-specific roster logic.
5. Ensure Scriptorium configured inputs resolve the derived players record by
the existing manifest-authoritative prepared-input lookup. Missing or
checksum-invalid generated bytes must fail at that owner.
6. Include party mode and projection schema version in prepare semantic
evidence so a legacy-to-canonical migration cannot reuse old prepared
inputs.
7. Complete the durable players integration document and update prepare,
artifact, Notarius, manifest, and operations documentation without copying
full maintained examples into prose.
### Tests And Exit Criteria
- Prepare tests compare canonical party bytes exactly, compare deterministic
projection bytes, and verify separate manifest checksums/source identity.
- Legacy prepare tests prove current opaque party and explicit players bytes
remain unchanged through the compatibility path.
- Extract/reference and analyze-input tests prove both source IDs resolve from
prepared manifest authority; missing/corrupt projection evidence fails
clearly without invoking live tools.
- Re-running prepare with identical inputs is byte-stable and does not perform
an unsafe partial write.
## Stage 13 — Basic Party-Driven Artifact Family Expansion
**Status: Completed**
### Goal
Expand a shared character family declaration into deterministic ordinary
Scriptorium artifacts with member-specific identity, paths, and variables.
### Required Work
1. Add strict `scriptorium.artifact_families` structures supporting the shared
concrete-artifact fields named by the feature roadmap plus
`for_each`, `output_path_pattern`, `member_vars`,
`member_dependencies`, and the later typed publish block. Reject unknown
fields through the existing strict pipeline decode.
2. Require the exact `for_each: party.characters`, a canonical party, a valid
family key, and exactly one literal `{character_id}` token in
`output_path_pattern`. Reject all other brace syntax before substitution.
3. For every family and canonical character sorted by family key then stable
character ID, create concrete key `<family>_<character_id>` and copy shared
enablement, prompt/profile, timeout, render-debug, ordinary dependencies,
inputs, and static variables into an ordinary
`ScriptoriumArtifactConfig`.
4. Resolve member variables only from the closed selectors in the roadmap and
merge their string values into concrete `vars`. Reject invalid selectors,
invalid destination variable names under the existing Scriptorium variable
rules, and collisions with static variables. Preserve the existing reserved
sticky-session variable ownership after expansion.
5. Substitute the stable character ID into output paths and then run the
existing configured-key, safe relative path, duplicate output, selected
executable-field, and Scriptorium variable validation. Do not create a new
runtime artifact type or loop.
6. Reject collisions among family keys, explicit concrete keys, generated
keys, and generated output paths. Expand disabled families too; ordinary
default effective selection will still omit their disabled members.
7. Store a runtime-only family catalog mapping family to sorted members and
each generated member to family/character origins and source ownership.
Remove resolution-only family declarations from the concrete runtime
Scriptorium configuration before adapters receive it.
8. Recompute the normalized effective digest after expansion so party member
values and generated concrete configuration are represented. Update
configuration and Scriptorium internal documentation for the implemented
fields and exact substitution/member-variable limits.
### Tests And Exit Criteria
- Tests cover two families over multiple characters, stable ordering, all
member selectors, multiclass/alias summaries, static vars, disabled families,
invalid iteration, legacy/no party, token errors, variable conflicts,
key/output collisions, unsafe paths, and ordinary post-expansion validation.
- Reordering source maps without changing party order semantics produces the
same concrete map, catalog, digest, and normalized output.
- Adapters and analyze planning receive only ordinary concrete artifacts and
current `map[string]any` string/bool variables.
- Adding or removing a character changes the generated member set
deterministically; reconciliation behavior is deferred to Stage 15.
## Stage 14 — Same-Member Dependencies And Member Artifact Sources
**Status: Completed**
### Goal
Support several coordinated character artifact families while resolving all
family-specific dependency syntax before runtime validation.
### Required Work
1. Validate `member_dependencies` as unique family keys. Each referenced family
must exist, use the exact same canonical party iteration source, and produce
the same character IDs. Reject self-dependency early with family/member
context.
2. For each generated member, append dependencies on the corresponding
`<dependency-family>_<character_id>` artifacts. Preserve separately declared
ordinary dependencies on shared concrete artifacts, normalize duplicates,
and let the existing concrete dependency planner detect transitive cycles.
3. Recognize `narratio.member_artifact.<family>` only while expanding family
inputs. Require the referenced family to appear in that declaration's
`member_dependencies`, then rewrite it to
`narratio.artifact.<family>_<character_id>` before artifact-policy and
analyze validation.
4. Reject member-artifact syntax in explicit concrete artifacts, references to
missing/non-member families, malformed suffixes, and any unresolved member
source reaching the concrete pipeline.
5. Attribute generated dependency/input paths to both the declaring family and
relevant party member in provenance. Do not register
`narratio.member_artifact.*` as a runtime artifact-policy source.
6. Update configuration, analyze, and artifact internals for the implemented
resolution boundary and examples no larger than needed to show the syntax.
### Tests And Exit Criteria
- Tests cover valid meta-to-items corresponding dependencies for every member,
shared concrete dependencies, missing families/members, self and transitive
cycles, duplicate dependencies, missing `member_dependencies` declarations,
malformed/member syntax in concrete artifacts, and deterministic rewritten
sources.
- Existing concrete dependency ordering, failure propagation, fingerprints,
and source validation work unchanged after expansion.
- A repository search/test assertion confirms no unresolved
`narratio.member_artifact.` source can reach stage or adapter configuration.
## Stage 15 — Family Selection, Origin Reporting, And Reconciliation
**Status: Completed**
### Goal
Make family declarations ergonomic at command boundaries while preserving
concrete execution, artifact-granular fingerprints, and manifest authority.
### Required Work
1. Extend application artifact selection normalization with the Stage 13 family
catalog. Expand an exact family key to all sorted member keys; retain an
exact generated/concrete key as one target; reject unknown values; and sort/
deduplicate mixed family/member/concrete selections before calling the
existing effective-artifact resolver.
2. Explicit family selection must pass every generated member through the
existing explicitly-selected-disabled validation. Default selection still
includes only enabled concrete artifacts. An empty canonical party is
already invalid and therefore never turns a family selection into a silent
no-op.
3. Extend effective-artifact metadata with optional typed `family` and
`character_id` origins while keeping `Keys()` and concrete lookup behavior
stable for existing callers.
4. Add backward-compatible optional family/character fields to the authoritative
per-artifact analyze session and run records. Populate them from resolved
configuration, not by splitting concrete names. Explicit non-family
artifacts leave them absent.
5. Include origin plus concrete identity in `session plan`, run/analyze
summaries, and `artifacts list` output. Family headings may summarize, but
every executable/reused/failed artifact must remain identifiable by concrete
key.
6. Ensure the existing analyze reconciliation treats a newly added party member
as one or more new configured artifacts and a removed member as removed
records. Preserve unrelated current members and archive/remove outputs only
through existing safe reconciliation policy.
7. Ensure member-specific resolved variables and dependency sources enter the
existing artifact fingerprint so changing a class or alias stales only
consumers of that value and their artifact dependents.
8. Update CLI, analyze, artifact, manifest, and operations documentation for
family versus member selection and concrete runtime identity.
### Tests And Exit Criteria
- Selection tests cover family, member, explicit concrete, mixed/duplicate,
unknown, enabled/disabled, and invalid executable member cases.
- Plan and runner tests prove family selection and its normalized concrete
selection are behaviorally equivalent; no adapter receives a family key.
- Reconciliation tests cover adding, changing, and removing a character while
preserving unrelated artifact records and respecting member dependencies.
- Manifest compatibility tests read old analyze records and round-trip new
optional origin fields without deriving origin from names.
## Stage 16 — Family Publish Rule Expansion
**Status: Completed**
### Goal
Turn one optional family publish declaration into existing concrete publish
rules before ordinary validation and publication.
### Required Work
1. Implement the strict family `publish` block with `enabled`, `required`, and
optional `dest_pattern`. An absent or disabled block emits no rules. An
enabled block requires a top-level publish configuration but does not change
top-level publish enablement or storage policy.
2. For each generated family member, create one ordinary publish rule whose
source is `narratio.artifact.<concrete-key>`, whose required bit is copied,
and whose destination is either the exactly-once substituted destination
pattern or the generated artifact output path.
3. Reject unknown/missing/repeated brace tokens, unsafe/escaping destinations,
conflicts with explicit rules for the same source, duplicate normalized
destinations, and sources unavailable under existing artifact policy.
4. Merge generated and explicit rules in deterministic destination/source order
before existing publish and static-lock validation. Runtime publish code
continues to receive a concrete list and performs no wildcard/family match.
5. Include generated rules in normalized effective configuration, effective
digest, source provenance, publish semantic fingerprint, plan output, and
static/remote lock applicability.
6. Update configuration, publish, artifact-policy, operations, and manifest
internals for the implemented expansion boundary.
### Tests And Exit Criteria
- Tests cover explicit destination patterns, omitted destination derivation,
disabled policy, missing top-level publish config, explicit/generated source
conflicts, duplicate destinations, token/path failures, required propagation,
static locks, and deterministic order.
- Publish stage tests use existing fake object storage to prove generated rules
enter the same staging/commit/lock path as explicit rules.
- No runtime source matcher or adapter accepts family wildcards.
## Stage 17 — Read-Only `config validate` And `config show`
**Status: Completed**
### Goal
Let operators validate and inspect the complete selected effective pipeline
without creating a session, workspace, run, or external adapter.
### Required Work
1. Add the top-level `config` command dispatcher and `validate`/`show`
subcommands. Reuse the shared explicit-presence profile flag and existing
pipeline/campaign selection parsers; reject session-only, stage-range,
force, and artifact-execution flags.
2. Add a shared read-only inspection resolver that uses the Stage 8 production
pipeline/campaign path but does not discover/load a session. If the selected
pipeline contains any party-driven family, require an explicit/unambiguous
campaign through the existing mechanisms and perform canonical party and
all family/publish expansion. Otherwise permit pipeline-only resolution.
3. `config validate` must run complete strict composition, selection, defaults,
path resolution, campaign/party loading when needed, expansion, concrete
artifact/publish validation, and effective digesting. On success, print a
concise root/profile/digest summary; on failure, retain source/field context.
4. `config show` must perform the same validation and emit one deterministic
normalized effective YAML document. Remove composition metadata,
resolution-only families, runtime provenance, and raw secret values; include
defaulted fields, expanded concrete artifacts, and generated publish rules.
5. Define a stable serializer over the logical effective representation rather
than marshaling runtime-only fields or relying on nondeterministic map
traversal. Use documented YAML scalar types and one trailing newline.
6. Both commands must be side-effect free: no workspace layout, manifest,
session lock, secret read, subprocess, network, object storage, or cleanup.
7. Register help/usage and update `docs/cli.md`, `docs/config.md`, and focused
command/config internals. `show` owns effective output, not source tracing or
profile comparison yet.
### Tests And Exit Criteria
- CLI tests cover help, unknown subcommands, default/explicit profiles,
pipeline-only success, campaign registry/file selection, family-without-
campaign errors, canonical party errors, and imported source diagnostics.
- `show` golden/semantic tests cover deterministic ordering and expanded
concrete values without overcoupling to incidental whitespace. Golden updates
require the repository's explicit-review convention.
- Side-effect tests prove no workspace/manifest is created and external fake
adapters/object stores are not invoked.
- `validate` and `show` resolve the same digest as plan/run for an equivalent
fully resolved pipeline and campaign.
## Stage 18 — Read-Only `config sources`
**Status: Completed**
### Goal
Expose enough deterministic source ownership to explain effective values and
generated configuration without leaking secrets or internal YAML machinery.
### Required Work
1. Add `config sources` on the Stage 17 resolver and flags. It must validate the
same complete effective configuration before reporting ownership.
2. Complete ownership propagation through defaults, overlay replacements,
canonical campaign/party resolution, derived players, concrete family
expansion, member dependencies/sources, and generated publish rules.
3. Emit sorted records with at least effective field path, source role, and
source path/identifier. Use repeat records when a generated value has both a
family declaration and party source. Mark centralized defaults explicitly
rather than assigning them to whichever source happened to be traversed.
4. Report the root, ordered imports, selected profile/selection source/overlay,
selected campaign, canonical or legacy party mode, party source, and
effective digest in a concise header. Legacy players provenance remains
visible only as a legacy source; canonical derived players point to party.
5. Normalize paths consistently, but never print raw file contents,
environment-resolved credential values, secrets directory contents, or
private adapter configuration contents.
6. Keep source reporting as a projection over configuration provenance. Do not
reparse files in the CLI or add annotations to runtime YAML structs solely
for formatting.
7. Update the CLI/config internal documentation and troubleshooting guidance
for diagnosing duplicate ownership and unexpected profile values.
### Tests And Exit Criteria
- Tests cover root/import ownership, profile replacement versus inherited
leaves, list ownership, defaults, campaign/party, derived players, family
variables, member sources/dependencies, generated publish rules, and stable
sorting.
- A source-aware conflict diagnostic and successful `config sources` output use
the same logical path convention.
- Secret sentinel values placed in environment/files never appear in output;
only configured indirect identifiers may appear where part of effective
configuration.
- The command remains side-effect free under the Stage 17 test harness.
## Stage 19 — Semantic `config diff`
**Status: Completed**
### Goal
Compare two fully resolved profiles by effective meaning rather than raw file
layout or formatting.
### Required Work
1. Add `config diff <left-profile> <right-profile>` with exactly two non-empty
positional profile names plus `--config`, `--campaign`, and
`--campaign-file`. Reject a separate `--profile`, duplicate singleton flags,
extra/missing profiles, and identical unknown selections through shared
profile rules.
2. Resolve the left and right independently from the same already parsed root
source set and the same explicitly selected campaign/party. Do not let one
profile's mutable structs or expansion metadata contaminate the other.
3. Flatten the two normalized secret-free effective mappings to logical field
paths. Emit sorted `added`, `removed`, and `changed` records with deterministic
compact representations of the relevant value(s). Treat an atomic list
replacement as one changed path unless semantic child paths are independently
addressable by the normalized model.
4. Compare expanded concrete artifacts and publish rules, not family source
text. Profile-only source movement with equal effective values produces no
semantic difference; differing complete digests with no emitted semantic
difference is an internal error.
5. Return success with an explicit `no differences` result when equal. Use
normal command errors for invalid resolution; do not use a non-zero exit
merely because differences exist unless Narratio already has a documented
CLI convention for that behavior.
6. Reuse Stage 17's side-effect-free resolver and Stage 1's canonical traversal.
Do not invoke an external `diff` process or add a raw-text diff dependency.
7. Update `docs/cli.md`, `docs/operations.md`, and focused config command
internals with the semantic output and migration/review use case.
### Tests And Exit Criteria
- Tests cover scalar changes, additions/removals, explicit false, list
replacement, keyed artifact additions, disabled artifacts, party-expanded
model/variable changes, generated publish differences, equal effective
profiles, invalid profile/campaign inputs, and deterministic order.
- Reformatting or moving an equal value between eligible base imports produces
no diff after successful composition.
- Profile resolution does not mutate cached/shared base nodes and remains
deterministic when left/right argument order is reversed.
- The command performs no runtime state or external side effects.
## Stage 20 — Maintained Split Configuration Bundle
**Status: Pending**
### Goal
Provide a copyable, validated production/testing bundle and move maintained
example data to the new party source of truth.
### Required Work
1. Add or migrate one maintained example bundle with root `pipeline.yml`,
explicit `conf.d` imports for stable concerns, production/testing overlays,
`production` as the default, and no auto-loaded fragment assumptions.
2. Use testing/production model selections that are obvious placeholders and
secret-free. Demonstrate at least one testing-only or testing-disabled
artifact without relying on deletion syntax or profile inheritance.
3. Migrate the maintained campaign to `narratio.party.v1` with stable IDs,
aliases, repeated-safe player semantics, and multiclass data. Remove its
separate `players_file` and provide at least two character families (meta
analysis and item tracking), a same-member dependency/source, and one family
publish policy.
4. Extend `TestExamplesLoadAndValidate` or its focused helpers to discover each
maintained root, validate every declared profile intentionally, select the
required campaign for families, and assert that examples remain offline and
secret-free. Retain at least one monolithic/explicit-concrete compatibility
fixture if the maintained set no longer naturally provides it.
5. Update `examples/README.md` and only the minimum canonical configuration or
integration links needed to make the new bundle discoverable. Do not defer
field/command behavior documentation from the stages that implemented it.
### Tests And Exit Criteria
- The maintained example test loads monolithic compatibility and every profile
of the split canonical bundle with its campaign, expands expected concrete
members/rules, and rejects no unknown fields.
- Documentation checks pass; complete examples exist only under `examples/`;
and no secret/private infrastructure values are added.
- A new operator can validate, show, source-trace, and diff the example profiles
using the documented commands without creating runtime state.
## Stage 21 — Canonical Documentation And Migration Convergence
**Status: Pending**
### Goal
Make current user, operator, integration, policy, and internal documentation
describe the completed feature once, at the correct canonical owners.
### Required Work
1. Review every document changed in earlier stages against the implementation
and maintained Stage 20 bundle. Correct commands, flags, fields, defaults,
schemas, paths, output conventions, compatibility limits, and resume claims;
do not repeat complete examples outside `examples/`.
2. Complete `docs/config.md`, `docs/cli.md`, `docs/operations.md`, and integration
links with a concise migration path: split additively, define/select
profiles, convert party, remove players, add families, inspect sources/show,
compare profiles, then run. Link to the maintained bundle rather than copy
it.
3. Revise `docs/policy/architecture.md` only for the durable implemented
invariants: root-owned deterministic composition, campaign-owned canonical
party, pre-runtime family expansion, one loading path, and semantic
stage/artifact resume evidence. Do not move field syntax or implementation
inventory into policy.
4. Update `docs/internal/overview.md` and focused configuration, manifest,
prepare, analyze, artifact, adapter, and publish documents so ownership and
cross-links describe the final implementation without parallel contracts.
5. Review troubleshooting and README orientation for only stable summaries and
links. Remove stale players-file guidance from canonical workflows while
retaining clearly labeled legacy migration documentation at its canonical
owner.
6. Check that documentation distinguishes Narratio-observable semantic
fingerprints from private external-tool content changes that still require
force, and that it never presents the profile name/effective digest as a
blanket resume key.
### Tests And Exit Criteria
- Documentation checks and maintained-example validation pass.
- Each contract has one canonical owner under the documentation policy; other
documents contain only a short stable summary and link.
- Current-behavior documents contain no pending implementation language, and
roadmap documents do not masquerade as current reference material.
- No commands, flags, schema fragments, secret values, private infrastructure,
or complete example bundles are duplicated into the wrong owner.
## Stage 22 — Assembled Workflow Regression And Final Validation
**Status: Pending**
### Goal
Prove the completed feature reaches existing pipeline boundaries correctly and
remove redundant implementation/test scaffolding before release review.
### Required Work
1. Add a small number of assembled offline workflow tests using real config,
filesystem, manifest, planner, and reconciliation owners plus fakes only for
external adapters/storage. Do not reproduce every lower-level parser case.
2. Prove one representative split bundle and canonical campaign can:
- select the production default and explicit testing profile;
- prepare unchanged party plus derived players;
- pass both prepared references to Notarius;
- expand two families with member variables, dependencies, and member input;
- select a full family or one concrete member;
- reconcile an added, changed, and removed party character; and
- expand and execute family publication through concrete rules.
3. Prove profile switching reuses semantically unchanged transcript stages,
stales an Audita polish result when its model changes, and limits analyze
regeneration to artifacts whose effective fingerprint changed. Verify
session/run provenance reports the selected profile/digest throughout.
4. Exercise `config validate/show/sources/diff`, `session plan`, and a bounded
run against the same fixtures and assert consistent effective digest,
concrete artifact identity, and family origin. Inspection remains
side-effect free.
5. Audit the final implementation for alternate production config loaders,
unresolved family/member syntax, reflection/whole-config cache keys,
compatibility code outside its marked boundary, duplicated merge or
projection helpers, stale documentation, and redundant tests. Consolidate
only where behavior remains unchanged.
6. Re-run formatting and the complete validation set from
`docs/development.md`:
- `go test ./...`
- `go test -race ./...`
- `go vet ./...`
- `go build ./...`
- `go test ./internal/doccheck`
- `go test ./internal/config -run '^TestExamplesLoadAndValidate$'`
7. Record any platform limitation exactly as current development policy does;
do not claim native macOS/Windows runtime evidence from cross-compilation.
### Tests And Exit Criteria
- The assembled tests protect cross-package wiring and meaningful state
transitions while lower-level tests retain ownership of detailed parse and
validation matrices.
- All repository validation commands pass offline and without real
credentials, paid APIs, mutable services, or live Notarius/Scriptorium calls.
- The implemented code, examples, manifests, CLI output, current documentation,
feature roadmap target state, and this plan agree on the final behavior.
- The worktree contains no generated test artifacts or obsolete compatibility
fixtures outside the intentionally retained legacy boundary.
## Open Questions
None. The accepted feature roadmap and the settled decisions above are
sufficient to implement every stage without another product or architecture
choice. If implementation reveals a genuinely new decision that would change
the accepted target state, stop that stage and revise the roadmap/plan with the
user rather than choosing a broader behavior implicitly.