diff --git a/docs/releases/README.md b/docs/releases/README.md index c69ee14..f2124ec 100644 --- a/docs/releases/README.md +++ b/docs/releases/README.md @@ -22,4 +22,5 @@ See the [release procedure](../release.md) for creating a candidate and tag. When asynchronous publication succeeds, the corresponding Gitea release is the canonical source for downloadable binaries and checksums. +- [v1.6.0](v1.6.0.md) - [v1.5.0](v1.5.0.md) diff --git a/docs/releases/v1.6.0.md b/docs/releases/v1.6.0.md new file mode 100644 index 0000000..22683b5 --- /dev/null +++ b/docs/releases/v1.6.0.md @@ -0,0 +1,74 @@ +# Narratio v1.6.0 + +Narratio v1.6.0 makes large pipeline configurations easier to organize, +inspect, and vary while adding character-oriented artifact generation and a +guarded, reproducible release procedure. + +## Summary + +Pipeline configuration can now be assembled from explicit additive imports and +a selected production or testing profile. Campaigns can own a canonical, +versioned party roster, and Scriptorium artifact families can expand one +definition into concrete per-character artifacts, dependencies, variables, and +publish rules. + +New read-only configuration commands expose the fully resolved pipeline, +source provenance, semantic digest, and profile differences before a session is +run. Stage reuse now records configuration-sensitive semantic evidence so +profile or configuration changes cannot silently reuse incompatible work. + +## Compatibility + +This is a backward-compatible feature release. Existing monolithic pipeline +files, concrete Scriptorium artifacts, publish rules, and configurations +without profiles remain supported. When profiles are declared and no explicit +profile is selected, Narratio uses the configured production default. + +An unversioned party file plus a separate players file remains available as an +isolated legacy compatibility path, but it cannot drive artifact families. New +campaigns and new party-oriented features should use the `narratio.party.v1` +schema. Existing manifests remain readable; missing legacy semantic evidence +is treated as stale rather than trusted. + +Narratio continues to consume the documented Notarius D&D pipeline contract. +The release workflow still cross-compiles Linux, macOS, and Windows binaries +for `amd64` and `arm64`; cross-compilation is not native runtime evidence for +macOS or Windows. + +## Upgrade + +No special action is required for existing monolithic configurations that do +not adopt profiles or artifact families. On the first run after upgrading, +stages recorded by older manifests may regenerate once because those records do +not contain the new semantic configuration evidence. + +To adopt the new configuration model, use the maintained +`examples/production-testing` bundle as a migration reference: split stable +settings into explicit imports, define a production default and optional +testing profile, convert campaign party data to `narratio.party.v1`, remove the +separate players file, and then introduce character artifact families. Review +the result with `narratio config validate`, `config show`, `config sources`, and +`config diff` before running a session. + +Campaign, session, previous-session, and run identifiers must satisfy the +documented portable identity grammar. Existing manifests or remote state with +unsafe legacy identifiers must be migrated before use. + +## Changes + +- Added root-owned, non-recursive additive pipeline imports with strict, + source-aware conflict detection. +- Added named pipeline profiles with an explicit production default and + deliberate command-line selection. +- Added `config validate`, `config show`, `config sources`, and `config diff` + for read-only inspection of resolved configuration and provenance. +- Added the strict `narratio.party.v1` campaign roster, including stable + character IDs, player and character names, optional aliases, and classes. +- Added deterministic players derivation and canonical party delivery to + downstream integrations. +- Added character-oriented artifact families, corresponding member + dependencies, family selection, and generated publish policies. +- Added semantic configuration fingerprints and stage-specific resume checks + across transcript and artifact stages. +- Added shared release candidate, asset build, and guarded tag-publication + scripts, with tag-triggered publication remaining asynchronous. diff --git a/docs/roadmap/implementation.md b/docs/roadmap/implementation.md deleted file mode 100644 index 984a618..0000000 --- a/docs/roadmap/implementation.md +++ /dev/null @@ -1,471 +0,0 @@ -# Release Procedure Upgrade Implementation Plan - -## Purpose And Status - -This document is the executable implementation plan for -[`release-procedure.md`](release-procedure.md). The feature roadmap owns the -accepted user intent, policy choices, required outcome, and responsibility -boundary. This plan translates that target into bounded implementation stages -suitable for one `gpt-5.6-terra` coding prompt apiece. - -All five stages were completed in numeric order. This document is retained as -implementation history; current release behavior is owned by the checked-in -scripts and canonical release documentation. This plan superseded the completed -pipeline-configuration implementation plan that previously occupied this path; -that feature's implemented behavior is now owned by current canonical -documentation and code. - -## Settled Implementation Decisions - -The following decisions are final for this implementation: - -- The synchronous release boundary is a successful push of the exact selected - tag ref to `origin`. Neither `scripts/release.sh` nor any instruction that - defines successful release completion may poll Woodpecker, query build - status, wait for a Gitea release, download assets, or verify asynchronous - publication. -- The supported command surfaces are exactly: - `scripts/build-release-assets.sh VERSION OUTPUT_DIR`, - `scripts/check-release-candidate.sh VERSION`, and - `scripts/release.sh VERSION`. -- All scripts are POSIX `sh`, use `set -eu`, derive the repository root from - their own checked-in location, and use narrowly named variables. They do not - rely on the caller's current directory and do not repurpose `HOME`, - `CODEX_HOME`, or common system-option variables. -- A private source-only `scripts/release-lib.sh` owns the stable semantic - version parser and shared fatal-error/reporting helpers. It has no behavior - merely from being sourced. Public scripts source it relative to their own - location so version syntax and error conventions do not drift. -- Stable versions match exactly - `v(0|[1-9][0-9]*)\.(0|[1-9][0-9]*)\.(0|[1-9][0-9]*)`. Prerelease and build - suffixes are rejected. -- `scripts/build-release-assets.sh` accepts exactly two arguments. Its output - path must be absolute, must not be `/` or the repository root, must not be a - symlink or non-directory, and must either not exist or be an empty directory. - The script creates the directory when absent, writes only its six named - assets and any narrowly scoped temporary smoke binary, and never recursively - clears a caller-supplied path. -- Official targets are CGO-disabled Linux, Darwin, and Windows builds for both - `amd64` and `arm64`. Filenames are - `narratio-VERSION-GOOS-GOARCH`, with `.exe` only on Windows. Builds use - `-trimpath`, `-s -w`, and - `-X gitea.maximumdirect.net/eric/narratio/internal/buildinfo.Version=VERSION`. -- The asset builder verifies `narratio version` on the current host. When the - host is one of the six official targets it executes that official asset; - otherwise it builds a separate temporary host smoke binary, executes it, - and removes only that file. Expected output is exactly - `narratio VERSION`. -- `scripts/check-release-candidate.sh` is read-only with respect to tracked - source and Git refs. It accepts exactly one version, validates the matching - note, performs repository and Go-module hygiene checks, runs every required - test/static/build/documentation/example check uncached where applicable, - verifies formatting and whitespace, and calls the shared asset builder in a - directory created with `mktemp -d`. Every Go command it owns runs with - `GOWORK=off`. -- The candidate checker deliberately does not require `main`, a clean checkout, - synchronized remote refs, or an unused tag. This allows it to validate a - candidate before commit and to run in tag CI's detached checkout. Those - mutable publication guards belong only to `scripts/release.sh`. -- `scripts/release.sh` accepts exactly one version, operates only against the - remote named `origin` and branch named `main`, and never accepts bypass, - force, alternate-remote, skip-check, or CI-wait flags. -- The publication script uses a two-phase guard. It checks cheap local and - upstream conditions before the potentially long candidate checker, then - fetches and rechecks the clean checkout, exact `HEAD == origin/main`, and - local/remote tag absence immediately before creating the tag. This prevents - a long validation run from publishing after the candidate or upstream tip - changed. -- Release tags remain explicitly unsigned lightweight tags. The script uses - `git -c tag.gpgSign=false tag VERSION COMMIT`, verifies that the ref type is - `commit` and resolves to the recorded commit, and pushes only - `refs/tags/VERSION:refs/tags/VERSION`. It never pushes `main` or uses - `git push --tags`. -- A failed push leaves the unpublished local tag intact for inspection. The - script never silently deletes or retries it. A successful push prints the - version and exact commit, returns zero immediately, and performs no later - network or CI action. -- Release-script behavioral tests use real temporary Git repositories and a - local bare `origin`, with a fixture-local stub candidate checker where a - full repository validation would add no confidence. They never touch the - developer's actual refs or any network service. -- Tag CI uses `golang:1.25.5` and - `woodpeckerci/plugin-release:0.3.1`. Its steps are named - `validate-release`, `build-release-assets`, and `publish-release` in that - dependency order. -- The release plugin explicitly sets title - `Narratio ${CI_COMMIT_TAG}`, note - `docs/releases/${CI_COMMIT_TAG}.md`, flattened SHA-256 checksums, - `file-exists: skip`, `overwrite: false`, and `prerelease: false`. Skipping an - existing remote release preserves idempotence and immutability; repairing a - failed published tag remains a new-patch-release operation. -- Release documentation describes optional later inspection but never makes - CI success, Gitea release creation, checksum download, or host asset execution - a required step after a successful tag push. -- This feature does not create the next release note, tag a version, push - `main`, or publish a release. Those actions occur only after this - implementation sprint is complete and a separate release candidate is - selected. - -## Instructions For Every Stage - -Before changing files in each stage: - -1. Read `docs/development.md`, all documents under `docs/policy/`, this plan, - and the relevant portions of - [`release-procedure.md`](release-procedure.md). -2. Inspect the current scripts, Woodpecker workflows, release notes, - `internal/buildinfo`, `internal/app/version.go`, and focused repository - checks relevant to the stage. Prefer the codebase knowledge graph for Go - discovery and use text search for shell, YAML, and Markdown. -3. Confirm the worktree state and preserve unrelated changes. -4. Re-read the responsibility boundary before editing any publication command: - successful upstream tag push ends the synchronous release process. - -During every stage: - -- Keep public shell interfaces and file ownership exactly as settled above. -- Use the Go standard library and ordinary POSIX tools already available in the - builder images; do not add a shell framework, Go dependency, release SaaS, - or package-manager tool. -- Keep tests deterministic, offline, and free of real credentials and remote - infrastructure. Use `t.TempDir()` and local bare Git repositories for - publication behavior. -- Test consequential behavior at the narrowest stable boundary. Avoid tests - that merely snapshot complete script text or duplicate every shell branch. -- Make failures identify the responsible script, invalid value, or failed - command without printing environment dumps or secret values. -- Do not document future behavior outside `docs/roadmap/` until the stage that - makes that behavior usable. Update canonical current-behavior documentation - in the same stage that completes the release procedure. -- Format changed Go files, run `sh -n` on changed shell files, use focused - tests while iterating, and finish each stage with `git diff --check` plus the - stage's exit criteria. -- Do not create, delete, move, or push a real Narratio tag while implementing - or testing this feature. - -## Stage 1 — Shared Release Library And Asset Builder - -**Status: Completed** - -### Goal - -Create the single safe implementation of version validation and official -release-asset construction without changing Git refs, CI, or current release -documentation. - -### Required Work - -1. Create `scripts/release-lib.sh` as a source-only POSIX library. Give its - functions a Narratio-specific prefix. At minimum it must provide: - - a fatal-error helper that prefixes errors with the calling release tool; - - exact stable-semver validation; and - - a helper for locating the repository root from a public script's absolute - or relative path without depending on the caller's working directory. - Keep the library private to the scripts; it is not a supported user-facing - API. -2. Create executable `scripts/build-release-assets.sh` with the exact - `VERSION OUTPUT_DIR` interface and output-directory contract in the settled - decisions. Reject missing/extra arguments, malformed versions, relative or - dangerously broad output paths, symlink/non-directory output, and a - nonempty output directory before building anything. -3. Build the six official binaries in deterministic target order. Use the - current command package and build-info linker symbol; do not duplicate or - change application version semantics. -4. Verify exact host version output as settled above. Ensure a failed build or - version mismatch is fatal and leaves no false success message. Temporary - smoke output must be inside the validated staging directory and removed - narrowly. -5. Add focused tests under a new `internal/releasecheck` test package. Protect - the meaningful asset-builder risks without making the default Go suite - compile all six targets repeatedly: - - argument and semver rejection; - - refusal of unsafe, symlink, non-directory, and nonempty destinations; - - creation only beneath an accepted empty destination; and - - exact naming/version behavior through one controlled host build or a - command shim, whichever is simpler and remains behavior-focused. - The operational six-target build is exercised directly in the stage exit - criteria and later by candidate validation. - -### Exit Criteria - -- `sh -n scripts/release-lib.sh scripts/build-release-assets.sh` succeeds. -- Focused `internal/releasecheck` tests pass. -- Running the asset builder with a synthetic stable version and a fresh - temporary output directory produces exactly six official assets and the - host version check succeeds. -- Invalid and unsafe output examples fail before creating release assets. -- `go test ./...`, `go vet ./...`, `go build ./...`, and - `git diff --check` succeed. - -## Stage 2 — Central Release-Candidate Checker - -**Status: Completed** - -### Goal - -Create one read-only source-candidate validator that maintainers and tag CI can -run identically, eliminating drift between documented preparation and tagged -source validation. - -### Required Work - -1. Create executable `scripts/check-release-candidate.sh` with the exact - one-version interface. Source `release-lib.sh`, resolve and enter the - repository root, and validate the version before using it in a path. -2. Require `docs/releases/VERSION.md` to be nonempty, begin with the exact - `# Narratio VERSION` heading, and contain exact - `## Summary`, `## Compatibility`, `## Upgrade`, and `## Changes` headings. - Do not parse prose or impose historical-note requirements on other tags. -3. Enforce repository/module hygiene: - - exact Narratio module declaration; - - no tracked `go.work` or `go.work.sum`; - - no `vendor` directory; - - no single-line or block `replace` declaration in `go.mod`; - - `GOWORK=off go mod tidy -diff` leaves module files unchanged; - - all tracked `*.go` files are `gofmt` clean; and - - both working-tree and cached whitespace checks pass. -4. Run the complete Narratio candidate checks with `GOWORK=off`: - - `go test -count=1 ./...`; - - `go test -race -count=1 ./...`; - - `go vet ./...`; - - `go build ./...`; - - `go test -count=1 ./internal/doccheck`; and - - `go test -count=1 ./internal/config -run '^TestExamplesLoadAndValidate$'`. -5. Create one temporary directory with `mktemp -d`, install a trap that removes - only that verified directory, and invoke `build-release-assets.sh` with the - selected version. The candidate checker owns no independent target matrix, - filenames, linker flags, or version smoke logic. -6. Extend `internal/releasecheck` with lean behavioral coverage for the - candidate checker's guards. Use isolated fixture repositories and command - shims where needed so default tests do not recursively run the complete - Narratio suite. Cover malformed version/note, wrong module identity, - disallowed workspace/vendor/replace state, propagation of a failing owned - validation command, and evidence that owned Go commands receive - `GOWORK=off`. -7. Ensure the checker itself does not inspect the active branch, fetch a - remote, require a clean status, check tag availability, mutate refs, or - contact Woodpecker/Gitea. Those are deliberately outside this script's - contract. - -### Exit Criteria - -- `sh -n scripts/release-lib.sh scripts/build-release-assets.sh scripts/check-release-candidate.sh` - succeeds. -- Focused `internal/releasecheck` tests pass, including failure propagation. -- A disposable fixture with a compliant synthetic release note exercises the - candidate checker successfully; malformed fixtures fail with actionable - script-owned errors. -- The repository-wide test, race, vet, build, documentation, example, - formatting, module-tidiness, and whitespace checks all pass independently. -- No Git ref, tracked file, or external service is mutated by the checker. - -## Stage 3 — Guarded Tag Publication Command - -**Status: Completed** - -### Goal - -Provide one safe, non-interactive command that publishes exactly one guarded -lightweight tag and stops immediately when the upstream Git push succeeds. - -### Required Work - -1. Create executable `scripts/release.sh` with the exact one-version interface. - Source the shared library and reject invalid arguments before running Git - operations. -2. Implement the first cheap guard phase: - - require branch `main`; - - require an empty `git status --porcelain` result; - - fetch `origin main --tags`; - - record `HEAD^{commit}` as the immutable candidate for this attempt; - - require it to equal `origin/main^{commit}`; - - require the matching note to exist in that candidate; and - - reject an existing local or exact upstream tag. -3. Invoke `scripts/check-release-candidate.sh VERSION` without suppressing its - output or weakening any check. -4. Implement the second guard phase immediately after validation: - - refetch `origin main --tags`; - - require the checkout still to be clean and on `main`; - - require `HEAD` and `origin/main` still to equal the originally recorded - candidate commit; and - - recheck exact local and upstream tag absence. - Do not silently select a newer commit or restart validation. -5. Create the explicitly unsigned lightweight tag against the recorded commit, - verify its object type and resolved commit, show a concise local summary, - and push only its fully qualified tag ref to `origin`. -6. On push success, print one concise completion message containing the tag and - commit and exit zero immediately. There must be no commands after the push - that query CI, Gitea, a release API, assets, or checksums. Do not sleep or - retry waiting for asynchronous state. -7. On failure before tag creation, leave refs unchanged. On failure after local - tag creation, retain the local tag and return a clear instruction that it is - unpublished and requires inspection. Never automatically delete, move, - force-push, or reuse a tag. -8. Add behavioral tests in `internal/releasecheck` using temporary working - repositories and local bare `origin` repositories. Put a successful stub - candidate checker at the fixture's expected checked-in path so tests isolate - tag policy rather than rerun compilation. Cover at least: - - wrong branch and dirty checkout rejection; - - local `HEAD` not equal to `origin/main`; - - candidate-checker failure; - - existing local and upstream tag rejection; - - upstream `main` changing between the two guard phases; - - successful creation of a lightweight remote tag pointing to the exact - candidate commit; - - push failure retaining the local tag; and - - absence of any pushed branch or unrelated tag. - Tests must never use the real repository's `origin` or network. - -### Exit Criteria - -- All release scripts pass `sh -n`. -- Focused publication tests pass and prove the exact remote-tag outcome and - failure behavior. -- A success-path test demonstrates that the release command returns as soon as - the local bare-remote tag push succeeds; it performs no CI/Gitea follow-up. -- `go test ./...`, `go test -race ./...`, `go vet ./...`, `go build ./...`, and - `git diff --check` succeed. -- No real Narratio tag or remote ref is created or changed. - -## Stage 4 — Asynchronous Woodpecker Release Integration - -**Status: Completed** - -### Goal - -Make tag CI consume the shared candidate and asset contracts while preserving -strict validation-before-publication ordering and complete independence from -the synchronous tag command. - -### Required Work - -1. Rewrite `.woodpecker/release.yml` around three explicit steps: - - `validate-release` in `golang:1.25.5`, invoking - `./scripts/check-release-candidate.sh "$CI_COMMIT_TAG"`; - - `build-release-assets` in `golang:1.25.5`, depending on - `validate-release`, requiring a fresh absent/empty absolute `$PWD/dist`, - and invoking the shared asset builder; and - - `publish-release`, depending on `build-release-assets` and using - `woodpeckerci/plugin-release:0.3.1`. - Remove the redundant inline target matrix, validation command list, and - separate cross-build step from the release workflow. -2. Configure publication exactly as settled: the existing release-token - secret, `dist/narratio-*` files, explicit title and tagged note, flattened - `SHA256SUMS`, `file-exists: skip`, no overwrite, and no prerelease. -3. Preserve the tag-only event trigger. Do not add a pipeline callback, status - endpoint, wait command, or any coupling from `scripts/release.sh` to this - workflow. -4. Update `internal/doccheck/doccheck_test.go` so the dependency assertion names - `validate-release` and still proves transitively that publication cannot run - after failed validation. Extend its minimal YAML model only as needed to - protect consequential release invariants: - - the validator calls the shared checker; - - asset publication depends on validated shared construction; - - the release plugin image is pinned; - - title and note use the selected tag; and - - overwrite/prerelease remain disabled. - Avoid snapshotting the whole workflow or duplicating every plugin setting. -5. Confirm `.woodpecker/verify.yml` and `.woodpecker/shuffle.yml` remain normal - push/PR and scheduled validation. Do not make either wait for or invoke a - release operation. - -### Exit Criteria - -- The Woodpecker YAML parses and the focused `internal/doccheck` workflow - dependency/invariant tests pass. -- The release workflow has one target matrix owner and one candidate-check - owner, both under `scripts/`. -- `publish-release` has a dependency path to `validate-release`; no failure in - validation or asset building can publish. -- Static inspection confirms `scripts/release.sh` has no reference to - Woodpecker, Gitea release status, or CI polling. -- `go test ./...`, `go test -race ./...`, `go vet ./...`, `go build ./...`, - documentation/example checks, and `git diff --check` succeed. - -## Stage 5 — Canonical Release Documentation And Final Audit - -**Status: Completed** - -### Goal - -Document the now-implemented maintainer workflow in one canonical location, -make it discoverable, and perform a complete offline acceptance audit without -creating or publishing a real release. - -### Required Work - -1. Create `docs/release.md` as the canonical current-behavior procedure. It - must include: - - stable SemVer selection for a post-`v1.0.0` project; - - the exact future release-note structure; - - candidate preparation and ordinary `main` push expectations; - - direct use of `scripts/check-release-candidate.sh`; - - direct use and complete safety contract of `scripts/release.sh`; - - lightweight-tag immutability and correction through a new patch version; - - the explicit statement that successful upstream tag push completes the - release command; and - - a clearly optional asynchronous inspection section, separated from the - release steps and stating that a human may inspect CI/Gitea later but an - automated releaser must not wait for it. - Do not embed a second implementation of script guards as a long shell block; - the scripts own volatile mechanics and the document owns the maintainer - workflow and policy. -2. Update `docs/development.md` with a task-guide row pointing release work to - `docs/release.md`. Keep its validation summary concise and link to the - release procedure/checker rather than duplicating the candidate command - list. -3. Update `docs/policy/documentation.md` to assign canonical ownership for the - maintainer release procedure and version-matched historical notes. Preserve - the rule that release notes link to, rather than replace, current contract - documentation. -4. Update `docs/releases/README.md` to explain forward-looking note structure, - retain its existing `v1.5.0` entry, and identify the Gitea release collection - as the binary/checksum source when asynchronous publication succeeds. Do - not backfill or rewrite historical notes. -5. Review other current documentation for claims that release completion waits - on CI. Remove or link any conflicting duplication. Do not add release - mechanics to operations, CLI, architecture, or integration documents unless - they already own a directly affected current contract. -6. Mark `docs/roadmap/release-procedure.md` implemented only after scripts, - tests, CI, and canonical documentation match its target. Keep implementation - status in roadmap documents; current-behavior claims belong in - `docs/release.md` and the policy/developer routing documents. -7. Perform a final behavior and safety audit: - - inspect all scripts for unsafe path deletion, unresolved variables, - accidental secret output, broad ref pushes, force options, and commands - after the successful tag push; - - run all release-script behavioral tests against temporary local remotes; - - run the asset builder directly with a synthetic stable version and a - fresh temporary output directory, then verify the six filenames and host - version output; - - validate candidate-checker success in its isolated fixture and every - important failure class without creating a real note or tag; - - confirm the release workflow reads the note from the tagged tree and - cannot publish before validation; and - - confirm no test or documentation instruction requires live CI, Gitea, - credentials, or paid/external adapters. - -### Exit Criteria - -- All local Markdown links and Woodpecker dependency checks pass. -- `sh -n scripts/release-lib.sh scripts/build-release-assets.sh scripts/check-release-candidate.sh scripts/release.sh` - succeeds. -- Release-script tests and the direct six-target asset build pass without - touching real refs or external services. -- The full repository validation set from `docs/development.md` passes: - - `go test ./...`; - - `go test -race ./...`; - - `go vet ./...`; - - `go build ./...`; - - `go test ./internal/doccheck`; and - - `go test ./internal/config -run '^TestExamplesLoadAndValidate$'`. -- `go mod tidy -diff`, tracked-file `gofmt` inspection, - `git diff --check`, and `git diff --cached --check` succeed. -- The repository contains no synthetic release note, local test tag, build - asset, credential, temporary repository, or other generated test residue. -- No real release is tagged or pushed during this implementation plan. - -## Open Questions - -None. The feature roadmap and settled decisions above are sufficient to -implement the release procedure without additional product or policy choices. diff --git a/docs/roadmap/pipeline-configuration-ergonomics.md b/docs/roadmap/pipeline-configuration-ergonomics.md deleted file mode 100644 index fecdae0..0000000 --- a/docs/roadmap/pipeline-configuration-ergonomics.md +++ /dev/null @@ -1,739 +0,0 @@ -# Pipeline Configuration Ergonomics - -## Status - -Accepted target state. This document owns the intended feature scope, user -intent, policy choices, and target end state until the behavior is implemented. - -## Goal - -Make Narratio's pipeline configuration easier to organize, vary, inspect, and -reuse without weakening its strict configuration contract or turning it into a -general-purpose configuration or workflow language. - -The feature should let operators: - -- split a large pipeline configuration into explicitly imported partial files; -- maintain one production-default pipeline plus a selectable testing profile; -- keep stable settings separate from frequently edited models and artifacts; -- define a character-oriented artifact workflow once and expand it for every - character in the selected campaign; -- reuse one campaign-owned party roster across multiple artifact families and - the Notarius integration; and -- inspect and compare the fully resolved configuration before running a - session. - -All composition and expansion must produce the same concrete, validated -`PipelineConfig` and concrete configured-artifact map consumed by the existing -stage, planning, manifest, artifact, and adapter boundaries. - -## User Intent - -Pipeline configuration currently combines long-lived platform and transcript -settings with model choices, prompt-backed artifact definitions, and publish -rules that change frequently during development. Keeping all of those concerns -in one file makes edits noisy and makes production/testing variation difficult -to review. - -Character-specific artifacts introduce a second kind of repetition. Several -artifact families, including character meta-analysis and character-specific -item tracking, should execute once per campaign character. Their prompt, -inputs, dependencies, and output shape are shared, while player, character, -class, alias, and output identity vary by party member. Those campaign facts do -not belong in a global pipeline definition. - -The desired model is therefore: - -```text -pipeline.yml + explicit imports + one selected profile - │ -campaign.yml + party.yml ─┤ - ▼ - resolved effective configuration - │ - expand party-driven families - │ - defaults and strict validation - ▼ - existing concrete pipeline and artifact model -``` - -## Architectural Constraints - -The feature must preserve the configuration and orchestration policies in -[`docs/policy/architecture.md`](../policy/architecture.md): - -- YAML remains strict and rejects unknown fields. -- Defaults remain centralized and are applied exactly once. -- Composition and expansion are deterministic and testable. -- Session templating remains narrow; this feature does not introduce arbitrary - expressions, scripting, or a general template language. -- The canonical stage sequence and fixed invalidation relation remain - application-owned and non-configurable. -- Stages receive resolved Narratio-owned configuration and do not implement - import, profile, or family semantics independently. -- External tools continue to own their private runtime configuration and - defaults. -- Raw secrets remain indirect and are never introduced into composed YAML, - effective-configuration output, manifests, or diagnostics. - -Configuration composition belongs to `internal/config`. CLI selection and -production loading belong to `internal/app`. Party parsing must be a -Narratio-owned campaign-input contract. Artifact-family expansion must finish -before existing configured-artifact validation, catalog construction, -planning, reconciliation, or execution begins. - -## Configuration Imports - -### Root document - -Keep `pipeline.yml` as the selected pipeline entry point. Add an optional -top-level composition envelope with explicitly ordered imports: - -```yaml -composition: - imports: - - conf.d/platform.yml - - conf.d/transcripts.yml - - conf.d/extraction.yml - - conf.d/artifacts.yml - - conf.d/publish.yml - - default_profile: production - - profiles: - production: - overlay: profiles/production.yml - testing: - overlay: profiles/testing.yml -``` - -An existing monolithic `pipeline.yml` without `composition` remains valid and -retains its current behavior. The root document may contain ordinary pipeline -fields alongside `composition`, allowing incremental migration. - -`imports` is the public term. Imported files are partial pipeline -configuration documents rather than independently runnable pipelines. - -### Import boundaries - -- Only the root pipeline document may declare `composition`, imports, or - profiles. Imported and overlay documents cannot import other documents. -- Every import is explicit. Narratio does not automatically scan `conf.d` or - interpret unlisted files. -- Import paths are relative to the root pipeline file's directory, must remain - confined beneath that directory, and must resolve to supported regular YAML - files. Absolute paths, traversal, non-regular files, and cycles are rejected. -- All ordinary relative pipeline paths retain one stable base: the root - pipeline file's directory. Moving a field between imported files must not - silently change the meaning of its relative path. -- The declared import order is retained for diagnostics and provenance, but it - is not an implicit precedence mechanism between base imports. -- Composition metadata is removed before the effective mapping is decoded as - strict `PipelineConfig` data. - -### Additive merge contract - -Base configuration from the root document and its imports is additive: - -- mappings merge recursively when their child fields are disjoint; -- two files may therefore contribute different entries beneath a shared map - such as `scriptorium.artifacts`; -- defining the same final field or keyed entry in more than one base source is - an error, even when the values are identical; -- lists are atomic values and cannot be contributed more than once at the same - base configuration path; -- mapping/scalar, mapping/list, and other incompatible type collisions are - errors; and -- duplicate YAML keys within one source remain errors. - -Errors must identify the complete configuration path and every source file -that claims it. This contract prevents an import reorder or newly added file -from silently changing production behavior. - -Composition must operate on a presence-aware YAML representation before Go -struct decoding. Merging already decoded structs is not sufficient because it -cannot reliably distinguish omission from explicitly configured `false`, zero, -an empty collection, or another meaningful zero value. - -## Named Profiles - -### Selection and precedence - -Exactly one named profile may be active for an invocation. A composition root -may define `default_profile`; the maintained production-shaped configuration -uses `production`. Add `--profile ` to the common configuration flags for -every command that loads pipeline configuration: - -```bash -narratio run SESSION -narratio run SESSION --profile testing -narratio session plan SESSION --profile testing -narratio regenerate-artifacts SESSION --profile testing -``` - -The explicit CLI value overrides `default_profile`. An unknown profile, a -profile selection against a configuration that defines no such profile, an -empty profile name, or a default naming an undefined profile is an error before -stage composition. Do not add an environment-variable selector in this -feature; profile selection should remain visible in the command or root -configuration. - -Only one profile can be selected. Profiles cannot extend other profiles, and -callers cannot stack several profiles. A single overlay provides the required -production/testing variation without creating user-programmable precedence. - -### Overlay semantics - -The selected profile is the sole intentional override layer: - -- mappings recursively merge with the assembled base mapping; -- scalar and boolean leaves replace base leaves; -- lists replace the complete base list rather than concatenate implicitly; -- keyed maps such as configured artifacts merge by key, allowing a testing - profile to add experimental artifacts; -- an existing artifact can be disabled explicitly with `enabled: false`; -- an overlay cannot change a value's YAML kind incompatibly; and -- the fully overlaid result undergoes the same strict decode, defaults, path - resolution, expansion, and validation as a monolithic configuration. - -Generic deletion syntax is outside this feature. Shared definitions should -live in base imports, while production-only and testing-only definitions live -in their corresponding overlays. Existing enable/disable fields and complete -list replacement cover the required cases without a YAML patch language. - -### Effective identity and provenance - -Every loaded configuration must retain bounded, non-secret provenance: - -- the root pipeline path; -- the selected profile name, including whether it came from the default or - CLI; -- the ordered imported and overlay source paths; -- a deterministic digest of the normalized effective pipeline configuration; - and -- enough source ownership information to explain composition errors and - effective values. - -Run reporting and invocation state should identify the selected profile and -effective configuration digest. The profile name is provenance, not a blanket -cache key: changing profiles must invalidate only work whose result-affecting -effective configuration changed. - -## Canonical Campaign Party Contract - -### Ownership and schema - -Make `party.yml` a strict, versioned Narratio campaign-input contract. A -canonical file has this shape: - -```yaml -schema_version: narratio.party.v1 - -characters: - arannis: - player: - name: Eric - character: - name: Arannis - alias: - - Ari - - The Grey Owl - classes: - - name: wizard - level: 8 - - brenna: - player: - name: Jane - character: - name: Brenna - classes: - - name: paladin - level: 6 - - name: warlock - level: 2 -``` - -The contract has these semantics: - -- `schema_version` is required and must equal `narratio.party.v1`. -- `characters` is a non-empty mapping. -- Each mapping key is the stable `character_id` and must satisfy the existing - configured-artifact key grammar. -- `player.name` and `character.name` are required, non-empty display strings. -- `character.alias` is optional and accepts a list of zero or more non-empty - alias strings. The singular field spelling `alias` is intentional. -- Leading or trailing whitespace and control characters in names, aliases, and - class names are rejected rather than silently normalized. -- A character alias cannot equal that character's primary name, another alias, - or another character's primary name or alias under case-insensitive - comparison. This keeps party-member grounding unambiguous. Player names may - repeat because one player may control more than one character. -- `character.classes` is required and non-empty. Every entry has a required - free-form `name` and an optional positive integer `level`. -- Class names are not restricted to a Narratio-owned D&D enumeration. -- Duplicate class names for one character are rejected case-insensitively. -- Class order and alias order are preserved. -- Narratio derives a deterministic class summary such as - `paladin 6 / warlock 2` and an alias summary that joins declared aliases with - `, ` in their declared order. -- Unknown fields and trailing YAML documents are rejected. - -The stable `character_id`, rather than a display name, owns generated artifact -identity. Changing a display name, alias, class, or level retains that identity -and changes the relevant semantic inputs. Changing the mapping key is an -intentional remove-and-add operation. - -### Load and prepare behavior - -The selected campaign's effective `party_file` resolves relative to -`campaign.yml`. Narratio must load and validate canonical party data during -combined pipeline/campaign resolution so `session plan`, configuration -inspection, and artifact-family expansion see the same roster before stage -execution. - -`prepare` continues to materialize the canonical document as -`inputs/party.yml`, record its source and content identity, and expose it as -`narratio.input.party`. The same canonical file is passed unchanged to -Notarius's `party` reference slot, which currently accepts YAML reference -material without imposing a competing roster schema. - -### Players projection and compatibility - -The canonical party document becomes the eventual single source of truth for -player-to-character relationships. Narratio derives a deterministic, -documented players-only YAML projection during prepare, materializes it at -`inputs/players.yml`, records its content identity, and exposes it through the -existing `narratio.input.players` source. This preserves the separate Notarius -`players` reference and existing Scriptorium source without requiring every -campaign to maintain duplicate facts. - -The projection uses this versioned shape, with entries sorted by stable -character ID: - -```yaml -schema_version: narratio.players.v1 -players: - - name: Eric - character: - id: arannis - name: Arannis - alias: - - Ari - - The Grey Owl -``` - -There is one projection entry per character rather than one grouped entry per -display player name. This permits one player to control several characters and -does not conflate distinct players who happen to share a display name. - -Backward compatibility must be narrow and removable: - -- an unversioned legacy party file continues to be treated as opaque reference - material when the existing `players_file` is also configured; -- legacy mode preserves the current prepared `party` and `players` behavior - but cannot drive artifact-family expansion; -- configuring a party-driven artifact family with a legacy roster produces a - clear migration error; -- canonical `narratio.party.v1` mode derives players and rejects a separate - `players_file`, preventing contradictory authorities; -- parsing, validation, documentation, and tests for legacy mode live behind a - clearly identified compatibility boundary with comments stating that it is - intended for removal after migration; and -- the maintained examples migrate to canonical mode and demonstrate no - separate players file. - -The derived players projection is a durable integration contract and must be -documented under `docs/integrations/` rather than left as incidental generated -YAML. - -## Party-Driven Artifact Families - -### Configuration model - -Add `scriptorium.artifact_families`, separate from the existing concrete -`scriptorium.artifacts` map: - -```yaml -scriptorium: - artifact_families: - character_meta: - enabled: true - for_each: party.characters - prompt_id: dnd.character_meta - profile_id: production - output_path_pattern: artifacts/characters/{character_id}/meta.md - inputs: - transcript: - source: narratio.transcript.final_trimmed - required: true - member_vars: - player_name: player.name - character_name: character.name - character_class: character.class_summary - character_aliases: character.alias_summary - - character_items: - enabled: true - for_each: party.characters - prompt_id: dnd.character_items - profile_id: production - output_path_pattern: artifacts/characters/{character_id}/items.md - inputs: - transcript: - source: narratio.transcript.final_trimmed - required: true - item_occurrences: - source: narratio.extraction.item_occurrences - required: true - member_vars: - character_name: character.name -``` - -Families support the ordinary result-affecting fields shared by concrete -Scriptorium artifacts: enablement, prompt and profile IDs, timeout, -render-debug policy, ordinary dependencies, inputs, and static variables. -`output_path_pattern` replaces concrete `output_path` at the family level. - -`for_each` is not an expression language. This feature accepts only the exact -source `party.characters`. `member_vars` maps a Scriptorium variable name to -one of a closed set of canonical values: - -- `character_id`; -- `player.name`; -- `character.name`; -- `character.class_summary`; and -- `character.alias_summary`. - -Static `vars` and resolved member variables merge deterministically. Duplicate -variable names across the two maps are rejected rather than assigned implicit -precedence. Narratio's existing reserved sticky-session variable remains -application-owned and is applied after expansion under its current rules. - -### Narrow substitution - -`{character_id}` is the only family substitution supported in this feature. -It is required in `output_path_pattern` and may appear exactly where documented -for family-owned publish destinations. Unknown, repeated in an invalid -position, unresolved, or malformed substitutions are errors. Names, aliases, -classes, environment values, and arbitrary YAML paths cannot be interpolated -into configuration strings. - -### Concrete expansion - -For each family and party character, resolution creates one ordinary concrete -artifact key by joining the normalized family key, an underscore, and the -stable character ID: - -```text -character_meta_arannis -character_items_arannis -``` - -Expansion must be sorted, deterministic, and completed before existing -configured-artifact defaults and validation. Every generated artifact then -uses the existing catalog, dependency planner, fingerprinting, manifest, -reconciliation, execution, materialization, and publish boundaries. - -Validation rejects: - -- invalid family keys or character IDs; -- generated keys that fail the existing configured-artifact grammar; -- collisions between generated artifacts, explicit concrete artifacts, or - family names; -- duplicate or escaping output paths; -- a family with no canonical party source; -- invalid member-variable selectors; -- a selected profile that leaves an executable family incomplete; and -- any expanded artifact that fails ordinary concrete-artifact validation. - -An added party character creates missing concrete artifacts. Removing a -character removes those artifacts from effective configuration so existing -analyze reconciliation classifies their old records as removed. A change to a -member field changes only fingerprints that actually consume its resolved -value or the party document as an input. - -### Dependencies between families - -Ordinary `depends_on` continues to name shared concrete artifacts such as -`session_recap`. Add a typed `member_dependencies` list for corresponding -members of another family: - -```yaml -character_items: - member_dependencies: - - character_meta -``` - -For `arannis`, this expands to a dependency on -`character_meta_arannis`. The referenced family must use the same party source -and generate the same character ID. Cycles and missing members are rejected by -the existing concrete dependency validation after expansion. - -When a family input consumes the corresponding member output, provide a narrow -pre-expansion member-artifact source form owned by configuration resolution. -The syntax is: - -```yaml -inputs: - prior_meta: - source: narratio.member_artifact.character_meta - required: true -``` - -For the `arannis` member, this resolves to -`narratio.artifact.character_meta_arannis` before ordinary runtime validation. -The referenced family must also appear in `member_dependencies`. The -`narratio.member_artifact.*` form must not survive into stage or adapter -configuration as a new runtime artifact kind. - -### Selection and reporting - -Extend artifact selection so an exact family key selects all of that family's -concrete members. An exact generated key selects only that member. Explicitly -selecting a disabled family follows the existing rule for explicitly selected -disabled concrete artifacts: all generated targets must still have valid -executable fields. Mixed family and concrete selections are normalized and -deduplicated after expansion. - -Planning, run summaries, artifact listing, and effective-configuration output -must show both family origin and concrete artifact identity. Execution remains -entirely concrete; a family is not a dynamic pipeline stage or runtime loop. - -### Family publish rules - -Allow a family to declare one typed publish policy so operators need not repeat -one publish output per character: - -```yaml -publish: - enabled: true - required: false - dest_pattern: artifacts/characters/{character_id}/meta.md -``` - -Resolution expands this into ordinary concrete publish rules. When -`dest_pattern` is omitted, the generated artifact's output path is used where -the existing publish contract permits derivation. Conflicts with explicit -publish rules, duplicate destinations, unsafe paths, and generated sources -that are unavailable are rejected by the existing publish policy after -expansion. Family publish behavior must not introduce wildcard source matching -at runtime. - -## Profile-Safe Resume And Invalidation - -Named profiles are safe only if result-affecting model and configuration -changes cannot silently reuse incompatible successful stage results. Artifact -analysis already has a versioned semantic fingerprint that includes effective -prompt/profile identifiers, variables, dependencies, inputs, and output -identity. The broader pipeline requires the same principle at every applicable -stage boundary. - -Audit each stage's resume contract and add a versioned semantic configuration -fingerprint wherever current evidence does not already cover all -Narratio-observable result-affecting settings. In particular, switching an -Audita model through a profile must not reuse polish output produced by the -testing model. Conversely, changing only an operational timeout, binary path, -workspace path, diagnostic location, or profile name must not invalidate -byte-equivalent semantic work unless that value genuinely affects the stage's -canonical result. - -The selected profile name and complete effective-config digest are retained as -provenance. Resume decisions use stage-specific effective semantics rather -than the profile name as a blanket invalidator. This permits production and -testing profiles to share unaffected transcript work while correctly staling -changed stages and their fixed transitive dependents. - -Private configuration, prompt, model, executable, and module contents loaded -inside an external tool remain outside Narratio's observable fingerprint -boundary unless an existing integration contract exposes their identity. -Changing such private inputs continues to require explicit force, and the -documentation must not claim otherwise. - -## Configuration Inspection - -Add a `config` command family that performs composition without running a -session: - -```bash -narratio config validate [--config ] [--profile ] -narratio config show [--config ] [--profile ] -narratio config sources [--config ] [--profile ] -narratio config diff [--config ] -``` - -Required behavior: - -- `validate` composes imports and the selected profile, resolves campaign data - when required for party-driven families, expands artifacts and publish rules, - applies defaults, and runs strict validation without creating session or run - state. -- `show` emits deterministic normalized effective YAML, including generated - concrete artifacts, without raw credentials or environment values. -- `sources` reports root, import, profile, campaign, and party provenance and - identifies which source owns effective configuration paths. -- `diff` compares normalized effective configurations rather than raw files - and clearly identifies additions, removals, and value changes. -- output order is deterministic and suitable for review, but output formatting - is not a substitute for the effective configuration digest. - -If family expansion requires a campaign, these commands accept the existing -campaign selection mechanisms or report clearly that pipeline-only validation -cannot complete the party-dependent portion. They must not silently choose an -unintended campaign. - -## Required Application Changes - -The implemented feature will require coordinated changes across existing -owners rather than a parallel configuration subsystem: - -- `internal/config`: presence-aware YAML composition, import confinement, - profile overlays, provenance, effective digesting, party parsing and - validation, players projection, family expansion, defaults, and final strict - validation; -- `internal/app`: `--profile` plumbing through every configuration-consuming - command, `config` inspection commands, campaign-aware inspection, profile - reporting, and one shared loading path; -- `internal/artifactpolicy` and `internal/artifacts`: generated concrete source - registration, family-aware selection metadata, derived-player identity, and - reuse of the existing configured-artifact grammar and path policy; -- `internal/stage`: consumption only of expanded concrete artifacts, party and - derived-player preparation, and stage-specific semantic resume evidence; -- `internal/manifest`: bounded profile/effective-config provenance and any - versioned stage semantic fingerprints required for safe reuse; -- publish planning: expansion of family publish declarations into existing - concrete rules before ordinary validation and execution; -- Notarius composition: continue passing prepared `party` and `players` - sources, with canonical mode sourcing both from the party-owned contract; - and -- Scriptorium composition: receive only resolved concrete artifacts and - string/bool variables under the existing adapter contract. - -There must be one production configuration-loading path. Remote session -loading, restore, status, planning, helper commands, and execution must not -independently reimplement import, profile, campaign, or family selection. - -## Documentation And Examples - -When behavior is implemented, update the canonical owners in the same change: - -- [`docs/config.md`](../config.md): import, merge, profile, party selection, - family fields, defaults, and validation contracts; -- [`docs/cli.md`](../cli.md): `--profile` and `config` command syntax and output - behavior; -- [`docs/operations.md`](../operations.md): recommended production/testing - bundle layout, migration, inspection, and profile-switching workflow; -- [`docs/policy/architecture.md`](../policy/architecture.md): only the durable - composition, campaign-data ownership, and semantic-resume invariants; -- `docs/integrations/`: the canonical party and derived players YAML contracts; -- [`docs/internal/overview.md`](../internal/overview.md) and focused internal - configuration, manifest, analyze, prepare, and publish documents: implemented - ownership and mechanics; and -- `examples/`: a validated split configuration bundle, production/testing - overlays, canonical party roster with aliases and multiclass data, at least - two party-driven artifact families, and representative family publishing. - -Do not copy complete example bundles into reference prose. Maintained examples -must be secret-free and loaded by the repository's example validation test. - -## Testing Expectations - -Testing should protect the public contracts and high-risk composition and -state-reuse behavior without coupling to private merge helpers. Required -confidence includes: - -- monolithic configuration compatibility; -- recursive mapping composition and all duplicate/type/list conflict classes; -- import confinement, missing files, non-regular files, unsupported nesting, - source-aware diagnostics, deterministic ordering, and duplicate YAML keys; -- explicit zero-value profile overrides, list replacement, keyed-map additions, - unknown profiles, invalid defaults, and absence of profile stacking; -- canonical party schema validation, aliases, alias/name ambiguity, multiclass - summaries, stable IDs, malformed data, and legacy/canonical mode separation; -- deterministic players projection and prepared-input provenance; -- deterministic multi-family expansion, generated identities and paths, - variable binding, same-member dependencies, collisions, additions, - removals, and family selection; -- expansion into ordinary analyze planning, fingerprints, manifests, partial - selection, failure handling, and publish rules; -- model/profile changes staling only stages whose semantic configuration - changed while unaffected transcript work remains reusable; -- representative CLI validation, show, sources, diff, plan, and run behavior; - and -- maintained monolithic, imported, production, and testing examples loading - and validating offline. - -Use package-level behavioral tests through the narrowest stable owner. A small -number of assembled workflow tests should prove that composed profiles and -party-driven families reach the existing pipeline correctly; higher-level -tests should not repeat every parser and validation case. - -## Compatibility And Migration - -- Existing monolithic pipeline files remain supported. -- Omitting `--profile` retains existing behavior for configurations without - profiles and selects the declared production default for composed bundles. -- Existing explicit concrete Scriptorium artifacts and publish rules remain - valid and may coexist with non-colliding families. -- Legacy opaque party plus players files remain temporarily usable only through - the isolated compatibility mode and cannot power party-driven families. -- Migration consists of converting the campaign party document to - `narratio.party.v1`, removing the separate players file, and then enabling - party-driven families. -- Configuration inspection must make the migration result reviewable before a - session run. -- Compatibility code and documentation must clearly identify the eventual - removal boundary; new features must target canonical party mode rather than - extend legacy semantics. - -## Non-Goals And Deferred Work - -This feature does not introduce: - -- automatic loading of every file in `conf.d`; -- recursive imports, imports in profile overlays, or profile inheritance; -- multiple simultaneously active profiles; -- environment-selected profiles; -- generic YAML patch/delete operations; -- arbitrary string interpolation, expressions, scripting, or loops; -- artifact expansion from arbitrary files, Notarius outputs, or runtime data; -- configurable pipeline stages or a workflow DAG; -- runtime wildcard artifact or publish sources; -- automatic hashing of private external-tool configuration, prompts, modules, - models, or executables; or -- immediate removal of the isolated legacy party/player compatibility path. - -Named reusable parameter sets beyond the canonical party roster may be -considered later if a second non-party use case demonstrates a need. Additional -party-member types, campaign-domain fields, or family iteration sources should -be added only through a versioned schema decision rather than an untyped -extension map. - -## Target End State - -The feature is complete when: - -- operators can organize a pipeline as a root file plus explicit additive - imports with deterministic, source-aware conflict errors; -- one production-default or explicitly selected testing profile applies a - deliberate override without duplicating stable configuration; -- every command uses the same resolved profile and reports its identity; -- a campaign owns one strict, versioned party roster containing stable - character IDs, players, character names, optional alias lists, and one or - more classes; -- Narratio derives the existing players input from that roster and supplies the - canonical campaign context to Notarius; -- multiple Scriptorium artifact families expand once per campaign character - into ordinary concrete artifacts, dependencies, variables, output paths, and - optional publish rules; -- adding, changing, or removing a character produces deterministic and - manifest-correct artifact reconciliation; -- profile model changes cannot silently reuse semantically incompatible stage - outputs and do not unnecessarily invalidate unaffected transcript work; -- operators can validate, display, trace, and compare effective configuration - without executing a session; -- existing monolithic and concrete-artifact configurations remain compatible; - and -- current user, operator, integration, architecture, internal, and example - documentation accurately owns the implemented contracts. diff --git a/docs/roadmap/release-procedure.md b/docs/roadmap/release-procedure.md deleted file mode 100644 index c2304e1..0000000 --- a/docs/roadmap/release-procedure.md +++ /dev/null @@ -1,303 +0,0 @@ -# Release Procedure Upgrade - -## Status - -Implemented. This document records the accepted release-process scope, -maintainer policy, and delivered end state. - -## Goal - -Give Narratio one reproducible, guarded minor/patch release procedure that -validates a source candidate before publication, creates the existing binary -release inputs consistently, and makes the exact upstream tag push the clear -completion boundary. - -The procedure should borrow WeatherReporter's binary-publication model and -Notarius's centralized source-candidate validation and immutable-tag guards, -while retaining Narratio's own repository-wide checks and six supported build -targets. - -## Responsibility Boundary - -A Narratio release is complete when the selected tag has been successfully -pushed to its exact tag ref in the upstream Git repository. The release command -must then return successfully without polling Woodpecker, waiting for a -pipeline, querying a Gitea release, downloading assets, or otherwise making its -result depend on CI availability. - -The tag event may asynchronously cause Woodpecker to validate the tagged tree, -build binaries and checksums, and create a Gitea release. That work is outside -the synchronous release procedure. CI failure or delay does not change whether -the Git tag was successfully published, and release tooling must never wait -indefinitely for infrastructure outside the upstream Git push. - -An operator may inspect CI or published assets later. Such inspection is an -optional asynchronous follow-up, not a release completion condition and not a -step that an automated coding agent must perform after pushing the tag. - -## Release Model - -- Releases use unused stable semantic-version tags in the form - `vMAJOR.MINOR.PATCH`; prereleases are outside this feature. -- Narratio is past `v1.0.0`, so a minor release adds backward-compatible - functionality, a patch release contains backward-compatible fixes, and an - intentional breaking change to a documented public contract requires a new - major version. -- Release tags remain lightweight, matching Narratio's established tag - history, and are bound explicitly to a guarded commit on `main`. -- Every new release has a nonempty checked-in note at - `docs/releases/.md` in the tagged tree. -- Published tags are immutable. They are never moved, reused, overwritten, or - deleted, even when asynchronous CI fails to create a complete binary - release. -- The Gitea release, when asynchronous publication succeeds, remains the - canonical source for downloadable binaries and checksums. The Git tag and - checked-in note remain valid source-release history independently of CI. -- Existing historical tags and notes are not retroactively rewritten or - backfilled to satisfy the new procedure. - -## Canonical Documentation - -Add `docs/release.md` as the canonical maintainer procedure. Update -`docs/development.md` to route release preparation, tagging, and publication to -it, and update the documentation policy so it assigns ownership as follows: - -- `docs/release.md` owns version selection, candidate preparation, validation, - tag guards, tag publication, the responsibility boundary, and failure - recovery; -- `docs/releases/.md` owns the immutable historical summary for one - release; -- `docs/releases/README.md` owns the release-note index and points to the Gitea - release collection for downloadable assets; and -- current CLI, configuration, operation, integration, and internal contracts - remain in their existing canonical documents rather than being restated in - release notes. - -The maintainer procedure must be written as a complete safe workflow suitable -for either a human maintainer or a coding agent. Commands must be explicit, -non-interactive, and scoped to the selected version and tag ref. - -## Release Notes - -Require each future note to use this minimum structure: - -```markdown -# Narratio vMAJOR.MINOR.PATCH - -This release ... - -## Summary - -## Compatibility - -## Upgrade - -## Changes -``` - -The compatibility section must identify changes to documented CLI, -configuration, durable artifact, integration, or operating contracts. The -upgrade section must state required operator actions or explicitly say that no -special action is required. When the Notarius consumer contract changes, the -note should identify the supported/tested contract version and any required -Notarius upgrade. - -Release notes summarize a historical candidate and link to current canonical -documentation for exact behavior. They must not substitute for updating that -documentation, include credentials or private infrastructure details, or make -claims that have not been validated against the candidate. - -## Shared Release Scripts - -### Source-candidate checker - -Add the POSIX-shell checker: - -```text -scripts/check-release-candidate.sh vMAJOR.MINOR.PATCH -``` - -It must be deterministic, non-interactive, safe to run repeatedly, and usable -both by maintainers before tagging and by tag CI in a detached checkout. It -must not create or push Git tags, modify tracked source, contact CI, or require -live WhisperX, LLM, Notarius, Scriptorium, object-storage, or notification -services. - -The checker must fail clearly unless all of the following hold: - -- the argument is a stable semantic version; -- `go.mod` declares Narratio's expected module path; -- the matching release note exists, has the exact Narratio/version heading, - and contains the required sections; -- Go workspace use is disabled for candidate commands; -- tracked `go.work`/`go.work.sum`, vendoring, and `go.mod` replacement - directives are absent; -- uncached repository tests, race tests, vet, build, documentation checks, and - maintained-example validation succeed; -- `go mod tidy -diff` reports no module-file change; -- every tracked Go source file is `gofmt` clean; -- working-tree and cached whitespace checks succeed; and -- all supported release binaries cross-compile with the selected version - embedded, with the host-platform binary reporting exactly - `narratio ` through `narratio version`. - -The checker must preserve the distinction already documented by Narratio: -cross-compilation is compilation evidence, not native macOS or Windows runtime -evidence. - -### Asset builder - -Add `scripts/build-release-assets.sh`, used by both the candidate checker and -release CI. It must build exactly these CGO-disabled targets: - -- Linux `amd64` and `arm64`; -- macOS `amd64` and `arm64`; and -- Windows `amd64` and `arm64`. - -Asset names must remain deterministic and include the Narratio version, -operating system, architecture, and `.exe` suffix for Windows. Build output -must use a caller-supplied, narrowly validated staging directory and must not -perform broad or ambiguous deletion. Official assets use `-trimpath` and the -established release version linker override. - -### Tag-publication command - -Add the narrowly scoped release command: - -```text -scripts/release.sh vMAJOR.MINOR.PATCH -``` - -The command owns local validation, publication guards, lightweight tag -creation, local tag verification, and pushing only the selected tag ref. It -must not commit candidate changes or push `main`; the candidate must already be -committed and present on upstream `main`. - -Before creating a tag, the command must: - -- run the shared candidate checker; -- require the current branch to be `main`; -- require a clean worktree and index; -- fetch upstream `main` and tags; -- record the exact candidate commit and require local `HEAD` to equal - `origin/main`; -- require the version-matched release note from that commit; and -- reject an existing local or upstream tag of the selected name. - -It must create an explicitly lightweight tag against the recorded commit, -verify that the local tag resolves directly to that commit, and push only: - -```text -refs/tags/:refs/tags/ -``` - -It must never use `git push --tags`. If the tag push succeeds, the command must -report the version and commit and exit successfully immediately. In -particular, it must not invoke a CI client, poll a build status, wait for a -Gitea release, or verify release assets. - -If a failure occurs before the upstream tag is published, no release has -completed. An unpublished local tag may be removed only after the maintainer -has inspected the failure. The script should not silently delete it. A -successful upstream push is final even if later asynchronous work fails. - -## Woodpecker Release Workflow - -Retain tag-triggered asynchronous release automation, but make it consume the -same checked-in contracts as local preparation: - -- validate the tag syntax and matching release note before asset work; -- call the shared candidate checker rather than maintaining an independent - drifting validation list; -- use the shared asset builder for the six official binaries; -- pin the Go builder image to `golang:1.25.5`, an explicit patch version - compatible with the current `go.mod` language version; -- pin the release plugin to the reviewed explicit version - `woodpeckerci/plugin-release:0.3.1` rather than a floating image; -- set the Gitea release title explicitly to `Narratio `; -- use `docs/releases/.md` from the tagged tree as the release body; -- generate a flattened `SHA256SUMS` file for all official binaries; -- keep prerelease publication disabled and overwrite disabled; and -- keep validation, asset building, and publication dependency-ordered so a - failed CI validation cannot publish assets. - -The workflow must not require a callback to, acknowledgement from, or status -update in the release command. The two sides have intentionally separate -responsibilities: - -```text -local release command asynchronous infrastructure ---------------------- --------------------------- -validate candidate -guard main and exact commit -create lightweight tag -push exact upstream tag ────────► validate tagged tree -return success immediately build assets and checksums - create Gitea release -``` - -## Failure And Correction Policy - -Failures before tag publication are corrected on `main`, followed by complete -candidate revalidation and fresh guards. - -After a tag has been pushed, neither CI failure nor a defect permits changing -that tag. Correct the cause on `main`, choose a new patch version, write a new -release note, and repeat the complete release procedure. Do not move or delete -the published tag, manually overwrite generated assets, or make the release -command wait for repaired infrastructure. - -A maintainer may later inspect the remote tag, Gitea note, assets, checksums, -and host binary. Documentation may provide these as optional asynchronous -inspection commands, clearly outside the release responsibility boundary. No -automated release instruction may require them after a successful tag push. - -## Security And Operational Constraints - -- Local scripts never read or print the Gitea release token; the token remains - a Woodpecker-owned secret supplied only to the release plugin. -- Candidate validation remains offline with respect to paid APIs, mutable - application services, and private user data. -- Temporary build output is confined to a validated temporary or staging - directory and cleaned safely. -- Scripts avoid environment dumps and never record credentials in release - notes, logs, repository files, or command output. -- The release command does not edit tracked files, create commits, or push - branches. Apart from fetching refs for its guards, its only release-state - mutations are creating the selected local tag and pushing that exact tag. - -## Target End State - -When this feature is complete: - -1. A maintainer or coding agent prepares and commits a standardized release - note plus any final canonical-documentation updates. -2. The candidate is pushed to `main` through the ordinary development - workflow. -3. One shared checker proves the exact tagged source is clean, self-contained, - formatted, tested, race-tested, statically checked, documented, module - tidy, cross-buildable, and version-correct. -4. One guarded release command refuses ambiguous or unpublished candidates, - creates the established lightweight tag, and pushes only that tag. -5. The command completes successfully at the upstream tag push and performs no - CI or Gitea status wait. -6. Woodpecker independently reuses the shared checks and asset builder and may - publish six versioned binaries, release notes, and SHA-256 checksums. -7. Published tags remain immutable, and failures after tag publication are - corrected only through a new version. -8. The release procedure is documented in one canonical location, linked from - contributor guidance, enforced by scripts and CI, and clear about the - boundary between guaranteed Git publication and best-effort asynchronous - binary publication. - -## Non-Goals - -This feature does not: - -- repair, replace, monitor, or improve Woodpecker reliability; -- wait for or synchronously verify CI or Gitea release completion; -- introduce prereleases, signed tags, containers, package-manager publication, - installers, auto-updaters, or native macOS/Windows test runners; -- publish `main` automatically or create release-note content automatically; -- run live external adapters or paid model calls during release validation; or -- rewrite historical tags or manufacture retrospective release notes.