diff --git a/docs/roadmap/application-upgrade-sequence.md b/docs/roadmap/application-upgrade-sequence.md index 205a7ee..50a82b6 100644 --- a/docs/roadmap/application-upgrade-sequence.md +++ b/docs/roadmap/application-upgrade-sequence.md @@ -40,7 +40,7 @@ independent review and closeout. - [x] [Feature 0019: Audita adoption](features/0019-audita-v2-adoption.md) — complete through role `c3dd59f`; independent review found no remediation issues. - [x] [Feature 0020: Notarius adoption](features/0020-notarius-v0-8-adoption.md) — complete through role `6aef48b`; independent review found no remediation issues; suite unchanged. - [x] [Feature 0021: Narratio adoption](features/0021-narratio-v3-adoption.md) — complete through role `3c0700b`; independent review found no remediation issues; suite unchanged. -- [ ] Suite integration — feature roadmap not yet created. +- [ ] [Feature 0022: Coordinated suite integration](features/0022-coordinated-suite-upgrade.md) — roadmap, default-inheritance policy, and staged plan approved; ready for implementation. ## Adoption Boundaries diff --git a/docs/roadmap/features/0022-coordinated-suite-upgrade.md b/docs/roadmap/features/0022-coordinated-suite-upgrade.md new file mode 100644 index 0000000..7f04bd7 --- /dev/null +++ b/docs/roadmap/features/0022-coordinated-suite-upgrade.md @@ -0,0 +1,247 @@ +# Feature 0022: Integrate the Reviewed Application Upgrades + +Status: ready + +## Summary + +Bring the four reviewed standalone upgrades into the D&D pipeline role, +collection, and deployment consumer as one compatible configuration set. +This completes the agreed +[application upgrade sequence](../application-upgrade-sequence.md). + +## Problem or Opportunity + +The standalone roles now support the new releases, but the suite intentionally +retains its previous pins and configuration. Audita still receives v1 settings; +Narratio and Notarius testing selection still uses retired profile terminology. +Updating executable versions alone would break the deployed configuration. + +## Scope + +- Adopt the reviewed application/role versions together. +- Provide Audita v2 backend configuration and suite-owned PromptKit profiles. +- Migrate Narratio credential forwarding and Narratio/Notarius testing overlays. +- Preserve D&D processing, artifact composition, storage/publishing, shared + filesystem, and operator-access behavior. +- Update the collection snapshot, public dependency manifests, consumer pins, + operator guidance, and focused offline integration coverage. + +## Non-goals + +- Deploying to the operational controller or `pipeline-dnd` host, running real + transcripts, or calling inference, WhisperX, or object-storage services. +- New models, prompts, artifact families, extraction lanes, or processing policy + beyond migration choices expressly approved here. +- Automatic manifest conversion, cache deletion, forced stage reruns, broad + directory cleanup, compatibility aliases, or runtime version/integrity probes. +- Changing generic standalone-role ownership or adding production validation. +- A newly tagged collection release or replacement of published 0.1.0 assets. + This feature updates the public development compatibility set; formal release + packaging remains a separate decision. + +## Target End State + +### Coordinated Versions and Distribution + +| Application | Suite version | Reviewed standalone role revision | +| --- | --- | --- | +| Scriptorium | v0.15.0 | `b174c8c` | +| Audita | v2.0.1 | `c3dd59f` | +| Notarius | v0.8.1 | `6aef48b` | +| Narratio | v3.0.0 | `3c0700b` | +| Seriatim | v1.5.0, unchanged | existing v0.1.0 role pin | + +Dependency manifests use full immutable revisions resolved from these reviewed +commits. The pipeline role remains authoritative; the collection imports its +completed revision through the established public Gitea subtree workflow, never +by editing `roles/pipeline` directly. The collection continues bundling the +pipeline role and declaring standalone role dependencies separately. + +Relevant reviewed commits must be published to their existing public origins +before downstream manifests/subtree imports depend on them. Publication of the +required Git commits is part of the eventual implementation workflow, not this +roadmap-writing pass. Do not rewrite tags or publish unrelated changes. + +The consumer keeps its minimal playbook and host-vars organization. Its role +and collection pins select the same compatibility set. No controller install +or remote playbook execution is an acceptance gate. + +### Audita Backend and Profile Ownership + +The pipeline role supplies `audita_backends`, profile selection, and schema-2 +settings to the standalone Audita role. The pipeline owns the missing profile +assets under `/usr/local/etc/audita/profiles/`, using its runtime owner/group and +configuration modes. This directory contains PromptKit LLM profiles, not +ConfigKit overlays; it must not be renamed to `overlays`. + +Approved suite interface: + +- `dnd_pipeline_audita_backends`: native backend map, defaulting to one backend + named `dnd`, endpoint `https://inference.ai.rakestrawhome.com/v1`, credential + reference `AUDITA_LLM_API_KEY`, and shared concurrency `4`. Omit queue capacity + to inherit the native default. +- `dnd_pipeline_audita_profiles`: a native profile mapping keyed by file/profile + ID, with complete `dnd-proposal` and `dnd-validation` entries. Each selects + backend `dnd`, model `google/gemma-4-31b-it`, and per-attempt + `timeout_seconds: 900`. Other model controls inherit native behavior. +- `dnd_pipeline_audita_llm_proposal_profile` and + `dnd_pipeline_audita_llm_validation_profile` select those respective IDs. + +Render each supplied profile as one safely serialized native mapping with its +matching ID. Document profile-map keys as filename-safe IDs. Do not introduce +per-provider templates, profile merging, or discovery of external assets. +Callers can compose native maps in inventory and replace the supplied maps. +As with existing managed secrets, removing a map entry does not imply pruning +arbitrary files; only selected profiles participate in normal execution. + +Preserve the ordered module sequence, `audita-v1` output, chunking values, +thresholds, context description, and normalization limits. Convert normalization +duration defaults from `3s`, `3s`, and `240s` to numeric `3.0`, `3.0`, and +`240.0` seconds. Preserve meaningful setting names where native semantics are +unchanged; update argument types accordingly. + +Remove retired per-role endpoint/model/credential/concurrency/retry variables +in favor of the native backend/profile interface. Document their replacements; +do not infer a conversion from arbitrary old inventory. Remove diagnostics +retention; leave debug disabled through native defaults. Retain the existing +diagnostics work-directory choice where still supported. Reporting follows +Audita's native configuration; Narratio consumes its ordinary output contract. + +Production and testing continue using the same Audita endpoint and model. +No separate Audita testing overlay is needed for an identical configuration. + +### Credential Lists and Overlay Selection + +Replace the suite's singular Narratio Notarius credential variable with +`dnd_pipeline_narratio_notarius_credentials`, default `[OPENROUTER_API_KEY]`. +Add `dnd_pipeline_narratio_audita_credentials`, default `[AUDITA_LLM_API_KEY]`. +Pass both through the reviewed Narratio role's native list variables. These +lists contain names, never values, and can be replaced with `[]`. Operators +overriding Audita backends must explicitly list all referenced credentials; +the suite does not inspect configurations to discover names. + +Preserve `dnd_pipeline_secrets`, existing secret names, Vault references, +protected secret directory/files, and ordinary world-readable nonsecret +configuration. No credential rotation, secret relocation, or forwarding logic +is required in Ansible. + +Deploy the suite-owned testing files at: + +- `/usr/local/etc/narratio/overlays/testing.yml` +- `/usr/local/etc/notarius/overlays/testing.yml` + +Narratio testing sets `notarius.config_overlay: testing`; production omits the +selector. Notarius remains enabled in both modes. Its production pipeline +continues selecting `gpt-terra-latest`, testing selects `gpt-luna-latest`, and +the existing caller-overridable pipeline map/ID behavior remains intact. +Scriptorium testing still selects `gemini-flash-latest`; production continues +using the prompt assets' selections. Retain separate artifact/family fragments. + +Use Narratio `--overlay testing` and Notarius `--config-overlay testing` in +tests and operator examples. Deploy the new files declaratively; leave ignored +legacy `profiles/testing.yml` files alone and document that they are obsolete. +Do not delete `profiles/` directories or conflate them with PromptKit profiles. + +### Runtime Paths and Existing Suite Behavior + +Keep Narratio's explicit downstream binaries/config paths, output handoffs, +Notarius `/var/cache/notarius` roots and resume behavior, shared writable +directories/ACLs, service user, operator enrollment, and S3/publishing choices. +Audita profiles are configuration assets, not shared output directories. +No new runtime storage architecture is required. + +### Retry and Timeout Policy + +Retain the previously selected per-attempt Audita profile timeouts of 900 +seconds while inheriting the new applications' retry, +repair, and whole-operation defaults. Expose nullable suite pass-throughs for +`audita_llm_transient_retries`, `audita_llm_repair_attempts`, +`audita_llm_operation_timeout`, `notarius_promptkit_transient_retries`, and +`scriptorium_transient_retries`, prefixed with `dnd_pipeline_` in the public +suite interface. Null means omit; explicit zero is preserved where valid. + +Do not mechanically translate old `max_retries: 3` into both transport retries +and structural repairs. Preserve explicitly authored Notarius module retries; +they are a separate behavior. Keep Narratio stage timeouts unchanged. Explain +the nested budgets and the changed semantics, not a guarantee of identical +provider cost or timing. + +## Repository Ownership + +- `ansible-role-dnd-pipeline`: suite defaults, argument specs, dependency wiring, + Audita profile assets, testing overlays, documentation, and integration tests. +- `ansible-collection-dnd`: imported pipeline snapshot, dependency manifest, + collection tests, and compatibility guidance. +- `rakestraw-ansible`: matching dependency pins and deployment guidance; adjust + host variables only where required. Preserve Vault content and unrelated + inventory. Keep the hosts-and-roles playbook minimal. +- Standalone role repositories: reviewed commits are reused/published, not + redesigned. Application and campaign/prompt repositories are read-only + contracts/assets unless a concrete incompatibility is raised for discussion. +- Workspace: roadmap, later staged plan, decisions, progress, and review. + +## Constraints and Settled Decisions + +Follow the [Ansible](../../ansible.md), [testing](../../testing.md), and +[planning](../../planning.md) policies, [MVP roadmap](../mvp.md), and +[upgrade sequence](../application-upgrade-sequence.md). New D&D policy remains +in the pipeline role; optional generic application defaults remain null. + +## Simplest Viable Implementation + +Update dependency variables/pins and existing templates, add an ordinary +Audita profile directory task and one looped template deployment, and change +testing destinations to `overlays/`. Reuse existing declarative convergence, +secret delivery, and collection assembly. No production validation, state +fingerprints, migration framework, or generated credential inference is needed. + +## Acceptance Criteria + +- Pipeline and collection installations produce a compatible set of the four + upgraded applications while Seriatim remains unchanged. +- Native offline inspection/validation accepts Audita settings and selected + profiles, Notarius base/testing configurations, Scriptorium configuration, + and Narratio base/testing composition with synthetic credential inputs. + Direct Audita validation receives credential environment variables explicitly: + it does not bootstrap Narratio's secrets directory on its own. +- Focused cases prove caller-supplied backend/profile configuration and + credential lists, testing selection, and existing important composition and + path handoffs. Assert supplied values, not fixed copies of today's defaults. +- Immediate repeated convergence is clean. Existing meaningful shared-user, + cache, artifact-composition, and installed-collection behavior remains covered + without expanding into upstream module/runtime regression matrices. +- Collection build/install and consumer syntax checks use the established + public dependency manifests. No sibling application/role checkout is needed + by an external consumer. Required referenced commits exist on public Gitea. +- Consumer inventory retains comments/Vault references for unknown secrets; + no actual keys or operational deployments are required for completion. +- Guidance covers breaking inventory fields, new CLI selectors, obsolete + testing-file locations, and upstream manifest migration. Back up and migrate + local provenance without deleting progress or editing immutable snapshots; + document normal named-selector invalidation rather than forcing reruns. + +## Open Questions + +None. The user approved deferring to application defaults wherever possible, +with nullable inventory overrides, and confirmed retaining the existing +900-second Audita per-attempt policy. Do not freeze native retry, repair, or +operation-deadline defaults in suite variables. Operators should note that an +attempt may consume the entire logical-operation budget; these are separate +controls, not additive timeout guarantees. + +## Dependencies and Follow-up Work + +Features 0018–0021 provide the reviewed standalone revisions. This roadmap is +approved; execution is defined in the [implementation plan](../implementation.md). +Final review precedes feature closeout. Actual deployment, +session migration, live processing, and a formal tagged release are separate +operator actions, not completion gates for this feature. + +## Upstream References + +- [Audita v2 migration](https://gitea.maximumdirect.net/eric/audita/src/tag/v2.0.1/docs/migrations/v2.md) +- [Audita configuration](https://gitea.maximumdirect.net/eric/audita/src/tag/v2.0.1/docs/config.md) +- [Narratio v3 release](https://gitea.maximumdirect.net/eric/narratio/src/tag/v3.0.0/docs/releases/v3.0.0.md) +- [Narratio operations and migration](https://gitea.maximumdirect.net/eric/narratio/src/tag/v3.0.0/docs/operations.md) +- [Notarius v0.8.0 migration](https://gitea.maximumdirect.net/eric/notarius/src/tag/v0.8.0/docs/releases/v0.8.0.md) +- [Scriptorium v0.14.0 migration](https://gitea.maximumdirect.net/eric/scriptorium/src/tag/v0.14.0/docs/releases/v0.14.0.md) diff --git a/docs/roadmap/implementation.md b/docs/roadmap/implementation.md index 8795d38..6c90062 100644 --- a/docs/roadmap/implementation.md +++ b/docs/roadmap/implementation.md @@ -1,12 +1,387 @@ -# Implementation Plan +# Implementation Plan 0022: Coordinated Suite Upgrade -No implementation plan is currently active. Agree on the next feature roadmap -before preparing its staged plan. +Status: ready -Feature 0021 is complete and independently reviewed with no remediation -findings. See [its roadmap](features/0021-narratio-v3-adoption.md) for completion -evidence. Git history preserves the completed plan. +Target executor: GPT-5.6-Terra -Coordinated suite integration is next in the agreed -[application upgrade sequence](application-upgrade-sequence.md). Suite and -consumer pins remain unchanged until that integration feature. +Feature roadmap: [Feature 0022](features/0022-coordinated-suite-upgrade.md) + +## Objective + +Integrate the reviewed application upgrades into the pipeline role, assembled +collection, and consumer repository. The approved roadmap owns scope, target +state, and policy. No operational deployment or tagged release is included. + +## Execution Rules and Starting State + +- Execute Stage 1 through Stage 6 in order. Before each stage, reread its + listed policies, check applicable repository instructions, and confirm its + owning repository's working-tree state. Preserve unrelated changes. +- Commit valid completed stages with the stated plain English intent. Do not + advance past failed validation. Distinguish actual blockers from unavailable + optional tools or operational credentials that this plan does not require. +- Use workspace `.venv/bin` Ansible tooling. Missing PATH entries are not a + missing controller. Existing integration scripts choose local tooling and + install dependencies into temporary paths. +- Pipeline/collection integration runs use local sudo and mutate the local + development installation, including source/binaries, configuration, fixture + secrets, and test users. Read harness cleanup and verify these locations are + disposable first. Do not run operational playbooks or contact live inference, + WhisperX, or S3 services. Public Git/dependency downloads are expected. +- Use ordinary declarative modules; no production validation, preliminary + state probes, compatibility aliases, manifest conversion, pruning, or new + credential-discovery mechanism. New tasks are limited to Audita profile + directory creation and looped profile templating. +- Publish only the required reviewed role revisions and feature commits to + existing origins as specified below. Check remote URLs and outgoing commits + before each push. Never force-push or move tags. If unrelated commits would + be published or credentials prevent publication, report that boundary; do + not substitute local paths in consumer manifests or subtree imports. +- Do not edit standalone application-role implementations, upstream apps, + campaign/prompt assets, unrelated consumer inventory, Vault content, or + vendored consumer roles. Required code owners are pipeline, collection, and + consumer; workspace owns progress. Keep Seriatim and native collection pins. +- The new retry/repair/deadline pass-throughs default to null. Preserve the + existing Audita 900-second per-attempt profile policy. Do not copy native + defaults into suite variables. + +## Stage 1: Make Reviewed Standalone Dependencies Public + +### Required Policy Review + +Read `AGENTS.md`, `docs/orientation.md`, `docs/conventions.md`, and +`docs/planning.md` before starting. + +### Owning Repositories and Preconditions + +The four standalone role repositories and workspace. Features 0018–0021 are +complete. Confirm these exact revisions and their existing Gitea origins: + +| Role | Revision | +| --- | --- | +| Scriptorium | `b174c8c2990bb79f8149d8f288c5e06ccb42bd73` | +| Audita | `c3dd59f5ac9b942306ffe540ae9e71d493998137` | +| Notarius | `6aef48b7e12e3651971444b8c224d96ff1879e6c` | +| Narratio | `3c0700b5323293d44e3b1e67a50074d82d7a3bff` | + +### Work and Validation + +Mark F0022 `in_progress`. For each repository, fetch origin, inspect its status +and outgoing history, and publish the reviewed revision to `origin/main` if +not already reachable there. Use a normal non-force push of the exact revision +to `refs/heads/main` where a fast-forward is appropriate. A remote already +containing it needs no push. Verify reachability after fetch; do not replace a +selected pin merely because a newer remote commit exists. + +Commands, substituting the role and exact revision from the table: + +```sh +git -C ROLE remote -v +git -C ROLE fetch origin +git -C ROLE log --oneline origin/main..REVISION +git -C ROLE push origin REVISION:refs/heads/main +git -C ROLE fetch origin +git -C ROLE merge-base --is-ancestor REVISION origin/main +``` + +### Completion and Commit + +All four pinned revisions are publicly available. No standalone code change or +new role commit is needed. Record evidence and commit workspace progress as +`Record the coordinated suite upgrade in progress`. + +## Stage 2: Migrate Suite Configuration and Existing Integration Cases + +### Required Policy Review + +Read `AGENTS.md`, `docs/orientation.md`, `docs/conventions.md`, +`docs/ansible.md`, and `docs/testing.md` before starting. + +### Owning Repository and Preconditions + +`ansible-role-dnd-pipeline`. Stage 1 is complete. Inspect current defaults, +specs, dependency vars, tasks, templates, and tests; read tagged Audita v2.0.1 +migration/configuration and Narratio v3.0.0 contracts linked by the roadmap. + +### Required Changes + +1. Pin the four standalone roles in `requirements.yml` to Stage 1 revisions. + Set application defaults to Scriptorium v0.15.0, Audita v2.0.1, Notarius + v0.8.1, Narratio v3.0.0. Preserve Seriatim v1.5.0 and its role pin. +2. Add `dnd_pipeline_audita_backends` (dict), default `dnd` with endpoint + `https://inference.ai.rakestrawhome.com/v1`, credential `AUDITA_LLM_API_KEY`, + concurrency `4`; omit queue capacity. Add `dnd_pipeline_audita_profiles` + (dict) with `dnd-proposal` and `dnd-validation` keys. Each value is a complete + native mapping containing matching `id`, backend `dnd`, model + `google/gemma-4-31b-it`, and `timeout_seconds: 900`. +3. Add string defaults `dnd_pipeline_audita_llm_proposal_profile: dnd-proposal` + and `dnd_pipeline_audita_llm_validation_profile: dnd-validation`. Map these + and backends into the standalone role. Set `audita_profiles_dir` to + `/usr/local/etc/audita/profiles`. +4. Add a `file` directory task and one looped `template` task over + `dnd_pipeline_audita_profiles | dict2items`. Write `profiles/.yml` using + an `audita-profile.yml.j2` template containing `item.value | to_nice_yaml`. + Use runtime owner/group, suite configuration modes, and task-level become. + Document that keys are filename-safe IDs matching each mapping's `id`. + Do not merge/rewrite profile values or delete removed entries. +5. Remove old `dnd_pipeline_audita_llm_{proposal,validation}_{base_url,model, + api_key_env,timeout,max_retries}`, `dnd_pipeline_audita_concurrency_*`, and + diagnostics retention from defaults, specs, dependency wiring, and active + examples. Preserve diagnostics work directory and unchanged domain tuning. + Convert normalization max gap/ellipsis gap/max duration to numeric + `3.0`/`3.0`/`240.0` with float specs. Leave debug/reporting at native defaults. +6. Add these optional nullable suite variables, forwarding to the corresponding + standalone variable by removing `dnd_pipeline_`: + - `dnd_pipeline_audita_llm_transient_retries`: int. + - `dnd_pipeline_audita_llm_repair_attempts`: int. + - `dnd_pipeline_audita_llm_operation_timeout`: str. + - `dnd_pipeline_notarius_promptkit_transient_retries`: int. + - `dnd_pipeline_scriptorium_transient_retries`: int. + Keep nulls and explicit zero; do not supply a fallback native default. +7. Add list-of-string credential defaults + `dnd_pipeline_narratio_audita_credentials: [AUDITA_LLM_API_KEY]` and + `dnd_pipeline_narratio_notarius_credentials: [OPENROUTER_API_KEY]`; wire to + Narratio's credential lists. Remove the old suite Notarius singular-key + variable and obsolete standalone-role arguments. Preserve secrets delivery. +8. Change Narratio/Notarius directory creation and testing-file destinations + from `profiles` to `overlays`. Change Narratio testing to + `notarius.config_overlay: testing`. Leave old ignored files alone. Preserve + Audita's LLM `profiles/`, artifact fragments, model selections, caches, + paths, publication, ACLs, and operator behavior. +9. Migrate existing `tests/integration` selectors and parsed assertions: + Narratio `--overlay`, Notarius `--config-overlay`, `config_overlay`, and + credential lists replace retired fields. Direct Audita validation receives + the harness's synthetic `AUDITA_LLM_API_KEY` explicitly in its environment; + do not assume Narratio's bootstrap applies to that process. Add Scriptorium + native config validation. Keep old meaningful scenarios, clean repeats, + and fixture cleanup; do not test upstream forwarding internals. +10. Update README public variables and migration mapping in the same commit + so the role's current interface is documented. Explain defaults versus + inventory overrides and retain existing native configuration ownership. + +### Validation + +From workspace root: + +```sh +bash -n ansible-role-dnd-pipeline/tests/integration +ansible-role-dnd-pipeline/tests/integration +git -C ansible-role-dnd-pipeline diff --check +``` + +The harness installs public pinned dependencies and includes syntax/lint. +Require native acceptance of base/testing configuration and existing meaningful +behavior with clean repeats, without live service calls. If an unrelated asset +contract incompatibility appears, report it rather than rewriting assets. + +### Completion and Commit + +Suite configuration, specs, docs, and existing tests are compatible together. +Commit as `Integrate the reviewed application upgrades into the suite`. + +## Stage 3: Verify Inventory Overrides and Finish Suite Migration Guidance + +### Required Policy Review + +Read `AGENTS.md`, `docs/orientation.md`, `docs/conventions.md`, +`docs/ansible.md`, and `docs/testing.md` before starting. + +### Owning Repository and Preconditions + +`ansible-role-dnd-pipeline`. Stage 2 passes. Extend its existing harness rather +than introducing a test framework or changing production convergence. + +### Required Changes + +1. Add one representative override case with a fixture backend endpoint, + caller-composed native profile map, selected proposal/validation IDs, and + an explicit custom credential-name list. Use synthetic values only; provide + required names both through managed fixture secrets for Narratio and direct + test environment for Audita. Use simple offline-valid profile values. +2. In that case supply zero for the four retry/repair integers and a valid + explicit Audita operation timeout, such as `20m`. Inspect native effective + settings/profile views to verify supplied values and selected models. Inspect + Narratio configuration to verify the credential names. Do not assert current + application default retry counts or model catalogs. +3. Explicitly null the five new pass-throughs in a baseline case and inspect + rendered application YAML for omission (effective views may include native + defaults). Require an immediate unchanged repeat for the override case. + Retain existing pipeline-map composition, testing selection, shared-user, + cache, storage, and artifact tests; do not add module metadata checks. +4. Document exact old-to-new inventory choices, shared backend concurrency, + filename/profile ID contract, credential list ownership, and CLI changes. + Explain the retained 900-second attempt setting and independent native + operation deadline. Link upstream manifest migration; do not automate it. + Explain ignored legacy testing files and normal named-selector invalidation. + +### Validation + +```sh +bash -n ansible-role-dnd-pipeline/tests/integration +ansible-role-dnd-pipeline/tests/integration +git -C ansible-role-dnd-pipeline diff --check +``` + +All native checks and immediate repeats pass. Review the final pipeline diff +against roadmap scope, including no live-provider test or new production probe. + +### Completion and Commit + +Commit as `Verify suite configuration overrides and document migration`. +Record the final pipeline commit as the assembly revision. Inspect outgoing +commits and publish that revision to the pipeline's existing `origin/main` +with a normal push; verify remote reachability before Stage 4. + +## Stage 4: Assemble and Validate the Collection + +### Required Policy Review + +Read `AGENTS.md`, `docs/orientation.md`, `docs/conventions.md`, +`docs/ansible.md`, and `docs/testing.md` before starting. + +### Owning Repository and Preconditions + +`ansible-collection-dnd`. The validated Stage 3 pipeline commit is public. +Read the collection README's subtree and dependency workflow. + +### Required Changes + +1. Import that exact revision using the authoritative public Gitea origin: + + ```sh + git -C ansible-collection-dnd subtree pull --prefix=roles/pipeline \ + https://gitea.maximumdirect.net/eric/ansible-role-dnd-pipeline.git \ + PIPELINE_REVISION --squash + ``` + + Substitute the Stage 3 hash; never a local sibling path. Subtree-generated + commits are expected. Do not edit imported role files directly. +2. Match the standalone revisions in root `requirements.yml` to Stage 1. + Keep collection metadata version 0.1.0 and published release instructions + intact; describe the new development snapshot separately. +3. Migrate collection `tests/integration` CLI/field assertions and direct Audita + synthetic environment handling as in Stage 2. Validate Audita profiles and + Scriptorium configuration natively. Keep collection tests focused on the + installed artifact and dependency boundary, base/testing selection, and + clean repeat; do not duplicate Stage 3's complete override matrix. +4. Update compatibility guidance and reference the pipeline migration notes. + A fresh public dependency install must work without sibling role repos. + Existing local campaign/prompt fixture overrides in the test harness remain + acceptable; production manifests must not acquire those local paths. + +### Validation + +```sh +bash -n ansible-collection-dnd/tests/integration +ansible-collection-dnd/tests/integration +git -C ansible-collection-dnd diff --check +``` + +Require successful build, disposable artifact install, syntax/lint, native +configuration checks, and immediate clean repeat using public role dependencies. + +### Completion and Commit + +Commit remaining changes as `Assemble the upgraded D&D suite` after the subtree +import commits. Inspect and publish feature commits to `origin/main`; record +and verify the final public collection hash for Stage 5. No tags or release +assets are created or replaced. + +## Stage 5: Update the Consumer Without Deploying + +### Required Policy Review + +Read `AGENTS.md`, `docs/orientation.md`, `docs/conventions.md`, +`docs/ansible.md`, and `docs/testing.md` before starting. + +### Owning Repository and Preconditions + +`rakestraw-ansible`. Stage 4's collection and all referenced dependencies are +public. Read applicable consumer instructions and `docs/pipeline-dnd.md`. + +### Required Changes + +1. Update only the relevant role pins in `requirements.yml` to Stage 1 and its + D&D collection pin to Stage 4. Preserve unrelated requirements. +2. Keep `playbooks/pipeline/dnd.yml` minimal. Existing host-vars credential + names remain valid; preserve encrypted Vault content and all unrelated + values. Do not add copied suite defaults to inventory. If an obsolete + feature-owned setting is present, replace only that setting and document it. +3. Update `docs/pipeline-dnd.md` to the coordinated versions, exact collection + refresh hash, new testing invocation, and migration prerequisites. Preserve + the explicit refresh guidance for an equal-version development collection. + Remove misleading claims that the v3 upgrade needs no configuration or + manifest migration. Distinguish future operator commands from checks run. + +### Validation + +Use the workspace tools and disposable role/collection paths. From consumer +root, set `ANSIBLE_ROLES_PATH` and `ANSIBLE_COLLECTIONS_PATH` to directories +under a new `mktemp -d` path, then run: + +```sh +/home/eric/Workspace/ansible-dnd/.venv/bin/ansible-galaxy install -r requirements.yml +/home/eric/Workspace/ansible-dnd/.venv/bin/ansible-playbook -i inventory playbooks/pipeline/dnd.yml --list-hosts --limit pipeline-dnd +/home/eric/Workspace/ansible-dnd/.venv/bin/ansible-playbook -i inventory playbooks/pipeline/dnd.yml --syntax-check --limit pipeline-dnd +git diff --check +``` + +These commands must not connect to the target or execute role tasks. If loading +encrypted inventory requires unavailable Vault credentials, validate syntax +with a temporary inventory containing only `pipeline-dnd`, a synthetic WhisperX +URL, and synthetic required inputs instead. Inspect the real playbook target +and inventory entry read-only; report this limited check accurately. Do not +decrypt Vault, require real keys, add ansible-doc gates, or execute deployment. + +### Completion and Commit + +Public dependencies resolve and consumer syntax/target checks pass (or the +documented synthetic-inventory fallback is used). Commit as +`Prepare the consumer for the coordinated suite upgrade`. Leave this consumer +commit local unless separately asked to push; it is not a downstream dependency. + +## Stage 6: Record Evidence and Hand Off for Independent Review + +### Required Policy Review + +Read `AGENTS.md`, `docs/orientation.md`, `docs/conventions.md`, +`docs/ansible.md`, `docs/testing.md`, and `docs/planning.md` before starting. + +### Owning Repository and Preconditions + +Workspace. Stages 1–5 are complete and committed. No new code or deployment +work belongs in this stage. + +### Required Changes and Validation + +Record final hashes, public availability, commands/results, environment limits, +and any plan deviations in the feature roadmap and concise progress notes. +Update `docs/status.md` and the upgrade sequence to implementation complete, +review pending, using `in_progress` until independent review. Review that pins +across pipeline, collection, and consumer agree, and no standalone role or +application/asset code changed outside scope. + +```sh +git diff --check +scripts/status +``` + +Reuse earlier integration evidence for documentation-only changes. Report +which commits were pushed and which remain local; a clean working tree does +not imply a synchronized remote. Keep deployment, session migration, and formal +release work explicitly deferred. + +### Completion and Commit + +Commit as `Record coordinated suite upgrade validation`. All changed working +trees are clean, and implementation is ready for independent review. Do not +mark F0022 done, begin another feature, or push unrelated workspace history. + +## Progress Notes + +Plan approved; implementation has not started. + +## Open Questions + +None. The user approved application-owned retry/repair/deadline defaults with +nullable inventory overrides and the retained Audita per-attempt policy. diff --git a/docs/status.md b/docs/status.md index 0b76899..d0dd4d7 100644 --- a/docs/status.md +++ b/docs/status.md @@ -4,8 +4,8 @@ Last updated: 2026-09-29 Current milestone: Operational D&D pipeline deployment -Current focus: Plan coordinated suite integration of the reviewed application -upgrades. The suite remains unchanged until that feature. +Current focus: Implement Feature 0022's coordinated suite upgrade. The roadmap, +default-inheritance policy, and staged plan are approved; implementation has not started. Features 0018 through 0021 are complete after independent review with no remediation findings. Follow the agreed [application upgrade sequence](roadmap/application-upgrade-sequence.md): @@ -45,6 +45,7 @@ is complete. | F0019 | Adopt Audita v2.0.1 | Audita role and workspace | done | Role `c3dd59f`, following `6137ff2` and `52f67fe`, passed independent review with no findings; [roadmap](roadmap/features/0019-audita-v2-adoption.md) records validation; suite pins unchanged | | F0020 | Adopt Notarius v0.8.1 | Notarius role and workspace | done | Role `6aef48b`, following `b1b922c` and `380b6a3`, passed independent review with no findings; [feature roadmap](roadmap/features/0020-notarius-v0-8-adoption.md) records evidence; suite pins unchanged | | F0021 | Adopt Narratio v3.0.0 | Narratio role and workspace | done | Role `3c0700b`, following `a28f22c` and `45f6836`, passed independent review with no findings; [feature roadmap](roadmap/features/0021-narratio-v3-adoption.md) records evidence; suite pins unchanged | +| F0022 | Integrate the reviewed application upgrades | Pipeline role, collection, consumer, and workspace | ready | [Feature roadmap](roadmap/features/0022-coordinated-suite-upgrade.md) and [staged plan](roadmap/implementation.md) approved; ready for implementation | | A0001 | Audit completed standalone-role test suites | workspace and standalone roles | done | Findings recorded in `docs/roadmap/test-suite-audit-findings.md` | | R0001 | Simplify completed standalone-role test suites | workspace and standalone roles | done | Role commits `e08c63c`, `d2d0b9d`, and `cfecf37` passed implementation and independent review |