# Implementation Plan 0016: Deploy the D&D Pipeline to `pipeline-dnd` Status: in_progress Target executor: GPT-5.6-Terra Feature roadmap: `docs/roadmap/features/0016-deploy-dnd-pipeline-to-pipeline-dnd.md` Active plan location: `docs/roadmap/implementation.md` ## Objective Deploy the reviewed collection through a minimal consumer playbook, host inventory, and Ansible Vault. Demonstrate clean repeated convergence and one operator-selected session through real publication. The feature roadmap owns scope, rationale, target state, and acceptance criteria. ## Execution Rules - Execute stages in order. Before each stage, reread its Required Policy Review and check the owning repository's working tree. Preserve unrelated changes. Validate and commit each completed stage with its stated intent. - Keep the playbook limited to hosts and roles. Use existing interfaces; add no wrapper, preflight framework, custom module, or test harness. - Inherit suite storage, publication, caching, cleanup, version, and asset defaults. Set only the roadmap-selected host choices. - Never print decrypted inventory, credential values, or Vault passwords in commands, logs, documentation, or Git. Do not use decrypted inventory dumps as validation evidence. The operator supplies secrets directly through Ansible Vault's encrypted-file workflow, using an established secure editor. Do not solicit secrets in chat or create placeholder credentials. - Stop dependent work when operational inputs are missing or validation fails. Record the concrete blocker; local checks do not prove a live deployment. - Scope blockers to the operation that requires the missing input. Complete Stages 1–2 locally without remote SSH, production Vault values, or an acceptance session. Controller/SSH/Vault inputs gate Stage 3; session and audio selection gate Stage 4. Continue independent authorized preparation. - A missing executable on `PATH` is not proof that tooling is uninstalled. Check the workspace `.venv/bin` and repository test drivers before reporting a missing dependency. Use the verified executable path in diagnostics. - An Ansible controller is the environment running Ansible; it requires no controller daemon or `/etc/ansible` directory. Use explicit configuration, inventory, role, and collection paths for local checks. Do not recreate the operational controller layout or install system-wide tooling for that purpose. - Retain the historical playbook and proof-of-concept role. Role, collection, application, and asset repositories remain read-only during this feature. - Deployment and paid provider execution occur only during the approved sprint with the selected controller, target, credentials, and session. Do not substitute an arbitrary session or broaden the host limit. ## Starting State and Preconditions - Feature 0015 passed independent review without findings. Its pipeline commit is `51566afcf69083cdbb5b5718f2544c86b079001c`; the reviewed public collection commit is `46913446765e86ecfb29ebad9955d579ef6a4c7e`. - `rakestraw-ansible` starts clean on `main` at `d86ec29a9cbd821fead248736171b5cdd8440ad6`. Inspect and preserve any subsequent changes before adapting this plan. - Root `requirements.yml` already has an unpinned Gitea `rakestraw.dnd` entry alongside unrelated collections. The historical playbook is `playbooks/apps/dnd.yml`; it invokes `pipeline_dnd` on `automation-dnd`. - Host `pipeline-dnd` already connects as `eric` to `pipeline-dnd.rakestrawhome.com`. Preserve its connection settings. - `inventory/host_vars/pipeline-dnd/narratio.yml` contains the legacy campaign and S3 variables to migrate. - The selected collection still declares version `0.1.0`. An installed release with that number is not sufficient evidence of the selected content. - This plan is approved, with this revision permitting local preparation before operational inputs are resolved. Do not invent those inputs; resolve them at the stage that needs them. - Local tooling is already installed and verified: Ansible Core 2.21.3 in `/home/eric/Workspace/ansible-dnd/.venv`. Use its `bin/ansible`, `bin/ansible-playbook`, `bin/ansible-galaxy`, and `bin/ansible-doc`, or prepend that `bin` directory to `PATH`. No activation is required for absolute paths. ## Repository Scope - `rakestraw-ansible`: requirements, playbook, host configuration, encrypted credentials, and `docs/pipeline-dnd.md` runbook linked from README. - Workspace: feature status, implementation progress, and review handoff. - Other repositories: read-only compatibility and application references. ## Stage 1: Establish Local Preparation and Record Deployment Inputs Completion: complete (consumer `716a80ef68208537dacfa5f0f22259bf42ae6279`) ### Objective and Owning Repository Record known and pending deployment inputs in `rakestraw-ansible` and close the reviewed Feature 0015 in the workspace. ### Required Policy Review Read `AGENTS.md`, `docs/orientation.md`, `docs/conventions.md`, `docs/ansible.md`, `docs/testing.md`, `docs/planning.md`, the Feature 0016 roadmap, and `rakestraw-ansible/README.md` before this stage. ### Preconditions The plan is approved and repositories have been checked. Local tooling is available; no remote access or credential/session input is required. ### Required Changes 1. Create `rakestraw-ansible/docs/pipeline-dnd.md` and link it from README. Record the local validation setup and known operational details. List the operational controller/checkout, effective paths, Vault selection, and acceptance session as pending where unknown, with their consuming stage. Do not fabricate completed checks or turn missing values into blockers here. 2. Confirm local Ansible Core is at least 2.18 and the public suite repositories and Galaxy are reachable. Inspect effective configuration without dumping decrypted host variables. Existing repository defaults are `/etc/ansible/inventory`, `/etc/ansible/roles`, and `/etc/ansible/collections`; document these as operational defaults, not required local directories. 3. Record the reported target SSH rejection as a Stage 3 prerequisite. Defer target Ubuntu and sudo checks until authorized access is available. Repeated attempts with the same rejected identity do not advance local preparation. 4. Mark Feature 0015 `done` in its roadmap and workspace status, citing its reviewed commits and the successful independent standalone/collection integration runs. Leave historical remediation records unchanged. Mark Feature 0016 `in_progress` when execution begins. ### Validation and Completion Criteria Local tooling works and the runbook distinguishes known facts from pending operational inputs, assigning each to Stage 3 or Stage 4. Run `git diff --check` in both changed repositories. No deployment or provider request is necessary for this stage. ### Commit Consumer: `Document the D&D deployment context`. Workspace: `Close storage review and begin host deployment`. ## Stage 2: Prepare the Consumer Playbook and Inventory ### Objective and Owning Repository Make `rakestraw-ansible` a complete consumer of the reviewed suite snapshot. ### Required Policy Review Read `AGENTS.md`, `docs/orientation.md`, `docs/conventions.md`, `docs/ansible.md`, `docs/testing.md`, `docs/planning.md`, the Feature 0016 roadmap, `ansible-collection-dnd/README.md`, its `requirements.yml`, and the Stage 1 runbook before this stage. ### Preconditions Stage 1 is complete. Production credentials and remote access are not required. ### Required Changes 1. Add `playbooks/pipeline/dnd.yml` in the roadmap's exact minimal shape: one play targeting `pipeline-dnd` with only `rakestraw.dnd.pipeline` in its roles list. No play variables, tasks, or play-level privilege escalation. 2. Replace the legacy contents of `inventory/host_vars/pipeline-dnd/narratio.yml` with the roadmap's public inventory example: operator `eric`, the confirmed WhisperX URL, `narratio_campaigns_default_campaign_id: dilfs`, and the four Vault-backed `dnd_pipeline_secrets` mappings. Remove obsolete `narratio_s3_*` and `narratio_default_campaign_id` settings. Preserve unrelated intervening edits and the separate connection file. Do not copy suite defaults into inventory. 3. Document `inventory/host_vars/pipeline-dnd/vault.yml` and its four required `vault_dnd_*` names. Keep the inventory references unresolved until the operator provisions that encrypted file in Stage 3. Do not create an empty Vault file or substitute credentials to make local checks appear complete. 4. Pin the existing `rakestraw.dnd` Git entry in root `requirements.yml` to `46913446765e86ecfb29ebad9955d579ef6a4c7e` with its existing public URL. 5. Copy the five `roles` entries from that collection's compatibility manifest, retaining role names, public Gitea URLs, `scm: git`, and `v0.1.0` tags. Reconcile native collection entries to `ansible.posix: 2.2.0` and `community.general: 13.3.0`, without duplicates. Preserve unrelated dependencies; report demonstrated conflicts rather than silently changing the reviewed suite pins. 6. Extend the runbook with the concrete commands for dependency installation, collection refresh, Vault selection, deployment, fresh login, and native validation used below. Link upstream Narratio v2.1.1 operator documentation. ### Validation Parse edited plaintext YAML and run `git diff --check`. In isolated controller dependency paths, install from public sources with `ansible-galaxy install -r requirements.yml`. Configure role and collection destinations through Ansible configuration/environment; avoid a role-only custom path that would skip the collection section of the mixed manifest. Confirm resolution with `ansible-doc -t role rakestraw.dnd.pipeline` and the five `rakestraw.` role names. For local checks, use the workspace executables and the checked-out inventory. Configure isolated role and collection paths explicitly. From the consumer checkout, run: ```text ansible-playbook -i inventory playbooks/pipeline/dnd.yml --limit pipeline-dnd --list-hosts ansible-playbook -i inventory playbooks/pipeline/dnd.yml --limit pipeline-dnd --syntax-check ``` Host listing and syntax checking do not deploy or establish that credentials resolve. Run them without evaluating or printing secret values; do not supply fake Vault variables. If an existing unrelated encrypted inventory file needs an unavailable password, preserve it and use a disposable local inventory with only the target alias and its non-secret host variables for syntax checking. Report that narrower scope explicitly; full encrypted-inventory validation belongs to Stage 3. Do not change operational inventory to bypass encryption. Do not impose whole-repository lint changes on the established minimal-playbook style. ### Completion Criteria Only the intended host is selected. Public dependencies resolve without local sibling checkouts, local syntax checking succeeds within its recorded scope, and the diff contains no plaintext secrets. ### Commit `Add the D&D pipeline deployment playbook`. ## Stage 3: Deploy and Verify Clean Convergence ### Objective and Owning Repository Deploy from the operational controller; record results in the consumer runbook. ### Required Policy Review Read `AGENTS.md`, `docs/orientation.md`, `docs/conventions.md`, `docs/ansible.md`, `docs/testing.md`, `docs/planning.md`, the Feature 0016 roadmap, and `rakestraw-ansible/docs/pipeline-dnd.md` before this stage. ### Preconditions Stage 2 passed. The operational controller/checkout and delivery workflow are identified, and target SSH access and the Vault mechanism are available. Deliver committed changes through that workflow without overwriting a dirty checkout. Acceptance-session selection is not a prerequisite for this stage. ### Required Changes and Validation Before installation, confirm the operational controller's Ansible version and effective inventory, role, and collection paths. Confirm Ubuntu 26.04 and the existing `eric` account's required sudo access through read-only target checks. Have the operator create `inventory/host_vars/pipeline-dnd/vault.yml` encrypted in full, defining the four roadmap credential variables. Commit ciphertext with intent `Add encrypted D&D host credentials` after checking the diff exposes no plaintext. Rerun host listing and syntax validation with the real encrypted inventory and recorded Vault options before convergence. 1. Install root requirements at the confirmed controller paths. Explicitly refresh only the D&D collection from its selected commit, even when `0.1.0` is already installed: ```text ansible-galaxy collection install git+https://gitea.maximumdirect.net/eric/ansible-collection-dnd.git,46913446765e86ecfb29ebad9955d579ef6a4c7e --force --no-deps ``` Use the configured collection path; dependencies come from the manifest. Do not force-update unrelated unpinned collections. Confirm role resolution and playbook syntax on this operational controller as in Stage 2. 2. Run `ansible-playbook playbooks/pipeline/dnd.yml --limit pipeline-dnd` using the recorded inventory, Vault, and any required sudo-password options. Immediately repeat the identical command after success. Require `changed=0`, `failed=0`, and no unreachable hosts on the repeat. 3. On the target, execute `sudo -u narratio /usr/local/bin/narratio config validate`, then validate with `--profile testing`. The managed default campaign should make an explicit campaign selector unnecessary. 4. Start a fresh login as `eric` and execute the same default native validation without switching user, demonstrating usable supplementary group access. 5. Record consumer commit, target, date, repeat recap, and native validation results. Diagnose failures at the owning boundary; report required changes to roles/applications separately rather than expanding this feature. ### Completion Criteria The actual host converges cleanly and both runtime identities validate the configuration. Run `git diff --check` on the evidence/runbook changes. ### Commit `Record successful D&D host deployment`. ## Stage 4: Exercise One Real Session ### Objective and Owning Repository Demonstrate production processing and publication; record acceptance in `rakestraw-ansible/docs/pipeline-dnd.md`. ### Required Policy Review Read `AGENTS.md`, `docs/orientation.md`, `docs/conventions.md`, `docs/testing.md`, `docs/planning.md`, the Feature 0016 roadmap, the runbook, and Narratio v2.1.1's `docs/config.md`, `docs/cli.md`, and `docs/operations.md` before this stage. Follow the sibling application's `AGENTS.md` and development reading guide when consulting its repository. ### Preconditions Stage 3 passed. Actual credentials, the operator-selected campaign/session, session file, and audio are accessible. The exercise must demonstrate real transcription and publication, not merely resume an already complete run. Do not erase existing state to create a fixture. ### Required Changes and Validation 1. As `eric` in a fresh authorized login, run: ```text narratio session validate SESSION_ID --campaign CAMPAIGN --session SESSION_FILE narratio run SESSION_ID --campaign CAMPAIGN --session SESSION_FILE narratio session artifacts SESSION_ID --remote --campaign CAMPAIGN --session SESSION_FILE ``` Substitute the operator-recorded values and use the production profile. The session document remains operational input; no session-management task belongs in the playbook. Observe transcription, processing, and successful publication. Do not force reprocessing or reduce the selected artifact set merely to bypass a failure. 2. Observe a representative published transcript or recap at the configured destination through the existing storage/operator interface. Successful cleanup may remove local workspace content; local survival is not required. 3. Record identity, input location, runtime user, completion outcome, and non-secret publication evidence. Avoid committing raw provider logs or credential-bearing URLs. Do not repeat paid work just to duplicate evidence or expand acceptance into a restore/locking test matrix. ### Completion Criteria A real session completes and its published output is observable. Record any provider/application blocker and leave acceptance incomplete if it cannot finish. Run `git diff --check` on the runbook changes. ### Commit `Record the first published D&D session`. ## Stage 5: Prepare the Independent Review Handoff ### Objective and Owning Repository Record completed implementation evidence in the workspace. ### Required Policy Review Read `AGENTS.md`, `docs/orientation.md`, `docs/conventions.md`, `docs/testing.md`, `docs/planning.md`, and the Feature 0016 roadmap before this stage. ### Preconditions Stages 1–4 completed with committed changes and actual deployment/session evidence. A blocked live exercise does not satisfy this stage. ### Required Changes and Validation Update `docs/status.md` and this plan's progress notes with owning commits, runbook link, validation outcomes, and deviations. Keep Feature 0016 `in_progress` pending independent review. Report any additional complexity and its authorizing requirement. Run `git diff --check` and confirm changed repositories are clean after committing. Do not rerun role integration matrices or paid sessions without a new change or failure requiring it. ### Completion Criteria Evidence is recorded and implementation is ready for independent review. ### Commit `Record the D&D deployment implementation`. ## Final Validation and Handoff Report dependency revision, consumer commits, controller/target, public dependency and syntax results, immediate clean convergence, native validation for both identities, and session publication evidence. Distinguish completed work from operator-dependent blockers. Do not mark Feature 0016 `done` before independent review. ## Progress Notes - Stage 1 is complete in consumer commit `716a80ef68208537dacfa5f0f22259bf42ae6279`. The linked deployment runbook records the Ansible Core 2.21.3 local toolchain, the effective repository defaults, successful public-source reachability, and the inputs pending for live deployment and session acceptance. - Feature 0015 is closed after independent review found no remediation issue. Its reviewed pipeline commit is `51566afcf69083cdbb5b5718f2544c86b079001c`; collection fixture, squash, and merge commits are `cf3de016ee09edf91afc3bc1addfd926d4a172ad`, `77f67adbc0f700e88bc15554e02052eaa22f1033`, and `46913446765e86ecfb29ebad9955d579ef6a4c7e`. - Stage 2 local consumer preparation is next. The rejected target SSH identity, operational controller details, and Vault workflow/values remain Stage 3 prerequisites. The acceptance session/audio selection remains a Stage 4 prerequisite. No live deployment has begun. ## Open Questions Questions 1–2 gate Stage 3 only. Question 3 gates Stage 4 only. None prevents local preparation in Stages 1–2. 1. Which controller/checkout executes deployment and how do committed changes reach it? Recommended: use the existing operational controller and its established `/etc/ansible/` layout, keeping one operational inventory. Alternative: a dedicated checkout with explicit inventory and dependency paths, if the operator prefers isolation and records those paths consistently. 2. Which Vault identity/password mechanism is available, and can the operator populate all four values? Recommended: reuse the controller's established Vault workflow and populate the encrypted file directly. Alternative: an interactive Vault password prompt for this file if no identity is established. Both keep passwords outside Git; neither requires secrets in chat. 3. Which campaign/session/audio supplies acceptance? Recommended: an operator-selected unprocessed `dilfs` session with accessible audio, matching the retained default campaign and avoiding forced reprocessing. Alternative: a selected session from another deployed campaign with an explicit selector. Record the session file and audio location before execution; do not invent sample production input.