ansible-workspace-dnd/docs/roadmap/implementation.md

Implementation Plan 0016: Deploy the D&D Pipeline to pipeline-dnd

Status: ready

Target executor: GPT-5.6-Terra

Feature roadmap: docs/roadmap/features/0016-deploy-dnd-pipeline-to-pipeline-dnd.md

Active plan location: docs/roadmap/implementation.md

Objective

Deploy the reviewed collection through a minimal consumer playbook, host inventory, and Ansible Vault. Demonstrate clean repeated convergence and one operator-selected session through real publication. The feature roadmap owns scope, rationale, target state, and acceptance criteria.

Execution Rules

• Execute stages in order. Before each stage, reread its Required Policy Review and check the owning repository's working tree. Preserve unrelated changes. Validate and commit each completed stage with its stated intent.
• Keep the playbook limited to hosts and roles. Use existing interfaces; add no wrapper, preflight framework, custom module, or test harness.
• Inherit suite storage, publication, caching, cleanup, version, and asset defaults. Set only the roadmap-selected host choices.
• Never print decrypted inventory, credential values, or Vault passwords in commands, logs, documentation, or Git. Do not use decrypted inventory dumps as validation evidence. The operator supplies secrets directly through Ansible Vault's encrypted-file workflow, using an established secure editor. Do not solicit secrets in chat or create placeholder credentials.
• Stop dependent work when operational inputs are missing or validation fails. Record the concrete blocker; local checks do not prove a live deployment.
• Retain the historical playbook and proof-of-concept role. Role, collection, application, and asset repositories remain read-only during this feature.
• Deployment and paid provider execution occur only during the approved sprint with the selected controller, target, credentials, and session. Do not substitute an arbitrary session or broaden the host limit.

Starting State and Preconditions

• Feature 0015 passed independent review without findings. Its pipeline commit is 51566afcf69083cdbb5b5718f2544c86b079001c; the reviewed public collection commit is 46913446765e86ecfb29ebad9955d579ef6a4c7e.
• rakestraw-ansible starts clean on main at d86ec29a9cbd821fead248736171b5cdd8440ad6. Inspect and preserve any subsequent changes before adapting this plan.
• Root requirements.yml already has an unpinned Gitea rakestraw.dnd entry alongside unrelated collections. The historical playbook is playbooks/apps/dnd.yml; it invokes pipeline_dnd on automation-dnd.
• Host pipeline-dnd already connects as eric to pipeline-dnd.rakestrawhome.com. Preserve its connection settings.
• inventory/host_vars/pipeline-dnd/narratio.yml contains the legacy campaign and S3 variables to migrate.
• The selected collection still declares version 0.1.0. An installed release with that number is not sufficient evidence of the selected content.
• Approve and commit the planning documents before execution. Resolve the operational inputs under Open Questions before the sprint, as required by the roadmap; do not invent controller topology, credentials, or session data.

Repository Scope

• rakestraw-ansible: requirements, playbook, host configuration, encrypted credentials, and docs/pipeline-dnd.md runbook linked from README.
• Workspace: feature status, implementation progress, and review handoff.
• Other repositories: read-only compatibility and application references.

Stage 1: Establish the Operational Context

Objective and Owning Repository

Record deployment prerequisites in rakestraw-ansible and close the reviewed Feature 0015 in the workspace.

Required Policy Review

Read AGENTS.md, docs/orientation.md, docs/conventions.md, docs/ansible.md, docs/testing.md, docs/planning.md, the Feature 0016 roadmap, and rakestraw-ansible/README.md before this stage.

Preconditions

The plan is approved, repositories have been checked, and the operator has supplied the missing operational inputs below.

Required Changes

1. Create rakestraw-ansible/docs/pipeline-dnd.md and link it from README. Record the actual controller, checkout, effective Ansible configuration, inventory, role/collection paths, Vault selection method, and acceptance campaign/session/audio location. Record only non-secret information.
2. Confirm controller Ansible Core is at least 2.18 and the public suite repositories and Galaxy are reachable. Inspect effective configuration without dumping decrypted host variables. Existing repository defaults are /etc/ansible/inventory, /etc/ansible/roles, and /etc/ansible/collections; establish how the operational checkout supplies those paths.
3. Use ordinary read-only host checks to confirm Ubuntu 26.04 and the existing eric account's SSH and required sudo access. Record the outcome. Report discrepancies before deployment rather than changing platform/account policy.
4. Mark Feature 0015 done in its roadmap and workspace status, citing its reviewed commits and the successful independent standalone/collection integration runs. Leave historical remediation records unchanged. Mark Feature 0016 in_progress when execution begins.

Validation and Completion Criteria

Operational inputs are concrete and target platform/access checks pass. Run git diff --check in both changed repositories. No deployment or provider request is necessary for this stage.

Commit

Consumer: Document the D&D deployment context. Workspace: Close storage review and begin host deployment.

Stage 2: Prepare the Consumer Playbook and Inventory

Objective and Owning Repository

Make rakestraw-ansible a complete consumer of the reviewed suite snapshot.

Required Policy Review

Read AGENTS.md, docs/orientation.md, docs/conventions.md, docs/ansible.md, docs/testing.md, docs/planning.md, the Feature 0016 roadmap, ansible-collection-dnd/README.md, its requirements.yml, and the Stage 1 runbook before this stage.

Preconditions

Stage 1 is complete. The operator can populate and unlock the encrypted host file using the recorded Vault mechanism.

Required Changes

1. Add playbooks/pipeline/dnd.yml in the roadmap's exact minimal shape: one play targeting pipeline-dnd with only rakestraw.dnd.pipeline in its roles list. No play variables, tasks, or play-level privilege escalation.
2. Replace the legacy contents of inventory/host_vars/pipeline-dnd/narratio.yml with the roadmap's public inventory example: operator eric, the confirmed WhisperX URL, narratio_campaigns_default_campaign_id: dilfs, and the four Vault-backed dnd_pipeline_secrets mappings. Remove obsolete narratio_s3_* and narratio_default_campaign_id settings. Preserve unrelated intervening edits and the separate connection file. Do not copy suite defaults into inventory.
3. Have the operator create inventory/host_vars/pipeline-dnd/vault.yml, fully Ansible Vault-encrypted, defining the four exact vault_dnd_* names from the roadmap. Commit ciphertext only.
4. Pin the existing rakestraw.dnd Git entry in root requirements.yml to 46913446765e86ecfb29ebad9955d579ef6a4c7e with its existing public URL.
5. Copy the five roles entries from that collection's compatibility manifest, retaining role names, public Gitea URLs, scm: git, and v0.1.0 tags. Reconcile native collection entries to ansible.posix: 2.2.0 and community.general: 13.3.0, without duplicates. Preserve unrelated dependencies; report demonstrated conflicts rather than silently changing the reviewed suite pins.
6. Extend the runbook with the concrete commands for dependency installation, collection refresh, Vault selection, deployment, fresh login, and native validation used below. Link upstream Narratio v2.1.1 operator documentation.

Validation

Parse edited plaintext YAML and run git diff --check. In isolated controller dependency paths, install from public sources with ansible-galaxy install -r requirements.yml. Configure role and collection destinations through Ansible configuration/environment; avoid a role-only custom path that would skip the collection section of the mixed manifest. Confirm resolution with ansible-doc -t role rakestraw.dnd.pipeline and the five rakestraw.<app> role names.

From the consumer checkout, using the established Vault options, run:

ansible-playbook -i inventory playbooks/pipeline/dnd.yml --limit pipeline-dnd --list-hosts
ansible-playbook -i inventory playbooks/pipeline/dnd.yml --limit pipeline-dnd --syntax-check

Use the concrete inventory path from Stage 1 if different. Do not impose whole-repository lint changes on the established minimal-playbook style.

Completion Criteria

Only the intended host is selected. Public dependencies resolve without local sibling checkouts, syntax checking succeeds with encrypted inventory, and the diff contains no plaintext secrets.

Commit

Add the D&D pipeline deployment playbook.

Stage 3: Deploy and Verify Clean Convergence

Objective and Owning Repository

Deploy from the operational controller; record results in the consumer runbook.

Required Policy Review

Read AGENTS.md, docs/orientation.md, docs/conventions.md, docs/ansible.md, docs/testing.md, docs/planning.md, the Feature 0016 roadmap, and rakestraw-ansible/docs/pipeline-dnd.md before this stage.

Preconditions

Stage 2 passed. The operational checkout contains its committed changes and Vault is usable. Deliver changes through the established checkout workflow; do not overwrite a dirty operational checkout.

Required Changes and Validation

1. Install root requirements at the confirmed controller paths. Explicitly refresh only the D&D collection from its selected commit, even when 0.1.0 is already installed:

   ansible-galaxy collection install git+https://gitea.maximumdirect.net/eric/ansible-collection-dnd.git,46913446765e86ecfb29ebad9955d579ef6a4c7e --force --no-deps
   

   Use the configured collection path; dependencies come from the manifest. Do not force-update unrelated unpinned collections. Confirm role resolution and playbook syntax on this operational controller as in Stage 2.

2. Run ansible-playbook playbooks/pipeline/dnd.yml --limit pipeline-dnd using the recorded inventory, Vault, and any required sudo-password options. Immediately repeat the identical command after success. Require changed=0, failed=0, and no unreachable hosts on the repeat.

3. On the target, execute sudo -u narratio /usr/local/bin/narratio config validate, then validate with --profile testing. The managed default campaign should make an explicit campaign selector unnecessary.

4. Start a fresh login as eric and execute the same default native validation without switching user, demonstrating usable supplementary group access.

5. Record consumer commit, target, date, repeat recap, and native validation results. Diagnose failures at the owning boundary; report required changes to roles/applications separately rather than expanding this feature.

Completion Criteria

The actual host converges cleanly and both runtime identities validate the configuration. Run git diff --check on the evidence/runbook changes.

Commit

Record successful D&D host deployment.

Stage 4: Exercise One Real Session

Objective and Owning Repository

Demonstrate production processing and publication; record acceptance in rakestraw-ansible/docs/pipeline-dnd.md.

Required Policy Review

Read AGENTS.md, docs/orientation.md, docs/conventions.md, docs/testing.md, docs/planning.md, the Feature 0016 roadmap, the runbook, and Narratio v2.1.1's docs/config.md, docs/cli.md, and docs/operations.md before this stage. Follow the sibling application's AGENTS.md and development reading guide when consulting its repository.

Preconditions

Stage 3 passed. Actual credentials, the operator-selected campaign/session, session file, and audio are accessible. The exercise must demonstrate real transcription and publication, not merely resume an already complete run. Do not erase existing state to create a fixture.

Required Changes and Validation

1. As eric in a fresh authorized login, run:

   narratio session validate SESSION_ID --campaign CAMPAIGN --session SESSION_FILE
   narratio run SESSION_ID --campaign CAMPAIGN --session SESSION_FILE
   narratio session artifacts SESSION_ID --remote --campaign CAMPAIGN --session SESSION_FILE
   

   Substitute the operator-recorded values and use the production profile. The session document remains operational input; no session-management task belongs in the playbook. Observe transcription, processing, and successful publication. Do not force reprocessing or reduce the selected artifact set merely to bypass a failure.

2. Observe a representative published transcript or recap at the configured destination through the existing storage/operator interface. Successful cleanup may remove local workspace content; local survival is not required.

3. Record identity, input location, runtime user, completion outcome, and non-secret publication evidence. Avoid committing raw provider logs or credential-bearing URLs. Do not repeat paid work just to duplicate evidence or expand acceptance into a restore/locking test matrix.

Completion Criteria

A real session completes and its published output is observable. Record any provider/application blocker and leave acceptance incomplete if it cannot finish. Run git diff --check on the runbook changes.

Commit

Record the first published D&D session.

Stage 5: Prepare the Independent Review Handoff

Objective and Owning Repository

Record completed implementation evidence in the workspace.

Required Policy Review

Read AGENTS.md, docs/orientation.md, docs/conventions.md, docs/testing.md, docs/planning.md, and the Feature 0016 roadmap before this stage.

Preconditions

Stages 1–4 completed with committed changes and actual deployment/session evidence. A blocked live exercise does not satisfy this stage.

Required Changes and Validation

Update docs/status.md and this plan's progress notes with owning commits, runbook link, validation outcomes, and deviations. Keep Feature 0016 in_progress pending independent review. Report any additional complexity and its authorizing requirement. Run git diff --check and confirm changed repositories are clean after committing. Do not rerun role integration matrices or paid sessions without a new change or failure requiring it.

Completion Criteria

Evidence is recorded and implementation is ready for independent review.

Commit

Record the D&D deployment implementation.

Final Validation and Handoff

Report dependency revision, consumer commits, controller/target, public dependency and syntax results, immediate clean convergence, native validation for both identities, and session publication evidence. Distinguish completed work from operator-dependent blockers. Do not mark Feature 0016 done before independent review.

Progress Notes

Plan approved; no Feature 0016 implementation or deployment has begun. Architecture is settled; the operational inputs below remain to be supplied.

Open Questions

1. Which controller/checkout executes deployment and how do committed changes reach it? Recommended: use the existing operational controller and its established /etc/ansible/ layout, keeping one operational inventory. Alternative: a dedicated checkout with explicit inventory and dependency paths, if the operator prefers isolation and records those paths consistently.
2. Which Vault identity/password mechanism is available, and can the operator populate all four values? Recommended: reuse the controller's established Vault workflow and populate the encrypted file directly. Alternative: an interactive Vault password prompt for this file if no identity is established. Both keep passwords outside Git; neither requires secrets in chat.
3. Which campaign/session/audio supplies acceptance? Recommended: an operator-selected unprocessed dilfs session with accessible audio, matching the retained default campaign and avoiding forced reprocessing. Alternative: a selected session from another deployed campaign with an explicit selector. Record the session file and audio location before execution; do not invent sample production input.