365 lines
16 KiB
Markdown
365 lines
16 KiB
Markdown
# Implementation Plan 0016: Deploy the D&D Pipeline to `pipeline-dnd`
|
||
|
||
Status: ready
|
||
|
||
Target executor: GPT-5.6-Terra
|
||
|
||
Feature roadmap: `docs/roadmap/features/0016-deploy-dnd-pipeline-to-pipeline-dnd.md`
|
||
|
||
Active plan location: `docs/roadmap/implementation.md`
|
||
|
||
## Objective
|
||
|
||
Deploy the reviewed collection through a minimal consumer playbook, host
|
||
inventory, and Ansible Vault. Demonstrate clean repeated convergence and one
|
||
operator-selected session through real publication. The feature roadmap owns
|
||
scope, rationale, target state, and acceptance criteria.
|
||
|
||
## Execution Rules
|
||
|
||
- Execute stages in order. Before each stage, reread its Required Policy
|
||
Review and check the owning repository's working tree. Preserve unrelated
|
||
changes. Validate and commit each completed stage with its stated intent.
|
||
- Keep the playbook limited to hosts and roles. Use existing interfaces;
|
||
add no wrapper, preflight framework, custom module, or test harness.
|
||
- Inherit suite storage, publication, caching, cleanup, version, and asset
|
||
defaults. Set only the roadmap-selected host choices.
|
||
- Never print decrypted inventory, credential values, or Vault passwords in
|
||
commands, logs, documentation, or Git. Do not use decrypted inventory dumps
|
||
as validation evidence. The operator supplies secrets directly through
|
||
Ansible Vault's encrypted-file workflow, using an established secure editor.
|
||
Do not solicit secrets in chat or create placeholder credentials.
|
||
- Stop dependent work when operational inputs are missing or validation fails.
|
||
Record the concrete blocker; local checks do not prove a live deployment.
|
||
- Retain the historical playbook and proof-of-concept role. Role, collection,
|
||
application, and asset repositories remain read-only during this feature.
|
||
- Deployment and paid provider execution occur only during the approved sprint
|
||
with the selected controller, target, credentials, and session. Do not
|
||
substitute an arbitrary session or broaden the host limit.
|
||
|
||
## Starting State and Preconditions
|
||
|
||
- Feature 0015 passed independent review without findings. Its pipeline commit
|
||
is `51566afcf69083cdbb5b5718f2544c86b079001c`; the reviewed public collection
|
||
commit is `46913446765e86ecfb29ebad9955d579ef6a4c7e`.
|
||
- `rakestraw-ansible` starts clean on `main` at
|
||
`d86ec29a9cbd821fead248736171b5cdd8440ad6`. Inspect and preserve any subsequent
|
||
changes before adapting this plan.
|
||
- Root `requirements.yml` already has an unpinned Gitea `rakestraw.dnd` entry
|
||
alongside unrelated collections. The historical playbook is
|
||
`playbooks/apps/dnd.yml`; it invokes `pipeline_dnd` on `automation-dnd`.
|
||
- Host `pipeline-dnd` already connects as `eric` to
|
||
`pipeline-dnd.rakestrawhome.com`. Preserve its connection settings.
|
||
- `inventory/host_vars/pipeline-dnd/narratio.yml` contains the legacy campaign
|
||
and S3 variables to migrate.
|
||
- The selected collection still declares version `0.1.0`. An installed
|
||
release with that number is not sufficient evidence of the selected content.
|
||
- Approve and commit the planning documents before execution. Resolve the
|
||
operational inputs under Open Questions before the sprint, as required by
|
||
the roadmap; do not invent controller topology, credentials, or session data.
|
||
|
||
## Repository Scope
|
||
|
||
- `rakestraw-ansible`: requirements, playbook, host configuration, encrypted
|
||
credentials, and `docs/pipeline-dnd.md` runbook linked from README.
|
||
- Workspace: feature status, implementation progress, and review handoff.
|
||
- Other repositories: read-only compatibility and application references.
|
||
|
||
## Stage 1: Establish the Operational Context
|
||
|
||
### Objective and Owning Repository
|
||
|
||
Record deployment prerequisites in `rakestraw-ansible` and close the reviewed
|
||
Feature 0015 in the workspace.
|
||
|
||
### Required Policy Review
|
||
|
||
Read `AGENTS.md`, `docs/orientation.md`, `docs/conventions.md`,
|
||
`docs/ansible.md`, `docs/testing.md`, `docs/planning.md`, the Feature 0016
|
||
roadmap, and `rakestraw-ansible/README.md` before this stage.
|
||
|
||
### Preconditions
|
||
|
||
The plan is approved, repositories have been checked, and the operator has
|
||
supplied the missing operational inputs below.
|
||
|
||
### Required Changes
|
||
|
||
1. Create `rakestraw-ansible/docs/pipeline-dnd.md` and link it from README.
|
||
Record the actual controller, checkout, effective Ansible configuration,
|
||
inventory, role/collection paths, Vault selection method, and acceptance
|
||
campaign/session/audio location. Record only non-secret information.
|
||
2. Confirm controller Ansible Core is at least 2.18 and the public suite
|
||
repositories and Galaxy are reachable. Inspect effective configuration
|
||
without dumping decrypted host variables. Existing repository defaults are
|
||
`/etc/ansible/inventory`, `/etc/ansible/roles`, and `/etc/ansible/collections`;
|
||
establish how the operational checkout supplies those paths.
|
||
3. Use ordinary read-only host checks to confirm Ubuntu 26.04 and the existing
|
||
`eric` account's SSH and required sudo access. Record the outcome. Report
|
||
discrepancies before deployment rather than changing platform/account policy.
|
||
4. Mark Feature 0015 `done` in its roadmap and workspace status, citing its
|
||
reviewed commits and the successful independent standalone/collection
|
||
integration runs. Leave historical remediation records unchanged. Mark
|
||
Feature 0016 `in_progress` when execution begins.
|
||
|
||
### Validation and Completion Criteria
|
||
|
||
Operational inputs are concrete and target platform/access checks pass.
|
||
Run `git diff --check` in both changed repositories. No deployment or provider
|
||
request is necessary for this stage.
|
||
|
||
### Commit
|
||
|
||
Consumer: `Document the D&D deployment context`.
|
||
Workspace: `Close storage review and begin host deployment`.
|
||
|
||
## Stage 2: Prepare the Consumer Playbook and Inventory
|
||
|
||
### Objective and Owning Repository
|
||
|
||
Make `rakestraw-ansible` a complete consumer of the reviewed suite snapshot.
|
||
|
||
### Required Policy Review
|
||
|
||
Read `AGENTS.md`, `docs/orientation.md`, `docs/conventions.md`,
|
||
`docs/ansible.md`, `docs/testing.md`, `docs/planning.md`, the Feature 0016
|
||
roadmap, `ansible-collection-dnd/README.md`, its `requirements.yml`, and the
|
||
Stage 1 runbook before this stage.
|
||
|
||
### Preconditions
|
||
|
||
Stage 1 is complete. The operator can populate and unlock the encrypted host
|
||
file using the recorded Vault mechanism.
|
||
|
||
### Required Changes
|
||
|
||
1. Add `playbooks/pipeline/dnd.yml` in the roadmap's exact minimal shape:
|
||
one play targeting `pipeline-dnd` with only `rakestraw.dnd.pipeline` in its
|
||
roles list. No play variables, tasks, or play-level privilege escalation.
|
||
2. Replace the legacy contents of
|
||
`inventory/host_vars/pipeline-dnd/narratio.yml` with the roadmap's public
|
||
inventory example: operator `eric`, the confirmed WhisperX URL,
|
||
`narratio_campaigns_default_campaign_id: dilfs`, and the four Vault-backed
|
||
`dnd_pipeline_secrets` mappings. Remove obsolete `narratio_s3_*` and
|
||
`narratio_default_campaign_id` settings. Preserve unrelated intervening edits
|
||
and the separate connection file. Do not copy suite defaults into inventory.
|
||
3. Have the operator create `inventory/host_vars/pipeline-dnd/vault.yml`, fully
|
||
Ansible Vault-encrypted, defining the four exact `vault_dnd_*` names from
|
||
the roadmap. Commit ciphertext only.
|
||
4. Pin the existing `rakestraw.dnd` Git entry in root `requirements.yml` to
|
||
`46913446765e86ecfb29ebad9955d579ef6a4c7e` with its existing public URL.
|
||
5. Copy the five `roles` entries from that collection's compatibility manifest,
|
||
retaining role names, public Gitea URLs, `scm: git`, and `v0.1.0` tags.
|
||
Reconcile native collection entries to `ansible.posix: 2.2.0` and
|
||
`community.general: 13.3.0`, without duplicates. Preserve unrelated
|
||
dependencies; report demonstrated conflicts rather than silently changing
|
||
the reviewed suite pins.
|
||
6. Extend the runbook with the concrete commands for dependency installation,
|
||
collection refresh, Vault selection, deployment, fresh login, and native
|
||
validation used below. Link upstream Narratio v2.1.1 operator documentation.
|
||
|
||
### Validation
|
||
|
||
Parse edited plaintext YAML and run `git diff --check`. In isolated controller
|
||
dependency paths, install from public sources with
|
||
`ansible-galaxy install -r requirements.yml`. Configure role and collection
|
||
destinations through Ansible configuration/environment; avoid a role-only
|
||
custom path that would skip the collection section of the mixed manifest.
|
||
Confirm resolution with `ansible-doc -t role rakestraw.dnd.pipeline` and the
|
||
five `rakestraw.<app>` role names.
|
||
|
||
From the consumer checkout, using the established Vault options, run:
|
||
|
||
```text
|
||
ansible-playbook -i inventory playbooks/pipeline/dnd.yml --limit pipeline-dnd --list-hosts
|
||
ansible-playbook -i inventory playbooks/pipeline/dnd.yml --limit pipeline-dnd --syntax-check
|
||
```
|
||
|
||
Use the concrete inventory path from Stage 1 if different. Do not impose
|
||
whole-repository lint changes on the established minimal-playbook style.
|
||
|
||
### Completion Criteria
|
||
|
||
Only the intended host is selected. Public dependencies resolve without local
|
||
sibling checkouts, syntax checking succeeds with encrypted inventory, and the
|
||
diff contains no plaintext secrets.
|
||
|
||
### Commit
|
||
|
||
`Add the D&D pipeline deployment playbook`.
|
||
|
||
## Stage 3: Deploy and Verify Clean Convergence
|
||
|
||
### Objective and Owning Repository
|
||
|
||
Deploy from the operational controller; record results in the consumer runbook.
|
||
|
||
### Required Policy Review
|
||
|
||
Read `AGENTS.md`, `docs/orientation.md`, `docs/conventions.md`,
|
||
`docs/ansible.md`, `docs/testing.md`, `docs/planning.md`, the Feature 0016
|
||
roadmap, and `rakestraw-ansible/docs/pipeline-dnd.md` before this stage.
|
||
|
||
### Preconditions
|
||
|
||
Stage 2 passed. The operational checkout contains its committed changes and
|
||
Vault is usable. Deliver changes through the established checkout workflow;
|
||
do not overwrite a dirty operational checkout.
|
||
|
||
### Required Changes and Validation
|
||
|
||
1. Install root requirements at the confirmed controller paths. Explicitly
|
||
refresh only the D&D collection from its selected commit, even when `0.1.0`
|
||
is already installed:
|
||
|
||
```text
|
||
ansible-galaxy collection install git+https://gitea.maximumdirect.net/eric/ansible-collection-dnd.git,46913446765e86ecfb29ebad9955d579ef6a4c7e --force --no-deps
|
||
```
|
||
|
||
Use the configured collection path; dependencies come from the manifest.
|
||
Do not force-update unrelated unpinned collections. Confirm role resolution
|
||
and playbook syntax on this operational controller as in Stage 2.
|
||
2. Run `ansible-playbook playbooks/pipeline/dnd.yml --limit pipeline-dnd`
|
||
using the recorded inventory, Vault, and any required sudo-password options.
|
||
Immediately repeat the identical command after success. Require
|
||
`changed=0`, `failed=0`, and no unreachable hosts on the repeat.
|
||
3. On the target, execute
|
||
`sudo -u narratio /usr/local/bin/narratio config validate`, then validate
|
||
with `--profile testing`. The managed default campaign should make an
|
||
explicit campaign selector unnecessary.
|
||
4. Start a fresh login as `eric` and execute the same default native validation
|
||
without switching user, demonstrating usable supplementary group access.
|
||
5. Record consumer commit, target, date, repeat recap, and native validation
|
||
results. Diagnose failures at the owning boundary; report required changes
|
||
to roles/applications separately rather than expanding this feature.
|
||
|
||
### Completion Criteria
|
||
|
||
The actual host converges cleanly and both runtime identities validate the
|
||
configuration. Run `git diff --check` on the evidence/runbook changes.
|
||
|
||
### Commit
|
||
|
||
`Record successful D&D host deployment`.
|
||
|
||
## Stage 4: Exercise One Real Session
|
||
|
||
### Objective and Owning Repository
|
||
|
||
Demonstrate production processing and publication; record acceptance in
|
||
`rakestraw-ansible/docs/pipeline-dnd.md`.
|
||
|
||
### Required Policy Review
|
||
|
||
Read `AGENTS.md`, `docs/orientation.md`, `docs/conventions.md`,
|
||
`docs/testing.md`, `docs/planning.md`, the Feature 0016 roadmap, the runbook,
|
||
and Narratio v2.1.1's `docs/config.md`, `docs/cli.md`, and
|
||
`docs/operations.md` before this stage. Follow the sibling application's
|
||
`AGENTS.md` and development reading guide when consulting its repository.
|
||
|
||
### Preconditions
|
||
|
||
Stage 3 passed. Actual credentials, the operator-selected campaign/session,
|
||
session file, and audio are accessible. The exercise must demonstrate real
|
||
transcription and publication, not merely resume an already complete run.
|
||
Do not erase existing state to create a fixture.
|
||
|
||
### Required Changes and Validation
|
||
|
||
1. As `eric` in a fresh authorized login, run:
|
||
|
||
```text
|
||
narratio session validate SESSION_ID --campaign CAMPAIGN --session SESSION_FILE
|
||
narratio run SESSION_ID --campaign CAMPAIGN --session SESSION_FILE
|
||
narratio session artifacts SESSION_ID --remote --campaign CAMPAIGN --session SESSION_FILE
|
||
```
|
||
|
||
Substitute the operator-recorded values and use the production profile.
|
||
The session document remains operational input; no session-management task
|
||
belongs in the playbook. Observe transcription, processing, and successful
|
||
publication. Do not force reprocessing or reduce the selected artifact set
|
||
merely to bypass a failure.
|
||
2. Observe a representative published transcript or recap at the configured
|
||
destination through the existing storage/operator interface. Successful
|
||
cleanup may remove local workspace content; local survival is not required.
|
||
3. Record identity, input location, runtime user, completion outcome, and
|
||
non-secret publication evidence. Avoid committing raw provider logs or
|
||
credential-bearing URLs. Do not repeat paid work just to duplicate evidence
|
||
or expand acceptance into a restore/locking test matrix.
|
||
|
||
### Completion Criteria
|
||
|
||
A real session completes and its published output is observable. Record any
|
||
provider/application blocker and leave acceptance incomplete if it cannot
|
||
finish. Run `git diff --check` on the runbook changes.
|
||
|
||
### Commit
|
||
|
||
`Record the first published D&D session`.
|
||
|
||
## Stage 5: Prepare the Independent Review Handoff
|
||
|
||
### Objective and Owning Repository
|
||
|
||
Record completed implementation evidence in the workspace.
|
||
|
||
### Required Policy Review
|
||
|
||
Read `AGENTS.md`, `docs/orientation.md`, `docs/conventions.md`,
|
||
`docs/testing.md`, `docs/planning.md`, and the Feature 0016 roadmap before
|
||
this stage.
|
||
|
||
### Preconditions
|
||
|
||
Stages 1–4 completed with committed changes and actual deployment/session
|
||
evidence. A blocked live exercise does not satisfy this stage.
|
||
|
||
### Required Changes and Validation
|
||
|
||
Update `docs/status.md` and this plan's progress notes with owning commits,
|
||
runbook link, validation outcomes, and deviations. Keep Feature 0016
|
||
`in_progress` pending independent review. Report any additional complexity
|
||
and its authorizing requirement. Run `git diff --check` and confirm changed
|
||
repositories are clean after committing. Do not rerun role integration
|
||
matrices or paid sessions without a new change or failure requiring it.
|
||
|
||
### Completion Criteria
|
||
|
||
Evidence is recorded and implementation is ready for independent review.
|
||
|
||
### Commit
|
||
|
||
`Record the D&D deployment implementation`.
|
||
|
||
## Final Validation and Handoff
|
||
|
||
Report dependency revision, consumer commits, controller/target, public
|
||
dependency and syntax results, immediate clean convergence, native validation
|
||
for both identities, and session publication evidence. Distinguish completed
|
||
work from operator-dependent blockers. Do not mark Feature 0016 `done` before
|
||
independent review.
|
||
|
||
## Progress Notes
|
||
|
||
Plan approved; no Feature 0016 implementation or deployment has begun.
|
||
Architecture is settled; the operational inputs below remain to be supplied.
|
||
|
||
## Open Questions
|
||
|
||
1. Which controller/checkout executes deployment and how do committed changes
|
||
reach it? Recommended: use the existing operational controller and its
|
||
established `/etc/ansible/` layout, keeping one operational inventory.
|
||
Alternative: a dedicated checkout with explicit inventory and dependency
|
||
paths, if the operator prefers isolation and records those paths consistently.
|
||
2. Which Vault identity/password mechanism is available, and can the operator
|
||
populate all four values? Recommended: reuse the controller's established
|
||
Vault workflow and populate the encrypted file directly. Alternative: an
|
||
interactive Vault password prompt for this file if no identity is established.
|
||
Both keep passwords outside Git; neither requires secrets in chat.
|
||
3. Which campaign/session/audio supplies acceptance? Recommended: an
|
||
operator-selected unprocessed `dilfs` session with accessible audio, matching
|
||
the retained default campaign and avoiding forced reprocessing. Alternative:
|
||
a selected session from another deployed campaign with an explicit selector.
|
||
Record the session file and audio location before execution; do not invent
|
||
sample production input.
|