Limit the D&D feature to playbook preparation

This commit is contained in:
2026-09-21 05:14:06 +00:00
parent f9901f17d1
commit 1d1f34fc8b
3 changed files with 198 additions and 653 deletions

View File

@@ -1,88 +1,41 @@
# Feature 0016: Deploy the D&D Pipeline to `pipeline-dnd`
# Feature 0016: Prepare the D&D Deployment Playbook for `pipeline-dnd`
Status: in_progress
## Summary
Add a minimal playbook and host inventory to `rakestraw-ansible` that apply
`rakestraw.dnd.pipeline` to the existing `pipeline-dnd` host. Configure the
host through `host_vars`, enroll `eric` as the initial human operator, keep
credential values in Ansible Vault, and prove the complete pipeline through a
real repeatable deployment and representative D&D session.
[Feature 0015](0015-dnd-s3-storage-and-publishing.md) has implemented and passed
independent review of the storage and publishing behavior needed for an
operational end-to-end pipeline. Feature 0016 consumes that reviewed interface
and its suite defaults.
## Problem or Opportunity
The collection and its application roles are implemented and installable, but
the operational Ansible repository does not yet have a playbook for deploying
the new collection architecture. Its existing D&D playbook targets a retired
inventory name and invokes the proof-of-concept local `pipeline_dnd` role.
The current `pipeline-dnd` host inventory already records the connection
identity and useful legacy Narratio storage values. A production-shaped first
deployment now needs to express the new role's public interface, deliver
secrets safely, reconcile the controller's dependency manifest, and validate
the actual host rather than extending the role repositories with another
synthetic scenario.
Write and commit the consumer playbook, necessary host variables, dependency
manifest, and concise usage documentation in `rakestraw-ansible`. This feature
ends with repository preparation. Deployment to a controller or target host
and execution of the pipeline are separate operator work.
## Scope
- Add `rakestraw-ansible/playbooks/pipeline/dnd.yml`, targeting the existing
`pipeline-dnd` inventory host and applying only
`rakestraw.dnd.pipeline`.
- Keep the playbook limited to its host selection and role list. Put all
deployment choices in `inventory/host_vars/pipeline-dnd/`.
- Configure the external WhisperX transcription endpoint as
`https://transcription.ai.rakestrawhome.com/transcribe`.
- Enroll the existing `eric` account through
`dnd_pipeline_operator_users`.
- Map the role's `dnd_pipeline_secrets` interface to host-specific Vault
variables. Store credential values only in an Ansible Vault-encrypted host
variables file.
- Inherit Feature 0015's storage and publishing defaults, which already match
the intended Backblaze B2 deployment. Host inventory contains only actual
host choices and deliberate overrides, without copying suite defaults.
- Preserve the existing default campaign choice, `dilfs`, through the
standalone role's `narratio_campaigns_default_campaign_id` variable.
- Reconcile existing legacy Narratio host variables so they cannot silently
override the suite policy selected by the new pipeline role.
- Update `rakestraw-ansible/requirements.yml` so a clean controller can install
the reviewed collection snapshot, all five standalone role dependencies,
and required native collections from one manifest.
- Validate controller dependency resolution, playbook syntax, initial
convergence, immediate clean convergence, operator access, native Narratio
configuration, and one representative end-to-end pipeline run on the real
host.
- Document the operational invocation, Vault prerequisite, initial operator,
and expected validation path in `rakestraw-ansible`.
- `playbooks/pipeline/dnd.yml` targets the existing `pipeline-dnd` alias and
applies `rakestraw.dnd.pipeline`, using only hosts and roles.
- Host choices belong in `inventory/host_vars/pipeline-dnd/`: the WhisperX
endpoint, initial operator `eric`, default campaign `dilfs`, and credential
references. Unknown required values are explained in adjacent comments.
- Inherit the reviewed suite's storage, publication, caching, cleanup, and
application defaults. Retire obsolete legacy host variables.
- Root `requirements.yml` selects the reviewed public collection snapshot and
its role/native collection dependencies, preserving unrelated entries.
- Document local validation and eventual operator use. Commit the completed
repository changes without requiring remote access or real credentials.
## Non-goals
- Implementing storage or publishing behavior. Feature 0015 owns that
prerequisite.
- Changing application or role behavior merely to accommodate this
deployment.
- Replacing the existing inventory alias with `pipeline.dnd`, changing DNS,
or moving connection variables out of the current host inventory.
- Adding a wrapper role, pre-task layer, service, timer, scheduler, web UI, or
unattended session-ingestion mechanism.
- Creating or managing the `eric` account, its login policy, SSH access, or
sudo policy.
- Adding another production host, high availability, backup policy, or
disaster-recovery automation.
- Removing the proof-of-concept local `pipeline_dnd` role or its historical
playbook before the new deployment has demonstrated operational parity.
- Publishing a new coordinated suite release solely as part of this
deployment feature.
- Installing or updating anything on the operational controller or target.
- Connecting to `pipeline-dnd`, validating its platform or sudo access,
converging the host, checking runtime identities, or executing a real session.
- Obtaining API keys, choosing a Vault password, populating real credentials,
or selecting acceptance audio/session data.
- Changing application/role behavior, publishing a new suite release, adding
orchestration layers, or retiring the proof-of-concept role/playbook.
## Target End State
`rakestraw-ansible/playbooks/pipeline/dnd.yml` is an intentionally small
consumer playbook equivalent to:
The committed playbook is:
```yaml
---
@@ -93,12 +46,8 @@ consumer playbook equivalent to:
...
```
The existing inventory selects the host's connection identity. A dedicated
host-variable file selects the WhisperX endpoint, the initial operator list,
and the default campaign. Storage, caching, publication, and cleanup inherit
the reviewed suite defaults. The file maps runtime credential names to
Vault-backed variables without embedding credential values in the playbook or
ordinary plaintext inventory, for example:
Existing connection variables remain in the host's `main.yml`. Its
`narratio.yml` contains the following public configuration and comments:
```yaml
dnd_pipeline_operator_users:
@@ -109,6 +58,10 @@ dnd_pipeline_whisperx_transcribe_url: >-
narratio_campaigns_default_campaign_id: dilfs
# Before deployment, define the four vault_dnd_* variables below in an
# Ansible Vault-encrypted host_vars/pipeline-dnd/vault.yml file.
# Values are not known here; leave these references unresolved until supplied.
# Do not put API keys or object-storage credentials in this plaintext file.
dnd_pipeline_secrets:
AUDITA_LLM_API_KEY: "{{ vault_dnd_audita_llm_api_key }}"
OPENROUTER_API_KEY: "{{ vault_dnd_openrouter_api_key }}"
@@ -116,214 +69,77 @@ dnd_pipeline_secrets:
OBJECT_STORAGE_KEY: "{{ vault_dnd_object_storage_key }}"
```
The corresponding credential variables reside in a tracked file encrypted in
full with Ansible Vault. The Vault password or identity mechanism remains
outside Git and outside this feature's managed content.
The references and comments are a complete deliverable for unknown secrets.
No encrypted file, placeholder key, or real credential is required for feature
completion. Future operator-supplied values should be Vault-encrypted in Git;
the suite's existing restricted runtime secrets files remain the delivery path.
The controller dependency manifest selects public collection commit
`46913446765e86ecfb29ebad9955d579ef6a4c7e`, containing the reviewed Features
0014 and 0015 and pipeline commit
`51566afcf69083cdbb5b5718f2544c86b079001c`. Its compatibility manifest selects
the five standalone roles at `v0.1.0`, `ansible.posix` at `2.2.0`, and
`community.general` at `13.3.0`. A clean controller therefore does not depend
on sibling development checkouts or previously installed roles.
The manifest pins the collection to
`46913446765e86ecfb29ebad9955d579ef6a4c7e`, containing reviewed pipeline commit
`51566afcf69083cdbb5b5718f2544c86b079001c`. Its five standalone role dependencies
use their reviewed `v0.1.0` tags, with `ansible.posix` `2.2.0` and
`community.general` `13.3.0`. Suite Git dependencies use public Gitea URLs;
native collections come from Galaxy. Local sibling checkouts are not dependencies.
The published collection `v0.1.0` predates these features. The selected
development snapshot still declares collection version `0.1.0`, so the
deployment instructions explicitly refresh an already-installed collection
from the pinned commit instead of allowing an equal-version installation to
be skipped. Unrelated controller dependencies remain preserved.
On `pipeline-dnd`, the role converges the full application suite and its D&D
configuration. The `narratio` service identity can run the pipeline, and
`eric` can run it after starting a login session that observes the granted
supplementary group membership. A representative session can be transcribed,
processed, stored, and published through the configured production path. An
immediate identical Ansible run reports no changes.
The runbook explains that the selected collection snapshot still reports
`0.1.0`, so eventual operator installation must explicitly refresh it from
the pinned commit if an older `0.1.0` is installed. This is an instruction
for future use, not an operation required by this feature.
## Repository Ownership
- `rakestraw-ansible` owns the consumer playbook, host inventory, encrypted
host credentials, controller dependency manifest, and deployment runbook.
- `ansible-role-dnd-pipeline` owns the suite's public variables and the
opinionated cross-application behavior consumed by the deployment.
- `ansible-collection-dnd` owns the assembled `rakestraw.dnd.pipeline` role
snapshot and consumer-facing compatibility manifest.
- The standalone application roles continue to own installation and native
application configuration. Feature 0016 consumes their reviewed releases or
commits without reimplementing their behavior.
- `ansible-workspace-dnd` owns this roadmap, the later implementation plan,
progress tracking, and independent review record.
- Application and asset repositories are read-only sources of runtime
contracts and deployed content for this feature.
- `rakestraw-ansible`: consumer files, comments, dependency manifest, and runbook.
- Workspace: roadmap, staged plan, progress, and independent review record.
- Roles, collection, applications, and assets: read-only contract references.
## Constraints and Settled Decisions
- Feature 0015's implementation and independent review satisfy the storage
and publishing prerequisite. Its workspace completion record should be
closed out before the Feature 0016 sprint.
- The target is the inventory alias `pipeline-dnd` as currently defined in
`rakestraw-ansible`, not a new `pipeline.dnd` alias.
- The playbook follows the repository's established style: hosts and roles
only. Facts remain at Ansible's normal default, privilege escalation remains
task-owned by the roles, and deployment variables belong in host inventory.
- `eric` is the initial human operator and must already exist on the managed
host. The pipeline role grants only additive runtime-group membership.
- API credentials use the existing typed `dnd_pipeline_secrets` interface and
Ansible Vault. Plaintext secret values must not enter Git, command lines,
logs, examples, or ordinary variable files.
- Public Gitea repositories supply the suite's Git dependencies; native
collections come from Ansible Galaxy. The deployment must not reference
this workspace's sibling checkout paths.
- The collection is pinned to the reviewed public commit identified above.
Feature 0015 did not create a release; this deployment does not track an
unbounded branch or require a new coordinated release.
- The collection artifact does not itself install arbitrary Git-hosted role
dependencies. The operational manifest must therefore include the five
standalone roles as well as the collection and its native collection
dependencies.
- Remove the legacy `narratio_s3_*` host variables because their intended
values are supplied by the suite defaults. Replace
`narratio_default_campaign_id` with
`narratio_campaigns_default_campaign_id: dilfs`. The standalone role already
exposes this host-specific setting; no new pipeline variable is required.
- The initial real-host run is an operational acceptance exercise. It should
test observable pipeline behavior and convergence, not module task traces or
the current text of defaults.
The workspace [Ansible](../../ansible.md), [testing](../../testing.md), and
[planning](../../planning.md) policies apply. Feature 0015 has passed review
and supplies the required suite interface.
## Operational Prerequisites
Local repository preparation and syntax/dependency checks can proceed with the
workspace's existing Ansible virtual environment. Before live deployment,
establish the controller, target access, and Vault details below. Select the
session/audio before the end-to-end exercise, not before local implementation.
Record non-secret choices in the implementation plan or deployment runbook:
- Identify the operational Ansible controller and its deployment checkout.
The repository configuration currently selects `/etc/ansible/inventory`,
`/etc/ansible/roles`, and `/etc/ansible/collections`. Confirm the effective
paths so dependency installation and playbook execution use the same
inventory and installed content. Distinguish those paths from this
development workspace.
- The target must satisfy the roles' Ubuntu 26.04 and access requirements.
Normal connection, role guards, and privileged tasks can establish these;
separate platform or sudo probes are not required acceptance tests.
- Establish the controller's existing Vault password or identity mechanism
and operator availability to populate the four credential values in the
encrypted host file. Record how Vault is selected, never its password or
decrypted contents.
- Have the operator select the acceptance campaign, session identity, and
audio input, with the corresponding session file and accessible input
location. This is a real processing and publication exercise using the
selected providers; session selection remains an operator decision.
These are deployment inputs and operational checks, not additional production
tasks or a new automated preflight framework.
- Keep playbooks minimal and settings in inventory. Preserve `pipeline-dnd`
and its connection identity; do not rename it to `pipeline.dnd`.
- Preserve `dilfs` using the existing standalone Narratio variable. Remove
legacy `narratio_s3_*` and `narratio_default_campaign_id` host settings.
- Missing operational inputs are documented, not treated as implementation
blockers. No remote controller, SSH identity, Vault password, or session is
required for repository preparation.
- Local dependency installation and syntax/host-selection checks are permitted.
Existing passing evidence may be reused when relevant inputs are unchanged.
- `ansible-doc` rendering is not an acceptance gate. Missing role argument
descriptions are a separate documentation improvement.
## Simplest Viable Implementation
Use the reviewed collection revision and associated compatibility manifest to
update the controller's single root requirements file. Add the minimal playbook and one ordinary
host-variable file for the endpoint, operator, campaign, and credential
references, inheriting the suite's storage and publication defaults. Add one
fully encrypted host Vault file for the four credential values, populated by
the operator without exposing its plaintext during implementation or review.
Install or update dependencies from the root manifest at the confirmed
controller paths, explicitly refreshing the collection from its pinned commit
even when `0.1.0` is already installed. Confirm that Ansible
resolves `rakestraw.dnd.pipeline`, syntax-check the playbook, and converge only
`pipeline-dnd`. Start a fresh operator login after enrollment, validate the
native Narratio configuration under the intended runtime identity, rerun the
same play immediately to confirm clean convergence, and process one selected
session through the complete Feature 0015 publishing path. Record concise
operational instructions and results; do not add a second test harness around
the real deployment.
## Rationale and Alternatives
Keeping the playbook nearly declarative makes inventory the durable record of
what differs about this host and allows ordinary Ansible precedence to remain
the customization mechanism. Embedding endpoint, operator, storage, or secret
mapping variables in the playbook would make a one-host play harder to reuse
and would diverge from the established `rakestraw-ansible` style.
A local wrapper role could hide the collection interface, but it would create
another layer with no independent behavior. Applying the collection role by
FQCN proves the same consumer path that future hosts should use.
Tracking collection `main` would be convenient during development, but it
would make a later controller bootstrap select an unreviewed compatibility
set. An exact public commit retains development flexibility until the next
release while still making this deployment reproducible.
Feature 0015 now supplies the reviewed storage and publishing path needed for
meaningful end-to-end acceptance. Inheriting that policy keeps host inventory
focused on deployment choices and avoids maintaining a second copy of the
suite defaults.
Retain the already-committed minimal playbook, manifest, and public inventory.
Add adjacent comments for unresolved credentials, align the runbook with the
repository-only scope, perform appropriate local checks, and commit. No new
Ansible task, runtime test harness, or deployment automation is needed.
## Acceptance Criteria
- Feature 0015 is complete and independently reviewed before Feature 0016
implementation begins.
- `playbooks/pipeline/dnd.yml` targets only `pipeline-dnd`, applies only
`rakestraw.dnd.pipeline`, and contains no deployment variables or procedural
setup.
- Host inventory sets the confirmed WhisperX endpoint, enrolls only `eric`,
preserves `dilfs` as the default campaign through the existing standalone
role variable, and inherits Feature 0015's storage and publishing defaults.
- `dnd_pipeline_secrets` maps the required typed runtime keys to host Vault
variables. Credential values in the consumer repository are Vault-encrypted;
runtime delivery uses the suite's existing restricted secrets files.
- This feature introduces no plaintext credential values into commits, diffs,
command examples, validation output, or documentation. A repository-wide
historical secret audit is not an acceptance requirement.
- Legacy Narratio storage values are either represented through the current
pipeline interface or deliberately retired; no stale high-precedence host
variable can override the selected suite configuration accidentally.
- The root requirements manifest preserves unrelated existing dependencies
and installs the exact reviewed public collection snapshot, its five
standalone role dependencies, and its required native collections without
using local repository paths.
- The controller uses the selected collection commit even if collection
version `0.1.0` was already installed. The documented inventory, role, and
collection paths correspond to the operational controller.
- Public dependency installation succeeds, and the new playbook passes syntax
checking with the installed collection and roles. Documentation rendering
with `ansible-doc` is not required for deployment acceptance.
- The initial limited playbook run against `pipeline-dnd` completes
successfully using the encrypted inventory and ordinary sudo-capable remote
account.
- The immediate identical second run reports `changed=0` and `failed=0`.
- Narratio's native configuration validation succeeds under the intended
runtime identity, and a fresh session for `eric` can access and invoke the
shared pipeline through runtime-group membership.
- One operator-selected D&D session completes transcription, processing,
storage, and publishing, and its expected published output is observable at
the configured destination. Either supported runtime identity may execute
it. Production is the default profile; the operator may select the testing
profile while retaining the same processing and publication acceptance scope.
- The operational documentation explains dependency installation, Vault use,
deployment invocation, the login-session membership caveat, convergence
verification, and representative pipeline invocation without disclosing
credentials.
- The historical proof-of-concept role and playbook remain available during
the initial rollout; their later retirement requires a separate explicit
decision.
- The playbook targets only `pipeline-dnd` and applies only the collection role.
- Host inventory expresses the known choices above and documents unknown
credentials next to their references without real or fake secret values.
- The dependency manifest contains the reviewed public pins and preserves
unrelated dependencies. Public installation and local syntax/host-selection
evidence are available; no documentation-rendering check is required.
- The runbook distinguishes repository readiness from future deployment and
explains how the operator supplies missing values and eventually invokes it.
- Changes are committed in the owning repositories, with clean working trees
for the feature's changes. No controller deployment, target convergence,
clean-repeat run, or real pipeline execution is required or claimed.
- Historical role/playbook files remain available.
## Dependencies and Follow-up Work
[Feature 0015](0015-dnd-s3-storage-and-publishing.md) supplies the implemented
and independently reviewed pipeline interface, credentials, and publishing
boundary consumed here. Its administrative closeout remains before the
Feature 0016 sprint; its runtime implementation needs no further remediation.
Features 0013–0015 supply public dependencies, operator enrollment, and storage
policy. Feature 0016's consumer preparation already exists in `12b2462`, with
initial documentation in `716a80e`; remaining work aligns comments and docs
with the user's clarified scope.
Features 0013 and 0014 provide the public collection installation path and
existing-operator enrollment required here. Features 0009 and 0011 provide
the shared filesystem and managed credential-delivery policies.
After the real deployment demonstrates parity, a later cleanup feature may
retire the proof-of-concept `pipeline_dnd` role, its old playbook, and any
obsolete inventory values. Service scheduling, unattended ingestion,
additional human operators, backup policy, and additional hosts also remain
separate work.
The operator may later provision Vault values, install dependencies on their
controller, deploy to the host, and validate real processing/publication.
Those actions require a separate request and are not unfinished Feature 0016
acceptance requirements.

View File

@@ -1,6 +1,6 @@
# Implementation Plan 0016: Deploy the D&D Pipeline to `pipeline-dnd`
# Implementation Plan 0016: Prepare the D&D Deployment Playbook
Status: in_progress
Status: ready
Target executor: GPT-5.6-Terra
@@ -10,440 +10,169 @@ Active plan location: `docs/roadmap/implementation.md`
## Objective
Deploy the reviewed collection through a minimal consumer playbook, host
inventory, and Ansible Vault. Demonstrate clean repeated convergence and one
operator-selected session through real publication. The feature roadmap owns
scope, rationale, target state, and acceptance criteria.
Finish and commit repository preparation in `rakestraw-ansible`: minimal
playbook, host variables with comments for unknown values, reviewed dependency
pins, and concise operator instructions. This revision supersedes all previous
deployment and live-acceptance stages. Do not execute them.
## Execution Rules
- Execute stages in order. Before each stage, reread its Required Policy
Review and check the owning repository's working tree. Preserve unrelated
changes. Validate and commit each completed stage with its stated intent.
- Keep the playbook limited to hosts and roles. Use existing interfaces;
add no wrapper, preflight framework, custom module, or test harness.
- Inherit suite storage, publication, caching, cleanup, version, and asset
defaults. Set only the roadmap-selected host choices.
- Never print decrypted inventory, credential values, or Vault passwords in
commands, logs, documentation, or Git. Do not use decrypted inventory dumps
as validation evidence. The operator supplies secrets directly through
Ansible Vault's encrypted-file workflow, using an established secure editor.
Do not solicit secrets in chat or create placeholder credentials.
- Stop dependent work when operational inputs are missing or validation fails.
Record the concrete blocker; local checks do not prove a live deployment.
- Evaluate failures against the required outcome. Documentation rendering,
incidental warnings, and optional diagnostics are not deployment acceptance
gates. Report them separately unless they demonstrate a failure of required
installation, parsing, convergence, or application behavior. Do not add new
checks as mandatory conditions during execution.
- Reuse successful validation evidence while its inputs remain unchanged.
Rerun affected checks after changes; do not repeat entire stages solely to
produce another record or because a non-blocking diagnostic failed.
- Scope blockers to the operation that requires the missing input. Complete
Stages 1–2 locally without remote SSH, production Vault values, or an
acceptance session. Controller/SSH/Vault inputs gate Stage 3; session and
audio selection gate Stage 4. Continue independent authorized preparation.
- A missing executable on `PATH` is not proof that tooling is uninstalled.
Check the workspace `.venv/bin` and repository test drivers before reporting
a missing dependency. Use the verified executable path in diagnostics.
- An Ansible controller is the environment running Ansible; it requires no
controller daemon or `/etc/ansible` directory. Use explicit configuration,
inventory, role, and collection paths for local checks. Do not recreate the
operational controller layout or install system-wide tooling for that purpose.
- Retain the historical playbook and proof-of-concept role. Role, collection,
application, and asset repositories remain read-only during this feature.
- Deployment and paid provider execution occur only during the approved sprint
with the selected controller, target, credentials, and session. Do not
substitute an arbitrary session or broaden the host limit.
- Execute the stages below in order, rereading each stage's listed policies.
Check working trees first, preserve unrelated work, validate, and commit each
completed unit with its stated plain-English intent.
- Do not deploy to an operational controller or target, connect to the target,
populate actual credentials, or run a real pipeline session.
- Missing controller, SSH, Vault, and session inputs are not blockers. Express
unknown required values through commented instructions and variable references.
- Keep playbooks limited to hosts and roles. No new tasks, wrapper, probes,
runtime assertions, or automated test harness is required.
- Use existing workspace `.venv/bin` executables for local Ansible checks.
Core 2.21.3 is installed; a missing `PATH` entry or `/etc/ansible` directory
does not mean tooling is missing. Explicit local paths suffice.
- Reuse passing validation whose inputs are unchanged. Optional diagnostics
and `ansible-doc` documentation rendering are not acceptance gates.
- Keep credential values out of plaintext files, Git, commands, and logs.
Do not create fake keys or an empty Vault file to satisfy inventory references.
## Starting State and Preconditions
## Starting State and Repository Scope
- Feature 0015 passed independent review without findings. Its pipeline commit
is `51566afcf69083cdbb5b5718f2544c86b079001c`; the reviewed public collection
commit is `46913446765e86ecfb29ebad9955d579ef6a4c7e`.
- `rakestraw-ansible` starts clean on `main` at
`d86ec29a9cbd821fead248736171b5cdd8440ad6`. Inspect and preserve any subsequent
changes before adapting this plan.
- Root `requirements.yml` already has an unpinned Gitea `rakestraw.dnd` entry
alongside unrelated collections. The historical playbook is
`playbooks/apps/dnd.yml`; it invokes `pipeline_dnd` on `automation-dnd`.
- Host `pipeline-dnd` already connects as `eric` to
`pipeline-dnd.rakestrawhome.com`. Preserve its connection settings.
- `inventory/host_vars/pipeline-dnd/narratio.yml` contains the legacy campaign
and S3 variables to migrate.
- The selected collection still declares version `0.1.0`. An installed
release with that number is not sufficient evidence of the selected content.
- This plan is approved, with this revision permitting local preparation before
operational inputs are resolved. Do not invent those inputs; resolve them
at the stage that needs them.
- Local tooling is already installed and verified: Ansible Core 2.21.3 in
`/home/eric/Workspace/ansible-dnd/.venv`. Use its `bin/ansible`,
`bin/ansible-playbook` and `bin/ansible-galaxy`, or prepend
that `bin` directory to `PATH`. No activation is required for absolute paths.
- Feature 0015 is complete and independently reviewed.
- Consumer `716a80e` added the initial runbook; `12b2462` committed the new
playbook, host configuration, and dependencies. Inspect intervening commits;
do not repeat or discard completed work.
- Public dependency installation, target-only host listing, and syntax checking
passed for the prepared consumer configuration. The missing option descriptions
in three roles affect `ansible-doc` only and are not part of this feature.
- `rakestraw-ansible` owns remaining comments/runbook changes. The workspace
owns status and review handoff. All other repositories remain read-only.
- No credentials, controller access, target access, or session selection is
needed to complete this plan.
## Repository Scope
- `rakestraw-ansible`: requirements, playbook, host configuration, encrypted
credentials, and `docs/pipeline-dnd.md` runbook linked from README.
- Workspace: feature status, implementation progress, and review handoff.
- Other repositories: read-only compatibility and application references.
## Stage 1: Establish Local Preparation and Record Deployment Inputs
Completion: complete (consumer `716a80ef68208537dacfa5f0f22259bf42ae6279`)
## Stage 1: Finish Consumer Comments and Usage Documentation
### Objective and Owning Repository
Record known and pending deployment inputs in `rakestraw-ansible` and close the reviewed
Feature 0015 in the workspace.
Make the existing committed consumer configuration self-explanatory in
`rakestraw-ansible` and remove live execution as a completion requirement.
### Required Policy Review
Read `AGENTS.md`, `docs/orientation.md`, `docs/conventions.md`,
`docs/ansible.md`, `docs/testing.md`, `docs/planning.md`, the Feature 0016
roadmap, and `rakestraw-ansible/README.md` before this stage.
Before this stage, read `AGENTS.md`, `docs/orientation.md`,
`docs/conventions.md`, `docs/ansible.md`, `docs/testing.md`,
`docs/planning.md`, the Feature 0016 roadmap, and the consumer README/runbook.
### Preconditions
The plan is approved and repositories have been checked. Local tooling is
available; no remote access or credential/session input is required.
Inspect the current consumer changes and the two completed commits above.
Local Ansible tools are available. There are no remote prerequisites.
### Required Changes
1. Create `rakestraw-ansible/docs/pipeline-dnd.md` and link it from README.
Record the local validation setup and known operational details. List the
operational controller/checkout, effective paths, Vault selection, and
acceptance session as pending where unknown, with their consuming stage.
Do not fabricate completed checks or turn missing values into blockers here.
2. Confirm local Ansible Core is at least 2.18 and the public suite
repositories and Galaxy are reachable. Inspect effective configuration
without dumping decrypted host variables. Existing repository defaults are
`/etc/ansible/inventory`, `/etc/ansible/roles`, and `/etc/ansible/collections`;
document these as operational defaults, not required local directories.
3. Record the reported target SSH rejection as a Stage 3 prerequisite. Defer
target Ubuntu and sudo checks until authorized access is available. Repeated
attempts with the same rejected identity do not advance local preparation.
4. Mark Feature 0015 `done` in its roadmap and workspace status, citing its
reviewed commits and the successful independent standalone/collection
integration runs. Leave historical remediation records unchanged. Mark
Feature 0016 `in_progress` when execution begins.
### Validation and Completion Criteria
Local tooling works and the runbook distinguishes known facts from pending
operational inputs, assigning each to Stage 3 or Stage 4.
Run `git diff --check` in both changed repositories. No deployment or provider
request is necessary for this stage.
### Commit
Consumer: `Document the D&D deployment context`.
Workspace: `Close storage review and begin host deployment`.
## Stage 2: Prepare the Consumer Playbook and Inventory
Completion: complete (consumer `12b2462`)
### Objective and Owning Repository
Make `rakestraw-ansible` a complete consumer of the reviewed suite snapshot.
### Required Policy Review
Read `AGENTS.md`, `docs/orientation.md`, `docs/conventions.md`,
`docs/ansible.md`, `docs/testing.md`, `docs/planning.md`, the Feature 0016
roadmap, `ansible-collection-dnd/README.md`, its `requirements.yml`, and the
Stage 1 runbook before this stage.
### Preconditions
Stage 1 is complete. Production credentials and remote access are not required.
### Required Changes
1. Add `playbooks/pipeline/dnd.yml` in the roadmap's exact minimal shape:
one play targeting `pipeline-dnd` with only `rakestraw.dnd.pipeline` in its
roles list. No play variables, tasks, or play-level privilege escalation.
2. Replace the legacy contents of
`inventory/host_vars/pipeline-dnd/narratio.yml` with the roadmap's public
inventory example: operator `eric`, the confirmed WhisperX URL,
`narratio_campaigns_default_campaign_id: dilfs`, and the four Vault-backed
`dnd_pipeline_secrets` mappings. Remove obsolete `narratio_s3_*` and
`narratio_default_campaign_id` settings. Preserve unrelated intervening edits
and the separate connection file. Do not copy suite defaults into inventory.
3. Document `inventory/host_vars/pipeline-dnd/vault.yml` and its four required
`vault_dnd_*` names. Keep the inventory references unresolved until the
operator provisions that encrypted file in Stage 3. Do not create an empty
Vault file or substitute credentials to make local checks appear complete.
4. Pin the existing `rakestraw.dnd` Git entry in root `requirements.yml` to
`46913446765e86ecfb29ebad9955d579ef6a4c7e` with its existing public URL.
5. Copy the five `roles` entries from that collection's compatibility manifest,
retaining role names, public Gitea URLs, `scm: git`, and `v0.1.0` tags.
Reconcile native collection entries to `ansible.posix: 2.2.0` and
`community.general: 13.3.0`, without duplicates. Preserve unrelated
dependencies; report demonstrated conflicts rather than silently changing
the reviewed suite pins.
6. Extend the runbook with the concrete commands for dependency installation,
collection refresh, Vault selection, deployment, fresh login, and native
validation used below. Link upstream Narratio v2.1.1 operator documentation.
1. Retain `playbooks/pipeline/dnd.yml`, which contains only the `pipeline-dnd`
host and `rakestraw.dnd.pipeline` role. Retain reviewed dependency pins and
known host values. Compare against the revised roadmap; correct only actual
discrepancies, without adding suite-default copies or extra play content.
2. In `inventory/host_vars/pipeline-dnd/narratio.yml`, add the roadmap's comments
beside `dnd_pipeline_secrets`: define the four referenced `vault_dnd_*`
variables in an encrypted `vault.yml` before eventual deployment. Retain
unresolved references. Do not obtain secret values or create a Vault file.
3. Revise `docs/pipeline-dnd.md` to describe repository preparation as the
completed deliverable. Clearly label installation, credential provisioning,
and deployment commands as future operator instructions, not steps the
implementation agent must execute. Remove operational prerequisite checklists
and session-acceptance requirements from this feature's completion path.
4. Retain useful concise future-use instructions: the pinned collection refresh,
Vault variable names and selection options, and minimal playbook invocation.
Link application documentation for later pipeline operation rather than
maintaining a live-session test script or demanding session selection now.
5. Preserve connection inventory, unrelated dependencies, and the historical
proof-of-concept playbook/role.
### Validation
Parse edited plaintext YAML and run `git diff --check`. In isolated controller
dependency paths, install from public sources with
`ansible-galaxy install -r requirements.yml`. Configure role and collection
destinations through Ansible configuration/environment; avoid a role-only
custom path that would skip the collection section of the mixed manifest.
Successful installation and playbook syntax checking are the dependency
validation boundary. `ansible-doc` renders documentation and is not required.
Run `git diff --check` in `rakestraw-ansible` and review the changed documentation
and comments. Reuse the recorded successful public installation, syntax, and
host-listing results if only comments/documentation change. No runtime tests
or repeated dependency installation are necessary for those edits.
For local checks, use the workspace executables and the checked-out inventory.
Configure isolated role and collection paths explicitly. From the consumer
checkout, run:
If executable configuration changes, rerun only affected local checks using
the workspace tools and explicit local inventory/dependency paths:
```text
ansible-playbook -i inventory playbooks/pipeline/dnd.yml --limit pipeline-dnd --list-hosts
ansible-playbook -i inventory playbooks/pipeline/dnd.yml --limit pipeline-dnd --syntax-check
```
Host listing and syntax checking do not deploy or establish that credentials
resolve. Run them without evaluating or printing secret values; do not supply
fake Vault variables. If an existing unrelated encrypted inventory file needs
an unavailable password, preserve it and use a disposable local inventory with
only the target alias and its non-secret host variables for syntax checking.
Report that narrower scope explicitly; full encrypted-inventory validation
belongs to Stage 3. Do not change operational inventory to bypass encryption.
Do not impose
whole-repository lint changes on the established minimal-playbook style.
Use previously installed isolated public dependencies when available. If needed,
install from `requirements.yml` into disposable local role/collection paths.
Do not use operational `/etc/ansible` destinations or local sibling role
sources. Keep unresolved credentials untouched; local syntax checking does
not claim they resolve. If unrelated encrypted inventory prevents a local
check, use a disposable inventory containing the target alias and non-secret
variables, and report that narrower validation scope.
### Completion Criteria
Only the intended host is selected. Public dependencies resolve without local
sibling checkouts, local syntax checking succeeds within its recorded scope, and the
diff contains no plaintext secrets.
The required public files exist, unknown values have explanatory comments,
local validation evidence is sufficient, and the consumer changes are committed.
No deployment or real credential provisioning is required.
### Commit
`Add the D&D pipeline deployment playbook`.
`Document the remaining D&D deployment inputs`.
## Stage 3: Deploy and Verify Clean Convergence
## Stage 2: Record the Repository Preparation Handoff
### Objective and Owning Repository
Deploy from the operational controller; record results in the consumer runbook.
Record Feature 0016 implementation readiness for independent review in the
workspace, scoped entirely to committed repository content.
### Required Policy Review
Read `AGENTS.md`, `docs/orientation.md`, `docs/conventions.md`,
`docs/ansible.md`, `docs/testing.md`, `docs/planning.md`, the Feature 0016
roadmap, and `rakestraw-ansible/docs/pipeline-dnd.md` before this stage.
Before this stage, read `AGENTS.md`, `docs/orientation.md`,
`docs/conventions.md`, `docs/testing.md`, `docs/planning.md`, and the revised
Feature 0016 roadmap.
### Preconditions
Stage 2 passed. The operational controller/checkout and delivery workflow are
identified, and target SSH access and the Vault mechanism are available.
Deliver committed changes through that workflow without overwriting a dirty
checkout. Acceptance-session selection is not a prerequisite for this stage.
Stage 1 has passed local validation and is committed. No remote prerequisites.
### Required Changes and Validation
### Required Changes
Before installation, confirm the operational controller's Ansible version and
effective inventory, role, and collection paths. The target must support the
roles' Ubuntu 26.04 contract and required sudo operations; ordinary connection,
fact gathering, role platform guards, and privileged tasks establish these
conditions. Separate preflight probes are optional troubleshooting, not gates.
Have the operator create `inventory/host_vars/pipeline-dnd/vault.yml` encrypted
in full, defining the four roadmap credential variables. Commit ciphertext
with intent `Add encrypted D&D host credentials` after checking the diff
exposes no plaintext. Rerun host listing and syntax validation with the real
encrypted inventory and recorded Vault options before convergence.
1. Update `docs/status.md` and this plan's progress notes with the consumer
commit and local validation evidence. Describe real deployment as separately
deferred operator work, not a blocker or incomplete acceptance criterion.
2. Keep Feature 0016 `in_progress` pending independent review. Report any actual
repository issues; do not mark it `done` solely because implementation ended.
3. Confirm the feature's owning working trees are clean after committing.
Preserve any unrelated changes and identify them instead of discarding them.
1. Install root requirements at the confirmed controller paths. Explicitly
refresh only the D&D collection from its selected commit, even when `0.1.0`
is already installed:
### Validation and Completion Criteria
```text
ansible-galaxy collection install git+https://gitea.maximumdirect.net/eric/ansible-collection-dnd.git,46913446765e86ecfb29ebad9955d579ef6a4c7e --force --no-deps
```
Use the configured collection path; dependencies come from the manifest.
Do not force-update unrelated unpinned collections. Confirm role resolution
and playbook syntax on this operational controller as in Stage 2.
2. Run `ansible-playbook playbooks/pipeline/dnd.yml --limit pipeline-dnd`
using the recorded inventory, Vault, and any required sudo-password options.
Immediately repeat the identical command after success. Require
`changed=0`, `failed=0`, and no unreachable hosts on the repeat.
3. On the target, execute
`sudo -u narratio /usr/local/bin/narratio config validate`.
Testing-overlay validation is optional; it has already passed suite review
and is not changed by this deployment. The managed default campaign should make an
explicit campaign selector unnecessary.
4. Start a fresh login as `eric` and execute the same default native validation
without switching user, demonstrating usable supplementary group access.
5. Record consumer commit, target, date, repeat recap, and native validation
results. Diagnose failures at the owning boundary; report required changes
to roles/applications separately rather than expanding this feature.
### Completion Criteria
The actual host converges cleanly and both runtime identities validate the
configuration. Run `git diff --check` on the evidence/runbook changes.
Run `git diff --check`. Evidence points to committed consumer content and
appropriate local checks. The handoff makes no claims of live deployment or
runtime acceptance. No paid request or host access is necessary.
### Commit
`Record successful D&D host deployment`.
## Stage 4: Exercise One Real Session
### Objective and Owning Repository
Demonstrate production processing and publication; record acceptance in
`rakestraw-ansible/docs/pipeline-dnd.md`.
### Required Policy Review
Read `AGENTS.md`, `docs/orientation.md`, `docs/conventions.md`,
`docs/testing.md`, `docs/planning.md`, the Feature 0016 roadmap, the runbook,
and Narratio v2.1.1's `docs/config.md`, `docs/cli.md`, and
`docs/operations.md` before this stage. Follow the sibling application's
`AGENTS.md` and development reading guide when consulting its repository.
### Preconditions
Stage 3 passed. Actual credentials, the operator-selected campaign/session,
session file, and audio are accessible. The exercise must demonstrate real
transcription and publication, not merely resume an already complete run.
Do not erase existing state to create a fixture.
### Required Changes and Validation
1. Under either supported runtime identity (`narratio` or `eric` with effective
runtime-group membership), run:
```text
narratio session validate SESSION_ID --campaign CAMPAIGN --session SESSION_FILE
narratio run SESSION_ID --campaign CAMPAIGN --session SESSION_FILE
narratio session artifacts SESSION_ID --remote --campaign CAMPAIGN --session SESSION_FILE
```
Substitute the operator-recorded values. Production is the default; an
operator-selected testing profile is also valid if the exercise covers the
agreed processing and publication path. Record the selected profile.
The session document remains operational input; no session-management task
belongs in the playbook. Observe transcription, processing, and successful
publication. Do not force reprocessing or reduce the selected artifact set
merely to bypass a failure.
2. Observe a representative published transcript or recap at the configured
destination through the existing storage/operator interface. Successful
cleanup may remove local workspace content; local survival is not required.
3. Record identity, input location, runtime user, completion outcome, and
non-secret publication evidence. Avoid committing raw provider logs or
credential-bearing URLs. Do not repeat paid work just to duplicate evidence
or expand acceptance into a restore/locking test matrix.
### Completion Criteria
A real session completes and its published output is observable. Record any
provider/application blocker and leave acceptance incomplete if it cannot
finish. Run `git diff --check` on the runbook changes.
### Commit
`Record the first published D&D session`.
## Stage 5: Prepare the Independent Review Handoff
### Objective and Owning Repository
Record completed implementation evidence in the workspace.
### Required Policy Review
Read `AGENTS.md`, `docs/orientation.md`, `docs/conventions.md`,
`docs/testing.md`, `docs/planning.md`, and the Feature 0016 roadmap before
this stage.
### Preconditions
Record progress at any stopping point, including a live-operation blocker.
Full implementation handoff requires Stages 1–4 and actual deployment/session
evidence; a partial handoff must explicitly identify uncompleted stages.
### Required Changes and Validation
Update `docs/status.md` and this plan's progress notes with owning commits,
runbook link, validation outcomes, and deviations. Keep Feature 0016
`in_progress` pending independent review. Report any additional complexity
and its authorizing requirement. Run `git diff --check` and confirm changed
repositories are clean after committing. Do not rerun role integration
matrices or paid sessions without a new change or failure requiring it.
### Completion Criteria
Evidence is recorded and implementation is ready for independent review.
### Commit
`Record the D&D deployment implementation`.
`Record D&D playbook preparation for review`.
## Final Validation and Handoff
Report dependency revision, consumer commits, controller/target, public
dependency and syntax results, immediate clean convergence, native validation
for both identities, and session publication evidence. Distinguish completed
work from operator-dependent blockers. Do not mark Feature 0016 `done` before
independent review.
Report owning commits, changed consumer files, comments explaining missing
values, and local validation performed or reused. Feature review evaluates the
committed playbook/configuration, not operational deployment. No push is
required unless separately requested.
## Progress Notes
- Stage 1 is complete in consumer commit
`716a80ef68208537dacfa5f0f22259bf42ae6279`. The linked deployment runbook
records the Ansible Core 2.21.3 local toolchain, the effective repository
defaults, successful public-source reachability, and the inputs pending for
live deployment and session acceptance.
- Feature 0015 is closed after independent review found no remediation issue.
Its reviewed pipeline commit is
`51566afcf69083cdbb5b5718f2544c86b079001c`; collection fixture, squash, and
merge commits are `cf3de016ee09edf91afc3bc1addfd926d4a172ad`,
`77f67adbc0f700e88bc15554e02052eaa22f1033`, and
`46913446765e86ecfb29ebad9955d579ef6a4c7e`.
- Stage 2 local consumer preparation is complete. The rejected target SSH identity,
operational controller details, and Vault workflow/values remain Stage 3
prerequisites. The acceptance session/audio selection remains a Stage 4
prerequisite. No live deployment has begun.
- Stage 2 consumer changes are committed in `12b2462`. Installation from the
public manifest, host selection, and syntax checks passed. The former
`ansible-doc` gate was unnecessary and has been removed: missing argument
descriptions prevent documentation rendering, not the demonstrated role
loading. The prepared consumer diff was reviewed and whitespace checks passed
before committing. Resume at Stage 3 when its operational inputs are
available. No dependency pin change is needed.
## Non-blocking Follow-up
The pinned Notarius, Scriptorium, and Narratio `v0.1.0` roles lack option
descriptions required by `ansible-doc` documentation rendering. Consider adding
descriptions in a separate role documentation change and normal later release.
This is not a Feature 0016 acceptance criterion or a reason to change its pins.
The earlier consumer preparation is committed in `716a80e` and `12b2462`.
This revised two-stage plan finishes comments and documentation, then records
handoff. Earlier live-deployment stages and their open questions are withdrawn
from Feature 0016. No live deployment has been performed.
## Open Questions
Questions 1–2 gate Stage 3 only. Question 3 gates Stage 4 only. None prevents
local preparation in Stages 1–2.
1. Which controller/checkout executes deployment and how do committed changes
reach it? Recommended: use the existing operational controller and its
established `/etc/ansible/` layout, keeping one operational inventory.
Alternative: a dedicated checkout with explicit inventory and dependency
paths, if the operator prefers isolation and records those paths consistently.
2. Which Vault identity/password mechanism is available, and can the operator
populate all four values? Recommended: reuse the controller's established
Vault workflow and populate the encrypted file directly. Alternative: an
interactive Vault password prompt for this file if no identity is established.
Both keep passwords outside Git; neither requires secrets in chat.
3. Which campaign/session/audio supplies acceptance? Recommended: an
operator-selected unprocessed `dilfs` session with accessible audio, matching
the retained default campaign and avoiding forced reprocessing. Alternative:
a selected session from another deployed campaign with an explicit selector.
Record the session file and audio location before execution; do not invent
sample production input.
None required for this repository-only implementation. Unknown API keys and
future operational inputs are documented for the operator, not requested now.

View File

@@ -4,10 +4,10 @@ Last updated: 2026-09-21
Current milestone: Operational D&D pipeline deployment
Current focus: Resume Feature 0016 at Stage 3 once controller, SSH, and Vault
inputs are available. Stage 2 is committed in consumer `12b2462`. The
unnecessary `ansible-doc` gate has been removed; role documentation omissions
are a separate non-blocking follow-up.
Current focus: Finish Feature 0016 repository preparation under the revised
two-stage plan: comments for unknown host values and concise operator docs,
then commit and hand off for review. Controller deployment, target convergence,
and session execution are outside this feature; no operational input blocks it.
Feature 0015 is complete after independent review found no remediation issue.
| ID | Deliverable | Owner | State | Evidence or next step |
@@ -36,7 +36,7 @@ Feature 0015 is complete after independent review found no remediation issue.
| F0013 | Publish the first coordinated suite release | all Ansible repositories and workspace | done | Public [`rakestraw.dnd` 0.1.0 release](https://gitea.maximumdirect.net/eric/ansible-collection-dnd/releases/tag/v0.1.0); documentation remediation completed in pipeline `78afffe` and collection `5e695f3` |
| F0014 | Enroll existing pipeline operators | ansible-role-dnd-pipeline, ansible-collection-dnd, and workspace | done | Pipeline `c13f726` and collection import `c0b0ebd` passed their complete integration suites and independent review with no remediation findings |
| F0015 | Enable D&D S3 storage and publishing | ansible-role-dnd-pipeline, ansible-collection-dnd, and workspace | done | Pipeline `51566afcf69083cdbb5b5718f2544c86b079001c`, collection fixture `cf3de016ee09edf91afc3bc1addfd926d4a172ad`, collection squash `77f67adbc0f700e88bc15554e02052eaa22f1033`, and collection merge `46913446765e86ecfb29ebad9955d579ef6a4c7e` passed the complete repository suites and independent review with no remediation findings |
| F0016 | Deploy the D&D pipeline to `pipeline-dnd` | rakestraw-ansible and workspace | in_progress | Stage 1: consumer `716a80e`; Stage 2: consumer `12b2462`, with public installation, host selection, and syntax checks passed; resume Stage 3 when operational inputs are available |
| F0016 | Prepare the D&D deployment playbook for `pipeline-dnd` | rakestraw-ansible and workspace | in_progress | Consumer preparation is in `716a80e` and `12b2462`; revised [plan](roadmap/implementation.md) finishes comments/docs and repository handoff only; previous live-deployment stages withdrawn |
| A0001 | Audit completed standalone-role test suites | workspace and standalone roles | done | Findings recorded in `docs/roadmap/test-suite-audit-findings.md` |
| R0001 | Simplify completed standalone-role test suites | workspace and standalone roles | done | Role commits `e08c63c`, `d2d0b9d`, and `cfecf37` passed implementation and independent review |