Limit the D&D feature to playbook preparation

docs/roadmap/features/0016-deploy-dnd-pipeline-to-pipeline-dnd.md

@@ -1,88 +1,41 @@
-# Feature 0016: Deploy the D&D Pipeline to `pipeline-dnd`
+# Feature 0016: Prepare the D&D Deployment Playbook for `pipeline-dnd`
 
 Status: in_progress
 
 ## Summary
 
-Add a minimal playbook and host inventory to `rakestraw-ansible` that apply
-`rakestraw.dnd.pipeline` to the existing `pipeline-dnd` host. Configure the
-host through `host_vars`, enroll `eric` as the initial human operator, keep
-credential values in Ansible Vault, and prove the complete pipeline through a
-real repeatable deployment and representative D&D session.
-
-[Feature 0015](0015-dnd-s3-storage-and-publishing.md) has implemented and passed
-independent review of the storage and publishing behavior needed for an
-operational end-to-end pipeline. Feature 0016 consumes that reviewed interface
-and its suite defaults.
-
-## Problem or Opportunity
-
-The collection and its application roles are implemented and installable, but
-the operational Ansible repository does not yet have a playbook for deploying
-the new collection architecture. Its existing D&D playbook targets a retired
-inventory name and invokes the proof-of-concept local `pipeline_dnd` role.
-
-The current `pipeline-dnd` host inventory already records the connection
-identity and useful legacy Narratio storage values. A production-shaped first
-deployment now needs to express the new role's public interface, deliver
-secrets safely, reconcile the controller's dependency manifest, and validate
-the actual host rather than extending the role repositories with another
-synthetic scenario.
+Write and commit the consumer playbook, necessary host variables, dependency
+manifest, and concise usage documentation in `rakestraw-ansible`. This feature
+ends with repository preparation. Deployment to a controller or target host
+and execution of the pipeline are separate operator work.
 
 ## Scope
 
-- Add `rakestraw-ansible/playbooks/pipeline/dnd.yml`, targeting the existing
-  `pipeline-dnd` inventory host and applying only
-  `rakestraw.dnd.pipeline`.
-- Keep the playbook limited to its host selection and role list. Put all
-  deployment choices in `inventory/host_vars/pipeline-dnd/`.
-- Configure the external WhisperX transcription endpoint as
-  `https://transcription.ai.rakestrawhome.com/transcribe`.
-- Enroll the existing `eric` account through
-  `dnd_pipeline_operator_users`.
-- Map the role's `dnd_pipeline_secrets` interface to host-specific Vault
-  variables. Store credential values only in an Ansible Vault-encrypted host
-  variables file.
-- Inherit Feature 0015's storage and publishing defaults, which already match
-  the intended Backblaze B2 deployment. Host inventory contains only actual
-  host choices and deliberate overrides, without copying suite defaults.
-- Preserve the existing default campaign choice, `dilfs`, through the
-  standalone role's `narratio_campaigns_default_campaign_id` variable.
-- Reconcile existing legacy Narratio host variables so they cannot silently
-  override the suite policy selected by the new pipeline role.
-- Update `rakestraw-ansible/requirements.yml` so a clean controller can install
-  the reviewed collection snapshot, all five standalone role dependencies,
-  and required native collections from one manifest.
-- Validate controller dependency resolution, playbook syntax, initial
-  convergence, immediate clean convergence, operator access, native Narratio
-  configuration, and one representative end-to-end pipeline run on the real
-  host.
-- Document the operational invocation, Vault prerequisite, initial operator,
-  and expected validation path in `rakestraw-ansible`.
+- `playbooks/pipeline/dnd.yml` targets the existing `pipeline-dnd` alias and
+  applies `rakestraw.dnd.pipeline`, using only hosts and roles.
+- Host choices belong in `inventory/host_vars/pipeline-dnd/`: the WhisperX
+  endpoint, initial operator `eric`, default campaign `dilfs`, and credential
+  references. Unknown required values are explained in adjacent comments.
+- Inherit the reviewed suite's storage, publication, caching, cleanup, and
+  application defaults. Retire obsolete legacy host variables.
+- Root `requirements.yml` selects the reviewed public collection snapshot and
+  its role/native collection dependencies, preserving unrelated entries.
+- Document local validation and eventual operator use. Commit the completed
+  repository changes without requiring remote access or real credentials.
 
 ## Non-goals
 
-- Implementing storage or publishing behavior. Feature 0015 owns that
-  prerequisite.
-- Changing application or role behavior merely to accommodate this
-  deployment.
-- Replacing the existing inventory alias with `pipeline.dnd`, changing DNS,
-  or moving connection variables out of the current host inventory.
-- Adding a wrapper role, pre-task layer, service, timer, scheduler, web UI, or
-  unattended session-ingestion mechanism.
-- Creating or managing the `eric` account, its login policy, SSH access, or
-  sudo policy.
-- Adding another production host, high availability, backup policy, or
-  disaster-recovery automation.
-- Removing the proof-of-concept local `pipeline_dnd` role or its historical
-  playbook before the new deployment has demonstrated operational parity.
-- Publishing a new coordinated suite release solely as part of this
-  deployment feature.
+- Installing or updating anything on the operational controller or target.
+- Connecting to `pipeline-dnd`, validating its platform or sudo access,
+  converging the host, checking runtime identities, or executing a real session.
+- Obtaining API keys, choosing a Vault password, populating real credentials,
+  or selecting acceptance audio/session data.
+- Changing application/role behavior, publishing a new suite release, adding
+  orchestration layers, or retiring the proof-of-concept role/playbook.
 
 ## Target End State
 
-`rakestraw-ansible/playbooks/pipeline/dnd.yml` is an intentionally small
-consumer playbook equivalent to:
+The committed playbook is:
 
 ```yaml
 ---
@@ -93,12 +46,8 @@ consumer playbook equivalent to:
 ...
 ```
 
-The existing inventory selects the host's connection identity. A dedicated
-host-variable file selects the WhisperX endpoint, the initial operator list,
-and the default campaign. Storage, caching, publication, and cleanup inherit
-the reviewed suite defaults. The file maps runtime credential names to
-Vault-backed variables without embedding credential values in the playbook or
-ordinary plaintext inventory, for example:
+Existing connection variables remain in the host's `main.yml`. Its
+`narratio.yml` contains the following public configuration and comments:
 
 ```yaml
 dnd_pipeline_operator_users:
@@ -109,6 +58,10 @@ dnd_pipeline_whisperx_transcribe_url: >-
 
 narratio_campaigns_default_campaign_id: dilfs
 
+# Before deployment, define the four vault_dnd_* variables below in an
+# Ansible Vault-encrypted host_vars/pipeline-dnd/vault.yml file.
+# Values are not known here; leave these references unresolved until supplied.
+# Do not put API keys or object-storage credentials in this plaintext file.
 dnd_pipeline_secrets:
   AUDITA_LLM_API_KEY: "{{ vault_dnd_audita_llm_api_key }}"
   OPENROUTER_API_KEY: "{{ vault_dnd_openrouter_api_key }}"
@@ -116,214 +69,77 @@ dnd_pipeline_secrets:
   OBJECT_STORAGE_KEY: "{{ vault_dnd_object_storage_key }}"
 ```
 
-The corresponding credential variables reside in a tracked file encrypted in
-full with Ansible Vault. The Vault password or identity mechanism remains
-outside Git and outside this feature's managed content.
+The references and comments are a complete deliverable for unknown secrets.
+No encrypted file, placeholder key, or real credential is required for feature
+completion. Future operator-supplied values should be Vault-encrypted in Git;
+the suite's existing restricted runtime secrets files remain the delivery path.
 
-The controller dependency manifest selects public collection commit
-`46913446765e86ecfb29ebad9955d579ef6a4c7e`, containing the reviewed Features
-0014 and 0015 and pipeline commit
-`51566afcf69083cdbb5b5718f2544c86b079001c`. Its compatibility manifest selects
-the five standalone roles at `v0.1.0`, `ansible.posix` at `2.2.0`, and
-`community.general` at `13.3.0`. A clean controller therefore does not depend
-on sibling development checkouts or previously installed roles.
+The manifest pins the collection to
+`46913446765e86ecfb29ebad9955d579ef6a4c7e`, containing reviewed pipeline commit
+`51566afcf69083cdbb5b5718f2544c86b079001c`. Its five standalone role dependencies
+use their reviewed `v0.1.0` tags, with `ansible.posix` `2.2.0` and
+`community.general` `13.3.0`. Suite Git dependencies use public Gitea URLs;
+native collections come from Galaxy. Local sibling checkouts are not dependencies.
 
-The published collection `v0.1.0` predates these features. The selected
-development snapshot still declares collection version `0.1.0`, so the
-deployment instructions explicitly refresh an already-installed collection
-from the pinned commit instead of allowing an equal-version installation to
-be skipped. Unrelated controller dependencies remain preserved.
-
-On `pipeline-dnd`, the role converges the full application suite and its D&D
-configuration. The `narratio` service identity can run the pipeline, and
-`eric` can run it after starting a login session that observes the granted
-supplementary group membership. A representative session can be transcribed,
-processed, stored, and published through the configured production path. An
-immediate identical Ansible run reports no changes.
+The runbook explains that the selected collection snapshot still reports
+`0.1.0`, so eventual operator installation must explicitly refresh it from
+the pinned commit if an older `0.1.0` is installed. This is an instruction
+for future use, not an operation required by this feature.
 
 ## Repository Ownership
 
-- `rakestraw-ansible` owns the consumer playbook, host inventory, encrypted
-  host credentials, controller dependency manifest, and deployment runbook.
-- `ansible-role-dnd-pipeline` owns the suite's public variables and the
-  opinionated cross-application behavior consumed by the deployment.
-- `ansible-collection-dnd` owns the assembled `rakestraw.dnd.pipeline` role
-  snapshot and consumer-facing compatibility manifest.
-- The standalone application roles continue to own installation and native
-  application configuration. Feature 0016 consumes their reviewed releases or
-  commits without reimplementing their behavior.
-- `ansible-workspace-dnd` owns this roadmap, the later implementation plan,
-  progress tracking, and independent review record.
-- Application and asset repositories are read-only sources of runtime
-  contracts and deployed content for this feature.
+- `rakestraw-ansible`: consumer files, comments, dependency manifest, and runbook.
+- Workspace: roadmap, staged plan, progress, and independent review record.
+- Roles, collection, applications, and assets: read-only contract references.
 
 ## Constraints and Settled Decisions
 
-- Feature 0015's implementation and independent review satisfy the storage
-  and publishing prerequisite. Its workspace completion record should be
-  closed out before the Feature 0016 sprint.
-- The target is the inventory alias `pipeline-dnd` as currently defined in
-  `rakestraw-ansible`, not a new `pipeline.dnd` alias.
-- The playbook follows the repository's established style: hosts and roles
-  only. Facts remain at Ansible's normal default, privilege escalation remains
-  task-owned by the roles, and deployment variables belong in host inventory.
-- `eric` is the initial human operator and must already exist on the managed
-  host. The pipeline role grants only additive runtime-group membership.
-- API credentials use the existing typed `dnd_pipeline_secrets` interface and
-  Ansible Vault. Plaintext secret values must not enter Git, command lines,
-  logs, examples, or ordinary variable files.
-- Public Gitea repositories supply the suite's Git dependencies; native
-  collections come from Ansible Galaxy. The deployment must not reference
-  this workspace's sibling checkout paths.
-- The collection is pinned to the reviewed public commit identified above.
-  Feature 0015 did not create a release; this deployment does not track an
-  unbounded branch or require a new coordinated release.
-- The collection artifact does not itself install arbitrary Git-hosted role
-  dependencies. The operational manifest must therefore include the five
-  standalone roles as well as the collection and its native collection
-  dependencies.
-- Remove the legacy `narratio_s3_*` host variables because their intended
-  values are supplied by the suite defaults. Replace
-  `narratio_default_campaign_id` with
-  `narratio_campaigns_default_campaign_id: dilfs`. The standalone role already
-  exposes this host-specific setting; no new pipeline variable is required.
-- The initial real-host run is an operational acceptance exercise. It should
-  test observable pipeline behavior and convergence, not module task traces or
-  the current text of defaults.
+The workspace [Ansible](../../ansible.md), [testing](../../testing.md), and
+[planning](../../planning.md) policies apply. Feature 0015 has passed review
+and supplies the required suite interface.
 
-## Operational Prerequisites
-
-Local repository preparation and syntax/dependency checks can proceed with the
-workspace's existing Ansible virtual environment. Before live deployment,
-establish the controller, target access, and Vault details below. Select the
-session/audio before the end-to-end exercise, not before local implementation.
-Record non-secret choices in the implementation plan or deployment runbook:
-
-- Identify the operational Ansible controller and its deployment checkout.
-  The repository configuration currently selects `/etc/ansible/inventory`,
-  `/etc/ansible/roles`, and `/etc/ansible/collections`. Confirm the effective
-  paths so dependency installation and playbook execution use the same
-  inventory and installed content. Distinguish those paths from this
-  development workspace.
-- The target must satisfy the roles' Ubuntu 26.04 and access requirements.
-  Normal connection, role guards, and privileged tasks can establish these;
-  separate platform or sudo probes are not required acceptance tests.
-- Establish the controller's existing Vault password or identity mechanism
-  and operator availability to populate the four credential values in the
-  encrypted host file. Record how Vault is selected, never its password or
-  decrypted contents.
-- Have the operator select the acceptance campaign, session identity, and
-  audio input, with the corresponding session file and accessible input
-  location. This is a real processing and publication exercise using the
-  selected providers; session selection remains an operator decision.
-
-These are deployment inputs and operational checks, not additional production
-tasks or a new automated preflight framework.
+- Keep playbooks minimal and settings in inventory. Preserve `pipeline-dnd`
+  and its connection identity; do not rename it to `pipeline.dnd`.
+- Preserve `dilfs` using the existing standalone Narratio variable. Remove
+  legacy `narratio_s3_*` and `narratio_default_campaign_id` host settings.
+- Missing operational inputs are documented, not treated as implementation
+  blockers. No remote controller, SSH identity, Vault password, or session is
+  required for repository preparation.
+- Local dependency installation and syntax/host-selection checks are permitted.
+  Existing passing evidence may be reused when relevant inputs are unchanged.
+- `ansible-doc` rendering is not an acceptance gate. Missing role argument
+  descriptions are a separate documentation improvement.
 
 ## Simplest Viable Implementation
 
-Use the reviewed collection revision and associated compatibility manifest to
-update the controller's single root requirements file. Add the minimal playbook and one ordinary
-host-variable file for the endpoint, operator, campaign, and credential
-references, inheriting the suite's storage and publication defaults. Add one
-fully encrypted host Vault file for the four credential values, populated by
-the operator without exposing its plaintext during implementation or review.
-
-Install or update dependencies from the root manifest at the confirmed
-controller paths, explicitly refreshing the collection from its pinned commit
-even when `0.1.0` is already installed. Confirm that Ansible
-resolves `rakestraw.dnd.pipeline`, syntax-check the playbook, and converge only
-`pipeline-dnd`. Start a fresh operator login after enrollment, validate the
-native Narratio configuration under the intended runtime identity, rerun the
-same play immediately to confirm clean convergence, and process one selected
-session through the complete Feature 0015 publishing path. Record concise
-operational instructions and results; do not add a second test harness around
-the real deployment.
-
-## Rationale and Alternatives
-
-Keeping the playbook nearly declarative makes inventory the durable record of
-what differs about this host and allows ordinary Ansible precedence to remain
-the customization mechanism. Embedding endpoint, operator, storage, or secret
-mapping variables in the playbook would make a one-host play harder to reuse
-and would diverge from the established `rakestraw-ansible` style.
-
-A local wrapper role could hide the collection interface, but it would create
-another layer with no independent behavior. Applying the collection role by
-FQCN proves the same consumer path that future hosts should use.
-
-Tracking collection `main` would be convenient during development, but it
-would make a later controller bootstrap select an unreviewed compatibility
-set. An exact public commit retains development flexibility until the next
-release while still making this deployment reproducible.
-
-Feature 0015 now supplies the reviewed storage and publishing path needed for
-meaningful end-to-end acceptance. Inheriting that policy keeps host inventory
-focused on deployment choices and avoids maintaining a second copy of the
-suite defaults.
+Retain the already-committed minimal playbook, manifest, and public inventory.
+Add adjacent comments for unresolved credentials, align the runbook with the
+repository-only scope, perform appropriate local checks, and commit. No new
+Ansible task, runtime test harness, or deployment automation is needed.
 
 ## Acceptance Criteria
 
-- Feature 0015 is complete and independently reviewed before Feature 0016
-  implementation begins.
-- `playbooks/pipeline/dnd.yml` targets only `pipeline-dnd`, applies only
-  `rakestraw.dnd.pipeline`, and contains no deployment variables or procedural
-  setup.
-- Host inventory sets the confirmed WhisperX endpoint, enrolls only `eric`,
-  preserves `dilfs` as the default campaign through the existing standalone
-  role variable, and inherits Feature 0015's storage and publishing defaults.
-- `dnd_pipeline_secrets` maps the required typed runtime keys to host Vault
-  variables. Credential values in the consumer repository are Vault-encrypted;
-  runtime delivery uses the suite's existing restricted secrets files.
-- This feature introduces no plaintext credential values into commits, diffs,
-  command examples, validation output, or documentation. A repository-wide
-  historical secret audit is not an acceptance requirement.
-- Legacy Narratio storage values are either represented through the current
-  pipeline interface or deliberately retired; no stale high-precedence host
-  variable can override the selected suite configuration accidentally.
-- The root requirements manifest preserves unrelated existing dependencies
-  and installs the exact reviewed public collection snapshot, its five
-  standalone role dependencies, and its required native collections without
-  using local repository paths.
-- The controller uses the selected collection commit even if collection
-  version `0.1.0` was already installed. The documented inventory, role, and
-  collection paths correspond to the operational controller.
-- Public dependency installation succeeds, and the new playbook passes syntax
-  checking with the installed collection and roles. Documentation rendering
-  with `ansible-doc` is not required for deployment acceptance.
-- The initial limited playbook run against `pipeline-dnd` completes
-  successfully using the encrypted inventory and ordinary sudo-capable remote
-  account.
-- The immediate identical second run reports `changed=0` and `failed=0`.
-- Narratio's native configuration validation succeeds under the intended
-  runtime identity, and a fresh session for `eric` can access and invoke the
-  shared pipeline through runtime-group membership.
-- One operator-selected D&D session completes transcription, processing,
-  storage, and publishing, and its expected published output is observable at
-  the configured destination. Either supported runtime identity may execute
-  it. Production is the default profile; the operator may select the testing
-  profile while retaining the same processing and publication acceptance scope.
-- The operational documentation explains dependency installation, Vault use,
-  deployment invocation, the login-session membership caveat, convergence
-  verification, and representative pipeline invocation without disclosing
-  credentials.
-- The historical proof-of-concept role and playbook remain available during
-  the initial rollout; their later retirement requires a separate explicit
-  decision.
+- The playbook targets only `pipeline-dnd` and applies only the collection role.
+- Host inventory expresses the known choices above and documents unknown
+  credentials next to their references without real or fake secret values.
+- The dependency manifest contains the reviewed public pins and preserves
+  unrelated dependencies. Public installation and local syntax/host-selection
+  evidence are available; no documentation-rendering check is required.
+- The runbook distinguishes repository readiness from future deployment and
+  explains how the operator supplies missing values and eventually invokes it.
+- Changes are committed in the owning repositories, with clean working trees
+  for the feature's changes. No controller deployment, target convergence,
+  clean-repeat run, or real pipeline execution is required or claimed.
+- Historical role/playbook files remain available.
 
 ## Dependencies and Follow-up Work
 
-[Feature 0015](0015-dnd-s3-storage-and-publishing.md) supplies the implemented
-and independently reviewed pipeline interface, credentials, and publishing
-boundary consumed here. Its administrative closeout remains before the
-Feature 0016 sprint; its runtime implementation needs no further remediation.
+Features 0013–0015 supply public dependencies, operator enrollment, and storage
+policy. Feature 0016's consumer preparation already exists in `12b2462`, with
+initial documentation in `716a80e`; remaining work aligns comments and docs
+with the user's clarified scope.
 
-Features 0013 and 0014 provide the public collection installation path and
-existing-operator enrollment required here. Features 0009 and 0011 provide
-the shared filesystem and managed credential-delivery policies.
-
-After the real deployment demonstrates parity, a later cleanup feature may
-retire the proof-of-concept `pipeline_dnd` role, its old playbook, and any
-obsolete inventory values. Service scheduling, unattended ingestion,
-additional human operators, backup policy, and additional hosts also remain
-separate work.
+The operator may later provision Vault values, install dependencies on their
+controller, deploy to the host, and validate real processing/publication.
+Those actions require a separate request and are not unfinished Feature 0016
+acceptance requirements.