eric/ansible-workspace-dnd
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Plan D&D S3 storage and publishing

Browse Source
This commit is contained in:
Eric Rakestraw
2026-09-21 02:20:57 +00:00
parent ec69f5a03b
commit e262b61292
6 changed files with 1023 additions and 181 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
docs/roadmap/features/0014-existing-pipeline-operator-enrollment.md
View File

@@ -1,6 +1,6 @@
# Feature 0014: Enroll Existing Pipeline Operators
Status: in_progress
Status: done
## Summary

392
docs/roadmap/features/0015-dnd-s3-storage-and-publishing.md Normal file
View File

@@ -0,0 +1,392 @@
# Feature 0015: Enable D&D S3 Storage and Publishing
Status: ready
## Summary
Make the D&D pipeline operational with Narratio's current S3 storage,
publication, caching, cleanup, and restore capabilities. The pipeline role
will select the established Backblaze B2 destination, reference two typed
object-storage credentials delivered through its existing secrets directory,
publish the useful transcript and D&D artifact outputs, and retain maximum S3
audio caching.
The standalone Narratio role already exposes Narratio v2.1.0's complete
configuration surface for this behavior. This feature supplies D&D suite
policy through that existing interface; it does not add another storage
implementation or credential mechanism.
## Problem or Opportunity
The pipeline role currently installs Narratio v2.1.0 and configures the full
processing graph, but deliberately leaves Narratio storage and publishing at
their application-level defaults. Its README consequently describes storage,
publishing, and end-to-end external execution as deferred. The resulting suite
can validate configuration and process local state, but it does not select the
remote archive or publish the outputs needed for an operational deployment.
The prior test-machine configuration records the intended Backblaze B2
endpoint, bucket, prefix, credential names, publication set, and post-publish
workspace cleanup. Narratio's current contract has since replaced the legacy
credential field names and secrets configuration with ConfigKit typed
references and a conventional sibling `secrets/` directory. The Ansible suite
already implements that new credential-delivery boundary for Audita and
Notarius, so S3 needs only to join the same path.
## Governing Policies
- [The Ansible house style](../../ansible.md) requires suite policy to flow
through the standalone role's field-specific interface and optional fields
to remain application-owned unless the suite deliberately selects them.
- [The testing house style](../../testing.md) requires representative
public-interface and native-application validation without duplicating AWS
SDK, ConfigKit, or Ansible module behavior.
- [The MVP roadmap](../mvp.md) makes the pipeline role responsible for
cross-application configuration and end-to-end validation.
- [Feature 0011](0011-managed-pipeline-credential-delivery.md) establishes the
managed Narratio secrets directory and typed credential-forwarding policy.
- [Feature 0012](0012-collection-assembly.md) makes the standalone pipeline
repository authoritative and imports reviewed changes through public
Gitea.
- [Feature 0016](0016-deploy-dnd-pipeline-to-pipeline-dnd.md) consumes this
feature's reviewed interface for the first real-host deployment.
## Scope
- Add suite-owned pipeline variables for the selected Narratio workspace,
S3, cache, and publish settings.
- Select the S3 backend with these D&D deployment defaults:
| Setting | Suite default |
| --- | --- |
| Endpoint | `https://s3.us-east-005.backblazeb2.com` |
| Bucket | `rakestraw-assets` |
| Root prefix | `dnd` |
| Region | `us-east-1` |
| Force path style | `true` |
| Access-key reference | `OBJECT_STORAGE_KEY_ID` |
| Secret-key reference | `OBJECT_STORAGE_KEY` |
- Pass the two credential reference names to the standalone Narratio role.
Their values use the existing `dnd_pipeline_secrets` map and managed
`/usr/local/etc/narratio/secrets/` directory.
- Enable S3 audio caching explicitly so the suite retains Narratio's maximum
available audio reuse.
- Enable run publication and publish these complete native output rules:
| Source | Required |
| --- | --- |
| `narratio.transcript.final` | `true` |
| `narratio.transcript.final_trimmed` | `true` |
| `narratio.transcript.final_markdown` | `true` |
| `narratio.transcript.final_trimmed_markdown` | `true` |
| `narratio.artifact.session_recap` | `true` |
| `narratio.artifact.hostile_rumors` | `true` |
| `narratio.artifact.deranged_rumors` | `false` |
- Let Narratio derive each destination from the built-in transcript path or
configured artifact output path rather than duplicating those paths in the
publish list.
- Enable workspace cleanup only after Narratio records successful run
publication. Leave spool-audio deletion under Narratio's existing default
because the suite has selected no policy requiring source-audio removal.
- Keep the production and `testing` profile storage and publishing behavior
identical. The existing overlay remains limited to model-profile selection.
- Document the four credentials normally required by the operational suite:
`AUDITA_LLM_API_KEY`, `OPENROUTER_API_KEY`,
`OBJECT_STORAGE_KEY_ID`, and `OBJECT_STORAGE_KEY`.
- Document the operator-visible publication, restore, S3-session-audio, lock,
caching, and cleanup capabilities enabled by the selected configuration,
while linking Narratio's canonical CLI and operations documentation for
command details.
- Preserve every accepted Notarius lane as its existing
`narratio.extraction.<output_key>` source. Future Narratio artifact
definitions can consume those sources during analyze without first making
the extraction lanes separate published outputs.
- Adapt the existing pipeline integration scenario to resolve synthetic S3
credentials and validate one explicitly supplied representative storage and
publishing configuration through Narratio's native interface without
contacting object storage.
- Import the reviewed public pipeline commit into the collection through the
established squashed subtree workflow, adapt the collection's artifact-level
fixture to supply the added credential names, and validate the assembled
artifact.
## Non-goals
- Implementing an S3 client, publication engine, remote commit format,
restore mechanism, publish-lock mechanism, or audio cache in Ansible.
Narratio owns all of those behaviors.
- Placing credential values in role defaults, templates, documentation,
tests, ordinary inventory, or the generated pipeline configuration.
- Adding a second secret-delivery mechanism, environment wrapper, service
unit, credential synchronizer, or AWS configuration-file manager.
- Managing Backblaze buckets, object lifecycle policy, application keys,
retention policy, versioning, encryption, quotas, or network access.
- Managing per-session `session.yml` files or selecting local versus S3 audio
input for an individual session.
- Running a paid or networked publication test from a repository-local test
suite. Feature 0016 owns the representative live end-to-end exercise.
- Publishing every Notarius extraction lane as a named current output. The
lanes remain available as analyze inputs independently of publication; stable
remote destinations require an explicit operator need.
- Setting Narratio's optional `notarius.chunk_cache` override. The managed
Notarius configuration remains the single owner of its ordinary `auto`
cache policy, and Narratio v2.1 already requests refresh for forced
extraction.
- Adding notification providers. Narratio v2.1 supports only its existing
`noop` notification mode.
- Releasing new role or collection versions as part of this implementation.
## Target End State
The standalone pipeline role exposes these suite variables with the selected
defaults:
```yaml
dnd_pipeline_narratio_workspace_cleanup_after_publish: true
dnd_pipeline_narratio_storage_backend: s3
dnd_pipeline_narratio_storage_s3_bucket: rakestraw-assets
dnd_pipeline_narratio_storage_s3_root_prefix: dnd
dnd_pipeline_narratio_storage_s3_region: us-east-1
dnd_pipeline_narratio_storage_s3_endpoint: >-
https://s3.us-east-005.backblazeb2.com
dnd_pipeline_narratio_storage_s3_force_path_style: true
dnd_pipeline_narratio_storage_s3_access_key_id: OBJECT_STORAGE_KEY_ID
dnd_pipeline_narratio_storage_s3_secret_access_key: OBJECT_STORAGE_KEY
dnd_pipeline_narratio_cache_s3_audio: true
dnd_pipeline_narratio_publish_enabled: true
dnd_pipeline_narratio_publish_upload_run: true
dnd_pipeline_narratio_publish_outputs:
- source: narratio.transcript.final
required: true
- source: narratio.transcript.final_trimmed
required: true
- source: narratio.transcript.final_markdown
required: true
- source: narratio.transcript.final_trimmed_markdown
required: true
- source: narratio.artifact.session_recap
required: true
- source: narratio.artifact.hostile_rumors
required: true
- source: narratio.artifact.deranged_rumors
required: false
```
Each value is declared in the pipeline argument specification and passed
directly to its corresponding `narratio_*` variable in the pipeline role's
Narratio dependency. Narratio's existing root template renders the native
configuration. No new pipeline fragment or production task is required.
The ordinary generated configuration contains only the credential names:
```yaml
storage:
backend: s3
s3:
access_key_id: OBJECT_STORAGE_KEY_ID
secret_access_key: OBJECT_STORAGE_KEY
```
Consumers supply the two values beside the existing model-provider values
through encrypted inventory:
```yaml
dnd_pipeline_secrets:
AUDITA_LLM_API_KEY: "{{ vault_dnd_audita_llm_api_key }}"
OPENROUTER_API_KEY: "{{ vault_dnd_openrouter_api_key }}"
OBJECT_STORAGE_KEY_ID: "{{ vault_dnd_object_storage_key_id }}"
OBJECT_STORAGE_KEY: "{{ vault_dnd_object_storage_key }}"
```
The example defines references only; no roadmap, role, or test contains real
credential values. The existing credential task writes the selected entries
with the suite's established group-authorized secret modes. Omitted map keys
remain unmanaged, as defined by Feature 0011.
With valid credentials, Narratio can accept either local or S3 audio selected
by each session file, cache S3 audio beneath `/var/cache/narratio`, publish an
immutable run and the selected stable outputs, use remote publication locks,
restore committed state, and clean the published workspace. These are native
Narratio commands and lifecycle rules rather than additional Ansible tasks.
The ten accepted Notarius outputs continue to enter Narratio's artifact
catalog as `narratio.extraction.*` sources. Analyze-stage Scriptorium artifacts
may reference those sources directly in their input mappings. Feature 0015
does not yet add such artifact definitions, but its publication choices do not
remove, rename, or constrain this future composition path.
## Repository Ownership
- `ansible-role-dnd-pipeline` owns the D&D suite variables, opinionated
defaults, dependency mappings, consumer documentation, and representative
integration scenario.
- `ansible-role-narratio` remains the generic owner of installation and native
pipeline rendering. Its reviewed v2.1.0 interface already covers every
configuration field required by this feature and should remain unchanged.
- `ansible-collection-dnd` owns the faithful pipeline subtree import, the
unified consumer dependency manifest, and artifact-boundary validation.
- `ansible-workspace-dnd` owns this roadmap, the later implementation plan,
status, and independent review record.
- `narratio` is a read-only source of storage, publication, credential,
caching, cleanup, restore, and CLI contracts.
- `rakestraw-ansible` remains unchanged until Feature 0016 consumes this
interface on `pipeline-dnd`.
## Constraints and Settled Decisions
- The S3 endpoint, bucket, prefix, region, and path-style values are deliberate
D&D suite defaults. They are ordinary variables and remain overrideable by
inventory.
- Credential references are also policy defaults, but credential values have
no defaults. Operators must supply matching entries through
`dnd_pipeline_secrets` or another environment source supported by Narratio.
- The two credential-reference variables are nullable as a pair. Setting both
to `null` omits the references and preserves Narratio's AWS SDK provider-chain
path; configuring only one remains invalid under Narratio's native contract.
- The role does not assert that the secrets map contains particular keys.
Narratio's native loader owns typed-reference resolution, and external
environment values remain a supported higher-precedence source.
- `dnd_pipeline_narratio_publish_outputs` is one complete native list. A caller
replacing it assumes responsibility for the entire selected publication
set; the role does not merge rules.
- Publish destinations remain omitted where Narratio can derive them. The
configured source identity and artifact output path remain the single source
of truth.
- Workspace cleanup is gated by Narratio's own successful-publication
evidence. Ansible does not independently inspect remote state or remove
runtime content.
- S3 audio caching is enabled, and ordinary cleanup preserves the cache.
Explicit operator cache clearing remains a Narratio CLI operation.
- The testing profile does not silently disable Notarius, storage,
publication, cleanup, or credentials. It continues to differ only in the
already selected inexpensive Scriptorium model profile.
- S3 storage also enables Narratio's native restore and publication-lock
commands without another configuration surface. Ansible documentation may
describe availability but must not duplicate Narratio's detailed operating
manual.
- Notarius extraction-source availability and remote publication are separate
concerns. Analyze consumes `narratio.extraction.*` sources from the accepted
extraction catalog; `publish.outputs` selects only which sources also receive
stable published destinations.
## Simplest Viable Implementation
Add the selected variables to the pipeline defaults and argument specification,
then map them into the existing `rakestraw.narratio` dependency. Update the
pipeline README to document the policy, credential names, complete-list
replacement semantics, and native capabilities now available. Do not add a
template, task, handler, fact, probe, or storage-specific validation layer.
In the existing pipeline integration driver, extend the synthetic secrets map
with object-storage values and explicitly supply representative S3 and publish
variables. Use `narratio config validate` and its secret-safe effective
configuration output to prove that the public inputs compose into a valid
configuration. Retain the immediate clean repeat and secret non-disclosure
checks. Do not call a real S3 service or assert that the representative values
equal current role defaults.
Push the reviewed pipeline commit, import that exact public revision into the
collection through `git subtree pull --squash`, and extend only the
collection-level fixture inputs needed for native validation. The collection
suite should continue to prove artifact installation and one representative
assembled convergence rather than duplicate the detailed pipeline matrix.
## Rationale and Alternatives
Implementing storage behavior in Ansible would duplicate a mature native
application boundary. Narratio already owns credential resolution, AWS SDK
provider construction, immutable commits, verification, locks, restore,
caching, and cleanup. Passing declarative configuration through the generic
role is both smaller and more reliable.
Using the AWS SDK's ambient provider chain would avoid named secret files, but
the initial deployment has explicit Backblaze application credentials and the
suite already has a suitable managed typed-secret path. Keeping the two
credential-reference variables nullable preserves the ambient-provider
alternative: consumers can set both suite variables to `null` when their
deployment uses another provider.
Copying destinations into every publish rule would make the output layout more
visible locally, but it would duplicate paths already owned by Narratio's
built-in sources and the configured artifact definitions. Native destination
derivation keeps path coordination automatic.
Deleting spool audio after publication would minimize transient disk usage,
but no such retention policy has been selected, and local-audio sessions may
depend on that content. Leaving the field omitted preserves Narratio's default
while enabling the explicitly requested S3 audio cache.
Publishing all ten Notarius extraction lanes would create stable remote output
contracts that have not been requested. It is also unnecessary for the stated
future use: analyze-stage artifacts can consume `narratio.extraction.*`
directly. The immutable published run retains the processing result, while the
selected transcript and human-facing artifact outputs match the known
operational workflow.
## Acceptance Criteria
- The pipeline role declares and documents every suite variable shown in the
target state, with matching argument-spec entries and direct Narratio
dependency mappings.
- The two credential-reference variables accept paired `null` overrides so a
caller can deliberately select Narratio's ambient AWS SDK provider chain.
- The default effective Narratio configuration selects the specified
Backblaze-compatible S3 destination, typed credential names, S3 audio cache,
successful-publication workspace cleanup, run upload, and seven output
rules.
- The publish list relies on Narratio's native destination derivation and
retains `deranged_rumors` as optional while the other six outputs are
required.
- Spool-audio deletion remains unset by the pipeline role, and the testing
profile introduces no storage or publication override.
- `dnd_pipeline_secrets` remains a generic empty-by-default mapping; the role
documents all four expected operational entries without embedding any
credential value.
- No new production task, template, handler, fact, state file, wrapper, or
storage-validation command is introduced.
- The standalone Narratio role remains unchanged because its v2.1.0 interface
already renders the required native fields and preserves null omission.
- The existing pipeline integration scenario explicitly supplies
representative S3 and publish inputs, resolves synthetic values for all four
typed credentials, passes Narratio's native configuration validation, and
exposes no synthetic value in captured output.
- The immediate identical second pipeline application reports `changed=0`
and `failed=0`; syntax checking, production-profile lint, the complete
standalone integration suite, and `git diff --check` pass.
- The exact reviewed pipeline commit is pushed to public Gitea before the
collection imports it through the documented remote-only squashed subtree
workflow.
- The collection fixture supplies the added synthetic credential names, and
its existing build, isolated installation, FQCN resolution, native
validation, and clean-repeat suite passes without a duplicate S3 behavior
test.
- No real object-storage request, paid model request, application repository
change, asset change, release tag, or published artifact is required.
- Pipeline documentation links operators to Narratio's canonical procedures
for S3 session audio, publish, restore, locks, status, and cleanup and no
longer describes storage and publishing as deferred.
- Documentation distinguishes accepted Notarius extraction sources from
published outputs and confirms that future analyze-stage artifacts may use
`narratio.extraction.<output_key>` inputs without publishing those lanes.
## Dependencies and Follow-up Work
Features 0009, 0011, 0012, 0013, and 0014 are prerequisites. In particular,
Feature 0011 supplies the credential directory and Feature 0014 supplies the
human-operator access needed for the eventual live deployment.
[Feature 0016](0016-deploy-dnd-pipeline-to-pipeline-dnd.md) follows this work.
It will put the four actual values in encrypted host inventory, pin the exact
reviewed suite revision in `rakestraw-ansible`, converge `pipeline-dnd`, and
exercise one real session through remote publication. Bucket lifecycle policy,
backup policy, credential rotation, additional publish outputs, automated
scheduling, and cleanup of the proof-of-concept deployment remain separate
future work.
A later artifact-development feature may add D&D-specific Narratio artifact
definitions whose Scriptorium input mappings consume selected
`narratio.extraction.*` sources. That feature should define each artifact's
purpose, inputs, prompt, output contract, and publication choice; it does not
require a storage-interface change.

278
docs/roadmap/features/0016-deploy-dnd-pipeline-to-pipeline-dnd.md Normal file
View File

@@ -0,0 +1,278 @@
# Feature 0016: Deploy the D&D Pipeline to `pipeline-dnd`
Status: proposed
## Summary
Add a minimal playbook and host inventory to `rakestraw-ansible` that apply
`rakestraw.dnd.pipeline` to the existing `pipeline-dnd` host. Configure the
host through `host_vars`, enroll `eric` as the initial human operator, keep
credential values in Ansible Vault, and prove the complete pipeline through a
real repeatable deployment and representative D&D session.
[Feature 0015](0015-dnd-s3-storage-and-publishing.md) must first provide the
storage and publishing behavior needed for an operational end-to-end pipeline.
Feature 0016 consumes that reviewed interface; it does not invent a temporary
storage contract.
## Problem or Opportunity
The collection and its application roles are implemented and installable, but
the operational Ansible repository does not yet have a playbook for deploying
the new collection architecture. Its existing D&D playbook targets a retired
inventory name and invokes the proof-of-concept local `pipeline_dnd` role.
The current `pipeline-dnd` host inventory already records the connection
identity and useful legacy Narratio storage values. A production-shaped first
deployment now needs to express the new role's public interface, deliver
secrets safely, reconcile the controller's dependency manifest, and validate
the actual host rather than extending the role repositories with another
synthetic scenario.
## Scope
- Add `rakestraw-ansible/playbooks/pipeline/dnd.yml`, targeting the existing
`pipeline-dnd` inventory host and applying only
`rakestraw.dnd.pipeline`.
- Keep the playbook limited to its host selection and role list. Put all
deployment choices in `inventory/host_vars/pipeline-dnd/`.
- Configure the external WhisperX transcription endpoint as
`https://transcription.ai.rakestrawhome.com/transcribe`.
- Enroll the existing `eric` account through
`dnd_pipeline_operator_users`.
- Map the role's `dnd_pipeline_secrets` interface to host-specific Vault
variables. Store credential values only in an Ansible Vault-encrypted host
variables file.
- Carry the storage and publishing settings selected by Feature 0015 through
its final public pipeline interface. Preserve the operational intent of the
existing Backblaze B2 endpoint, bucket, prefix, region, and path-style
settings where they remain applicable.
- Reconcile existing legacy Narratio host variables so they cannot silently
override the suite policy selected by the new pipeline role.
- Update `rakestraw-ansible/requirements.yml` so a clean controller can install
the reviewed collection snapshot, all five standalone role dependencies,
and required native collections from one manifest.
- Validate controller dependency resolution, playbook syntax, initial
convergence, immediate clean convergence, operator access, native Narratio
configuration, and one representative end-to-end pipeline run on the real
host.
- Document the operational invocation, Vault prerequisite, initial operator,
and expected validation path in `rakestraw-ansible`.
## Non-goals
- Implementing storage or publishing behavior. Feature 0015 owns that
prerequisite.
- Changing application or role behavior merely to accommodate this
deployment.
- Replacing the existing inventory alias with `pipeline.dnd`, changing DNS,
or moving connection variables out of the current host inventory.
- Adding a wrapper role, pre-task layer, service, timer, scheduler, web UI, or
unattended session-ingestion mechanism.
- Creating or managing the `eric` account, its login policy, SSH access, or
sudo policy.
- Adding another production host, high availability, backup policy, or
disaster-recovery automation.
- Removing the proof-of-concept local `pipeline_dnd` role or its historical
playbook before the new deployment has demonstrated operational parity.
- Publishing a new coordinated suite release solely as part of this
deployment feature.
## Target End State
`rakestraw-ansible/playbooks/pipeline/dnd.yml` is an intentionally small
consumer playbook equivalent to:
```yaml
---
- hosts:
- pipeline-dnd
roles:
- rakestraw.dnd.pipeline
...
```
The existing inventory selects the host's connection identity. A dedicated
host-variable file selects the pipeline's non-secret public inputs, including
the WhisperX endpoint, the initial operator list, and the storage and
publishing values exposed by Feature 0015. It maps runtime credential names to
Vault-backed variables without embedding credential values in the playbook or
ordinary plaintext inventory, for example:
```yaml
dnd_pipeline_operator_users:
- eric
dnd_pipeline_whisperx_transcribe_url: >-
https://transcription.ai.rakestrawhome.com/transcribe
dnd_pipeline_secrets:
AUDITA_LLM_API_KEY: "{{ vault_dnd_audita_llm_api_key }}"
OPENROUTER_API_KEY: "{{ vault_dnd_openrouter_api_key }}"
OBJECT_STORAGE_KEY_ID: "{{ vault_dnd_object_storage_key_id }}"
OBJECT_STORAGE_KEY: "{{ vault_dnd_object_storage_key }}"
```
The corresponding credential variables reside in a tracked file encrypted in
full with Ansible Vault. The Vault password or identity mechanism remains
outside Git and outside this feature's managed content.
The controller dependency manifest selects one reviewed, publicly reachable
suite snapshot containing Features 0014 and 0015. It also installs that
snapshot's exact standalone role and native collection dependencies. A clean
controller therefore does not depend on sibling development checkouts or
previously installed roles.
On `pipeline-dnd`, the role converges the full application suite and its D&D
configuration. The `narratio` service identity can run the pipeline, and
`eric` can run it after starting a login session that observes the granted
supplementary group membership. A representative session can be transcribed,
processed, stored, and published through the configured production path. An
immediate identical Ansible run reports no changes.
## Repository Ownership
- `rakestraw-ansible` owns the consumer playbook, host inventory, encrypted
host credentials, controller dependency manifest, and deployment runbook.
- `ansible-role-dnd-pipeline` owns the suite's public variables and the
opinionated cross-application behavior consumed by the deployment.
- `ansible-collection-dnd` owns the assembled `rakestraw.dnd.pipeline` role
snapshot and consumer-facing compatibility manifest.
- The standalone application roles continue to own installation and native
application configuration. Feature 0016 consumes their reviewed releases or
commits without reimplementing their behavior.
- `ansible-workspace-dnd` owns this roadmap, the later implementation plan,
progress tracking, and independent review record.
- Application and asset repositories are read-only sources of runtime
contracts and deployed content for this feature.
## Constraints and Settled Decisions
- Feature 0015 is a prerequisite and must settle, implement, and review the
storage and publishing interface before Feature 0016 becomes ready for an
implementation sprint.
- The target is the inventory alias `pipeline-dnd` as currently defined in
`rakestraw-ansible`, not a new `pipeline.dnd` alias.
- The playbook follows the repository's established style: hosts and roles
only. Facts remain at Ansible's normal default, privilege escalation remains
task-owned by the roles, and deployment variables belong in host inventory.
- `eric` is the initial human operator and must already exist on the managed
host. The pipeline role grants only additive runtime-group membership.
- API credentials use the existing typed `dnd_pipeline_secrets` interface and
Ansible Vault. Plaintext secret values must not enter Git, command lines,
logs, examples, or ordinary variable files.
- The public Gitea repositories are the sole dependency sources. The
deployment must not reference this workspace's sibling checkout paths.
- The dependency manifest must select an immutable release when Feature 0015
produces one. Otherwise, it must pin the exact reviewed public Git commit;
it must not track an unbounded branch for the production deployment.
- The collection artifact does not itself install arbitrary Git-hosted role
dependencies. The operational manifest must therefore include the five
standalone roles as well as the collection and its native collection
dependencies.
- The legacy host variables must be compared with the final Feature 0015
interface. Applicable values move to the new suite configuration, while
obsolete or conflicting variables are removed rather than left to win by
inventory precedence.
- The initial real-host run is an operational acceptance exercise. It should
test observable pipeline behavior and convergence, not module task traces or
the current text of defaults.
## Simplest Viable Implementation
After Feature 0015 is reviewed, select its exact public collection revision
and use the associated compatibility manifest to update the controller's
single root requirements file. Add the minimal playbook and one ordinary
host-variable file for public inputs. Add one fully encrypted host Vault file
for the two credential values, populated by the operator without exposing its
plaintext during implementation or review.
Install or update dependencies from the root manifest, confirm that Ansible
resolves `rakestraw.dnd.pipeline`, syntax-check the playbook, and converge only
`pipeline-dnd`. Start a fresh operator login after enrollment, validate the
native Narratio configuration under the intended runtime identity, rerun the
same play immediately to confirm clean convergence, and process one selected
session through the complete Feature 0015 publishing path. Record concise
operational instructions and results; do not add a second test harness around
the real deployment.
## Rationale and Alternatives
Keeping the playbook nearly declarative makes inventory the durable record of
what differs about this host and allows ordinary Ansible precedence to remain
the customization mechanism. Embedding endpoint, operator, storage, or secret
mapping variables in the playbook would make a one-host play harder to reuse
and would diverge from the established `rakestraw-ansible` style.
A local wrapper role could hide the collection interface, but it would create
another layer with no independent behavior. Applying the collection role by
FQCN proves the same consumer path that future hosts should use.
Tracking collection `main` would be convenient during development, but it
would make a later controller bootstrap select an unreviewed compatibility
set. An exact public commit retains development flexibility until the next
release while still making this deployment reproducible.
Running the real-host deployment before Feature 0015 would establish only an
installation demonstration: the pipeline still could not complete its
required storage and publishing path. Making Feature 0015 an explicit
prerequisite avoids temporary inventory variables and gives the acceptance
run a meaningful end-to-end boundary.
## Acceptance Criteria
- Feature 0015 is complete and independently reviewed before Feature 0016
implementation begins.
- `playbooks/pipeline/dnd.yml` targets only `pipeline-dnd`, applies only
`rakestraw.dnd.pipeline`, and contains no deployment variables or procedural
setup.
- Host inventory sets the confirmed WhisperX endpoint, enrolls only `eric`,
and supplies the final Feature 0015 storage and publishing inputs.
- `dnd_pipeline_secrets` maps the required typed runtime keys to host Vault
variables, and the actual values exist only in a valid Ansible
Vault-encrypted file.
- No plaintext credential value is present in any tracked revision, diff,
command example, validation output, or deployment documentation.
- Legacy Narratio storage values are either represented through the current
pipeline interface or deliberately retired; no stale high-precedence host
variable can override the selected suite configuration accidentally.
- The root requirements manifest preserves unrelated existing dependencies
and installs the exact reviewed public collection snapshot, its five
standalone role dependencies, and its required native collections without
using local repository paths.
- On a controller bootstrapped from that manifest, `ansible-doc` resolves
`rakestraw.dnd.pipeline` and each role dependency, and the new playbook
passes syntax checking.
- The initial limited playbook run against `pipeline-dnd` completes
successfully using the encrypted inventory and ordinary sudo-capable remote
account.
- The immediate identical second run reports `changed=0` and `failed=0`.
- Narratio's native configuration validation succeeds under the intended
runtime identity, and a fresh session for `eric` can access and invoke the
shared pipeline through runtime-group membership.
- One operator-selected D&D session completes transcription, processing,
storage, and publishing, and its expected published output is observable at
the configured destination.
- The operational documentation explains dependency installation, Vault use,
deployment invocation, the login-session membership caveat, convergence
verification, and representative pipeline invocation without disclosing
credentials.
- The historical proof-of-concept role and playbook remain available during
the initial rollout; their later retirement requires a separate explicit
decision.
## Dependencies and Follow-up Work
[Feature 0015](0015-dnd-s3-storage-and-publishing.md) is the immediate next unit
of work and a hard prerequisite. It selects the pipeline interface,
credentials, and observable publishing boundary that Feature 0016 will deploy.
Features 0013 and 0014 provide the public collection installation path and
existing-operator enrollment required here. Features 0009 and 0011 provide
the shared filesystem and managed credential-delivery policies.
After the real deployment demonstrates parity, a later cleanup feature may
retire the proof-of-concept `pipeline_dnd` role, its old playbook, and any
obsolete inventory values. Service scheduling, unattended ingestion,
additional human operators, backup policy, and additional hosts also remain
separate work.