ansible-workspace-dnd/docs/roadmap/features/0012-collection-assembly.md

Feature 0012: Assemble the D&D Collection

Status: done

Summary

Turn the existing ansible-collection-dnd repository into a valid, buildable rakestraw.dnd Ansible collection that exposes the independently developed pipeline role as rakestraw.dnd.pipeline. The collection will carry a self-contained snapshot of the pipeline role, publish the collection dependencies Ansible can resolve natively, and provide one consumer bootstrap manifest for the five Git-hosted application roles.

Problem or Opportunity

The standalone application roles and the opinionated pipeline role are now implemented and independently tested, but the collection repository still contains only a placeholder README and repository metadata. A consumer cannot build or install rakestraw.dnd, invoke the intended fully qualified role name, or reproduce the reviewed combination of application-role dependencies from that repository.

The pipeline role must remain independently developed, versioned, and usable. At the same time, the collection artifact must contain the role it advertises; it cannot depend on a sibling checkout, a symbolic link, or workspace-only tooling at runtime. The assembly boundary therefore needs an explicit, repeatable source-import convention and focused integration coverage of the packaged result.

Governing Policies

• The Ansible house style governs collection metadata, dependency declarations, documentation, and the preference for native Ansible packaging behavior.
• The testing house style governs behavior-oriented, state-based validation and the avoidance of tests that merely restate file layout or implementation details.
• The MVP roadmap establishes the collection name, independently released role repositories, Git-hosted application-role dependencies, and the collection as the compatibility record for an assembled release.
• Features 0006 through 0011 define the pipeline role behavior being assembled; this feature does not reopen those contracts.

Scope

• Add valid Ansible Galaxy metadata for collection namespace rakestraw, name dnd, development version 0.1.0, the repository's existing license and README, and its supported Ansible version.
• Declare the exact ansible.posix and community.general versions already selected by the pipeline role as collection dependencies.
• Import the complete standalone pipeline-role repository into roles/pipeline as a squashed Git subtree at an explicitly reviewed commit.
• Treat ansible-role-dnd-pipeline as authoritative: changes are made and reviewed there first, then deliberately imported into the collection. Do not edit role behavior directly beneath the collection's subtree prefix.
• Document the initial subtree import and subsequent update commands so the source relationship is reproducible without custom tooling or private knowledge.
• Ensure the standalone pipeline role's collection-facing documentation distinguishes its standalone role name and invocation from rakestraw.dnd.pipeline use without changing its public behavior.
• Add meta/runtime.yml with requires_ansible: ">=2.18.0", matching the implemented roles' supported baseline.
• Add one root requirements.yml that combines the five Git-hosted application-role dependencies with the two collection dependencies needed to bootstrap the assembled suite.
• Pin application roles to the exact reviewed commits already selected by the standalone pipeline role during development. Keep the root bootstrap manifest and the imported role's dependency manifests consistent.
• Configure collection builds to omit development-only content embedded in the role subtree, including its integration suite, local Ansible configuration, and nested dependency manifests. Retain operator-facing role documentation and licensing in the source collection unless Ansible packaging requires otherwise.
• Replace the collection's placeholder README with concise instructions for dependency bootstrap, collection build and installation, FQCN invocation, subtree provenance and updates, supported platform, and the distinction between development commit pins and future released role tags.
• Add a focused collection integration suite that starts with isolated role and collection paths, installs the root bootstrap manifest and the locally built collection artifact, invokes rakestraw.dnd.pipeline, validates the assembled pipeline through the applications' native configuration interfaces, and verifies an immediate second application converges with no reported changes.

Non-goals

• Publishing the collection to Ansible Galaxy, Automation Hub, Gitea, or any other registry.
• Creating the first collection release tag or application-role release tags.
• Replacing development commit pins with immutable role tags before those tags exist.
• Vendoring any of the five standalone application roles into the collection.
• Moving authoritative pipeline-role development into the collection repository or permitting collection-local role forks.
• Changing pipeline defaults, host convergence, application versions, application configuration, asset deployment, credentials, or runtime behavior.
• Adding collection plugins, modules, filters, playbooks, execution environments, CI workflows, release automation, or a general-purpose sync framework.
• Repeating every standalone pipeline integration scenario inside the collection suite or asserting the exact contents of metadata and manifests as an implementation snapshot.
• Supporting installation without network access. The collection artifact is self-contained for the pipeline role, but its declared external role and collection dependencies must still be made available to Ansible.

Target End State

The collection repository has this durable shape:

ansible-collection-dnd/
├── galaxy.yml
├── meta/
│   └── runtime.yml
├── requirements.yml
├── roles/
│   └── pipeline/             imported pipeline-role subtree
├── tests/
│   ├── converge.yml
│   ├── integration           assembled-artifact behavior test
│   └── inventory
├── LICENSE
└── README.md

ansible-galaxy collection build produces a valid rakestraw-dnd-0.1.0.tar.gz artifact. After the documented bootstrap installs the root dependency manifest and the built artifact into clean Ansible paths, a consumer can invoke:

- name: Install and configure the D&D transcript pipeline
  hosts: pipeline_hosts
  roles:
    - role: rakestraw.dnd.pipeline

The collection contains ordinary role files at roles/pipeline; it has no runtime link to the standalone repository. Git history records that directory as a squashed subtree import from ansible-role-dnd-pipeline. Maintainers make pipeline changes in the standalone repository, validate and commit them there, then update the collection subtree from the desired reviewed commit.

The root bootstrap manifest is the consumer-facing compatibility manifest for development. It contains the same five application-role sources and reviewed commit pins as the imported pipeline role, plus exact versions of ansible.posix and community.general. The collection's Galaxy metadata also declares those two collection dependencies because Ansible can resolve them as native collection dependencies. This intentional duplication serves two different installation boundaries and must remain consistent.

Development-only files may remain in the subtree source for faithful provenance, but they are excluded from the built artifact. The collection's own integration suite exercises the artifact rather than the repository role directory, with role and collection paths isolated from ordinary user installations.

Repository Ownership

• ansible-collection-dnd owns collection metadata, the consumer bootstrap manifest, the imported pipeline snapshot, collection-facing documentation, build exclusions, and assembled-artifact integration coverage.
• ansible-role-dnd-pipeline remains authoritative for all pipeline-role code, defaults, templates, dependency metadata, standalone documentation, and its comprehensive integration suite. This feature changes only the wording needed to document both standalone and collection use.
• The five standalone application-role repositories remain authoritative for their roles and are read-only inputs to this feature.
• campaigns and narratio-prompts remain independently managed asset repositories and are read-only integration inputs.
• ansible-workspace-dnd owns this feature roadmap, progress tracking, and the later independent review record.
• Application source repositories and rakestraw-ansible remain unchanged.

Constraints and Settled Decisions

• The collection is rakestraw.dnd; the bundled role's FQCN is rakestraw.dnd.pipeline.
• A squashed Git subtree is the selected incorporation mechanism. It yields a normal, self-contained collection artifact while preserving an auditable upstream relationship and avoiding submodule checkout requirements, symbolic links, manual copying, or custom synchronization state.
• The subtree prefix is roles/pipeline, and the standalone pipeline repository remains the only place where role behavior is authored.
• The first development build uses collection version 0.1.0. This version enables ordinary Galaxy build and install validation but does not itself authorize publication or establish a release tag.
• Development dependencies use exact reviewed commits. A later release pass must replace all application-role commit pins with immutable role release tags and revalidate the assembled artifact before publishing.
• Collection dependencies remain exactly ansible.posix 2.2.0 and community.general 13.3.0, matching the existing reviewed pipeline manifests.
• Galaxy metadata cannot install arbitrary standalone Git roles. The root unified requirements manifest and documented bootstrap are therefore part of the collection's public installation contract.
• The collection integration suite must not receive dependencies from the developer's normal Ansible paths. Its success must demonstrate that the documented manifest and built artifact are sufficient.
• The collection suite owns only the packaging boundary: buildability, dependency bootstrap, FQCN resolution, representative assembled behavior, native configuration acceptance, and convergence. Detailed pipeline feature matrices remain in the standalone role's tests.
• The collection does not add probes, manifests, checksums, copied-file comparisons, or private state to police the subtree. Git and the documented subtree workflow are the provenance mechanism.

Simplest Viable Implementation

Use Ansible's conventional collection layout and metadata directly. Import the reviewed pipeline repository with ordinary git subtree add --squash, then exclude only the subtree's development fixtures and nested bootstrap files from Galaxy builds. No wrapper role, generated role, symbolic link, submodule, or synchronization script is required.

Derive the root unified dependency manifest directly from the pipeline role's two existing manifests. The only deliberate duplication is the collection dependency declaration in galaxy.yml, because the artifact and the source bootstrap have distinct native Ansible dependency interfaces.

Use one representative integration path: build into a temporary directory, install dependencies and the artifact into isolated temporary Ansible paths, apply rakestraw.dnd.pipeline using the same offline synthetic inputs already proven by the standalone suite, run native application configuration validation, and apply the play a second time to assert changed=0 and failed=0. This tests the behavior unique to collection assembly without duplicating the standalone role's full test matrix.

Rationale and Alternatives

A squashed subtree is preferable to a Git submodule because Ansible collection archives require real role content and should build correctly from an ordinary checkout without a second initialization step. It is preferable to a symbolic link because archive behavior and installed collection paths would be fragile. It is preferable to an unrecorded copy because subtree history retains the source relationship and provides a standard update command. Squashing avoids injecting the pipeline repository's entire development history into the collection while still recording each reviewed import.

Moving the role permanently into the collection would simplify assembly, but would discard the settled requirement that the suite role remain independently developed, tested, versioned, and usable. A thin collection wrapper that depends on rakestraw.dnd_pipeline would avoid source import but would not actually bundle the role or satisfy the desired FQCN without adding another runtime dependency layer.

Keeping one unified root requirements file gives consumers one bootstrap command for both dependency kinds. Retaining the imported role's own requirements files in source preserves standalone repository fidelity; build exclusion prevents those nested development interfaces from confusing users of the installed artifact.

Acceptance Criteria

• The collection repository is recognized by ansible-galaxy as rakestraw.dnd version 0.1.0 and builds successfully from a clean checkout.
• The built archive contains roles/pipeline and excludes the imported role's development-only integration suite, local Ansible configuration, and nested dependency manifests.
• The role content at roles/pipeline is traceable to a documented reviewed commit of ansible-role-dnd-pipeline through the repository's squashed subtree history.
• The standalone pipeline README accurately documents both rakestraw.dnd_pipeline and rakestraw.dnd.pipeline consumption without changing role behavior.
• meta/runtime.yml selects Ansible 2.18 or newer, and Galaxy metadata declares the exact two collection dependencies used by the pipeline.
• The root bootstrap manifest contains all five standalone application roles at the same reviewed commits used by the imported pipeline role and the exact two required collections.
• Collection documentation provides sufficient commands to install the bootstrap dependencies, build and install the artifact, invoke the pipeline FQCN, and update the subtree from a future reviewed pipeline commit.
• The integration suite starts with isolated Ansible role and collection paths, installs only the declared bootstrap inputs and built artifact, and resolves rakestraw.dnd.pipeline without access to a sibling role checkout.
• The representative assembled deployment passes the applications' native configuration validation and an immediate identical second run reports changed=0 and failed=0.
• Existing standalone pipeline-role validation continues to pass after its documentation-only change.
• No application role, application source, asset behavior, or pipeline host convergence behavior changes as part of this feature.

Dependencies and Follow-up Work

• Features 0001 through 0011 and their reviewed role commits are prerequisites.
• A subsequent release feature must create immutable role tags, replace development commit pins, select the release collection version, validate the tagged combination, create release notes and tags, and publish or distribute the artifact.
• CI automation may later run the collection integration suite after a local, repeatable workflow exists.

Potential Future Hardening

• Sign collection artifacts or publish checksums and software bills of materials when a distribution threat model requires them.
• Automate subtree provenance checks and dependency-pin synchronization if manual collection updates become frequent enough to justify that machinery.