ansible-workspace-dnd/docs/roadmap/implementation.md

Implementation Plan 0012: Assemble the D&D Collection

Status: proposed

Target executor: GPT-5.6-Terra

Feature roadmap: docs/roadmap/features/0012-collection-assembly.md

Active plan location: docs/roadmap/implementation.md

Objective

Build the initial rakestraw.dnd collection around the independently maintained pipeline role. The completed source repository must build and install as collection version 0.1.0, expose the imported role as rakestraw.dnd.pipeline, bootstrap the exact reviewed standalone role and collection dependencies, and prove the packaged boundary through one focused isolated integration path. The feature roadmap remains authoritative for scope, rationale, target state, and acceptance criteria.

Execution Rules

• Execute stages in numeric order.
• Number stages with consecutive integers beginning with Stage 1. Do not use zero-based, decimal, nested, or nonsequential stage identifiers.
• Before each stage, confirm its preconditions and the working-tree state of its owning repository.
• Before each stage, read every document listed in that stage's Required Policy Review. Do not rely on policy recalled from an earlier stage.
• Preserve unrelated and pre-existing changes.
• Do not make unresolved product or architecture decisions during execution; stop and report the decision required.
• Start from the feature roadmap's simplest viable implementation. Use native Galaxy metadata, dependency manifests, collection layout, and Git subtree behavior rather than adding wrappers, generators, synchronization scripts, or private state.
• Do not alter pipeline behavior in the standalone repository or beneath the imported roles/pipeline prefix. Any discovered behavioral defect belongs in the standalone pipeline repository and requires a separately reviewed change before a new subtree import.
• Do not add integrity, drift, release, publishing, signing, rollback, or offline-installation guarantees beyond the feature roadmap.
• Keep collection tests focused on the packaging boundary. Do not copy the standalone pipeline role's full integration matrix.
• Run every stage's validation before committing it. Stage 3 is the sole mechanical exception: git subtree add necessarily creates its provenance commit as part of the import, so run its validation immediately afterward and do not proceed if it fails.
• Commit each completed stage when it leaves the repository in a valid state, using the commit intent specified by that stage. Do not push any repository.
• Do not begin a dependent stage after an earlier stage fails validation.

Starting State and Preconditions

• The workspace planning changes for Feature 0012, including this plan, have been reviewed and committed before execution begins.

• ansible-workspace-dnd, ansible-role-dnd-pipeline, and ansible-collection-dnd begin on main with clean working trees.

• The standalone pipeline role begins at de1b191cd2b6895895d3915e7b453596994127cf. Stage 1 will create the new reviewed commit that Stage 3 imports.

• The collection begins at 976277b03374. It contains no role or Galaxy metadata yet; Feature 0012 planning files belong to the independent parent workspace repository.

• The pipeline dependency manifests select these exact application-role commits, which the collection bootstrap manifest must preserve:

  • rakestraw.seriatim at e08c63c0b078658bdfa2d10a2e6e1e4f9372a075
  • rakestraw.audita at 66ebea9a47ce82bfdbdc5e2855f3dfaaff8e2a16
  • rakestraw.notarius at 14a38e04e35ef524d6a97fa7fb5a04981a0a2161
  • rakestraw.scriptorium at 8b4c86e48e7d8655e479b26dc510bf01683a0c8f
  • rakestraw.narratio at 292d8669e7aad5cfe80b0c469d22b6177e8cf0f1

• Collection dependencies remain ansible.posix 2.2.0 and community.general 13.3.0.

• The workspace .venv provides Ansible Core 2.21.3 and ansible-lint 26.8.0. Invoke them through .venv/bin rather than assuming they are on PATH.

• Git includes the git subtree command.

• The local Ubuntu 26.04 proxy host supports passwordless noninteractive sudo, as required by the existing role integration suites.

• Network access is available for Galaxy dependencies and the pinned Git role sources. Local immutable file:// inputs from assets/campaigns and assets/narratio-prompts remain available for integration testing.

Repository Scope

• ansible-role-dnd-pipeline may change only its README to document the two supported consumption forms. It remains authoritative for the role.
• ansible-collection-dnd owns Galaxy and runtime metadata, the unified bootstrap manifest, collection documentation, build exclusions, the squashed pipeline-role subtree, and focused collection integration tests.
• ansible-workspace-dnd may change only planning and status documents during the final handoff stage.
• The five standalone application roles, both asset repositories, all application repositories, and rakestraw-ansible may be inspected but must remain unchanged.

Stage 1: Document Standalone and Collection Consumption

Objective

Make the authoritative pipeline role's README accurately describe both its standalone role name and its collection FQCN before taking the collection snapshot.

Required Policy Review

Before beginning this stage, read:

• AGENTS.md
• docs/orientation.md
• docs/conventions.md
• docs/ansible.md

Preconditions

• ansible-role-dnd-pipeline is clean at de1b191cd2b6895895d3915e7b453596994127cf.
• Feature 0012 is committed and records the standalone repository as the authoritative role source.

Owning Repository

ansible-role-dnd-pipeline

Required Changes

1. Revise README.md so it no longer describes collection exposure only as a future possibility.
2. Retain the existing standalone installation and rakestraw.dnd_pipeline invocation instructions.
3. Add a concise collection-use path that names rakestraw.dnd.pipeline and directs collection consumers to the collection repository's unified bootstrap instructions. Do not duplicate the collection's complete build, dependency, or subtree maintenance documentation here.
4. Explain that this repository remains authoritative and the collection carries a reviewed snapshot. Do not claim that a collection release has been published or tagged.
5. Make no changes to defaults, tasks, templates, metadata, dependency pins, tests, or role behavior.

Validation

Run from ansible-role-dnd-pipeline:

git diff --check
git diff -- README.md
git status --short

Expected result: the diff is limited to accurate README wording, contains no whitespace errors, preserves the standalone instructions, and introduces no claim of an already published collection release.

Completion Criteria

• Both role names and their distinct consumption contexts are documented.
• No executable or dependency file changed.
• The repository is ready to supply the exact commit for the subtree import.

Commit

Commit intent: Document standalone and collection pipeline usage

Stage 2: Establish the Collection Foundation

Objective

Create the valid, buildable collection metadata, reproducible dependency bootstrap, build policy, and operator documentation that will own the imported role.

Required Policy Review

Before beginning this stage, read:

• AGENTS.md
• docs/orientation.md
• docs/conventions.md
• docs/ansible.md

Preconditions

• Stage 1 is committed and its full pipeline-role commit hash is recorded.
• ansible-collection-dnd is clean at its current main HEAD.
• The exact application-role and collection dependency pins still match the values in Starting State and Preconditions. Stop if a pin has changed.

Owning Repository

ansible-collection-dnd

Required Changes

1. Add galaxy.yml with:

   • namespace rakestraw, name dnd, and development version 0.1.0;
   • README.md, Eric Rakestraw as author, license: [CC0-1.0], a concise D&D pipeline description, the tags dnd, pipeline, transcript, and processing, repository URL https://gitea.maximumdirect.net/eric/ansible-collection-dnd.git, and issue URL https://gitea.maximumdirect.net/eric/ansible-collection-dnd/issues;
   • exact collection dependencies ansible.posix: 2.2.0 and community.general: 13.3.0; and
   • build_ignore entries for the collection's tests, root .gitignore, and the imported role's .gitignore, ansible.cfg, collections, requirements.yml, and tests paths.

2. Add meta/runtime.yml with only the native runtime requirement requires_ansible: ">=2.18.0" and the normal YAML document marker.

3. Add one root requirements.yml using Ansible's unified roles: and collections: format. Copy the five Git sources, canonical installed role names, and exact reviewed commit pins from the pipeline manifests. Include the same exact two collection versions declared in galaxy.yml.

4. Replace the placeholder README with concise documentation covering:

   • purpose, Ubuntu 26.04 support, and the rakestraw.dnd.pipeline FQCN;
   • the unified ansible-galaxy install -r requirements.yml bootstrap;
   • collection build and local artifact installation with --no-deps after bootstrap;
   • a minimal FQCN play example;
   • the distinction between development commit pins and future released role tags; and
   • exact git subtree add and git subtree pull --squash maintenance examples using roles/pipeline and https://gitea.maximumdirect.net/eric/ansible-role-dnd-pipeline.git.

5. Make the subtree documentation explicit that role changes originate in ansible-role-dnd-pipeline; collection maintainers import a reviewed commit and do not edit the snapshot directly.

6. Extend .gitignore only as needed to keep locally built *.tar.gz collection artifacts untracked. The documented/tested workflow should still build into a temporary output directory.

7. Do not create a wrapper role, placeholder role, sync script, lock file, generated manifest, release automation, or publication configuration.

Validation

Run from ansible-collection-dnd:

build_dir=$(mktemp -d)
trap 'rm -rf -- "$build_dir"' EXIT
../.venv/bin/ansible-galaxy collection build --output-path "$build_dir"
test -f "$build_dir/rakestraw-dnd-0.1.0.tar.gz"
git diff --check
git status --short

Expected result: Ansible recognizes and builds collection version 0.1.0 without metadata errors, the expected archive is created outside the repository, and only the planned collection foundation files have changed. The artifact does not yet need to contain the pipeline role.

Completion Criteria

• The collection has valid Galaxy and runtime metadata.
• The root requirements file exactly reproduces both existing dependency manifests in unified form.
• The README gives a complete source-consumer and maintainer workflow without claiming a published release.
• No role code exists in or has been copied into the collection yet.

Commit

Commit intent: Establish the D&D collection foundation

Stage 3: Import the Authoritative Pipeline Role

Objective

Create the self-contained roles/pipeline collection role through one provenance-bearing squashed subtree import of the Stage 1 commit.

Required Policy Review

Before beginning this stage, read:

• AGENTS.md
• docs/orientation.md
• docs/conventions.md
• docs/ansible.md

Preconditions

• Stages 1 and 2 are committed and validated.
• Both ansible-role-dnd-pipeline and ansible-collection-dnd are clean.
• roles/pipeline does not exist.
• The exact Stage 1 pipeline commit is available from the sibling repository.

Owning Repository

ansible-collection-dnd

Required Changes

1. From the collection repository, capture and inspect the full Stage 1 pipeline HEAD, then import exactly that commit with:

   pipeline_commit=$(git -C ../ansible-role-dnd-pipeline rev-parse HEAD)
   git subtree add --prefix=roles/pipeline ../ansible-role-dnd-pipeline "$pipeline_commit" --squash
   

2. Retain the standard subtree-generated provenance commit and trailers. Do not replace the import with a manual copy, submodule, symbolic link, wrapper role, or custom synchronization record.

3. Do not edit, remove, or reformat any file beneath roles/pipeline. Files excluded from the built artifact remain in source so the imported snapshot stays faithful to the authoritative repository.

4. Do not change collection-owned metadata merely to hide an unexpected import result. Stop and report a mismatch between the roadmap, Stage 2 build policy, and the actual role source.

Validation

Run from ansible-collection-dnd immediately after the subtree command:

diff -ru --exclude=.git ../ansible-role-dnd-pipeline roles/pipeline
../.venv/bin/python -c 'import pathlib, yaml; root = yaml.safe_load(pathlib.Path("requirements.yml").read_text()); role = yaml.safe_load(pathlib.Path("roles/pipeline/requirements.yml").read_text()); collections = yaml.safe_load(pathlib.Path("roles/pipeline/collections/requirements.yml").read_text()); assert root["roles"] == role["roles"]; assert root["collections"] == collections["collections"]'
build_dir=$(mktemp -d)
trap 'rm -rf -- "$build_dir"' EXIT
../.venv/bin/ansible-galaxy collection build --output-path "$build_dir"
tar -tf "$build_dir/rakestraw-dnd-0.1.0.tar.gz" > "$build_dir/files.txt"
grep -Fx 'roles/pipeline/tasks/main.yml' "$build_dir/files.txt"
! grep -Eq '^roles/pipeline/(tests/|ansible.cfg$|collections/|requirements.yml$|\.gitignore$)' "$build_dir/files.txt"
git status --short
git log -2 --format=fuller

Expected result: source and imported role trees are byte-for-byte equivalent apart from Git metadata; both dependency sections match their authoritative pipeline manifests; the archive contains the pipeline task entry point but no imported development fixtures or nested manifests; and Git records the standard squashed subtree provenance. The collection working tree is clean because the subtree command created the import commit.

Completion Criteria

• roles/pipeline is an exact snapshot of the Stage 1 pipeline commit.
• The collection artifact is self-contained for the pipeline role.
• Build exclusions affect only the artifact and have not mutated the source snapshot.
• The dependency bootstrap remains consistent with the imported role.

Commit

Commit intent: retain the standard commit created by git subtree add; do not create a second manual import commit.

Stage 4: Validate the Installed Collection Boundary

Objective

Add one lean integration path that proves the built artifact, unified dependency bootstrap, FQCN resolution, representative native configuration, and clean repeated convergence.

Required Policy Review

Before beginning this stage, read:

• AGENTS.md
• docs/orientation.md
• docs/conventions.md
• docs/ansible.md
• docs/testing.md

Preconditions

• Stage 3 is committed and validated.
• ansible-collection-dnd is clean.
• The workspace .venv, passwordless sudo, and both local asset repositories are available.
• No developer-global Ansible installation may satisfy the test's role or collection dependencies.

Owning Repository

ansible-collection-dnd

Required Changes

1. Add tests/inventory for local connection and tests/converge.yml whose only role invocation is rakestraw.dnd.pipeline.

2. Add an executable tests/integration shell driver consistent with the standalone role repositories. It must:

   • use set -euo pipefail, locate the repository and workspace .venv without assuming the caller's working directory, and create one private mktemp -d root with cleanup on exit;
   • set ANSIBLE_ROLES_PATH and ANSIBLE_COLLECTIONS_PATH exclusively to directories beneath that temporary root, without appending the workspace, sibling repositories, or user-global paths;
   • run the unified ansible-galaxy install -r requirements.yml with those environment paths so both dependency sections are exercised;
   • build rakestraw-dnd-0.1.0.tar.gz into the temporary root and install that artifact into the isolated collection path with --no-deps because the unified bootstrap has already installed its exact dependencies;
   • confirm the archive includes the pipeline task entry point and excludes the development-only subtree paths selected in build_ignore;
   • run syntax checking and production-profile ansible-lint against tests/converge.yml only after the isolated dependencies and artifact are installed;
   • derive immutable local asset revisions from assets/campaigns and assets/narratio-prompts, pass their file:// URLs and revisions through the pipeline's public variables, and use the same syntactically valid local WhisperX URL and distinctive synthetic credential values as the standalone suite;
   • apply the collection role without contacting WhisperX or any model provider;
   • use the installed applications' native interfaces for one representative assembled validation: Seriatim reports a version, Audita validates its configuration, Notarius validates dnd-session, Scriptorium resolves a deployed prompt or profile through its configured roots, and Narratio validates one deployed campaign;
   • immediately apply the identical play and variables a second time, capture its play recap, and require changed=0 and failed=0; and
   • remove only its temporary root during cleanup.

3. Keep the test at this packaging boundary. Do not add alternate-version, mapping-composition, invalid-secret, human-operator, ACL, profile-matrix, or exhaustive artifact tests already owned by the standalone suite.

4. Do not assert current default values, task names or order, individual task results, module behavior, complete rendered files, or exact manifest contents. The small archive inclusion/exclusion check is permitted because packaged content is behavior owned specifically by this feature.

5. Do not place real credentials in the test, logs, inventory, or repository. Native configuration checks must remain offline.

Validation

Run from the workspace root:

bash -n ansible-collection-dnd/tests/integration
ansible-collection-dnd/tests/integration
ansible-role-dnd-pipeline/tests/integration
git -C ansible-collection-dnd diff --check
git -C ansible-role-dnd-pipeline status --short

Expected result: the collection driver builds and installs only the declared artifact and dependencies into isolated paths, resolves the FQCN, passes every native application check, and reports a clean immediate repeat. The existing standalone suite also passes, and the authoritative pipeline repository remains clean after testing.

Completion Criteria

• The source collection cannot accidentally satisfy its own FQCN or dependencies through workspace or user-global Ansible paths.
• One representative installed-artifact deployment and every selected native check pass without external service calls.
• The repeat application reports changed=0 and failed=0.
• The standalone pipeline integration suite remains green.
• The test suite contains no implementation snapshots or duplicated feature matrix.

Commit

Commit intent: Test the assembled D&D collection

Stage 5: Record the Implementation Handoff

Objective

Record the completed implementation stages and leave every changed repository clean for independent review without marking the feature complete.

Required Policy Review

Before beginning this stage, read:

• AGENTS.md
• docs/orientation.md
• docs/conventions.md
• docs/planning.md

Preconditions

• Stages 1 through 4 are committed and all validation has passed.
• ansible-role-dnd-pipeline and ansible-collection-dnd are clean.
• Full commit hashes for every implementation commit, including the subtree provenance commit, have been recorded.

Owning Repository

ansible-workspace-dnd

Required Changes

1. Change Feature 0012's roadmap status from ready to in_progress; do not mark it done before independent review.
2. Update docs/status.md so M1-04 and F0012 remain in_progress, the current focus is independent implementation review, and the evidence names the resulting pipeline and collection commit hashes.
3. Replace this document's Progress Notes placeholder with one concise entry per completed stage, including repository and commit hash. Record any deviation from the plan; write none if there were no deviations.
4. Do not create or modify docs/roadmap/remediation.md; independent review owns remediation findings.

Validation

Run from the workspace root:

git diff --check
./scripts/status
git status --short
git -C ansible-role-dnd-pipeline status --short
git -C ansible-collection-dnd status --short

Expected result: only the intended workspace progress documents are pending in the parent repository; both implementation repositories are clean; status points reviewers to concrete commits; and the feature remains in_progress.

Completion Criteria

• Every stage and deviation is recorded concisely.
• Workspace status requests independent review and does not claim completion.
• No remediation document was preemptively created.
• The workspace change is ready for its own commit.

Commit

Commit intent: Record the Feature 0012 implementation

Final Validation and Handoff

• Confirm the successful Stage 4 logs include collection build and isolated installation, FQCN syntax resolution, every selected native application check, and the clean repeat recap.
• Run ./scripts/status and confirm the workspace, ansible-role-dnd-pipeline, and ansible-collection-dnd are clean after the Stage 5 commit. All repositories outside those three must remain unchanged.
• Report the Stage 1 pipeline commit, the Stage 2 collection foundation commit, the Stage 3 subtree import commit, the Stage 4 collection test commit, and the Stage 5 workspace handoff commit.
• Summarize any deviations, unresolved findings, and deferred release work.
• Report any implementation complexity beyond native collection metadata, one unified requirements manifest, the standard subtree import, and the focused integration driver, together with the acceptance criterion that required it.
• Do not push commits and do not mark Feature 0012 done; implementation results require independent review first.

Progress Notes

• Stage 1 complete: ansible-role-dnd-pipeline 600eb21cd967831607c087f1773063636ff2c31b documents standalone and collection consumption without changing role behavior.
• Stage 2 complete: ansible-collection-dnd 3f7f82e40f63ee19e883052822c34dc806d8d51f establishes buildable Galaxy metadata, unified pinned dependencies, build exclusions, and source-use and subtree-maintenance documentation.
• Stage 3 complete: ansible-collection-dnd c84aa0db8feda65ac66af07bc53976a23a016486 imports pipeline commit 600eb21cd967831607c087f1773063636ff2c31b as an exact squashed subtree snapshot at roles/pipeline.