ansible-workspace-dnd/docs/roadmap/implementation.md

# Implementation Plan 0012: Assemble the D&D Collection

Status: proposed

Target executor: GPT-5.6-Terra

Feature roadmap:
`docs/roadmap/features/0012-collection-assembly.md`

Active plan location: `docs/roadmap/implementation.md`

## Objective

Build the initial `rakestraw.dnd` collection around the independently
maintained pipeline role. The completed source repository must build and
install as collection version `0.1.0`, expose the imported role as
`rakestraw.dnd.pipeline`, bootstrap the exact reviewed standalone role and
collection dependencies, and prove the packaged boundary through one focused
isolated integration path. The feature roadmap remains authoritative for
scope, rationale, target state, and acceptance criteria.

## Execution Rules

- Execute stages in numeric order.
- Number stages with consecutive integers beginning with `Stage 1`. Do not use
  zero-based, decimal, nested, or nonsequential stage identifiers.
- Before each stage, confirm its preconditions and the working-tree state of
  its owning repository.
- Before each stage, read every document listed in that stage's Required Policy
  Review. Do not rely on policy recalled from an earlier stage.
- Preserve unrelated and pre-existing changes.
- Do not make unresolved product or architecture decisions during execution;
  stop and report the decision required.
- Start from the feature roadmap's simplest viable implementation. Use native
  Galaxy metadata, dependency manifests, collection layout, and Git subtree
  behavior rather than adding wrappers, generators, synchronization scripts,
  or private state.
- Do not alter pipeline behavior in the standalone repository or beneath the
  imported `roles/pipeline` prefix. Any discovered behavioral defect belongs
  in the standalone pipeline repository and requires a separately reviewed
  change before a new subtree import.
- Do not add integrity, drift, release, publishing, signing, rollback, or
  offline-installation guarantees beyond the feature roadmap.
- Keep collection tests focused on the packaging boundary. Do not copy the
  standalone pipeline role's full integration matrix.
- Run every stage's validation before committing it. Stage 3 is the sole
  mechanical exception: `git subtree add` necessarily creates its provenance
  commit as part of the import, so run its validation immediately afterward
  and do not proceed if it fails.
- Commit each completed stage when it leaves the repository in a valid state,
  using the commit intent specified by that stage. Do not push any repository.
- Do not begin a dependent stage after an earlier stage fails validation.

## Starting State and Preconditions

- The workspace planning changes for Feature 0012, including this plan, have
  been reviewed and committed before execution begins.
- `ansible-workspace-dnd`, `ansible-role-dnd-pipeline`, and
  `ansible-collection-dnd` begin on `main` with clean working trees.
- The standalone pipeline role begins at `de1b191cd2b6895895d3915e7b453596994127cf`.
  Stage 1 will create the new reviewed commit that Stage 3 imports.
- The collection begins at `976277b03374`. It contains no role or Galaxy
  metadata yet; Feature 0012 planning files belong to the independent parent
  workspace repository.
- The pipeline dependency manifests select these exact application-role
  commits, which the collection bootstrap manifest must preserve:

  - `rakestraw.seriatim` at
    `e08c63c0b078658bdfa2d10a2e6e1e4f9372a075`
  - `rakestraw.audita` at
    `66ebea9a47ce82bfdbdc5e2855f3dfaaff8e2a16`
  - `rakestraw.notarius` at
    `14a38e04e35ef524d6a97fa7fb5a04981a0a2161`
  - `rakestraw.scriptorium` at
    `8b4c86e48e7d8655e479b26dc510bf01683a0c8f`
  - `rakestraw.narratio` at
    `292d8669e7aad5cfe80b0c469d22b6177e8cf0f1`

- Collection dependencies remain `ansible.posix` 2.2.0 and
  `community.general` 13.3.0.
- The workspace `.venv` provides Ansible Core 2.21.3 and ansible-lint 26.8.0.
  Invoke them through `.venv/bin` rather than assuming they are on `PATH`.
- Git includes the `git subtree` command.
- The local Ubuntu 26.04 proxy host supports passwordless noninteractive
  `sudo`, as required by the existing role integration suites.
- Network access is available for Galaxy dependencies and the pinned Git role
  sources. Local immutable `file://` inputs from `assets/campaigns` and
  `assets/narratio-prompts` remain available for integration testing.

## Repository Scope

- `ansible-role-dnd-pipeline` may change only its README to document the two
  supported consumption forms. It remains authoritative for the role.
- `ansible-collection-dnd` owns Galaxy and runtime metadata, the unified
  bootstrap manifest, collection documentation, build exclusions, the
  squashed pipeline-role subtree, and focused collection integration tests.
- `ansible-workspace-dnd` may change only planning and status documents during
  the final handoff stage.
- The five standalone application roles, both asset repositories, all
  application repositories, and `rakestraw-ansible` may be inspected but must
  remain unchanged.

## Stage 1: Document Standalone and Collection Consumption

### Objective

Make the authoritative pipeline role's README accurately describe both its
standalone role name and its collection FQCN before taking the collection
snapshot.

### Required Policy Review

Before beginning this stage, read:

- `AGENTS.md`
- `docs/orientation.md`
- `docs/conventions.md`
- `docs/ansible.md`

### Preconditions

- `ansible-role-dnd-pipeline` is clean at
  `de1b191cd2b6895895d3915e7b453596994127cf`.
- Feature 0012 is committed and records the standalone repository as the
  authoritative role source.

### Owning Repository

`ansible-role-dnd-pipeline`

### Required Changes

1. Revise `README.md` so it no longer describes collection exposure only as a
   future possibility.
2. Retain the existing standalone installation and
   `rakestraw.dnd_pipeline` invocation instructions.
3. Add a concise collection-use path that names `rakestraw.dnd.pipeline` and
   directs collection consumers to the collection repository's unified
   bootstrap instructions. Do not duplicate the collection's complete build,
   dependency, or subtree maintenance documentation here.
4. Explain that this repository remains authoritative and the collection
   carries a reviewed snapshot. Do not claim that a collection release has
   been published or tagged.
5. Make no changes to defaults, tasks, templates, metadata, dependency pins,
   tests, or role behavior.

### Validation

Run from `ansible-role-dnd-pipeline`:

```text
git diff --check
git diff -- README.md
git status --short
```

Expected result: the diff is limited to accurate README wording, contains no
whitespace errors, preserves the standalone instructions, and introduces no
claim of an already published collection release.

### Completion Criteria

- Both role names and their distinct consumption contexts are documented.
- No executable or dependency file changed.
- The repository is ready to supply the exact commit for the subtree import.

### Commit

Commit intent: `Document standalone and collection pipeline usage`

## Stage 2: Establish the Collection Foundation

### Objective

Create the valid, buildable collection metadata, reproducible dependency
bootstrap, build policy, and operator documentation that will own the imported
role.

### Required Policy Review

Before beginning this stage, read:

- `AGENTS.md`
- `docs/orientation.md`
- `docs/conventions.md`
- `docs/ansible.md`

### Preconditions

- Stage 1 is committed and its full pipeline-role commit hash is recorded.
- `ansible-collection-dnd` is clean at its current `main` HEAD.
- The exact application-role and collection dependency pins still match the
  values in Starting State and Preconditions. Stop if a pin has changed.

### Owning Repository

`ansible-collection-dnd`

### Required Changes

1. Add `galaxy.yml` with:

   - namespace `rakestraw`, name `dnd`, and development version `0.1.0`;
   - `README.md`, Eric Rakestraw as author, `license: [CC0-1.0]`, a concise D&D
     pipeline description, the tags `dnd`, `pipeline`, `transcript`, and
     `processing`, repository URL
     `https://gitea.maximumdirect.net/eric/ansible-collection-dnd.git`, and
     issue URL
     `https://gitea.maximumdirect.net/eric/ansible-collection-dnd/issues`;
   - exact collection dependencies `ansible.posix: 2.2.0` and
     `community.general: 13.3.0`; and
   - `build_ignore` entries for the collection's `tests`, root `.gitignore`,
     and the imported role's `.gitignore`, `ansible.cfg`, `collections`,
     `requirements.yml`, and `tests` paths.

2. Add `meta/runtime.yml` with only the native runtime requirement
   `requires_ansible: ">=2.18.0"` and the normal YAML document marker.
3. Add one root `requirements.yml` using Ansible's unified `roles:` and
   `collections:` format. Copy the five Git sources, canonical installed role
   names, and exact reviewed commit pins from the pipeline manifests. Include
   the same exact two collection versions declared in `galaxy.yml`.
4. Replace the placeholder README with concise documentation covering:

   - purpose, Ubuntu 26.04 support, and the `rakestraw.dnd.pipeline` FQCN;
   - the unified `ansible-galaxy install -r requirements.yml` bootstrap;
   - collection build and local artifact installation with `--no-deps` after
     bootstrap;
   - a minimal FQCN play example;
   - the distinction between development commit pins and future released role
     tags; and
   - exact `git subtree add` and `git subtree pull --squash` maintenance
     examples using `roles/pipeline` and
     `https://gitea.maximumdirect.net/eric/ansible-role-dnd-pipeline.git`.

5. Make the subtree documentation explicit that role changes originate in
   `ansible-role-dnd-pipeline`; collection maintainers import a reviewed commit
   and do not edit the snapshot directly.
6. Extend `.gitignore` only as needed to keep locally built `*.tar.gz`
   collection artifacts untracked. The documented/tested workflow should
   still build into a temporary output directory.
7. Do not create a wrapper role, placeholder role, sync script, lock file,
   generated manifest, release automation, or publication configuration.

### Validation

Run from `ansible-collection-dnd`:

```text
build_dir=$(mktemp -d)
trap 'rm -rf -- "$build_dir"' EXIT
../.venv/bin/ansible-galaxy collection build --output-path "$build_dir"
test -f "$build_dir/rakestraw-dnd-0.1.0.tar.gz"
git diff --check
git status --short
```

Expected result: Ansible recognizes and builds collection version `0.1.0`
without metadata errors, the expected archive is created outside the
repository, and only the planned collection foundation files have changed.
The artifact does not yet need to contain the pipeline role.

### Completion Criteria

- The collection has valid Galaxy and runtime metadata.
- The root requirements file exactly reproduces both existing dependency
  manifests in unified form.
- The README gives a complete source-consumer and maintainer workflow without
  claiming a published release.
- No role code exists in or has been copied into the collection yet.

### Commit

Commit intent: `Establish the D&D collection foundation`

## Stage 3: Import the Authoritative Pipeline Role

### Objective

Create the self-contained `roles/pipeline` collection role through one
provenance-bearing squashed subtree import of the Stage 1 commit.

### Required Policy Review

Before beginning this stage, read:

- `AGENTS.md`
- `docs/orientation.md`
- `docs/conventions.md`
- `docs/ansible.md`

### Preconditions

- Stages 1 and 2 are committed and validated.
- Both `ansible-role-dnd-pipeline` and `ansible-collection-dnd` are clean.
- `roles/pipeline` does not exist.
- The exact Stage 1 pipeline commit is available from the sibling repository.

### Owning Repository

`ansible-collection-dnd`

### Required Changes

1. From the collection repository, capture and inspect the full Stage 1
   pipeline HEAD, then import exactly that commit with:

   ```text
   pipeline_commit=$(git -C ../ansible-role-dnd-pipeline rev-parse HEAD)
   git subtree add --prefix=roles/pipeline ../ansible-role-dnd-pipeline "$pipeline_commit" --squash
   ```

2. Retain the standard subtree-generated provenance commit and trailers. Do
   not replace the import with a manual copy, submodule, symbolic link, wrapper
   role, or custom synchronization record.
3. Do not edit, remove, or reformat any file beneath `roles/pipeline`. Files
   excluded from the built artifact remain in source so the imported snapshot
   stays faithful to the authoritative repository.
4. Do not change collection-owned metadata merely to hide an unexpected import
   result. Stop and report a mismatch between the roadmap, Stage 2 build policy,
   and the actual role source.

### Validation

Run from `ansible-collection-dnd` immediately after the subtree command:

```text
diff -ru --exclude=.git ../ansible-role-dnd-pipeline roles/pipeline
../.venv/bin/python -c 'import pathlib, yaml; root = yaml.safe_load(pathlib.Path("requirements.yml").read_text()); role = yaml.safe_load(pathlib.Path("roles/pipeline/requirements.yml").read_text()); collections = yaml.safe_load(pathlib.Path("roles/pipeline/collections/requirements.yml").read_text()); assert root["roles"] == role["roles"]; assert root["collections"] == collections["collections"]'
build_dir=$(mktemp -d)
trap 'rm -rf -- "$build_dir"' EXIT
../.venv/bin/ansible-galaxy collection build --output-path "$build_dir"
tar -tf "$build_dir/rakestraw-dnd-0.1.0.tar.gz" > "$build_dir/files.txt"
grep -Fx 'roles/pipeline/tasks/main.yml' "$build_dir/files.txt"
! grep -Eq '^roles/pipeline/(tests/|ansible.cfg$|collections/|requirements.yml$|\.gitignore$)' "$build_dir/files.txt"
git status --short
git log -2 --format=fuller
```

Expected result: source and imported role trees are byte-for-byte equivalent
apart from Git metadata; both dependency sections match their authoritative
pipeline manifests; the archive contains the pipeline task entry point but no
imported development fixtures or nested manifests; and Git records the
standard squashed subtree provenance. The collection working tree is clean
because the subtree command created the import commit.

### Completion Criteria

- `roles/pipeline` is an exact snapshot of the Stage 1 pipeline commit.
- The collection artifact is self-contained for the pipeline role.
- Build exclusions affect only the artifact and have not mutated the source
  snapshot.
- The dependency bootstrap remains consistent with the imported role.

### Commit

Commit intent: retain the standard commit created by `git subtree add`; do not
create a second manual import commit.

## Stage 4: Validate the Installed Collection Boundary

### Objective

Add one lean integration path that proves the built artifact, unified
dependency bootstrap, FQCN resolution, representative native configuration,
and clean repeated convergence.

### Required Policy Review

Before beginning this stage, read:

- `AGENTS.md`
- `docs/orientation.md`
- `docs/conventions.md`
- `docs/ansible.md`
- `docs/testing.md`

### Preconditions

- Stage 3 is committed and validated.
- `ansible-collection-dnd` is clean.
- The workspace `.venv`, passwordless `sudo`, and both local asset
  repositories are available.
- No developer-global Ansible installation may satisfy the test's role or
  collection dependencies.

### Owning Repository

`ansible-collection-dnd`

### Required Changes

1. Add `tests/inventory` for local connection and `tests/converge.yml` whose
   only role invocation is `rakestraw.dnd.pipeline`.
2. Add an executable `tests/integration` shell driver consistent with the
   standalone role repositories. It must:

   - use `set -euo pipefail`, locate the repository and workspace `.venv`
     without assuming the caller's working directory, and create one private
     `mktemp -d` root with cleanup on exit;
   - set `ANSIBLE_ROLES_PATH` and `ANSIBLE_COLLECTIONS_PATH` exclusively to
     directories beneath that temporary root, without appending the workspace,
     sibling repositories, or user-global paths;
   - run the unified `ansible-galaxy install -r requirements.yml` with those
     environment paths so both dependency sections are exercised;
   - build `rakestraw-dnd-0.1.0.tar.gz` into the temporary root and install that
     artifact into the isolated collection path with `--no-deps` because the
     unified bootstrap has already installed its exact dependencies;
   - confirm the archive includes the pipeline task entry point and excludes
     the development-only subtree paths selected in `build_ignore`;
   - run syntax checking and production-profile ansible-lint against
     `tests/converge.yml` only after the isolated dependencies and artifact are
     installed;
   - derive immutable local asset revisions from `assets/campaigns` and
     `assets/narratio-prompts`, pass their `file://` URLs and revisions through
     the pipeline's public variables, and use the same syntactically valid
     local WhisperX URL and distinctive synthetic credential values as the
     standalone suite;
   - apply the collection role without contacting WhisperX or any model
     provider;
   - use the installed applications' native interfaces for one representative
     assembled validation: Seriatim reports a version, Audita validates its
     configuration, Notarius validates `dnd-session`, Scriptorium resolves a
     deployed prompt or profile through its configured roots, and Narratio
     validates one deployed campaign;
   - immediately apply the identical play and variables a second time, capture
     its play recap, and require `changed=0` and `failed=0`; and
   - remove only its temporary root during cleanup.

3. Keep the test at this packaging boundary. Do not add alternate-version,
   mapping-composition, invalid-secret, human-operator, ACL, profile-matrix,
   or exhaustive artifact tests already owned by the standalone suite.
4. Do not assert current default values, task names or order, individual task
   results, module behavior, complete rendered files, or exact manifest
   contents. The small archive inclusion/exclusion check is permitted because
   packaged content is behavior owned specifically by this feature.
5. Do not place real credentials in the test, logs, inventory, or repository.
   Native configuration checks must remain offline.

### Validation

Run from the workspace root:

```text
bash -n ansible-collection-dnd/tests/integration
ansible-collection-dnd/tests/integration
ansible-role-dnd-pipeline/tests/integration
git -C ansible-collection-dnd diff --check
git -C ansible-role-dnd-pipeline status --short
```

Expected result: the collection driver builds and installs only the declared
artifact and dependencies into isolated paths, resolves the FQCN, passes every
native application check, and reports a clean immediate repeat. The existing
standalone suite also passes, and the authoritative pipeline repository
remains clean after testing.

### Completion Criteria

- The source collection cannot accidentally satisfy its own FQCN or
  dependencies through workspace or user-global Ansible paths.
- One representative installed-artifact deployment and every selected native
  check pass without external service calls.
- The repeat application reports `changed=0` and `failed=0`.
- The standalone pipeline integration suite remains green.
- The test suite contains no implementation snapshots or duplicated feature
  matrix.

### Commit

Commit intent: `Test the assembled D&D collection`

## Stage 5: Record the Implementation Handoff

### Objective

Record the completed implementation stages and leave every changed repository
clean for independent review without marking the feature complete.

### Required Policy Review

Before beginning this stage, read:

- `AGENTS.md`
- `docs/orientation.md`
- `docs/conventions.md`
- `docs/planning.md`

### Preconditions

- Stages 1 through 4 are committed and all validation has passed.
- `ansible-role-dnd-pipeline` and `ansible-collection-dnd` are clean.
- Full commit hashes for every implementation commit, including the subtree
  provenance commit, have been recorded.

### Owning Repository

`ansible-workspace-dnd`

### Required Changes

1. Change Feature 0012's roadmap status from `ready` to `in_progress`; do not
   mark it `done` before independent review.
2. Update `docs/status.md` so M1-04 and F0012 remain `in_progress`, the current
   focus is independent implementation review, and the evidence names the
   resulting pipeline and collection commit hashes.
3. Replace this document's Progress Notes placeholder with one concise entry
   per completed stage, including repository and commit hash. Record any
   deviation from the plan; write `none` if there were no deviations.
4. Do not create or modify `docs/roadmap/remediation.md`; independent review
   owns remediation findings.

### Validation

Run from the workspace root:

```text
git diff --check
./scripts/status
git status --short
git -C ansible-role-dnd-pipeline status --short
git -C ansible-collection-dnd status --short
```

Expected result: only the intended workspace progress documents are pending in
the parent repository; both implementation repositories are clean; status
points reviewers to concrete commits; and the feature remains `in_progress`.

### Completion Criteria

- Every stage and deviation is recorded concisely.
- Workspace status requests independent review and does not claim completion.
- No remediation document was preemptively created.
- The workspace change is ready for its own commit.

### Commit

Commit intent: `Record the Feature 0012 implementation`

## Final Validation and Handoff

- Confirm the successful Stage 4 logs include collection build and isolated
  installation, FQCN syntax resolution, every selected native application
  check, and the clean repeat recap.
- Run `./scripts/status` and confirm the workspace,
  `ansible-role-dnd-pipeline`, and `ansible-collection-dnd` are clean after the
  Stage 5 commit. All repositories outside those three must remain unchanged.
- Report the Stage 1 pipeline commit, the Stage 2 collection foundation commit,
  the Stage 3 subtree import commit, the Stage 4 collection test commit, and
  the Stage 5 workspace handoff commit.
- Summarize any deviations, unresolved findings, and deferred release work.
- Report any implementation complexity beyond native collection metadata, one
  unified requirements manifest, the standard subtree import, and the focused
  integration driver, together with the acceptance criterion that required it.
- Do not push commits and do not mark Feature 0012 `done`; implementation
  results require independent review first.

## Progress Notes

- Stage 1 complete: `ansible-role-dnd-pipeline`
  `600eb21cd967831607c087f1773063636ff2c31b` documents standalone and
  collection consumption without changing role behavior.
- Stage 2 complete: `ansible-collection-dnd`
  `3f7f82e40f63ee19e883052822c34dc806d8d51f` establishes buildable Galaxy
  metadata, unified pinned dependencies, build exclusions, and source-use and
  subtree-maintenance documentation.
- Stage 3 complete: `ansible-collection-dnd`
  `c84aa0db8feda65ac66af07bc53976a23a016486` imports pipeline commit
  `600eb21cd967831607c087f1773063636ff2c31b` as an exact squashed subtree
  snapshot at `roles/pipeline`.
- Stage 4 complete: `ansible-collection-dnd`
  `10ad31c176b50a64dc30e4aae81167b359359ffe` adds an isolated installed-artifact
  integration path; both the collection test and standalone pipeline suite
  pass.
- Deviations: none.