ansible-workspace-dnd/docs/roadmap/implementation.md

# Implementation Plan R0001: Classical Test Suite Remediation

Status: ready

Target executor: GPT-5.6-Terra

Authoritative audit:
`docs/roadmap/test-suite-audit-findings.md`

Active plan location: `docs/roadmap/implementation.md`

## Objective

Replace the oversized, implementation-bound integration suites in the three
completed standalone role repositories with lean classical, state-based
suites. Retain tests of stable role-owned behavior and application outcomes;
remove mutable-default snapshots, task-trace assertions, tests of trusted
Ansible modules, redundant scenarios, and negative platform matrices.

Production role behavior is not changing. This plan changes only test
implementations, test-related role documentation, and the workspace planning
records identified by the audit.

## Execution Rules

- Execute stages in numeric order.
- Before each stage, confirm its owning repository is clean and record its
  starting revision.
- Preserve unrelated and pre-existing changes.
- Do not modify production tasks, defaults, templates, argument
  specifications, dependency pins, or role metadata. If a retained test
  exposes a production defect, stop and report it instead of expanding this
  remediation.
- Follow the normative testing house style in `docs/conventions.md` and the
  classifications in the authoritative audit.
- Exercise roles through their public variables and inspect final application
  behavior. Do not parse task names, task order, or individual task status.
- Retain only aggregate `changed=0` checks on specified immediate repeat runs.
  Do not require a nonzero change count on preceding runs.
- Do not verify ownership, permissions, path types, checkout commits, or other
  postconditions already declared through trusted modules.
- Do not add a test merely to replace each removed test one-for-one. The target
  is the smallest suite that protects the retained behavior.
- Keep each repository's shell driver self-contained. Do not create a shared
  cross-repository test helper or abstraction.
- Run each stage's full repository integration entry point before committing.
- Commit each completed stage separately with the plain-English intent given
  in that stage.
- Do not begin a dependent stage after an earlier stage fails validation.

## Starting State and Preconditions

- The workspace conventions, audit plan, audit findings, status update, and
  this implementation plan must be reviewed and committed in the workspace
  repository before Stage 1 begins.
- `ansible-role-seriatim` starts clean at `fb8dc41`.
- `ansible-role-audita` starts clean at `847c4c3`.
- `ansible-role-notarius` starts clean at `eb4f511`.
- The workspace `.venv` provides Ansible Core and ansible-lint, and the local
  Ubuntu 26.04 `x86_64` host remains the approved MVP integration proxy.
- The executor can use passwordless privilege escalation for the existing
  `/usr/local` integration-test paths and can reach Ansible Galaxy, the three
  application Git repositories, Snap infrastructure, and Go module sources.
- Existing installed application state may be present. Each driver must remain
  repeatable from that state and restore its default role deployment through a
  best-effort cleanup trap.
- Audita exposes no native application-release identity command. Its tags
  `v0.9.0` and `v0.9.1` currently resolve to the same commit. This plan
  deliberately omits an Audita version-selection test rather than substituting
  a raw-commit or Git-module assertion.

## Repository Scope

Repositories that change:

- `ansible-role-seriatim`: lean integration driver and verifier, removal of
  the unsupported-platform playbook, and accurate validation documentation.
- `ansible-role-audita`: lean integration driver and verifiers, removal of the
  unsupported-platform playbook, and accurate validation documentation.
- `ansible-role-notarius`: lean integration driver and verifiers, removal of
  the unsupported-platform playbook, and accurate validation documentation.
- Workspace control repository: test-specific corrections to Features 0001,
  0002, and 0003, remediation status, and cross-repository validation evidence.

Repositories that may be inspected but must remain unchanged:

- `../seriatim`, `../audita`, and `../notarius` for native CLI behavior.
- Every other application, role, collection, and proof-of-concept repository.

## Stage 1: Simplify the Seriatim Test Suite

### Objective

Reduce Seriatim's integration suite to static validation and representative
tests of default usability, explicit version selection, missing-binary
recovery, caller-supplied Go, and clean repeated convergence.

### Preconditions

- The workspace planning commit described above exists.
- `ansible-role-seriatim` is clean at `fb8dc41`.
- The existing `tests/integration` baseline passes.

### Owning Repository

`ansible-role-seriatim`

### Required Changes

1. Retain `tests/integration` as the single entry point, its private temporary
   directory, pinned collection installation, syntax check, production-profile
   ansible-lint, and best-effort cleanup convergence.
2. Retain a default role convergence as a smoke test. Exercise
   `/usr/local/bin/seriatim --version` successfully without comparing it to the
   current default version, then immediately apply the default role again and
   require only the aggregate recap `changed=0` and `failed=0`.
3. Add or retain one explicit version-selection scenario using
   `seriatim_version: v1.4.0`. Verify through the native CLI that the result is
   exactly `seriatim version v1.4.0`. Do not resolve or assert the corresponding
   Git commit.
4. Combine missing-binary recovery and caller-supplied build tooling into one
   representative scenario:
   - create an executable temporary wrapper that records invocation and then
     executes `/snap/bin/go` with the original arguments;
   - remove `/usr/local/bin/seriatim`;
   - apply the role with `seriatim_version: v1.4.0`,
     `seriatim_manage_build_dependencies: false`, and
     `seriatim_go_binary` set to the wrapper;
   - require the wrapper log to show that the supplied executable was used;
   - verify native output `seriatim version v1.4.0`; and
   - immediately repeat with the same variables and require aggregate
     `changed=0` and `failed=0`.
5. Delete `tests/verify.yml` and perform the narrow native version assertions
   directly in the shell driver. Do not retain source, commit, ownership,
   permission, file-type, or checksum inspection.
6. Remove the binary-mode mutation and repair scenario, every
   `require_changed_recap` use, every task-name or individual-status grep, the
   default and previous commit constants, redundant default-restoration
   assertions, and final duplicate shell assertions.
7. Delete `tests/unsupported.yml` and remove it from lint and execution. Do not
   replace its distribution, release, or architecture cases.
8. Keep a cleanup trap that reapplies the default role and removes only the
   private temporary directory. Cleanup output may remain suppressed and must
   not become another behavioral assertion.
9. Rewrite the README validation section to describe only the retained static
   checks and behavioral scenarios. Do not advertise exact-default, metadata,
   task-arrangement, or platform-matrix coverage.

### Validation

Run:

```text
tests/integration
git diff --check
git status --short
```

Expected result: static validation and every retained Seriatim scenario pass;
the specified repeat runs report no changes; cleanup completes; and the only
working-tree changes are the planned test and README edits.

### Completion Criteria

- No test asserts Seriatim's current default version or a Git commit.
- No test inspects task names, individual task status, ownership, permissions,
  or the platform assertion branches.
- Explicit `v1.4.0` selection is verified through Seriatim itself.
- One scenario proves both missing-binary recovery and use of the configured Go
  executable, followed by a clean repeat.
- The complete integration entry point passes.

### Commit

Commit intent: `Simplify the Seriatim integration tests`

## Stage 2: Simplify the Audita Test Suite

### Objective

Reduce Audita's integration suite to static validation and representative
tests of default usability, native configuration validity, explicit
configuration rendering, missing-binary recovery, caller-supplied Go, and clean
repeated convergence.

### Preconditions

- Stage 1 is complete and committed.
- `ansible-role-audita` is clean at `847c4c3`.
- The existing `tests/integration` baseline passes.

### Owning Repository

`ansible-role-audita`

### Required Changes

1. Retain `tests/integration` as the single entry point, its private temporary
   directory, pinned collection installation, syntax check, production-profile
   ansible-lint, and best-effort cleanup convergence.
2. Retain a default role convergence as a smoke test. Validate the managed
   configuration with
   `/usr/local/bin/audita config validate --config /usr/local/etc/audita/config.yml`,
   then immediately repeat the default role and require only aggregate
   `changed=0` and `failed=0`.
3. Replace the exhaustive configuration document and separate falsey scenario
   with one representative explicit configuration. Supply these values:
   - `audita_config_version: 1`;
   - `audita_pipeline_modules: ["glossary"]`;
   - `audita_llm_proposal_base_url: "https://openrouter.ai/api/v1"`;
   - `audita_llm_proposal_model: "openrouter/google/gemma-4-31b-it"`;
   - `audita_llm_proposal_timeout: "10s"`;
   - `audita_llm_proposal_max_retries: 0`;
   - `audita_context_description: ""`;
   - `audita_diagnostics_retention: "never"`;
   - `audita_llm_validation_model: null`;
   - `audita_output_schema: null`; and
   - `audita_diagnostics_work_dir: null`.
4. In `tests/verify-config.yml`, parse the rendered YAML and assert only that:
   - `version` is `1`;
   - `pipeline.modules` is `["glossary"]`;
   - the selected proposal URL, model, timeout, and zero retry count are
     present;
   - the empty context description and diagnostics retention value are
     present;
   - `llm.validation`, `output`, and `diagnostics.work_dir` are absent; and
   - Audita's native configuration validation command succeeds.
   Do not compare the complete parsed document to an expected mapping.
5. Immediately repeat the representative configuration convergence with the
   same explicit variables and require aggregate `changed=0` and `failed=0`.
6. Combine missing-binary recovery and caller-supplied build tooling into one
   representative scenario. Create a logging wrapper around `/snap/bin/go`,
   remove `/usr/local/bin/audita`, converge with
   `audita_manage_build_dependencies: false` and `audita_go_binary` set to the
   wrapper, verify wrapper use, validate the managed configuration through
   Audita, and immediately repeat the same convergence with aggregate
   `changed=0` and `failed=0`.
7. Delete `tests/verify.yml` and invoke Audita's native configuration validator
   directly from the shell driver for the smoke and recovery scenarios.
   Successful command completion is sufficient; do not separately assert the
   validator's fixed success text.
8. Remove the exact default configuration comparison, source-commit checks,
   source/binary/configuration metadata checks, binary-mode repair scenario,
   all `require_changed_recap` uses, and every task-name or individual-status
   grep.
9. Remove the raw alternate-commit scenario entirely. Do not replace it with a
   `git rev-parse`, tag-resolution, embedded-string, or Go build-metadata test.
   Do not add an Audita version-selection test until the application provides a
   stable native release-identity interface.
10. Delete `tests/unsupported.yml` and remove it from lint and execution. Do not
    replace its platform cases.
11. Retain default restoration only in the cleanup trap. Remove its fixed
    commit, exact document, task-status, and extra idempotence assertions.
12. Rewrite the README validation section to describe only the retained static
    checks and behavioral scenarios and to avoid claiming application-version,
    metadata, task-arrangement, or platform-matrix coverage.

### Validation

Run:

```text
tests/integration
git diff --check
git status --short
```

Expected result: static validation and every retained Audita scenario pass;
both default and representative configurations validate natively; the
specified repeats report no changes; cleanup completes; and the only
working-tree changes are the planned test and README edits.

### Completion Criteria

- No test asserts Audita's current default, a source commit, or a complete
  rendered configuration document.
- No test inspects task traces, trusted-module postconditions, or platform
  assertion branches.
- One focused explicit case proves representative nesting, null omission, and
  falsey preservation and passes Audita's validator.
- One scenario proves both missing-binary recovery and use of the configured Go
  executable, followed by a clean repeat.
- The absence of native Audita version coverage is documented accurately and
  is not replaced by an implementation proxy.
- The complete integration entry point passes.

### Commit

Commit intent: `Simplify the Audita integration tests`

## Stage 3: Simplify the Notarius Test Suite

### Objective

Reduce Notarius's integration suite to static validation and representative
tests of default usability, explicit version selection, missing-binary
recovery, caller-supplied Go, configuration behavior, caller-side pipeline
composition, native validation, and clean repeated convergence.

### Preconditions

- Stages 1 and 2 are complete and committed.
- `ansible-role-notarius` is clean at `eb4f511`.
- The existing `tests/integration` baseline passes.

### Owning Repository

`ansible-role-notarius`

### Required Changes

1. Retain `tests/integration` as the single entry point, its private temporary
   directory, pinned collection installation, syntax check, production-profile
   ansible-lint, and best-effort cleanup convergence.
2. Retain a default convergence as a smoke test. Require
   `/usr/local/bin/notarius --version` and
   `/usr/local/bin/notarius config validate --config /usr/local/etc/notarius/config.yml`
   to succeed without comparing the version output or complete configuration
   to current defaults. Immediately repeat the default role and require only
   aggregate `changed=0` and `failed=0`.
3. Retain one explicit version-selection scenario with
   `notarius_version: v0.5.0` and verify exact native output
   `notarius v0.5.0`. Do not resolve or assert the corresponding Git commit.
4. Combine missing-binary recovery and caller-supplied build tooling into one
   scenario. Create a logging wrapper around `/snap/bin/go`, remove
   `/usr/local/bin/notarius`, converge with `notarius_version: v0.5.0`,
   `notarius_manage_build_dependencies: false`, and `notarius_go_binary` set to
   the wrapper, verify wrapper use and native output `notarius v0.5.0`, then
   immediately repeat with aggregate `changed=0` and `failed=0`.
5. Replace the exhaustive configuration, separate profile-source cases,
   separate falsey case, and separate pipeline-composition case with one
   representative explicit configuration. Define two caller-side pipeline
   mappings and set `notarius_pipelines` to their ordinary `combine` result.
   Supply these additional values:
   - `notarius_config_version: 4`;
   - `notarius_promptkit_local_backend_endpoint: "http://localhost:8000/v1"`;
   - `notarius_promptkit_local_backend_concurrency_limit: 0`;
   - `notarius_cache_checkpoints_enabled: false`;
   - `notarius_cache_chunk_plans_directory: ""`;
   - `notarius_output_directory: "/tmp/notarius-output"`;
   - `notarius_promptkit_profile_dir: null`;
   - `notarius_promptkit_profile_file: null`; and
   - `notarius_debug_directory: null`.
   Use the two valid `dnd-session` and `dnd-archive` pipeline definitions from
   the existing composition scenario.
6. In `tests/verify-config.yml`, parse the rendered YAML and assert only that:
   - `version` is `4`;
   - local-backend endpoint and zero concurrency are present;
   - checkpoints are explicitly disabled and the empty chunk-plan directory is
     present;
   - the selected output directory is present;
   - both composed pipeline IDs and their supplied definitions are present
     without rewriting;
   - `promptkit.profile_dir`, `promptkit.profile_file`, and the top-level
     `debug` section are absent; and
   - Notarius's native configuration validation succeeds.
   Do not compare the complete document to an expected mapping.
7. Immediately repeat the representative configuration convergence with the
   same explicit variables and require aggregate `changed=0` and `failed=0`.
8. Delete `tests/verify.yml`. Perform native CLI smoke, configuration
   validation, and the explicit `v0.5.0` comparison directly in the shell
   driver.
9. Remove current-default version and commit assertions, complete default
   configuration comparisons, all source/binary/configuration metadata checks,
   the binary-mode repair scenario, every `require_changed_recap` use,
   `require_configuration_change_without_build`, and every task-name or
   individual-status grep.
10. Delete the standalone `profile_dir` and `profile_file` scenarios. Their
    simple leaf mappings do not require dedicated replacements.
11. Delete `tests/unsupported.yml` and remove it from lint and execution. Do not
    replace its platform cases.
12. Retain default restoration only in the cleanup trap and remove duplicate
    final assertions.
13. Rewrite the README validation section to describe only the retained static
    checks and behavioral scenarios. Do not advertise current-default,
    metadata, task-arrangement, profile-branch, or platform-matrix coverage.

### Validation

Run:

```text
tests/integration
git diff --check
git status --short
```

Expected result: static validation and every retained Notarius scenario pass;
the explicit tag is verified through Notarius; representative falsey, null,
and composed-pipeline behavior is verified and validates natively; the
specified repeats report no changes; cleanup completes; and only the planned
test and README changes remain.

### Completion Criteria

- No test snapshots Notarius's current default release, commit, or complete
  default configuration.
- No test inspects task traces, trusted-module postconditions, or platform
  assertion branches.
- Explicit `v0.5.0` selection is verified through Notarius itself.
- One scenario proves both missing-binary recovery and use of the configured Go
  executable, followed by a clean repeat.
- One representative configuration case proves null omission, falsey
  preservation, caller-side pipeline composition, and native validity.
- The complete integration entry point passes.

### Commit

Commit intent: `Simplify the Notarius integration tests`

## Stage 4: Reconcile Workspace Documentation and Validate the Suite

### Objective

Align durable test-specific roadmap language with the canonical testing policy,
record the remediation implementation for review, and validate all three lean
suites together.

### Preconditions

- Stages 1 through 3 are complete and committed in their owning repositories.
- The workspace repository is clean at the planning commit created before
  Stage 1.
- The three role repositories contain only their committed remediation changes.

### Owning Repository

Workspace control repository (`ansible-workspace-dnd`)

### Required Changes

1. Revise only test-specific language in Features 0001, 0002, and 0003. Retain
   their durable production requirements, including current defaults, paths,
   ownership, permissions, platform support, and convergence behavior.
2. In Feature 0001, describe the exact native version assertion as coverage of
   an explicitly selected immutable tag rather than an assertion about the
   role's current default.
3. In Feature 0002, remove the incorrect claim that tests execute Audita to
   verify its installed application version. State that the current CLI has no
   native release-identity command and that the suite deliberately avoids a
   raw-commit proxy. Retain native configuration validation.
4. In Feature 0003, describe exact version coverage using explicitly selected
   `v0.5.0`, not the current default `v0.6.0`. Keep the current default as a
   production requirement rather than a frozen test expectation.
5. Remove any implication in the three roadmaps that each platform assertion
   must have a negative integration-test matrix. Do not remove platform support
   itself from scope or acceptance criteria.
6. Update `docs/roadmap/test-suite-audit-findings.md` to say remediation was
   implemented and is pending review. Add the three role commit hashes and the
   final validation results; do not erase the audit classifications.
7. Update `docs/status.md` so `R0001` remains `in_progress` pending review, its
   evidence names the three role commits, and the current focus is review of
   the implemented test-suite remediation. Do not mark remediation `done`.
8. Add concise progress notes to this plan with the four stage commits and any
   deviations. Do not replace the active plan with a pending notice before
   review.

### Validation

Run sequentially from the workspace root:

```text
(cd ansible-role-seriatim && tests/integration)
(cd ansible-role-audita && tests/integration)
(cd ansible-role-notarius && tests/integration)
git diff --check
scripts/status
```

Expected result: all static and behavioral checks pass; every role cleanup path
completes; the three role repositories are clean and contain their remediation
commits; no unrelated repository changed; and the workspace has only the
planned documentation changes before its stage commit.

### Completion Criteria

- All three full integration suites pass sequentially from the workspace.
- Feature roadmaps distinguish production defaults from stable test inputs.
- Audita's native version limitation is stated accurately.
- Audit findings contain implementation and validation evidence without being
  rewritten as an implementation log.
- Workspace status leaves remediation pending review.
- Every repository changed by the implementation is clean after its owning
  commit.

### Commit

Commit intent: `Record the test suite remediation results`

## Final Validation and Handoff

- Report the Stage 1 through Stage 4 commit hashes and confirm the working-tree
  state of every repository in `workspace.yml`.
- Summarize the retained scenarios for each role and identify any deviation
  from this plan.
- Report the Audita version-identity limitation; do not silently fill it with a
  source-control assertion.
- Report all validation commands and results.
- Do not push commits unless the user separately authorizes a push.
- Do not mark `R0001` done or replace this plan with a pending notice. Final
  completion requires review of the implemented suites.

## Progress Notes

- Planning complete; implementation has not started.
- [x] Stage 1 complete — `ansible-role-seriatim` commit `e08c63c`; full
  `tests/integration` passed. No deviations.
- [x] Stage 2 complete — `ansible-role-audita` commit `d2d0b9d`; full
  `tests/integration` passed. No deviations.
- [x] Stage 3 complete — `ansible-role-notarius` commit `cfecf37`; full
  `tests/integration` passed. No deviations.