ansible-workspace-dnd/docs/roadmap/implementation.md

Implementation Plan 0017: Notarius Testing Overlay

Status: ready

Target executor: GPT-5.6-Terra

Feature roadmap: 0017-notarius-testing-overlay.md

Objective

Implement the approved roadmap: Narratio testing explicitly selects Notarius testing, with gpt-luna-latest for testing and gpt-terra-latest for production. Keep the standalone roles generic and distribute a coherent public dependency set. The roadmap owns scope and rationale; this document owns execution order.

Execution Rules and Starting State

• Execute stages in order. Before each stage reread its Required Policy Review, inspect owning working trees, and preserve unrelated changes. Paths to policy below are relative to the workspace, not the nested repository.
• Commit validated units with plain-English messages. Record resulting full commit IDs in Progress Notes; never invent future hashes or move release tags.
• Use the existing .venv/bin Ansible tools. Missing PATH entries, a local /etc/ansible, or controller access are not blockers. ansible-doc rendering is not an acceptance gate.
• Repository integration scripts converge this development host and use sudo; use this established local test environment, not the operational controller or pipeline-dnd. Do not provision real credentials or run paid inference.
• Read the scripts before running them. Preserve existing fixture and cleanup conventions; report an actual missing prerequisite rather than expanding scope.
• Trust Ansible modules. Add no runtime validation commands, state fingerprints, cache deletion, force-extraction automation, or profile discovery framework.
• Local validation can use local role sources where the existing suite does; distribution validation must use the declared public revisions.
• Publishing changed role commits to their existing Gitea main branches is required before public dependency installation and subtree import. Publish only completed, validated work; if publication is unavailable, preserve the commits and report that specific distribution blocker.
• No new release/tag or replacement of the published 0.1.0 artifact is required. Use full pushed commit pins for this development compatibility set.

Current versions: Narratio v2.2.0; Notarius v0.7.0, to become v0.7.1. The latter supplies PromptKit v0.10.0 and OpenRouter catalog v2.20260922.1, which contains both selected profile IDs. Existing OPENROUTER_API_KEY delivery is sufficient. The standalone role tests deliberately select older application versions in some cases; do not update those merely to match a new default.

Repository scope: change the Narratio and Notarius roles, pipeline role, collection, consumer rakestraw-ansible, and workspace planning records. Applications, other standalone roles, and asset repositories are read-only.

Stage 1: Expose Narratio's Native Profile Selector

Required Policy Review

Read AGENTS.md, docs/orientation.md, docs/conventions.md, docs/ansible.md, docs/testing.md, and the Feature 0017 roadmap.

Owning Repository and Preconditions

ansible-role-narratio. Its default already selects Narratio v2.2.0. Read that tag's configuration contract for notarius.config_profile.

Required Changes

1. Add narratio_notarius_config_profile: null to defaults and an optional string entry to meta/argument_specs.yml; document the public variable.
2. Include the variable in the enclosing Notarius-section condition in templates/pipeline.yml.j2. Render config_profile with to_json only when the variable is not none. Preserve explicit empty strings and exact values. Add no name validation or file discovery.
3. Extend the existing representative configuration tests with a supplied nonempty selector, null omission, and intentional empty-string rendering. Keep fixtures small; no downstream inference or overlay file is necessary merely to validate Narratio's native selector field. Existing tests against older application versions must leave the new field unset.

Validation and Completion

From this repository run tests/integration and git diff --check. The public variable renders correctly and native configuration validation and clean repeat pass. Extend existing tests rather than creating a parallel harness.

Commit

Expose the Notarius configuration profile in the Narratio role. Push this validated commit to origin/main and record its full ID as NARRATIO_ROLE.

Stage 2: Adopt Notarius v0.7.1

Required Policy Review

Read AGENTS.md, docs/orientation.md, docs/conventions.md, docs/ansible.md, docs/testing.md, and the Feature 0017 roadmap.

Owning Repository and Preconditions

ansible-role-notarius. Stage 1 is complete. Inspect v0.7.1 release notes and the role's existing integration entry point.

Required Changes

Set notarius_version to v0.7.1 and update the README's current default. No role configuration or task change is needed for this dependency upgrade. Retain explicitly selected historical-version tests. Do not copy new provider catalogs or model policy into this generic role.

Validation and Completion

Run tests/integration and git diff --check in this repository. The normal default convergence uses v0.7.1, native checks pass, and repeat convergence is clean. Preserve tests of caller version selection instead of asserting the default.

Commit

Install Notarius v0.7.1 by default. Push the validated commit and record its full ID as NOTARIUS_ROLE.

Stage 3: Compose and Validate the Suite Testing Overlay

Required Policy Review

Read AGENTS.md, docs/orientation.md, docs/conventions.md, docs/ansible.md, docs/testing.md, and the Feature 0017 roadmap.

Owning Repository and Preconditions

ansible-role-dnd-pipeline. Stages 1-2 are validated and publicly available. Review tests/integration, especially its profile diff, inventory override, and alternate pipeline composition cases.

Required Changes

1. Pin the Narratio and Notarius roles in requirements.yml to NARRATIO_ROLE and NOTARIUS_ROLE. Leave unrelated dependencies unchanged.

2. Set dnd_pipeline_notarius_version: v0.7.1. In the existing default dnd_pipeline_notarius_pipelines mapping, change the base dnd-session llm_profile to gpt-terra-latest. Do not introduce a second production profile variable: the mapping remains its inventory override interface.

3. Add dnd_pipeline_notarius_testing_llm_profile: gpt-luna-latest, its string argument specification, and README documentation.

4. Extend ordinary directory management to create /usr/local/etc/notarius/profiles. Add one template for /usr/local/etc/notarius/profiles/testing.yml containing only:

   pipelines:
     <selected suite pipeline ID>:
       llm_profile: <testing LLM profile variable>
   

   Serialize both dynamic key and value safely. Use the existing dnd_pipeline_narratio_notarius_pipeline_id for the key. Manage the file and directory with existing runtime owner/group and configuration modes. Do not hard-code dnd-session or copy the full pipeline into the overlay.

5. Add notarius: {config_profile: testing} to Narratio's existing testing template without changing its Scriptorium settings. Leave the base Narratio selector unset. No extra suite-level selector variable or PromptKit asset is needed. ConfigKit finds the overlay normally; do not import it into base.

6. Document base/testing choices, mapping overrides, native binding precedence, moving catalog IDs, unchanged credential reference, and v0.7.1 checkpoint compatibility. Keep Notarius enabled and caching unchanged.

7. Extend the existing integration script to inspect base and testing effective configurations through the applications' native commands. Use explicit caller-supplied values for assertions; for example, explicitly select gpt-luna-latest in the production fixture and gpt-terra-latest in the testing fixture to demonstrate independence from defaults. Validate the selected pipeline with Notarius --config-profile testing, without inference. Exercise the existing alternate pipeline ID case with the testing overlay to prove the key follows inventory. Do not add a binding-precedence matrix.

8. Replace the old test restriction that Narratio testing differences may only end in .profile_id: it excludes the new .notarius.config_profile setting. Prefer assertions of the selected effective fields over an exhaustive diff allowlist. Retain useful Scriptorium override coverage and clean-repeat checks.

Validation and Completion

Run tests/integration and git diff --check. Required outcomes: native validation passes for base and testing; selected inventory values are visible; base remains independent of testing; alternate pipeline composition works; immediate repeat is clean. No real keys or provider requests are required.

Commit

Add the Notarius testing overlay and production model defaults. Push the validated commit and record its full ID as PIPELINE_ROLE.

Stage 4: Assemble the Public Collection Snapshot

Required Policy Review

Read AGENTS.md, docs/orientation.md, docs/conventions.md, docs/ansible.md, docs/testing.md, and the collection README's subtree policy.

Owning Repository and Preconditions

ansible-collection-dnd. Stage 3 and its dependency commits are pushed.

Required Changes

1. Import PIPELINE_ROLE from Gitea using the documented workflow:

   git subtree pull --prefix=roles/pipeline \
     https://gitea.maximumdirect.net/eric/ansible-role-dnd-pipeline.git \
     FULL_PIPELINE_ROLE_COMMIT --squash \
     -m 'Bundle the Notarius testing overlay'
   

   Substitute the actual full commit ID. Do not edit the subtree directly.

2. Update collection-root requirements.yml to the same Narratio/Notarius role commits used by the pipeline manifest. Keep unrelated pins unchanged.

3. Extend the collection's existing installed-artifact smoke coverage to validate Notarius testing and inspect Narratio's effective downstream selector. Reuse Stage 3's detailed behavior evidence rather than copying its whole test suite.

4. Keep version 0.1.0 and published tags/artifacts unchanged. Document the current development snapshot where necessary without rewriting historical release claims.

Validation and Completion

Run tests/integration and git diff --check. This must build/install the collection with its public declared roles and pass native base/testing checks and clean repeat. Compare git rev-parse HEAD:roles/pipeline with the source pipeline commit's root tree: they must match. No sibling-role substitution.

Commit

The subtree command creates its own commit. Commit remaining changes as Validate and pin the testing overlay collection dependencies. Push the completed collection and record its full final ID as COLLECTION.

Stage 5: Update Consumer Pins and Record Review Handoff

Required Policy Review

Read AGENTS.md, docs/orientation.md, docs/conventions.md, docs/ansible.md, docs/testing.md, docs/planning.md, and Feature 0017.

Owning Repositories and Preconditions

rakestraw-ansible and workspace. Stage 4 is validated and pushed.

Required Changes

1. In the consumer manifest pin the collection to COLLECTION, and Narratio and Notarius roles to their corresponding public commits. Preserve unrelated pins.
2. Update the collection refresh command in docs/pipeline-dnd.md to COLLECTION. Explain the testing overlay behavior and v0.7.1 cache compatibility concisely. Preserve the minimal hosts/roles playbook, host inventory, and secret references. No host-variable copies of the new suite defaults are needed.
3. Update workspace status and this plan's Progress Notes with commit IDs and validation evidence. Mark Feature 0017 in_progress pending independent review, not done. Preserve the durable roadmap; no new feature scope is introduced.

Validation and Completion

Run git diff --check in both repositories. Reuse Stage 4's successful public dependency installation for the same dependency set. With those isolated role and collection paths and workspace .venv/bin tools, run from the consumer:

ansible-playbook -i inventory playbooks/pipeline/dnd.yml --limit pipeline-dnd --list-hosts
ansible-playbook -i inventory playbooks/pipeline/dnd.yml --limit pipeline-dnd --syntax-check

Supply explicit ANSIBLE_ROLES_PATH and ANSIBLE_COLLECTIONS_PATH for the isolated installation. Do not install to the controller's /etc/ansible or connect to the target. If unrelated encrypted inventory prevents local checks, use a disposable inventory with the target alias and non-secret settings and report that scope. Missing real Vault values are not a feature blocker; do not populate them.

Commits

Consumer: Select the collection with Notarius testing profiles. Workspace: Record the Notarius testing overlay implementation for review. Push the consumer update so the controller can retrieve the published pins.

Final Validation and Handoff

Report all resulting commits, validation performed, and any real blockers or deviations. Confirm changed working trees are clean. Distinguish repository readiness from an operational deployment; none is required or claimed. Note any complexity added beyond the planned declarative tasks and its requirement.

Progress Notes

Implementation plan approved by the user; implementation has not started.

Open Questions

None. Production and testing selections, upstream versions, public commit-pin distribution, and ownership boundaries are settled for this plan.