eric/ansible-workspace-dnd
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Plan the Notarius testing overlay implementation

Browse Source
This commit is contained in:
Eric Rakestraw
2026-09-22 18:23:20 +00:00
parent 58738e4efc
commit 38dfb0422c
3 changed files with 480 additions and 14 deletions
Download Patch File Download Diff File Expand all files Collapse all files

298
docs/roadmap/implementation.md
View File

@@ -1,12 +1,292 @@
# Implementation Plan
# Implementation Plan 0017: Notarius Testing Overlay
No implementation plan is currently active. Select and agree on the next unit
of work before replacing this notice with its staged plan.
Status: ready
Feature 0016 is complete. Its durable scope and completion evidence are in
[the feature roadmap](features/0016-deploy-dnd-pipeline-to-pipeline-dnd.md)
and [workspace status](../status.md). Git history preserves the completed plan.
Target executor: GPT-5.6-Terra
Completion covers the committed deployment playbook, host configuration,
dependency manifest, and operator documentation—not deployment or runtime
acceptance on the operational host.
Feature roadmap: [0017-notarius-testing-overlay.md](features/0017-notarius-testing-overlay.md)
## Objective
Implement the approved roadmap: Narratio testing explicitly selects Notarius
testing, with `gpt-luna-latest` for testing and `gpt-terra-latest` for production.
Keep the standalone roles generic and distribute a coherent public dependency
set. The roadmap owns scope and rationale; this document owns execution order.
## Execution Rules and Starting State
- Execute stages in order. Before each stage reread its Required Policy Review,
inspect owning working trees, and preserve unrelated changes. Paths to policy
below are relative to the workspace, not the nested repository.
- Commit validated units with plain-English messages. Record resulting full
commit IDs in Progress Notes; never invent future hashes or move release tags.
- Use the existing `.venv/bin` Ansible tools. Missing PATH entries, a local
`/etc/ansible`, or controller access are not blockers. `ansible-doc` rendering
is not an acceptance gate.
- Repository integration scripts converge this development host and use sudo;
use this established local test environment, not the operational controller
or `pipeline-dnd`. Do not provision real credentials or run paid inference.
- Read the scripts before running them. Preserve existing fixture and cleanup
conventions; report an actual missing prerequisite rather than expanding scope.
- Trust Ansible modules. Add no runtime validation commands, state fingerprints,
cache deletion, force-extraction automation, or profile discovery framework.
- Local validation can use local role sources where the existing suite does;
distribution validation must use the declared public revisions.
- Publishing changed role commits to their existing Gitea main branches is
required before public dependency installation and subtree import. Publish
only completed, validated work; if publication is unavailable, preserve the
commits and report that specific distribution blocker.
- No new release/tag or replacement of the published 0.1.0 artifact is required.
Use full pushed commit pins for this development compatibility set.
Current versions: Narratio v2.2.0; Notarius v0.7.0, to become v0.7.1.
The latter supplies PromptKit v0.10.0 and OpenRouter catalog v2.20260922.1,
which contains both selected profile IDs. Existing OPENROUTER_API_KEY delivery
is sufficient. The standalone role tests deliberately select older application
versions in some cases; do not update those merely to match a new default.
Repository scope: change the Narratio and Notarius roles, pipeline role,
collection, consumer `rakestraw-ansible`, and workspace planning records.
Applications, other standalone roles, and asset repositories are read-only.
## Stage 1: Expose Narratio's Native Profile Selector
### Required Policy Review
Read `AGENTS.md`, `docs/orientation.md`, `docs/conventions.md`,
`docs/ansible.md`, `docs/testing.md`, and the Feature 0017 roadmap.
### Owning Repository and Preconditions
`ansible-role-narratio`. Its default already selects Narratio v2.2.0.
Read that tag's configuration contract for `notarius.config_profile`.
### Required Changes
1. Add `narratio_notarius_config_profile: null` to defaults and an optional
string entry to `meta/argument_specs.yml`; document the public variable.
2. Include the variable in the enclosing Notarius-section condition in
`templates/pipeline.yml.j2`. Render `config_profile` with `to_json` only
when the variable is not none. Preserve explicit empty strings and exact
values. Add no name validation or file discovery.
3. Extend the existing representative configuration tests with a supplied
nonempty selector, null omission, and intentional empty-string rendering.
Keep fixtures small; no downstream inference or overlay file is necessary
merely to validate Narratio's native selector field. Existing tests against
older application versions must leave the new field unset.
### Validation and Completion
From this repository run `tests/integration` and `git diff --check`.
The public variable renders correctly and native configuration validation and
clean repeat pass. Extend existing tests rather than creating a parallel harness.
### Commit
`Expose the Notarius configuration profile in the Narratio role`.
Push this validated commit to origin/main and record its full ID as NARRATIO_ROLE.
## Stage 2: Adopt Notarius v0.7.1
### Required Policy Review
Read `AGENTS.md`, `docs/orientation.md`, `docs/conventions.md`,
`docs/ansible.md`, `docs/testing.md`, and the Feature 0017 roadmap.
### Owning Repository and Preconditions
`ansible-role-notarius`. Stage 1 is complete. Inspect v0.7.1 release notes and
the role's existing integration entry point.
### Required Changes
Set `notarius_version` to v0.7.1 and update the README's current default.
No role configuration or task change is needed for this dependency upgrade.
Retain explicitly selected historical-version tests. Do not copy new provider
catalogs or model policy into this generic role.
### Validation and Completion
Run `tests/integration` and `git diff --check` in this repository. The normal
default convergence uses v0.7.1, native checks pass, and repeat convergence is
clean. Preserve tests of caller version selection instead of asserting the default.
### Commit
`Install Notarius v0.7.1 by default`.
Push the validated commit and record its full ID as NOTARIUS_ROLE.
## Stage 3: Compose and Validate the Suite Testing Overlay
### Required Policy Review
Read `AGENTS.md`, `docs/orientation.md`, `docs/conventions.md`,
`docs/ansible.md`, `docs/testing.md`, and the Feature 0017 roadmap.
### Owning Repository and Preconditions
`ansible-role-dnd-pipeline`. Stages 1-2 are validated and publicly available.
Review `tests/integration`, especially its profile diff, inventory override,
and alternate pipeline composition cases.
### Required Changes
1. Pin the Narratio and Notarius roles in `requirements.yml` to NARRATIO_ROLE
and NOTARIUS_ROLE. Leave unrelated dependencies unchanged.
2. Set `dnd_pipeline_notarius_version: v0.7.1`. In the existing default
`dnd_pipeline_notarius_pipelines` mapping, change the base dnd-session
`llm_profile` to `gpt-terra-latest`. Do not introduce a second production
profile variable: the mapping remains its inventory override interface.
3. Add `dnd_pipeline_notarius_testing_llm_profile: gpt-luna-latest`, its string
argument specification, and README documentation.
4. Extend ordinary directory management to create
`/usr/local/etc/notarius/profiles`. Add one template for
`/usr/local/etc/notarius/profiles/testing.yml` containing only:
```yaml
pipelines:
<selected suite pipeline ID>:
llm_profile: <testing LLM profile variable>
```
Serialize both dynamic key and value safely. Use the existing
`dnd_pipeline_narratio_notarius_pipeline_id` for the key. Manage the file
and directory with existing runtime owner/group and configuration modes.
Do not hard-code dnd-session or copy the full pipeline into the overlay.
5. Add `notarius: {config_profile: testing}` to Narratio's existing testing
template without changing its Scriptorium settings. Leave the base Narratio
selector unset. No extra suite-level selector variable or PromptKit asset
is needed. ConfigKit finds the overlay normally; do not import it into base.
6. Document base/testing choices, mapping overrides, native binding precedence,
moving catalog IDs, unchanged credential reference, and v0.7.1 checkpoint
compatibility. Keep Notarius enabled and caching unchanged.
7. Extend the existing integration script to inspect base and testing effective
configurations through the applications' native commands. Use explicit
caller-supplied values for assertions; for example, explicitly select
`gpt-luna-latest` in the production fixture and `gpt-terra-latest` in the
testing fixture to demonstrate independence from defaults. Validate the
selected pipeline with Notarius `--config-profile testing`, without inference.
Exercise the existing alternate pipeline ID case with the testing overlay
to prove the key follows inventory. Do not add a binding-precedence matrix.
8. Replace the old test restriction that Narratio testing differences may only
end in `.profile_id`: it excludes the new `.notarius.config_profile` setting.
Prefer assertions of the selected effective fields over an exhaustive diff
allowlist. Retain useful Scriptorium override coverage and clean-repeat checks.
### Validation and Completion
Run `tests/integration` and `git diff --check`. Required outcomes: native
validation passes for base and testing; selected inventory values are visible;
base remains independent of testing; alternate pipeline composition works;
immediate repeat is clean. No real keys or provider requests are required.
### Commit
`Add the Notarius testing overlay and production model defaults`.
Push the validated commit and record its full ID as PIPELINE_ROLE.
## Stage 4: Assemble the Public Collection Snapshot
### Required Policy Review
Read `AGENTS.md`, `docs/orientation.md`, `docs/conventions.md`,
`docs/ansible.md`, `docs/testing.md`, and the collection README's subtree policy.
### Owning Repository and Preconditions
`ansible-collection-dnd`. Stage 3 and its dependency commits are pushed.
### Required Changes
1. Import PIPELINE_ROLE from Gitea using the documented workflow:
```sh
git subtree pull --prefix=roles/pipeline \
https://gitea.maximumdirect.net/eric/ansible-role-dnd-pipeline.git \
FULL_PIPELINE_ROLE_COMMIT --squash \
-m 'Bundle the Notarius testing overlay'
```
Substitute the actual full commit ID. Do not edit the subtree directly.
2. Update collection-root `requirements.yml` to the same Narratio/Notarius
role commits used by the pipeline manifest. Keep unrelated pins unchanged.
3. Extend the collection's existing installed-artifact smoke coverage to validate
Notarius testing and inspect Narratio's effective downstream selector. Reuse
Stage 3's detailed behavior evidence rather than copying its whole test suite.
4. Keep version 0.1.0 and published tags/artifacts unchanged. Document the current
development snapshot where necessary without rewriting historical release claims.
### Validation and Completion
Run `tests/integration` and `git diff --check`. This must build/install the
collection with its public declared roles and pass native base/testing checks
and clean repeat. Compare `git rev-parse HEAD:roles/pipeline` with the source
pipeline commit's root tree: they must match. No sibling-role substitution.
### Commit
The subtree command creates its own commit. Commit remaining changes as
`Validate and pin the testing overlay collection dependencies`.
Push the completed collection and record its full final ID as COLLECTION.
## Stage 5: Update Consumer Pins and Record Review Handoff
### Required Policy Review
Read `AGENTS.md`, `docs/orientation.md`, `docs/conventions.md`,
`docs/ansible.md`, `docs/testing.md`, `docs/planning.md`, and Feature 0017.
### Owning Repositories and Preconditions
`rakestraw-ansible` and workspace. Stage 4 is validated and pushed.
### Required Changes
1. In the consumer manifest pin the collection to COLLECTION, and Narratio and
Notarius roles to their corresponding public commits. Preserve unrelated pins.
2. Update the collection refresh command in `docs/pipeline-dnd.md` to COLLECTION.
Explain the testing overlay behavior and v0.7.1 cache compatibility concisely.
Preserve the minimal hosts/roles playbook, host inventory, and secret references.
No host-variable copies of the new suite defaults are needed.
3. Update workspace status and this plan's Progress Notes with commit IDs and
validation evidence. Mark Feature 0017 in_progress pending independent review,
not done. Preserve the durable roadmap; no new feature scope is introduced.
### Validation and Completion
Run `git diff --check` in both repositories. Reuse Stage 4's successful public
dependency installation for the same dependency set. With those isolated role
and collection paths and workspace `.venv/bin` tools, run from the consumer:
```text
ansible-playbook -i inventory playbooks/pipeline/dnd.yml --limit pipeline-dnd --list-hosts
ansible-playbook -i inventory playbooks/pipeline/dnd.yml --limit pipeline-dnd --syntax-check
```
Supply explicit ANSIBLE_ROLES_PATH and ANSIBLE_COLLECTIONS_PATH for the isolated
installation. Do not install to the controller's /etc/ansible or connect to the
target. If unrelated encrypted inventory prevents local checks, use a disposable
inventory with the target alias and non-secret settings and report that scope.
Missing real Vault values are not a feature blocker; do not populate them.
### Commits
Consumer: `Select the collection with Notarius testing profiles`.
Workspace: `Record the Notarius testing overlay implementation for review`.
Push the consumer update so the controller can retrieve the published pins.
## Final Validation and Handoff
Report all resulting commits, validation performed, and any real blockers or
deviations. Confirm changed working trees are clean. Distinguish repository
readiness from an operational deployment; none is required or claimed. Note
any complexity added beyond the planned declarative tasks and its requirement.
## Progress Notes
Implementation plan approved by the user; implementation has not started.
## Open Questions
None. Production and testing selections, upstream versions, public commit-pin
distribution, and ownership boundaries are settled for this plan.