diff --git a/docs/roadmap/features/0017-notarius-testing-overlay.md b/docs/roadmap/features/0017-notarius-testing-overlay.md index 9f2dc44..ceb9eae 100644 --- a/docs/roadmap/features/0017-notarius-testing-overlay.md +++ b/docs/roadmap/features/0017-notarius-testing-overlay.md @@ -1,6 +1,6 @@ # Feature 0017: Notarius Testing Overlay -Status: in_progress +Status: done ## Summary @@ -172,11 +172,27 @@ profile propagation would couple unrelated profile names and is not selected. ## Dependencies and Follow-up Work The upstream selector shipped in Narratio v2.2.0; the downstream baseline is -now Notarius v0.7.1. The staged implementation plan is maintained separately -at [implementation.md](../implementation.md). +now Notarius v0.7.1. Git history preserves the completed implementation plan. Future testing-specific Notarius settings belong in its overlay but are not implicitly included in this feature. +## Completion and Review Evidence + +Completed and published in Narratio role `330afe7`, Notarius role `a82a83d`, +pipeline role `c56a942`, collection `ce5b838`, and consumer `0ced48c`. + +Implementation reported passing full integration suites for both changed +application roles, the pipeline, and the collection, including clean repeated +convergence. Independent review found no code-quality issues, architecture gaps, +or unnecessarily strengthened requirements requiring remediation. + +The review independently verified collection build and public dependency +installation, syntax and shell checks, native configuration validation, +consistent dependency pins, and exact subtree/source agreement. It reviewed +the recorded integration results without rerunning host-mutating suites. +Operational deployment and paid inference were neither required nor performed +as part of this feature review. + ## Open Questions None. The user selected `gpt-luna-latest` for testing and `gpt-terra-latest` diff --git a/docs/roadmap/implementation.md b/docs/roadmap/implementation.md index ca66d80..2707921 100644 --- a/docs/roadmap/implementation.md +++ b/docs/roadmap/implementation.md @@ -1,326 +1,9 @@ -# Implementation Plan 0017: Notarius Testing Overlay +# Implementation Plan -Status: in_progress +No implementation plan is currently active. Select and agree on the next unit +of work before replacing this notice with its staged plan. -Target executor: GPT-5.6-Terra - -Feature roadmap: [0017-notarius-testing-overlay.md](features/0017-notarius-testing-overlay.md) - -## Objective - -Implement the approved roadmap: Narratio testing explicitly selects Notarius -testing, with `gpt-luna-latest` for testing and `gpt-terra-latest` for production. -Keep the standalone roles generic and distribute a coherent public dependency -set. The roadmap owns scope and rationale; this document owns execution order. - -## Execution Rules and Starting State - -- Execute stages in order. Before each stage reread its Required Policy Review, - inspect owning working trees, and preserve unrelated changes. Paths to policy - below are relative to the workspace, not the nested repository. -- Commit validated units with plain-English messages. Record resulting full - commit IDs in Progress Notes; never invent future hashes or move release tags. -- Use the existing `.venv/bin` Ansible tools. Missing PATH entries, a local - `/etc/ansible`, or controller access are not blockers. `ansible-doc` rendering - is not an acceptance gate. -- Repository integration scripts converge this development host and use sudo; - use this established local test environment, not the operational controller - or `pipeline-dnd`. Do not provision real credentials or run paid inference. -- Read the scripts before running them. Preserve existing fixture and cleanup - conventions; report an actual missing prerequisite rather than expanding scope. -- Trust Ansible modules. Add no runtime validation commands, state fingerprints, - cache deletion, force-extraction automation, or profile discovery framework. -- Local validation can use local role sources where the existing suite does; - distribution validation must use the declared public revisions. -- Publishing changed role commits to their existing Gitea main branches is - required before public dependency installation and subtree import. Publish - only completed, validated work; if publication is unavailable, preserve the - commits and report that specific distribution blocker. -- No new release/tag or replacement of the published 0.1.0 artifact is required. - Use full pushed commit pins for this development compatibility set. - -Current versions: Narratio v2.2.0; Notarius v0.7.0, to become v0.7.1. -The latter supplies PromptKit v0.10.0 and OpenRouter catalog v2.20260922.1, -which contains both selected profile IDs. Existing OPENROUTER_API_KEY delivery -is sufficient. The standalone role tests deliberately select older application -versions in some cases; do not update those merely to match a new default. - -Repository scope: change the Narratio and Notarius roles, pipeline role, -collection, consumer `rakestraw-ansible`, and workspace planning records. -Applications, other standalone roles, and asset repositories are read-only. - -## Stage 1: Expose Narratio's Native Profile Selector - -Completion: complete — Narratio role commit -`330afe7856da94b2402a75447f4b60aba767843f` is published on `origin/main`; -`tests/integration` and `git diff --check` passed. - -### Required Policy Review - -Read `AGENTS.md`, `docs/orientation.md`, `docs/conventions.md`, -`docs/ansible.md`, `docs/testing.md`, and the Feature 0017 roadmap. - -### Owning Repository and Preconditions - -`ansible-role-narratio`. Its default already selects Narratio v2.2.0. -Read that tag's configuration contract for `notarius.config_profile`. - -### Required Changes - -1. Add `narratio_notarius_config_profile: null` to defaults and an optional - string entry to `meta/argument_specs.yml`; document the public variable. -2. Include the variable in the enclosing Notarius-section condition in - `templates/pipeline.yml.j2`. Render `config_profile` with `to_json` only - when the variable is not none. Preserve explicit empty strings and exact - values. Add no name validation or file discovery. -3. Extend the existing representative configuration tests with a supplied - nonempty selector, null omission, and intentional empty-string rendering. - Keep fixtures small; no downstream inference or overlay file is necessary - merely to validate Narratio's native selector field. Existing tests against - older application versions must leave the new field unset. - -### Validation and Completion - -From this repository run `tests/integration` and `git diff --check`. -The public variable renders correctly and native configuration validation and -clean repeat pass. Extend existing tests rather than creating a parallel harness. - -### Commit - -`Expose the Notarius configuration profile in the Narratio role`. -Push this validated commit to origin/main and record its full ID as NARRATIO_ROLE. - -## Stage 2: Adopt Notarius v0.7.1 - -Completion: complete — Notarius role commit -`a82a83dfc23bb80a59f45f94f78d6754c47285e7` is published on `origin/main`; -`tests/integration` and `git diff --check` passed. - -### Required Policy Review - -Read `AGENTS.md`, `docs/orientation.md`, `docs/conventions.md`, -`docs/ansible.md`, `docs/testing.md`, and the Feature 0017 roadmap. - -### Owning Repository and Preconditions - -`ansible-role-notarius`. Stage 1 is complete. Inspect v0.7.1 release notes and -the role's existing integration entry point. - -### Required Changes - -Set `notarius_version` to v0.7.1 and update the README's current default. -No role configuration or task change is needed for this dependency upgrade. -Retain explicitly selected historical-version tests. Do not copy new provider -catalogs or model policy into this generic role. - -### Validation and Completion - -Run `tests/integration` and `git diff --check` in this repository. The normal -default convergence uses v0.7.1, native checks pass, and repeat convergence is -clean. Preserve tests of caller version selection instead of asserting the default. - -### Commit - -`Install Notarius v0.7.1 by default`. -Push the validated commit and record its full ID as NOTARIUS_ROLE. - -## Stage 3: Compose and Validate the Suite Testing Overlay - -Completion: complete — pipeline role commit -`c56a9425d30ef714166f1ff7675b202590f86425` is published on `origin/main`; -`tests/integration`, `bash -n tests/integration`, and `git diff --check` passed. - -### Required Policy Review - -Read `AGENTS.md`, `docs/orientation.md`, `docs/conventions.md`, -`docs/ansible.md`, `docs/testing.md`, and the Feature 0017 roadmap. - -### Owning Repository and Preconditions - -`ansible-role-dnd-pipeline`. Stages 1-2 are validated and publicly available. -Review `tests/integration`, especially its profile diff, inventory override, -and alternate pipeline composition cases. - -### Required Changes - -1. Pin the Narratio and Notarius roles in `requirements.yml` to NARRATIO_ROLE - and NOTARIUS_ROLE. Leave unrelated dependencies unchanged. -2. Set `dnd_pipeline_notarius_version: v0.7.1`. In the existing default - `dnd_pipeline_notarius_pipelines` mapping, change the base dnd-session - `llm_profile` to `gpt-terra-latest`. Do not introduce a second production - profile variable: the mapping remains its inventory override interface. -3. Add `dnd_pipeline_notarius_testing_llm_profile: gpt-luna-latest`, its string - argument specification, and README documentation. -4. Extend ordinary directory management to create - `/usr/local/etc/notarius/profiles`. Add one template for - `/usr/local/etc/notarius/profiles/testing.yml` containing only: - - ```yaml - pipelines: - : - llm_profile: - ``` - - Serialize both dynamic key and value safely. Use the existing - `dnd_pipeline_narratio_notarius_pipeline_id` for the key. Manage the file - and directory with existing runtime owner/group and configuration modes. - Do not hard-code dnd-session or copy the full pipeline into the overlay. -5. Add `notarius: {config_profile: testing}` to Narratio's existing testing - template without changing its Scriptorium settings. Leave the base Narratio - selector unset. No extra suite-level selector variable or PromptKit asset - is needed. ConfigKit finds the overlay normally; do not import it into base. -6. Document base/testing choices, mapping overrides, native binding precedence, - moving catalog IDs, unchanged credential reference, and v0.7.1 checkpoint - compatibility. Keep Notarius enabled and caching unchanged. -7. Extend the existing integration script to inspect base and testing effective - configurations through the applications' native commands. Use explicit - caller-supplied values for assertions; for example, explicitly select - `gpt-luna-latest` in the production fixture and `gpt-terra-latest` in the - testing fixture to demonstrate independence from defaults. Validate the - selected pipeline with Notarius `--config-profile testing`, without inference. - Exercise the existing alternate pipeline ID case with the testing overlay - to prove the key follows inventory. Do not add a binding-precedence matrix. -8. Replace the old test restriction that Narratio testing differences may only - end in `.profile_id`: it excludes the new `.notarius.config_profile` setting. - Prefer assertions of the selected effective fields over an exhaustive diff - allowlist. Retain useful Scriptorium override coverage and clean-repeat checks. - -### Validation and Completion - -Run `tests/integration` and `git diff --check`. Required outcomes: native -validation passes for base and testing; selected inventory values are visible; -base remains independent of testing; alternate pipeline composition works; -immediate repeat is clean. No real keys or provider requests are required. - -### Commit - -`Add the Notarius testing overlay and production model defaults`. -Push the validated commit and record its full ID as PIPELINE_ROLE. - -## Stage 4: Assemble the Public Collection Snapshot - -Completion: complete — subtree import `b6d7130` and final collection commit -`ce5b838d95050f7e6c0fde4d483d269daf5d6e0a` are published on `origin/main`; -`tests/integration`, subtree tree-identity comparison, `bash -n tests/integration`, -and `git diff --check` passed. - -### Required Policy Review - -Read `AGENTS.md`, `docs/orientation.md`, `docs/conventions.md`, -`docs/ansible.md`, `docs/testing.md`, and the collection README's subtree policy. - -### Owning Repository and Preconditions - -`ansible-collection-dnd`. Stage 3 and its dependency commits are pushed. - -### Required Changes - -1. Import PIPELINE_ROLE from Gitea using the documented workflow: - - ```sh - git subtree pull --prefix=roles/pipeline \ - https://gitea.maximumdirect.net/eric/ansible-role-dnd-pipeline.git \ - FULL_PIPELINE_ROLE_COMMIT --squash \ - -m 'Bundle the Notarius testing overlay' - ``` - - Substitute the actual full commit ID. Do not edit the subtree directly. -2. Update collection-root `requirements.yml` to the same Narratio/Notarius - role commits used by the pipeline manifest. Keep unrelated pins unchanged. -3. Extend the collection's existing installed-artifact smoke coverage to validate - Notarius testing and inspect Narratio's effective downstream selector. Reuse - Stage 3's detailed behavior evidence rather than copying its whole test suite. -4. Keep version 0.1.0 and published tags/artifacts unchanged. Document the current - development snapshot where necessary without rewriting historical release claims. - -### Validation and Completion - -Run `tests/integration` and `git diff --check`. This must build/install the -collection with its public declared roles and pass native base/testing checks -and clean repeat. Compare `git rev-parse HEAD:roles/pipeline` with the source -pipeline commit's root tree: they must match. No sibling-role substitution. - -### Commit - -The subtree command creates its own commit. Commit remaining changes as -`Validate and pin the testing overlay collection dependencies`. -Push the completed collection and record its full final ID as COLLECTION. - -## Stage 5: Update Consumer Pins and Record Review Handoff - -Completion: complete — consumer commit -`0ced48c99010772b2cbf9a2df5eb694af6ae77f6` is published on `origin/main`; -public dependency installation, target-only host listing, syntax checking, and -`git diff --check` passed. The workspace review handoff is recorded by this change. - -### Required Policy Review - -Read `AGENTS.md`, `docs/orientation.md`, `docs/conventions.md`, -`docs/ansible.md`, `docs/testing.md`, `docs/planning.md`, and Feature 0017. - -### Owning Repositories and Preconditions - -`rakestraw-ansible` and workspace. Stage 4 is validated and pushed. - -### Required Changes - -1. In the consumer manifest pin the collection to COLLECTION, and Narratio and - Notarius roles to their corresponding public commits. Preserve unrelated pins. -2. Update the collection refresh command in `docs/pipeline-dnd.md` to COLLECTION. - Explain the testing overlay behavior and v0.7.1 cache compatibility concisely. - Preserve the minimal hosts/roles playbook, host inventory, and secret references. - No host-variable copies of the new suite defaults are needed. -3. Update workspace status and this plan's Progress Notes with commit IDs and - validation evidence. Mark Feature 0017 in_progress pending independent review, - not done. Preserve the durable roadmap; no new feature scope is introduced. - -### Validation and Completion - -Run `git diff --check` in both repositories. Reuse Stage 4's successful public -dependency installation for the same dependency set. With those isolated role -and collection paths and workspace `.venv/bin` tools, run from the consumer: - -```text -ansible-playbook -i inventory playbooks/pipeline/dnd.yml --limit pipeline-dnd --list-hosts -ansible-playbook -i inventory playbooks/pipeline/dnd.yml --limit pipeline-dnd --syntax-check -``` - -Supply explicit ANSIBLE_ROLES_PATH and ANSIBLE_COLLECTIONS_PATH for the isolated -installation. Do not install to the controller's /etc/ansible or connect to the -target. If unrelated encrypted inventory prevents local checks, use a disposable -inventory with the target alias and non-secret settings and report that scope. -Missing real Vault values are not a feature blocker; do not populate them. - -### Commits - -Consumer: `Select the collection with Notarius testing profiles`. -Workspace: `Record the Notarius testing overlay implementation for review`. -Push the consumer update so the controller can retrieve the published pins. - -## Final Validation and Handoff - -Report all resulting commits, validation performed, and any real blockers or -deviations. Confirm changed working trees are clean. Distinguish repository -readiness from an operational deployment; none is required or claimed. Note -any complexity added beyond the planned declarative tasks and its requirement. - -## Progress Notes - -- NARRATIO_ROLE: `330afe7856da94b2402a75447f4b60aba767843f` -- NOTARIUS_ROLE: `a82a83dfc23bb80a59f45f94f78d6754c47285e7` -- PIPELINE_ROLE: `c56a9425d30ef714166f1ff7675b202590f86425` -- Collection subtree merge: `b6d7130046e25c7551e1412e805a28460f841900` -- COLLECTION: `ce5b838d95050f7e6c0fde4d483d269daf5d6e0a` -- Consumer: `0ced48c99010772b2cbf9a2df5eb694af6ae77f6` -- Full integration suites passed in the Narratio, Notarius, pipeline, and - collection repositories. The collection subtree tree matched the source - pipeline tree. The consumer installed the public manifest into isolated - paths and passed target-only host listing and syntax checking. -- Feature 0017 remains `in_progress` pending independent review. No operational - controller deployment, target connection, credential provisioning, or paid - inference was performed or claimed. - -## Open Questions - -None. Production and testing selections, upstream versions, public commit-pin -distribution, and ownership boundaries are settled for this plan. +Feature 0017 is complete and independently reviewed with no remediation +findings. Its scope and completion evidence are recorded in +[the feature roadmap](features/0017-notarius-testing-overlay.md) and +[workspace status](../status.md). Git history preserves the completed plan. diff --git a/docs/status.md b/docs/status.md index cc2f9b5..61b0e53 100644 --- a/docs/status.md +++ b/docs/status.md @@ -4,8 +4,8 @@ Last updated: 2026-09-22 Current milestone: Operational D&D pipeline deployment -Current focus: Feature 0017 implementation is complete and awaiting independent -review. The published development set selects `gpt-terra-latest` for production, +Current focus: Select the next unit of work. Feature 0017 passed independent +review with no remediation findings. The published development set selects `gpt-terra-latest` for production, `gpt-luna-latest` for testing, and Notarius v0.7.1. Consumer commit `0ced48c` pins collection commit `ce5b838` and the compatible Narratio and Notarius role commits. Feature 0016 is complete; deployment remains separate operator work. @@ -37,7 +37,7 @@ commits. Feature 0016 is complete; deployment remains separate operator work. | F0014 | Enroll existing pipeline operators | ansible-role-dnd-pipeline, ansible-collection-dnd, and workspace | done | Pipeline `c13f726` and collection import `c0b0ebd` passed their complete integration suites and independent review with no remediation findings | | F0015 | Enable D&D S3 storage and publishing | ansible-role-dnd-pipeline, ansible-collection-dnd, and workspace | done | Pipeline `51566afcf69083cdbb5b5718f2544c86b079001c`, collection fixture `cf3de016ee09edf91afc3bc1addfd926d4a172ad`, collection squash `77f67adbc0f700e88bc15554e02052eaa22f1033`, and collection merge `46913446765e86ecfb29ebad9955d579ef6a4c7e` passed the complete repository suites and independent review with no remediation findings | | F0016 | Prepare the D&D deployment playbook for `pipeline-dnd` | rakestraw-ansible and workspace | done | Consumer `716a80e`, `12b2462`, and `7f8353d` passed independent review with no findings; public dependency installation, target-only host listing, syntax and whitespace checks passed. Follow-up `7f4c90a` pins collection `2a53979` for Narratio v2.1.2; collection build and syntax checks passed | -| F0017 | Compose the Notarius testing overlay | Narratio role, pipeline role, collection, consumer, and workspace | in_progress | Implementation is published through Narratio `330afe7`, Notarius `a82a83d`, pipeline `c56a942`, collection `ce5b838`, and consumer `0ced48c`; independent review is next | +| F0017 | Compose the Notarius testing overlay | Narratio role, pipeline role, collection, consumer, and workspace | done | Published through Narratio `330afe7`, Notarius `a82a83d`, pipeline `c56a942`, collection `ce5b838`, and consumer `0ced48c`; independent review found no remediation issues; validation evidence is in the feature roadmap | | A0001 | Audit completed standalone-role test suites | workspace and standalone roles | done | Findings recorded in `docs/roadmap/test-suite-audit-findings.md` | | R0001 | Simplify completed standalone-role test suites | workspace and standalone roles | done | Role commits `e08c63c`, `d2d0b9d`, and `cfecf37` passed implementation and independent review |