eric/ansible-workspace-dnd
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Close out the coordinated suite upgrade

Browse Source
This commit is contained in:
Eric Rakestraw
2026-09-29 15:17:31 +00:00
parent 6829439807
commit 43ecdebcc5
4 changed files with 38 additions and 449 deletions
Download Patch File Download Diff File Expand all files Collapse all files

445
docs/roadmap/implementation.md
View File

@@ -1,438 +1,13 @@
# Implementation Plan 0022: Coordinated Suite Upgrade
# Implementation Plan
Status: in_progress
No implementation plan is currently active.
Target executor: GPT-5.6-Terra
[Feature 0022](features/0022-coordinated-suite-upgrade.md) was closed on
2026-09-29 after implementation, independent review, and resolution of all
three findings. Its roadmap and [remediation record](remediation.md) retain
the completion evidence; Git history preserves the completed staged plan.
Feature roadmap: [Feature 0022](features/0022-coordinated-suite-upgrade.md)
## Objective
Integrate the reviewed application upgrades into the pipeline role, assembled
collection, and consumer repository. The approved roadmap owns scope, target
state, and policy. No operational deployment or tagged release is included.
## Execution Rules and Starting State
- Execute Stage 1 through Stage 6 in order. Before each stage, reread its
listed policies, check applicable repository instructions, and confirm its
owning repository's working-tree state. Preserve unrelated changes.
- Commit valid completed stages with the stated plain English intent. Do not
advance past failed validation. Distinguish actual blockers from unavailable
optional tools or operational credentials that this plan does not require.
- Use workspace `.venv/bin` Ansible tooling. Missing PATH entries are not a
missing controller. Existing integration scripts choose local tooling and
install dependencies into temporary paths.
- Pipeline/collection integration runs use local sudo and mutate the local
development installation, including source/binaries, configuration, fixture
secrets, and test users. Read harness cleanup and verify these locations are
disposable first. Do not run operational playbooks or contact live inference,
WhisperX, or S3 services. Public Git/dependency downloads are expected.
- Use ordinary declarative modules; no production validation, preliminary
state probes, compatibility aliases, manifest conversion, pruning, or new
credential-discovery mechanism. New tasks are limited to Audita profile
directory creation and looped profile templating.
- Publish only the required reviewed role revisions and feature commits to
existing origins as specified below. Check remote URLs and outgoing commits
before each push. Never force-push or move tags. If unrelated commits would
be published or credentials prevent publication, report that boundary; do
not substitute local paths in consumer manifests or subtree imports.
- Do not edit standalone application-role implementations, upstream apps,
campaign/prompt assets, unrelated consumer inventory, Vault content, or
vendored consumer roles. Required code owners are pipeline, collection, and
consumer; workspace owns progress. Keep Seriatim and native collection pins.
- The new retry/repair/deadline pass-throughs default to null. Preserve the
existing Audita 900-second per-attempt profile policy. Do not copy native
defaults into suite variables.
## Stage 1: Make Reviewed Standalone Dependencies Public
Completion: complete (2026-09-29).
### Required Policy Review
Read `AGENTS.md`, `docs/orientation.md`, `docs/conventions.md`, and
`docs/planning.md` before starting.
### Owning Repositories and Preconditions
The four standalone role repositories and workspace. Features 0018–0021 are
complete. Confirm these exact revisions and their existing Gitea origins:
| Role | Revision |
| --- | --- |
| Scriptorium | `b174c8c2990bb79f8149d8f288c5e06ccb42bd73` |
| Audita | `c3dd59f5ac9b942306ffe540ae9e71d493998137` |
| Notarius | `6aef48b7e12e3651971444b8c224d96ff1879e6c` |
| Narratio | `3c0700b5323293d44e3b1e67a50074d82d7a3bff` |
### Work and Validation
Mark F0022 `in_progress`. For each repository, fetch origin, inspect its status
and outgoing history, and publish the reviewed revision to `origin/main` if
not already reachable there. Use a normal non-force push of the exact revision
to `refs/heads/main` where a fast-forward is appropriate. A remote already
containing it needs no push. Verify reachability after fetch; do not replace a
selected pin merely because a newer remote commit exists.
Commands, substituting the role and exact revision from the table:
```sh
git -C ROLE remote -v
git -C ROLE fetch origin
git -C ROLE log --oneline origin/main..REVISION
git -C ROLE push origin REVISION:refs/heads/main
git -C ROLE fetch origin
git -C ROLE merge-base --is-ancestor REVISION origin/main
```
### Completion and Commit
All four pinned revisions are publicly available. No standalone code change or
new role commit is needed. Record evidence and commit workspace progress as
`Record the coordinated suite upgrade in progress`.
## Stage 2: Migrate Suite Configuration and Existing Integration Cases
Completion: complete (2026-09-29).
### Required Policy Review
Read `AGENTS.md`, `docs/orientation.md`, `docs/conventions.md`,
`docs/ansible.md`, and `docs/testing.md` before starting.
### Owning Repository and Preconditions
`ansible-role-dnd-pipeline`. Stage 1 is complete. Inspect current defaults,
specs, dependency vars, tasks, templates, and tests; read tagged Audita v2.0.1
migration/configuration and Narratio v3.0.0 contracts linked by the roadmap.
### Required Changes
1. Pin the four standalone roles in `requirements.yml` to Stage 1 revisions.
Set application defaults to Scriptorium v0.15.0, Audita v2.0.1, Notarius
v0.8.1, Narratio v3.0.0. Preserve Seriatim v1.5.0 and its role pin.
2. Add `dnd_pipeline_audita_backends` (dict), default `dnd` with endpoint
`https://inference.ai.rakestrawhome.com/v1`, credential `AUDITA_LLM_API_KEY`,
concurrency `4`; omit queue capacity. Add `dnd_pipeline_audita_profiles`
(dict) with `dnd-proposal` and `dnd-validation` keys. Each value is a complete
native mapping containing matching `id`, backend `dnd`, model
`google/gemma-4-31b-it`, and `timeout_seconds: 900`.
3. Add string defaults `dnd_pipeline_audita_llm_proposal_profile: dnd-proposal`
and `dnd_pipeline_audita_llm_validation_profile: dnd-validation`. Map these
and backends into the standalone role. Set `audita_profiles_dir` to
`/usr/local/etc/audita/profiles`.
4. Add a `file` directory task and one looped `template` task over
`dnd_pipeline_audita_profiles | dict2items`. Write `profiles/<key>.yml` using
an `audita-profile.yml.j2` template containing `item.value | to_nice_yaml`.
Use runtime owner/group, suite configuration modes, and task-level become.
Document that keys are filename-safe IDs matching each mapping's `id`.
Do not merge/rewrite profile values or delete removed entries.
5. Remove old `dnd_pipeline_audita_llm_{proposal,validation}_{base_url,model,
api_key_env,timeout,max_retries}`, `dnd_pipeline_audita_concurrency_*`, and
diagnostics retention from defaults, specs, dependency wiring, and active
examples. Preserve diagnostics work directory and unchanged domain tuning.
Convert normalization max gap/ellipsis gap/max duration to numeric
`3.0`/`3.0`/`240.0` with float specs. Leave debug/reporting at native defaults.
6. Add these optional nullable suite variables, forwarding to the corresponding
standalone variable by removing `dnd_pipeline_`:
- `dnd_pipeline_audita_llm_transient_retries`: int.
- `dnd_pipeline_audita_llm_repair_attempts`: int.
- `dnd_pipeline_audita_llm_operation_timeout`: str.
- `dnd_pipeline_notarius_promptkit_transient_retries`: int.
- `dnd_pipeline_scriptorium_transient_retries`: int.
Keep nulls and explicit zero; do not supply a fallback native default.
7. Add list-of-string credential defaults
`dnd_pipeline_narratio_audita_credentials: [AUDITA_LLM_API_KEY]` and
`dnd_pipeline_narratio_notarius_credentials: [OPENROUTER_API_KEY]`; wire to
Narratio's credential lists. Remove the old suite Notarius singular-key
variable and obsolete standalone-role arguments. Preserve secrets delivery.
8. Change Narratio/Notarius directory creation and testing-file destinations
from `profiles` to `overlays`. Change Narratio testing to
`notarius.config_overlay: testing`. Leave old ignored files alone. Preserve
Audita's LLM `profiles/`, artifact fragments, model selections, caches,
paths, publication, ACLs, and operator behavior.
9. Migrate existing `tests/integration` selectors and parsed assertions:
Narratio `--overlay`, Notarius `--config-overlay`, `config_overlay`, and
credential lists replace retired fields. Direct Audita validation receives
the harness's synthetic `AUDITA_LLM_API_KEY` explicitly in its environment;
do not assume Narratio's bootstrap applies to that process. Add Scriptorium
native config validation. Keep old meaningful scenarios, clean repeats,
and fixture cleanup; do not test upstream forwarding internals.
10. Update README public variables and migration mapping in the same commit
so the role's current interface is documented. Explain defaults versus
inventory overrides and retain existing native configuration ownership.
### Validation
From workspace root:
```sh
bash -n ansible-role-dnd-pipeline/tests/integration
ansible-role-dnd-pipeline/tests/integration
git -C ansible-role-dnd-pipeline diff --check
```
The harness installs public pinned dependencies and includes syntax/lint.
Require native acceptance of base/testing configuration and existing meaningful
behavior with clean repeats, without live service calls. If an unrelated asset
contract incompatibility appears, report it rather than rewriting assets.
### Completion and Commit
Suite configuration, specs, docs, and existing tests are compatible together.
Commit as `Integrate the reviewed application upgrades into the suite`.
## Stage 3: Verify Inventory Overrides and Finish Suite Migration Guidance
Completion: complete (2026-09-29). Pipeline assembly revision: `ea8b09e190419586ede6df3815d1ea999a0f4391`.
### Required Policy Review
Read `AGENTS.md`, `docs/orientation.md`, `docs/conventions.md`,
`docs/ansible.md`, and `docs/testing.md` before starting.
### Owning Repository and Preconditions
`ansible-role-dnd-pipeline`. Stage 2 passes. Extend its existing harness rather
than introducing a test framework or changing production convergence.
### Required Changes
1. Add one representative override case with a fixture backend endpoint,
caller-composed native profile map, selected proposal/validation IDs, and
an explicit custom credential-name list. Use synthetic values only; provide
required names both through managed fixture secrets for Narratio and direct
test environment for Audita. Use simple offline-valid profile values.
2. In that case supply zero for the four retry/repair integers and a valid
explicit Audita operation timeout, such as `20m`. Inspect native effective
settings/profile views to verify supplied values and selected models. Inspect
Narratio configuration to verify the credential names. Do not assert current
application default retry counts or model catalogs.
3. Explicitly null the five new pass-throughs in a baseline case and inspect
rendered application YAML for omission (effective views may include native
defaults). Require an immediate unchanged repeat for the override case.
Retain existing pipeline-map composition, testing selection, shared-user,
cache, storage, and artifact tests; do not add module metadata checks.
4. Document exact old-to-new inventory choices, shared backend concurrency,
filename/profile ID contract, credential list ownership, and CLI changes.
Explain the retained 900-second attempt setting and independent native
operation deadline. Link upstream manifest migration; do not automate it.
Explain ignored legacy testing files and normal named-selector invalidation.
### Validation
```sh
bash -n ansible-role-dnd-pipeline/tests/integration
ansible-role-dnd-pipeline/tests/integration
git -C ansible-role-dnd-pipeline diff --check
```
All native checks and immediate repeats pass. Review the final pipeline diff
against roadmap scope, including no live-provider test or new production probe.
### Completion and Commit
Commit as `Verify suite configuration overrides and document migration`.
Record the final pipeline commit as the assembly revision. Inspect outgoing
commits and publish that revision to the pipeline's existing `origin/main`
with a normal push; verify remote reachability before Stage 4.
## Stage 4: Assemble and Validate the Collection
Completion: complete (2026-09-29). Public collection revision: `3d77eb382bcb4c1742015d3c32e23630bb8c73bf`.
### Required Policy Review
Read `AGENTS.md`, `docs/orientation.md`, `docs/conventions.md`,
`docs/ansible.md`, and `docs/testing.md` before starting.
### Owning Repository and Preconditions
`ansible-collection-dnd`. The validated Stage 3 pipeline commit is public.
Read the collection README's subtree and dependency workflow.
### Required Changes
1. Import that exact revision using the authoritative public Gitea origin:
```sh
git -C ansible-collection-dnd subtree pull --prefix=roles/pipeline \
https://gitea.maximumdirect.net/eric/ansible-role-dnd-pipeline.git \
PIPELINE_REVISION --squash
```
Substitute the Stage 3 hash; never a local sibling path. Subtree-generated
commits are expected. Do not edit imported role files directly.
2. Match the standalone revisions in root `requirements.yml` to Stage 1.
Keep collection metadata version 0.1.0 and published release instructions
intact; describe the new development snapshot separately.
3. Migrate collection `tests/integration` CLI/field assertions and direct Audita
synthetic environment handling as in Stage 2. Validate Audita profiles and
Scriptorium configuration natively. Keep collection tests focused on the
installed artifact and dependency boundary, base/testing selection, and
clean repeat; do not duplicate Stage 3's complete override matrix.
4. Update compatibility guidance and reference the pipeline migration notes.
A fresh public dependency install must work without sibling role repos.
Existing local campaign/prompt fixture overrides in the test harness remain
acceptable; production manifests must not acquire those local paths.
### Validation
```sh
bash -n ansible-collection-dnd/tests/integration
ansible-collection-dnd/tests/integration
git -C ansible-collection-dnd diff --check
```
Require successful build, disposable artifact install, syntax/lint, native
configuration checks, and immediate clean repeat using public role dependencies.
### Completion and Commit
Commit remaining changes as `Assemble the upgraded D&D suite` after the subtree
import commits. Inspect and publish feature commits to `origin/main`; record
and verify the final public collection hash for Stage 5. No tags or release
assets are created or replaced.
## Stage 5: Update the Consumer Without Deploying
Completion: complete (2026-09-29). Local consumer revision: `f694dad9426ee21138b175e44a3855d42be99d4b`.
### Required Policy Review
Read `AGENTS.md`, `docs/orientation.md`, `docs/conventions.md`,
`docs/ansible.md`, and `docs/testing.md` before starting.
### Owning Repository and Preconditions
`rakestraw-ansible`. Stage 4's collection and all referenced dependencies are
public. Read applicable consumer instructions and `docs/pipeline-dnd.md`.
### Required Changes
1. Update only the relevant role pins in `requirements.yml` to Stage 1 and its
D&D collection pin to Stage 4. Preserve unrelated requirements.
2. Keep `playbooks/pipeline/dnd.yml` minimal. Existing host-vars credential
names remain valid; preserve encrypted Vault content and all unrelated
values. Do not add copied suite defaults to inventory. If an obsolete
feature-owned setting is present, replace only that setting and document it.
3. Update `docs/pipeline-dnd.md` to the coordinated versions, exact collection
refresh hash, new testing invocation, and migration prerequisites. Preserve
the explicit refresh guidance for an equal-version development collection.
Remove misleading claims that the v3 upgrade needs no configuration or
manifest migration. Distinguish future operator commands from checks run.
### Validation
Use the workspace tools and disposable role/collection paths. From consumer
root, set `ANSIBLE_ROLES_PATH` and `ANSIBLE_COLLECTIONS_PATH` to directories
under a new `mktemp -d` path, then run:
```sh
/home/eric/Workspace/ansible-dnd/.venv/bin/ansible-galaxy install -r requirements.yml
/home/eric/Workspace/ansible-dnd/.venv/bin/ansible-playbook -i inventory playbooks/pipeline/dnd.yml --list-hosts --limit pipeline-dnd
/home/eric/Workspace/ansible-dnd/.venv/bin/ansible-playbook -i inventory playbooks/pipeline/dnd.yml --syntax-check --limit pipeline-dnd
git diff --check
```
These commands must not connect to the target or execute role tasks. If loading
encrypted inventory requires unavailable Vault credentials, validate syntax
with a temporary inventory containing only `pipeline-dnd`, a synthetic WhisperX
URL, and synthetic required inputs instead. Inspect the real playbook target
and inventory entry read-only; report this limited check accurately. Do not
decrypt Vault, require real keys, add ansible-doc gates, or execute deployment.
### Completion and Commit
Public dependencies resolve and consumer syntax/target checks pass (or the
documented synthetic-inventory fallback is used). Commit as
`Prepare the consumer for the coordinated suite upgrade`. Leave this consumer
commit local unless separately asked to push; it is not a downstream dependency.
## Stage 6: Record Evidence and Hand Off for Independent Review
Completion: complete (2026-09-29).
### Required Policy Review
Read `AGENTS.md`, `docs/orientation.md`, `docs/conventions.md`,
`docs/ansible.md`, `docs/testing.md`, and `docs/planning.md` before starting.
### Owning Repository and Preconditions
Workspace. Stages 1–5 are complete and committed. No new code or deployment
work belongs in this stage.
### Required Changes and Validation
Record final hashes, public availability, commands/results, environment limits,
and any plan deviations in the feature roadmap and concise progress notes.
Update `docs/status.md` and the upgrade sequence to implementation complete,
review pending, using `in_progress` until independent review. Review that pins
across pipeline, collection, and consumer agree, and no standalone role or
application/asset code changed outside scope.
```sh
git diff --check
scripts/status
```
Reuse earlier integration evidence for documentation-only changes. Report
which commits were pushed and which remain local; a clean working tree does
not imply a synchronized remote. Keep deployment, session migration, and formal
release work explicitly deferred.
### Completion and Commit
Commit as `Record coordinated suite upgrade validation`. All changed working
trees are clean, and implementation is ready for independent review. Do not
mark F0022 done, begin another feature, or push unrelated workspace history.
## Progress Notes
Plan approved. Stage 1 publication was initially blocked. On 2026-09-29, all four exact
reviewed revisions were present locally; clean worktrees and expected Gitea
origins were confirmed. After fetching, each `origin/main` was three reviewed
feature commits behind the selected revision and could fast-forward. The first
normal push, for Scriptorium, failed because HTTPS Git could not read a
username. At that time no role revision was published and no downstream stage
had started.
- Stage 1 resumed on 2026-09-29: authenticated pushes of the exact reviewed
Scriptorium `b174c8c`, Audita `c3dd59f`, Notarius `6aef48b`, and Narratio
`3c0700b` revisions to their existing Gitea `origin/main` branches succeeded.
A fresh fetch and ancestor check verified all four public references. No
standalone code or dependency pin changed in this stage.
- Stage 2: pipeline migration and existing integration cases passed `bash -n`,
the full role harness, and `git diff --check`; role commit `8ea8621`.
- Stage 3: representative override, nullable omission, native effective views,
and clean repeats passed the full role harness; public assembly revision
`ea8b09e190419586ede6df3815d1ea999a0f4391`. A subsequent fixture-JSON
line formatting change passed shell syntax, JSON parsing, and whitespace
checks without another full harness run.
- Stage 4: the exact public pipeline revision was subtree-imported. The
collection artifact harness passed with public dependencies, native base and
testing validation, and a clean repeat; public collection revision
`3d77eb382bcb4c1742015d3c32e23630bb8c73bf`.
- Stage 5: public consumer dependencies installed into disposable paths;
real-inventory `--list-hosts` selected `pipeline-dnd`, and `--syntax-check`
and `git diff --check` passed without Vault decryption or deployment. Consumer
revision `f694dad9426ee21138b175e44a3855d42be99d4b` remains local.
- Stage 6: cross-repository pins and owning repository worktrees were checked;
implementation is complete, independent review pending. Operational
deployment, live processing, session migration, and tagged release remain
deferred.
## Remediation Follow-up
The independent review's three findings are resolved in public pipeline
`37b1cf6`, public collection `b3bc045`, and local consumer `15f0f4e`.
See [remediation.md](remediation.md) for per-finding evidence and validation
limits. Runtime implementation and application defaults are unchanged;
feature closeout remains pending.
## Open Questions
None. The user approved application-owned retry/repair/deadline defaults with
nullable inventory overrides and the retained Audita per-attempt policy.
The [application upgrade sequence](application-upgrade-sequence.md) is complete.
Discuss and select the next unit of work before preparing a new feature
roadmap or staged implementation plan. Operational deployment and a new tagged
release are not authorized by this notice.