395 lines
19 KiB
Markdown
395 lines
19 KiB
Markdown
# Implementation Plan 0022: Coordinated Suite Upgrade
|
||
|
||
Status: blocked — reviewed role publication requires Gitea Git authentication
|
||
|
||
Target executor: GPT-5.6-Terra
|
||
|
||
Feature roadmap: [Feature 0022](features/0022-coordinated-suite-upgrade.md)
|
||
|
||
## Objective
|
||
|
||
Integrate the reviewed application upgrades into the pipeline role, assembled
|
||
collection, and consumer repository. The approved roadmap owns scope, target
|
||
state, and policy. No operational deployment or tagged release is included.
|
||
|
||
## Execution Rules and Starting State
|
||
|
||
- Execute Stage 1 through Stage 6 in order. Before each stage, reread its
|
||
listed policies, check applicable repository instructions, and confirm its
|
||
owning repository's working-tree state. Preserve unrelated changes.
|
||
- Commit valid completed stages with the stated plain English intent. Do not
|
||
advance past failed validation. Distinguish actual blockers from unavailable
|
||
optional tools or operational credentials that this plan does not require.
|
||
- Use workspace `.venv/bin` Ansible tooling. Missing PATH entries are not a
|
||
missing controller. Existing integration scripts choose local tooling and
|
||
install dependencies into temporary paths.
|
||
- Pipeline/collection integration runs use local sudo and mutate the local
|
||
development installation, including source/binaries, configuration, fixture
|
||
secrets, and test users. Read harness cleanup and verify these locations are
|
||
disposable first. Do not run operational playbooks or contact live inference,
|
||
WhisperX, or S3 services. Public Git/dependency downloads are expected.
|
||
- Use ordinary declarative modules; no production validation, preliminary
|
||
state probes, compatibility aliases, manifest conversion, pruning, or new
|
||
credential-discovery mechanism. New tasks are limited to Audita profile
|
||
directory creation and looped profile templating.
|
||
- Publish only the required reviewed role revisions and feature commits to
|
||
existing origins as specified below. Check remote URLs and outgoing commits
|
||
before each push. Never force-push or move tags. If unrelated commits would
|
||
be published or credentials prevent publication, report that boundary; do
|
||
not substitute local paths in consumer manifests or subtree imports.
|
||
- Do not edit standalone application-role implementations, upstream apps,
|
||
campaign/prompt assets, unrelated consumer inventory, Vault content, or
|
||
vendored consumer roles. Required code owners are pipeline, collection, and
|
||
consumer; workspace owns progress. Keep Seriatim and native collection pins.
|
||
- The new retry/repair/deadline pass-throughs default to null. Preserve the
|
||
existing Audita 900-second per-attempt profile policy. Do not copy native
|
||
defaults into suite variables.
|
||
|
||
## Stage 1: Make Reviewed Standalone Dependencies Public
|
||
|
||
### Required Policy Review
|
||
|
||
Read `AGENTS.md`, `docs/orientation.md`, `docs/conventions.md`, and
|
||
`docs/planning.md` before starting.
|
||
|
||
### Owning Repositories and Preconditions
|
||
|
||
The four standalone role repositories and workspace. Features 0018–0021 are
|
||
complete. Confirm these exact revisions and their existing Gitea origins:
|
||
|
||
| Role | Revision |
|
||
| --- | --- |
|
||
| Scriptorium | `b174c8c2990bb79f8149d8f288c5e06ccb42bd73` |
|
||
| Audita | `c3dd59f5ac9b942306ffe540ae9e71d493998137` |
|
||
| Notarius | `6aef48b7e12e3651971444b8c224d96ff1879e6c` |
|
||
| Narratio | `3c0700b5323293d44e3b1e67a50074d82d7a3bff` |
|
||
|
||
### Work and Validation
|
||
|
||
Mark F0022 `in_progress`. For each repository, fetch origin, inspect its status
|
||
and outgoing history, and publish the reviewed revision to `origin/main` if
|
||
not already reachable there. Use a normal non-force push of the exact revision
|
||
to `refs/heads/main` where a fast-forward is appropriate. A remote already
|
||
containing it needs no push. Verify reachability after fetch; do not replace a
|
||
selected pin merely because a newer remote commit exists.
|
||
|
||
Commands, substituting the role and exact revision from the table:
|
||
|
||
```sh
|
||
git -C ROLE remote -v
|
||
git -C ROLE fetch origin
|
||
git -C ROLE log --oneline origin/main..REVISION
|
||
git -C ROLE push origin REVISION:refs/heads/main
|
||
git -C ROLE fetch origin
|
||
git -C ROLE merge-base --is-ancestor REVISION origin/main
|
||
```
|
||
|
||
### Completion and Commit
|
||
|
||
All four pinned revisions are publicly available. No standalone code change or
|
||
new role commit is needed. Record evidence and commit workspace progress as
|
||
`Record the coordinated suite upgrade in progress`.
|
||
|
||
## Stage 2: Migrate Suite Configuration and Existing Integration Cases
|
||
|
||
### Required Policy Review
|
||
|
||
Read `AGENTS.md`, `docs/orientation.md`, `docs/conventions.md`,
|
||
`docs/ansible.md`, and `docs/testing.md` before starting.
|
||
|
||
### Owning Repository and Preconditions
|
||
|
||
`ansible-role-dnd-pipeline`. Stage 1 is complete. Inspect current defaults,
|
||
specs, dependency vars, tasks, templates, and tests; read tagged Audita v2.0.1
|
||
migration/configuration and Narratio v3.0.0 contracts linked by the roadmap.
|
||
|
||
### Required Changes
|
||
|
||
1. Pin the four standalone roles in `requirements.yml` to Stage 1 revisions.
|
||
Set application defaults to Scriptorium v0.15.0, Audita v2.0.1, Notarius
|
||
v0.8.1, Narratio v3.0.0. Preserve Seriatim v1.5.0 and its role pin.
|
||
2. Add `dnd_pipeline_audita_backends` (dict), default `dnd` with endpoint
|
||
`https://inference.ai.rakestrawhome.com/v1`, credential `AUDITA_LLM_API_KEY`,
|
||
concurrency `4`; omit queue capacity. Add `dnd_pipeline_audita_profiles`
|
||
(dict) with `dnd-proposal` and `dnd-validation` keys. Each value is a complete
|
||
native mapping containing matching `id`, backend `dnd`, model
|
||
`google/gemma-4-31b-it`, and `timeout_seconds: 900`.
|
||
3. Add string defaults `dnd_pipeline_audita_llm_proposal_profile: dnd-proposal`
|
||
and `dnd_pipeline_audita_llm_validation_profile: dnd-validation`. Map these
|
||
and backends into the standalone role. Set `audita_profiles_dir` to
|
||
`/usr/local/etc/audita/profiles`.
|
||
4. Add a `file` directory task and one looped `template` task over
|
||
`dnd_pipeline_audita_profiles | dict2items`. Write `profiles/<key>.yml` using
|
||
an `audita-profile.yml.j2` template containing `item.value | to_nice_yaml`.
|
||
Use runtime owner/group, suite configuration modes, and task-level become.
|
||
Document that keys are filename-safe IDs matching each mapping's `id`.
|
||
Do not merge/rewrite profile values or delete removed entries.
|
||
5. Remove old `dnd_pipeline_audita_llm_{proposal,validation}_{base_url,model,
|
||
api_key_env,timeout,max_retries}`, `dnd_pipeline_audita_concurrency_*`, and
|
||
diagnostics retention from defaults, specs, dependency wiring, and active
|
||
examples. Preserve diagnostics work directory and unchanged domain tuning.
|
||
Convert normalization max gap/ellipsis gap/max duration to numeric
|
||
`3.0`/`3.0`/`240.0` with float specs. Leave debug/reporting at native defaults.
|
||
6. Add these optional nullable suite variables, forwarding to the corresponding
|
||
standalone variable by removing `dnd_pipeline_`:
|
||
- `dnd_pipeline_audita_llm_transient_retries`: int.
|
||
- `dnd_pipeline_audita_llm_repair_attempts`: int.
|
||
- `dnd_pipeline_audita_llm_operation_timeout`: str.
|
||
- `dnd_pipeline_notarius_promptkit_transient_retries`: int.
|
||
- `dnd_pipeline_scriptorium_transient_retries`: int.
|
||
Keep nulls and explicit zero; do not supply a fallback native default.
|
||
7. Add list-of-string credential defaults
|
||
`dnd_pipeline_narratio_audita_credentials: [AUDITA_LLM_API_KEY]` and
|
||
`dnd_pipeline_narratio_notarius_credentials: [OPENROUTER_API_KEY]`; wire to
|
||
Narratio's credential lists. Remove the old suite Notarius singular-key
|
||
variable and obsolete standalone-role arguments. Preserve secrets delivery.
|
||
8. Change Narratio/Notarius directory creation and testing-file destinations
|
||
from `profiles` to `overlays`. Change Narratio testing to
|
||
`notarius.config_overlay: testing`. Leave old ignored files alone. Preserve
|
||
Audita's LLM `profiles/`, artifact fragments, model selections, caches,
|
||
paths, publication, ACLs, and operator behavior.
|
||
9. Migrate existing `tests/integration` selectors and parsed assertions:
|
||
Narratio `--overlay`, Notarius `--config-overlay`, `config_overlay`, and
|
||
credential lists replace retired fields. Direct Audita validation receives
|
||
the harness's synthetic `AUDITA_LLM_API_KEY` explicitly in its environment;
|
||
do not assume Narratio's bootstrap applies to that process. Add Scriptorium
|
||
native config validation. Keep old meaningful scenarios, clean repeats,
|
||
and fixture cleanup; do not test upstream forwarding internals.
|
||
10. Update README public variables and migration mapping in the same commit
|
||
so the role's current interface is documented. Explain defaults versus
|
||
inventory overrides and retain existing native configuration ownership.
|
||
|
||
### Validation
|
||
|
||
From workspace root:
|
||
|
||
```sh
|
||
bash -n ansible-role-dnd-pipeline/tests/integration
|
||
ansible-role-dnd-pipeline/tests/integration
|
||
git -C ansible-role-dnd-pipeline diff --check
|
||
```
|
||
|
||
The harness installs public pinned dependencies and includes syntax/lint.
|
||
Require native acceptance of base/testing configuration and existing meaningful
|
||
behavior with clean repeats, without live service calls. If an unrelated asset
|
||
contract incompatibility appears, report it rather than rewriting assets.
|
||
|
||
### Completion and Commit
|
||
|
||
Suite configuration, specs, docs, and existing tests are compatible together.
|
||
Commit as `Integrate the reviewed application upgrades into the suite`.
|
||
|
||
## Stage 3: Verify Inventory Overrides and Finish Suite Migration Guidance
|
||
|
||
### Required Policy Review
|
||
|
||
Read `AGENTS.md`, `docs/orientation.md`, `docs/conventions.md`,
|
||
`docs/ansible.md`, and `docs/testing.md` before starting.
|
||
|
||
### Owning Repository and Preconditions
|
||
|
||
`ansible-role-dnd-pipeline`. Stage 2 passes. Extend its existing harness rather
|
||
than introducing a test framework or changing production convergence.
|
||
|
||
### Required Changes
|
||
|
||
1. Add one representative override case with a fixture backend endpoint,
|
||
caller-composed native profile map, selected proposal/validation IDs, and
|
||
an explicit custom credential-name list. Use synthetic values only; provide
|
||
required names both through managed fixture secrets for Narratio and direct
|
||
test environment for Audita. Use simple offline-valid profile values.
|
||
2. In that case supply zero for the four retry/repair integers and a valid
|
||
explicit Audita operation timeout, such as `20m`. Inspect native effective
|
||
settings/profile views to verify supplied values and selected models. Inspect
|
||
Narratio configuration to verify the credential names. Do not assert current
|
||
application default retry counts or model catalogs.
|
||
3. Explicitly null the five new pass-throughs in a baseline case and inspect
|
||
rendered application YAML for omission (effective views may include native
|
||
defaults). Require an immediate unchanged repeat for the override case.
|
||
Retain existing pipeline-map composition, testing selection, shared-user,
|
||
cache, storage, and artifact tests; do not add module metadata checks.
|
||
4. Document exact old-to-new inventory choices, shared backend concurrency,
|
||
filename/profile ID contract, credential list ownership, and CLI changes.
|
||
Explain the retained 900-second attempt setting and independent native
|
||
operation deadline. Link upstream manifest migration; do not automate it.
|
||
Explain ignored legacy testing files and normal named-selector invalidation.
|
||
|
||
### Validation
|
||
|
||
```sh
|
||
bash -n ansible-role-dnd-pipeline/tests/integration
|
||
ansible-role-dnd-pipeline/tests/integration
|
||
git -C ansible-role-dnd-pipeline diff --check
|
||
```
|
||
|
||
All native checks and immediate repeats pass. Review the final pipeline diff
|
||
against roadmap scope, including no live-provider test or new production probe.
|
||
|
||
### Completion and Commit
|
||
|
||
Commit as `Verify suite configuration overrides and document migration`.
|
||
Record the final pipeline commit as the assembly revision. Inspect outgoing
|
||
commits and publish that revision to the pipeline's existing `origin/main`
|
||
with a normal push; verify remote reachability before Stage 4.
|
||
|
||
## Stage 4: Assemble and Validate the Collection
|
||
|
||
### Required Policy Review
|
||
|
||
Read `AGENTS.md`, `docs/orientation.md`, `docs/conventions.md`,
|
||
`docs/ansible.md`, and `docs/testing.md` before starting.
|
||
|
||
### Owning Repository and Preconditions
|
||
|
||
`ansible-collection-dnd`. The validated Stage 3 pipeline commit is public.
|
||
Read the collection README's subtree and dependency workflow.
|
||
|
||
### Required Changes
|
||
|
||
1. Import that exact revision using the authoritative public Gitea origin:
|
||
|
||
```sh
|
||
git -C ansible-collection-dnd subtree pull --prefix=roles/pipeline \
|
||
https://gitea.maximumdirect.net/eric/ansible-role-dnd-pipeline.git \
|
||
PIPELINE_REVISION --squash
|
||
```
|
||
|
||
Substitute the Stage 3 hash; never a local sibling path. Subtree-generated
|
||
commits are expected. Do not edit imported role files directly.
|
||
2. Match the standalone revisions in root `requirements.yml` to Stage 1.
|
||
Keep collection metadata version 0.1.0 and published release instructions
|
||
intact; describe the new development snapshot separately.
|
||
3. Migrate collection `tests/integration` CLI/field assertions and direct Audita
|
||
synthetic environment handling as in Stage 2. Validate Audita profiles and
|
||
Scriptorium configuration natively. Keep collection tests focused on the
|
||
installed artifact and dependency boundary, base/testing selection, and
|
||
clean repeat; do not duplicate Stage 3's complete override matrix.
|
||
4. Update compatibility guidance and reference the pipeline migration notes.
|
||
A fresh public dependency install must work without sibling role repos.
|
||
Existing local campaign/prompt fixture overrides in the test harness remain
|
||
acceptable; production manifests must not acquire those local paths.
|
||
|
||
### Validation
|
||
|
||
```sh
|
||
bash -n ansible-collection-dnd/tests/integration
|
||
ansible-collection-dnd/tests/integration
|
||
git -C ansible-collection-dnd diff --check
|
||
```
|
||
|
||
Require successful build, disposable artifact install, syntax/lint, native
|
||
configuration checks, and immediate clean repeat using public role dependencies.
|
||
|
||
### Completion and Commit
|
||
|
||
Commit remaining changes as `Assemble the upgraded D&D suite` after the subtree
|
||
import commits. Inspect and publish feature commits to `origin/main`; record
|
||
and verify the final public collection hash for Stage 5. No tags or release
|
||
assets are created or replaced.
|
||
|
||
## Stage 5: Update the Consumer Without Deploying
|
||
|
||
### Required Policy Review
|
||
|
||
Read `AGENTS.md`, `docs/orientation.md`, `docs/conventions.md`,
|
||
`docs/ansible.md`, and `docs/testing.md` before starting.
|
||
|
||
### Owning Repository and Preconditions
|
||
|
||
`rakestraw-ansible`. Stage 4's collection and all referenced dependencies are
|
||
public. Read applicable consumer instructions and `docs/pipeline-dnd.md`.
|
||
|
||
### Required Changes
|
||
|
||
1. Update only the relevant role pins in `requirements.yml` to Stage 1 and its
|
||
D&D collection pin to Stage 4. Preserve unrelated requirements.
|
||
2. Keep `playbooks/pipeline/dnd.yml` minimal. Existing host-vars credential
|
||
names remain valid; preserve encrypted Vault content and all unrelated
|
||
values. Do not add copied suite defaults to inventory. If an obsolete
|
||
feature-owned setting is present, replace only that setting and document it.
|
||
3. Update `docs/pipeline-dnd.md` to the coordinated versions, exact collection
|
||
refresh hash, new testing invocation, and migration prerequisites. Preserve
|
||
the explicit refresh guidance for an equal-version development collection.
|
||
Remove misleading claims that the v3 upgrade needs no configuration or
|
||
manifest migration. Distinguish future operator commands from checks run.
|
||
|
||
### Validation
|
||
|
||
Use the workspace tools and disposable role/collection paths. From consumer
|
||
root, set `ANSIBLE_ROLES_PATH` and `ANSIBLE_COLLECTIONS_PATH` to directories
|
||
under a new `mktemp -d` path, then run:
|
||
|
||
```sh
|
||
/home/eric/Workspace/ansible-dnd/.venv/bin/ansible-galaxy install -r requirements.yml
|
||
/home/eric/Workspace/ansible-dnd/.venv/bin/ansible-playbook -i inventory playbooks/pipeline/dnd.yml --list-hosts --limit pipeline-dnd
|
||
/home/eric/Workspace/ansible-dnd/.venv/bin/ansible-playbook -i inventory playbooks/pipeline/dnd.yml --syntax-check --limit pipeline-dnd
|
||
git diff --check
|
||
```
|
||
|
||
These commands must not connect to the target or execute role tasks. If loading
|
||
encrypted inventory requires unavailable Vault credentials, validate syntax
|
||
with a temporary inventory containing only `pipeline-dnd`, a synthetic WhisperX
|
||
URL, and synthetic required inputs instead. Inspect the real playbook target
|
||
and inventory entry read-only; report this limited check accurately. Do not
|
||
decrypt Vault, require real keys, add ansible-doc gates, or execute deployment.
|
||
|
||
### Completion and Commit
|
||
|
||
Public dependencies resolve and consumer syntax/target checks pass (or the
|
||
documented synthetic-inventory fallback is used). Commit as
|
||
`Prepare the consumer for the coordinated suite upgrade`. Leave this consumer
|
||
commit local unless separately asked to push; it is not a downstream dependency.
|
||
|
||
## Stage 6: Record Evidence and Hand Off for Independent Review
|
||
|
||
### Required Policy Review
|
||
|
||
Read `AGENTS.md`, `docs/orientation.md`, `docs/conventions.md`,
|
||
`docs/ansible.md`, `docs/testing.md`, and `docs/planning.md` before starting.
|
||
|
||
### Owning Repository and Preconditions
|
||
|
||
Workspace. Stages 1–5 are complete and committed. No new code or deployment
|
||
work belongs in this stage.
|
||
|
||
### Required Changes and Validation
|
||
|
||
Record final hashes, public availability, commands/results, environment limits,
|
||
and any plan deviations in the feature roadmap and concise progress notes.
|
||
Update `docs/status.md` and the upgrade sequence to implementation complete,
|
||
review pending, using `in_progress` until independent review. Review that pins
|
||
across pipeline, collection, and consumer agree, and no standalone role or
|
||
application/asset code changed outside scope.
|
||
|
||
```sh
|
||
git diff --check
|
||
scripts/status
|
||
```
|
||
|
||
Reuse earlier integration evidence for documentation-only changes. Report
|
||
which commits were pushed and which remain local; a clean working tree does
|
||
not imply a synchronized remote. Keep deployment, session migration, and formal
|
||
release work explicitly deferred.
|
||
|
||
### Completion and Commit
|
||
|
||
Commit as `Record coordinated suite upgrade validation`. All changed working
|
||
trees are clean, and implementation is ready for independent review. Do not
|
||
mark F0022 done, begin another feature, or push unrelated workspace history.
|
||
|
||
## Progress Notes
|
||
|
||
Plan approved. Stage 1 publication is blocked. On 2026-09-29, all four exact
|
||
reviewed revisions were present locally; clean worktrees and expected Gitea
|
||
origins were confirmed. After fetching, each `origin/main` was three reviewed
|
||
feature commits behind the selected revision and could fast-forward. The first
|
||
normal push, for Scriptorium, failed because HTTPS Git could not read a
|
||
username. No role revision was published, no downstream stage was started, and
|
||
no stage is marked complete. Resume with authenticated access to the existing
|
||
Gitea origins, then re-fetch and recheck outgoing history before pushing.
|
||
|
||
## Open Questions
|
||
|
||
None. The user approved application-owned retry/repair/deadline defaults with
|
||
nullable inventory overrides and the retained Audita per-attempt policy.
|