# Feature 0022: Integrate the Reviewed Application Upgrades Status: done ## Summary Bring the four reviewed standalone upgrades into the D&D pipeline role, collection, and deployment consumer as one compatible configuration set. This completes the agreed [application upgrade sequence](../application-upgrade-sequence.md). ## Problem or Opportunity The standalone roles now support the new releases, but the suite intentionally retains its previous pins and configuration. Audita still receives v1 settings; Narratio and Notarius testing selection still uses retired profile terminology. Updating executable versions alone would break the deployed configuration. ## Scope - Adopt the reviewed application/role versions together. - Provide Audita v2 backend configuration and suite-owned PromptKit profiles. - Migrate Narratio credential forwarding and Narratio/Notarius testing overlays. - Preserve D&D processing, artifact composition, storage/publishing, shared filesystem, and operator-access behavior. - Update the collection snapshot, public dependency manifests, consumer pins, operator guidance, and focused offline integration coverage. ## Non-goals - Deploying to the operational controller or `pipeline-dnd` host, running real transcripts, or calling inference, WhisperX, or object-storage services. - New models, prompts, artifact families, extraction lanes, or processing policy beyond migration choices expressly approved here. - Automatic manifest conversion, cache deletion, forced stage reruns, broad directory cleanup, compatibility aliases, or runtime version/integrity probes. - Changing generic standalone-role ownership or adding production validation. - A newly tagged collection release or replacement of published 0.1.0 assets. This feature updates the public development compatibility set; formal release packaging remains a separate decision. ## Target End State ### Coordinated Versions and Distribution | Application | Suite version | Reviewed standalone role revision | | --- | --- | --- | | Scriptorium | v0.15.0 | `b174c8c` | | Audita | v2.0.1 | `c3dd59f` | | Notarius | v0.8.1 | `6aef48b` | | Narratio | v3.0.0 | `3c0700b` | | Seriatim | v1.5.0, unchanged | existing v0.1.0 role pin | Dependency manifests use full immutable revisions resolved from these reviewed commits. The pipeline role remains authoritative; the collection imports its completed revision through the established public Gitea subtree workflow, never by editing `roles/pipeline` directly. The collection continues bundling the pipeline role and declaring standalone role dependencies separately. Relevant reviewed commits must be published to their existing public origins before downstream manifests/subtree imports depend on them. Publication of the required Git commits is part of the eventual implementation workflow, not this roadmap-writing pass. Do not rewrite tags or publish unrelated changes. The consumer keeps its minimal playbook and host-vars organization. Its role and collection pins select the same compatibility set. No controller install or remote playbook execution is an acceptance gate. ### Audita Backend and Profile Ownership The pipeline role supplies `audita_backends`, profile selection, and schema-2 settings to the standalone Audita role. The pipeline owns the missing profile assets under `/usr/local/etc/audita/profiles/`, using its runtime owner/group and configuration modes. This directory contains PromptKit LLM profiles, not ConfigKit overlays; it must not be renamed to `overlays`. Approved suite interface: - `dnd_pipeline_audita_backends`: native backend map, defaulting to one backend named `dnd`, endpoint `https://inference.ai.rakestrawhome.com/v1`, credential reference `AUDITA_LLM_API_KEY`, and shared concurrency `4`. Omit queue capacity to inherit the native default. - `dnd_pipeline_audita_profiles`: a native profile mapping keyed by file/profile ID, with complete `dnd-proposal` and `dnd-validation` entries. Each selects backend `dnd`, model `google/gemma-4-31b-it`, and per-attempt `timeout_seconds: 900`. Other model controls inherit native behavior. - `dnd_pipeline_audita_llm_proposal_profile` and `dnd_pipeline_audita_llm_validation_profile` select those respective IDs. Render each supplied profile as one safely serialized native mapping with its matching ID. Document profile-map keys as filename-safe IDs. Do not introduce per-provider templates, profile merging, or discovery of external assets. Callers can compose native maps in inventory and replace the supplied maps. As with existing managed secrets, removing a map entry does not imply pruning arbitrary files; only selected profiles participate in normal execution. Preserve the ordered module sequence, `audita-v1` output, chunking values, thresholds, context description, and normalization limits. Convert normalization duration defaults from `3s`, `3s`, and `240s` to numeric `3.0`, `3.0`, and `240.0` seconds. Preserve meaningful setting names where native semantics are unchanged; update argument types accordingly. Remove retired per-role endpoint/model/credential/concurrency/retry variables in favor of the native backend/profile interface. Document their replacements; do not infer a conversion from arbitrary old inventory. Remove diagnostics retention; leave debug disabled through native defaults. Retain the existing diagnostics work-directory choice where still supported. Reporting follows Audita's native configuration; Narratio consumes its ordinary output contract. Production and testing continue using the same Audita endpoint and model. No separate Audita testing overlay is needed for an identical configuration. ### Credential Lists and Overlay Selection Replace the suite's singular Narratio Notarius credential variable with `dnd_pipeline_narratio_notarius_credentials`, default `[OPENROUTER_API_KEY]`. Add `dnd_pipeline_narratio_audita_credentials`, default `[AUDITA_LLM_API_KEY]`. Pass both through the reviewed Narratio role's native list variables. These lists contain names, never values, and can be replaced with `[]`. Operators overriding Audita backends must explicitly list all referenced credentials; the suite does not inspect configurations to discover names. Preserve `dnd_pipeline_secrets`, existing secret names, Vault references, protected secret directory/files, and ordinary world-readable nonsecret configuration. No credential rotation, secret relocation, or forwarding logic is required in Ansible. Deploy the suite-owned testing files at: - `/usr/local/etc/narratio/overlays/testing.yml` - `/usr/local/etc/notarius/overlays/testing.yml` Narratio testing sets `notarius.config_overlay: testing`; production omits the selector. Notarius remains enabled in both modes. Its production pipeline continues selecting `gpt-terra-latest`, testing selects `gpt-luna-latest`, and the existing caller-overridable pipeline map/ID behavior remains intact. Scriptorium testing still selects `gemini-flash-latest`; production continues using the prompt assets' selections. Retain separate artifact/family fragments. Use Narratio `--overlay testing` and Notarius `--config-overlay testing` in tests and operator examples. Deploy the new files declaratively; leave ignored legacy `profiles/testing.yml` files alone and document that they are obsolete. Do not delete `profiles/` directories or conflate them with PromptKit profiles. ### Runtime Paths and Existing Suite Behavior Keep Narratio's explicit downstream binaries/config paths, output handoffs, Notarius `/var/cache/notarius` roots and resume behavior, shared writable directories/ACLs, service user, operator enrollment, and S3/publishing choices. Audita profiles are configuration assets, not shared output directories. No new runtime storage architecture is required. ### Retry and Timeout Policy Retain the previously selected per-attempt Audita profile timeouts of 900 seconds while inheriting the new applications' retry, repair, and whole-operation defaults. Expose nullable suite pass-throughs for `audita_llm_transient_retries`, `audita_llm_repair_attempts`, `audita_llm_operation_timeout`, `notarius_promptkit_transient_retries`, and `scriptorium_transient_retries`, prefixed with `dnd_pipeline_` in the public suite interface. Null means omit; explicit zero is preserved where valid. Do not mechanically translate old `max_retries: 3` into both transport retries and structural repairs. Preserve explicitly authored Notarius module retries; they are a separate behavior. Keep Narratio stage timeouts unchanged. Explain the nested budgets and the changed semantics, not a guarantee of identical provider cost or timing. ## Repository Ownership - `ansible-role-dnd-pipeline`: suite defaults, argument specs, dependency wiring, Audita profile assets, testing overlays, documentation, and integration tests. - `ansible-collection-dnd`: imported pipeline snapshot, dependency manifest, collection tests, and compatibility guidance. - `rakestraw-ansible`: matching dependency pins and deployment guidance; adjust host variables only where required. Preserve Vault content and unrelated inventory. Keep the hosts-and-roles playbook minimal. - Standalone role repositories: reviewed commits are reused/published, not redesigned. Application and campaign/prompt repositories are read-only contracts/assets unless a concrete incompatibility is raised for discussion. - Workspace: roadmap, later staged plan, decisions, progress, and review. ## Constraints and Settled Decisions Follow the [Ansible](../../ansible.md), [testing](../../testing.md), and [planning](../../planning.md) policies, [MVP roadmap](../mvp.md), and [upgrade sequence](../application-upgrade-sequence.md). New D&D policy remains in the pipeline role; optional generic application defaults remain null. ## Simplest Viable Implementation Update dependency variables/pins and existing templates, add an ordinary Audita profile directory task and one looped template deployment, and change testing destinations to `overlays/`. Reuse existing declarative convergence, secret delivery, and collection assembly. No production validation, state fingerprints, migration framework, or generated credential inference is needed. ## Acceptance Criteria - Pipeline and collection installations produce a compatible set of the four upgraded applications while Seriatim remains unchanged. - Native offline inspection/validation accepts Audita settings and selected profiles, Notarius base/testing configurations, Scriptorium configuration, and Narratio base/testing composition with synthetic credential inputs. Direct Audita validation receives credential environment variables explicitly: it does not bootstrap Narratio's secrets directory on its own. - Focused cases prove caller-supplied backend/profile configuration and credential lists, testing selection, and existing important composition and path handoffs. Assert supplied values, not fixed copies of today's defaults. - Immediate repeated convergence is clean. Existing meaningful shared-user, cache, artifact-composition, and installed-collection behavior remains covered without expanding into upstream module/runtime regression matrices. - Collection build/install and consumer syntax checks use the established public dependency manifests. No sibling application/role checkout is needed by an external consumer. Required referenced commits exist on public Gitea. - Consumer inventory retains comments/Vault references for unknown secrets; no actual keys or operational deployments are required for completion. - Guidance covers breaking inventory fields, new CLI selectors, obsolete testing-file locations, and upstream manifest migration. Back up and migrate local provenance without deleting progress or editing immutable snapshots; document normal named-selector invalidation rather than forcing reruns. ## Open Questions None. The user approved deferring to application defaults wherever possible, with nullable inventory overrides, and confirmed retaining the existing 900-second Audita per-attempt policy. Do not freeze native retry, repair, or operation-deadline defaults in suite variables. Operators should note that an attempt may consume the entire logical-operation budget; these are separate controls, not additive timeout guarantees. ## Dependencies and Follow-up Work Features 0018–0021 provide the reviewed standalone revisions. Implementation, independent review, and remediation are complete. Git history preserves the completed staged implementation plan. Actual deployment, session migration, live processing, and a formal tagged release are separate operator actions, not completion gates for this feature. ## Implementation Evidence and Review Handoff On 2026-09-29, the four reviewed standalone role revisions were published to their existing public Gitea `origin/main` branches and verified after fetch: Scriptorium `b174c8c2990bb79f8149d8f288c5e06ccb42bd73`, Audita `c3dd59f5ac9b942306ffe540ae9e71d493998137`, Notarius `6aef48b7e12e3651971444b8c224d96ff1879e6c`, and Narratio `3c0700b5323293d44e3b1e67a50074d82d7a3bff`. The pipeline role at public revision `ea8b09e190419586ede6df3815d1ea999a0f4391` passed `bash -n`, its full `tests/integration` harness twice (configuration migration and representative override coverage), and `git diff --check`. The collection imported that exact public revision by subtree; public collection revision `3d77eb382bcb4c1742015d3c32e23630bb8c73bf` passed `bash -n`, its full artifact-based `tests/integration`, and `git diff --check`. Both harnesses used synthetic credentials and offline native application validation, with clean repeats. The consumer's local commit `f694dad9426ee21138b175e44a3855d42be99d4b` passed public dependency installation into disposable paths, real-inventory `--list-hosts` limited to `pipeline-dnd`, `--syntax-check`, and `git diff --check`. Its commit was not pushed. Seriatim and native collection pins remain unchanged, and manifests across the three repositories agree. The local integration harnesses converged only the development installation; there was no operational deployment or live inference, transcription, or object storage call. Vault values and target access were not needed or verified. After the override harness passed, a formatting-only split of its fixture JSON line was checked with `bash -n`, JSON parsing, and `git diff --check` without rerunning the long integration harness. Independent review subsequently found the three issues resolved below. No tagged release was created. ## Remediation Follow-up The three independent-review findings are resolved; see [remediation evidence](../remediation.md). Public pipeline `37b1cf6` and collection `b3bc045` remove default-identity assertions and correct migration and credential guidance. Local consumer `15f0f4e` selects the corrected public snapshot. Shell/whitespace checks, pipeline and installed-collection syntax and lint, collection build/install, and real-inventory consumer target/syntax checks passed. Full host-mutating convergence was not rerun for these documentation and test-only changes. Application versions/defaults and runtime behavior are unchanged. ## Closeout Closed on 2026-09-29 after independent review and resolution of all three findings. This completes the agreed application upgrade sequence. The active implementation plan is retired; no additional implementation is required for this feature. Consumer commits remain local. Operational upgrade, session migration, live production/testing checks, and any new tagged release require separate operator action and were not performed as part of closeout. ## Upstream References - [Audita v2 migration](https://gitea.maximumdirect.net/eric/audita/src/tag/v2.0.1/docs/migrations/v2.md) - [Audita configuration](https://gitea.maximumdirect.net/eric/audita/src/tag/v2.0.1/docs/config.md) - [Narratio v3 release](https://gitea.maximumdirect.net/eric/narratio/src/tag/v3.0.0/docs/releases/v3.0.0.md) - [Narratio operations and migration](https://gitea.maximumdirect.net/eric/narratio/src/tag/v3.0.0/docs/operations.md) - [Notarius v0.8.0 migration](https://gitea.maximumdirect.net/eric/notarius/src/tag/v0.8.0/docs/releases/v0.8.0.md) - [Scriptorium v0.14.0 migration](https://gitea.maximumdirect.net/eric/scriptorium/src/tag/v0.14.0/docs/releases/v0.14.0.md)