diff --git a/docs/roadmap/application-upgrade-sequence.md b/docs/roadmap/application-upgrade-sequence.md index c8a40a8..36c3905 100644 --- a/docs/roadmap/application-upgrade-sequence.md +++ b/docs/roadmap/application-upgrade-sequence.md @@ -39,7 +39,7 @@ independent review and closeout. - [x] [Feature 0018: Scriptorium adoption](features/0018-scriptorium-v0-15-adoption.md) — complete through role `b174c8c`; independent review found no remediation issues. - [x] [Feature 0019: Audita adoption](features/0019-audita-v2-adoption.md) — complete through role `c3dd59f`; independent review found no remediation issues. - [x] [Feature 0020: Notarius adoption](features/0020-notarius-v0-8-adoption.md) — complete through role `6aef48b`; independent review found no remediation issues; suite unchanged. -- [ ] Narratio adoption — feature roadmap not yet created. +- [ ] [Feature 0021: Narratio adoption](features/0021-narratio-v3-adoption.md) — roadmap and staged plan approved; ready for implementation. - [ ] Suite integration — feature roadmap not yet created. ## Adoption Boundaries diff --git a/docs/roadmap/features/0021-narratio-v3-adoption.md b/docs/roadmap/features/0021-narratio-v3-adoption.md new file mode 100644 index 0000000..49bc7bd --- /dev/null +++ b/docs/roadmap/features/0021-narratio-v3-adoption.md @@ -0,0 +1,214 @@ +# Feature 0021: Adopt Narratio v3.0.0 in the Standalone Role + +Status: ready + +## Summary + +Update the standalone Narratio role from v2.2.0 to v3.0.0 and align its +configuration interface with the new Audita and Notarius contracts. This is +the fourth feature in the agreed +[application upgrade sequence](../application-upgrade-sequence.md). Suite +integration remains a separate, subsequent feature. + +## Problem or Opportunity + +Narratio v3 replaces configuration-profile terminology with overlays, replaces +singular downstream credential references with lists, and delegates Audita v2 +configuration to Audita. The existing role still exposes retired fields that +v3 rejects. Advancing only the executable would therefore leave callers and +the representative configuration fixture with an incompatible interface. + +## Scope + +- Default the standalone role to Narratio v3.0.0. +- Align defaults, argument specifications, template, examples, and tests with + the changed downstream configuration fields. +- Document operator migration, ownership boundaries, and native compatibility + consequences without automating session or asset migration. +- Record the independently reviewed role revision for final suite integration. + +## Non-goals + +- Updating pipeline, collection, consumer, or other application-role pins. +- Deploying downstream applications, changing D&D defaults, or migrating + suite-owned testing overlays and credentials. +- Creating arbitrary overlay assets or selecting Narratio's runtime overlay + through an installer variable. +- Compatibility aliases, dual-version template branches, automatic migration + of existing manifests, cache deletion, or forced stage reruns. +- New production validation, credential discovery, runtime version gates, + forwarding/redaction machinery, or upstream application changes. +- Live providers, operational deployment, or publication. + +## Target End State + +### Standalone Installation + +`narratio_version` defaults to `v3.0.0`. Preserve source-build behavior, Go +selection, service account, filesystem paths, owner/group/mode interfaces, and +the required caller-supplied WhisperX URL. Configuration continues to use the +existing root document shape and one ordinary template task. Unchanged native +fields and caller composition remain supported. + +### Audita Interface + +Add the following optional variables with null defaults. Field paths below +refer to the root configuration document's `audita` section; upstream prose +also calls this `pipeline.audita`. + +| Role variable | Native field | Argument type | +| --- | --- | --- | +| `narratio_audita_credentials` | `audita.credentials` | list of strings | +| `narratio_audita_overlay` | `audita.overlay` | string | +| `narratio_audita_proposal_profile` | `audita.proposal_profile` | string | +| `narratio_audita_validation_profile` | `audita.validation_profile` | string | + +Remove these retired role variables and their rendered fields: + +- `narratio_audita_llm_api_key` +- `narratio_audita_base_url` +- `narratio_audita_model` +- `narratio_audita_total_llm_concurrency` +- `narratio_audita_proposal_llm_concurrency` +- `narratio_audita_validation_model` +- `narratio_audita_validation_llm_concurrency` +- `narratio_audita_work_dir_retention` +- `narratio_audita_report` + +Retain binary, timeout, modules, transcript description, config path, and output +schema variables. Null modules are omitted; explicit `[]` means no correction +modules. Audita owns backend definitions, model/profile contents, concurrency, +reporting, and debug policy. Document migration to Audita's own configuration +without trying to infer replacements for those policies. + +### Notarius Interface + +Replace `narratio_notarius_config_profile` with +`narratio_notarius_config_overlay`, an optional nullable string rendered as +`notarius.config_overlay`. Preserve supplied empty strings, `default`, and +named selectors exactly; null omits the selector. Selection is independent of +Narratio's own overlay and is not a PromptKit LLM-profile selection. + +Replace `narratio_notarius_llm_api_key` with +`narratio_notarius_credentials`, an optional nullable list of strings rendered +as `notarius.credentials`. Preserve other Notarius configuration interfaces, +including cache controls, references, and output mappings. + +### Rendering and Credential Ownership + +Use `is not none`, appropriate serialization, and parent-section conditions +covering the new variables. Null omits a field; empty lists and strings remain +explicit. Do not copy native defaults into optional role defaults. + +Credential lists contain environment-variable/secret-reference **names**, not +secret values. Narratio resolves and forwards them under their exact names. +The role neither resolves credentials nor infers them from downstream files. +Null omits the field and `[]` emits an explicit empty list; Narratio overlays +replace lists rather than append to them. Operators must list every backend +credential needed by the selected Audita configuration, including configured +but unselected backends. Notarius likewise requires names matching its selected +provider configuration. Resolution occurs at pipeline loading, even when a +downstream stage will not execute. + +Removing retired variables is a documented breaking role-interface migration. +Do not add alias precedence, version-dependent rendering, or special guards +for old inventory. Users retaining old deployments retain their old role pin. + +### Operator Upgrade Guidance + +- Replace Narratio `--profile` with `--overlay` and selected `profiles/` files + with `overlays/`. Explicit imports from a directory named `profiles` remain + valid; this is not a blanket filesystem rename or an LLM-profile rename. +- Migrate downstream Notarius selection to `config_overlay` and its selected + files to `overlays/`. Named selection needs a supporting Notarius executable; + v0.8.1 is the tested baseline, not a new unconditional role version gate. +- Audita v2 must be configured independently before running polish. Narratio + does not generate its config or infer downstream overlay/profile selectors. +- Link the tagged manifest migration procedure: back up old manifests and + migrate `effective_config.selected_profile` to `selected_overlay` while + preserving progress/checkpoint state. Do not rewrite immutable remote + snapshots or infer that every stage must rerun. +- Note the native extraction-resume consequence for renamed named Notarius + selectors: the next applicable run regenerates extraction with normal + dependent invalidation. Omitted/base selections do not gain invalidation + solely from this rename. The role does not implement this behavior. + +## Repository Ownership + +- `ansible-role-narratio`: standalone interface, template, documentation, and + repository-local tests. +- Workspace: roadmap, later staged plan, status, and independent review. +- `../narratio`: read-only authoritative v3.0.0 contracts. +- Pipeline, collection, consumer, assets, and other roles remain unchanged. + +## Constraints and Settled Decisions + +Follow the [Ansible house style](../../ansible.md), +[testing house style](../../testing.md), [MVP roadmap](../mvp.md), and +[upgrade sequence](../application-upgrade-sequence.md). Use tagged v3.0.0 +contracts rather than assuming a newer checkout has the same interface. + +Keep meaningful explicit v2.1.0 version-selection and supplied-Go recovery +tests where their baseline inputs remain compatible. Do not turn configurable +version selection into a promise that new configuration fields work on old +applications; new fields stay unset in those cases. + +## Simplest Viable Implementation + +Change the version default and the affected defaults, argument-spec entries, +and conditional template fields. Adapt existing representative fixtures and +native validation to the new public interface. No additional production tasks, +state probes, file migration, or configuration assembly mechanism is needed. + +## Rationale and Alternatives + +Expose the native v3 interface directly instead of retaining aliases or +translating legacy Audita tuning into guessed backend/profile definitions. +This preserves application ownership and avoids a compatibility framework in +the generic installer. Coordinated suite migration can then make explicit D&D +policy choices against already-reviewed standalone roles. + +## Acceptance Criteria + +- The default role installs Narratio v3.0.0; native offline validation accepts + baseline and representative rendered configuration. +- Added and renamed fields agree across defaults, argument specifications, + template, examples, and documentation. Retired native fields are not emitted. +- Focused behavioral cases demonstrate null omission, supplied credential-name + lists, explicit empty lists, named/empty downstream selectors, and existing + meaningful falsey and artifact-composition behavior. Keep this representative, + not a matrix of every native rule or a snapshot of today's defaults. +- Native validation uses synthetic credential values supplied only to the test + process as needed; managed configuration contains names, not values. No live + provider calls or reimplementation of upstream forwarding/redaction tests is + required. Preserve existing useful log checks without expanding their scope. +- Immediate identical role application reports no changes. Selected-version + and supplied-Go/missing-binary outcomes remain covered. +- Syntax/lint and local repository integration checks pass. No operational + controller, target deployment, downstream pipeline execution, or real keys + are prerequisites for standalone-role acceptance. +- Documentation explains breaking variable changes and links upstream upgrade + procedures. No production validation or automatic migration has been added. +- Suite, collection, and consumer pins remain unchanged. Independent review + records the standalone revision for coordinated integration. + +## Dependencies and Follow-up Work + +Standalone Scriptorium, Audita, and Notarius adoption is complete. This roadmap +is approved; execution is defined in the [implementation plan](../implementation.md). +After implementation and review, the final integration feature owns D&D +configuration, credentials, overlay assets, coordinated dependency pins, +collection assembly, and consumer migration guidance. It must deliberately +review retry/timeout budgets rather than copying retired tuning mechanically. + +## Open Questions + +None identified for this standalone scope. + +## Upstream References + +- [Narratio v3.0.0 release](https://gitea.maximumdirect.net/eric/narratio/src/tag/v3.0.0/docs/releases/v3.0.0.md) +- [Configuration](https://gitea.maximumdirect.net/eric/narratio/src/tag/v3.0.0/docs/config.md) +- [Audita integration](https://gitea.maximumdirect.net/eric/narratio/src/tag/v3.0.0/docs/integrations/audita.md) +- [Notarius integration](https://gitea.maximumdirect.net/eric/narratio/src/tag/v3.0.0/docs/integrations/notarius.md) +- [Operator migration procedures](https://gitea.maximumdirect.net/eric/narratio/src/tag/v3.0.0/docs/operations.md) diff --git a/docs/roadmap/implementation.md b/docs/roadmap/implementation.md index 6e5a842..a3d9758 100644 --- a/docs/roadmap/implementation.md +++ b/docs/roadmap/implementation.md @@ -1,12 +1,260 @@ -# Implementation Plan +# Implementation Plan 0021: Adopt Narratio v3.0.0 -No implementation plan is currently active. Agree on the next feature roadmap -before preparing its staged plan. +Status: ready -Feature 0020 is complete and independently reviewed with no remediation -findings. See [its roadmap](features/0020-notarius-v0-8-adoption.md) for -completion evidence. Git history preserves the completed plan. +Target executor: GPT-5.6-Terra -Narratio v3 standalone adoption is next in the agreed -[application upgrade sequence](application-upgrade-sequence.md). Suite and -consumer pins remain unchanged until coordinated integration. +Feature roadmap: [Feature 0021](features/0021-narratio-v3-adoption.md) + +## Objective + +Adopt Narratio v3.0.0 in its standalone role with native downstream credential +lists and overlay/profile selectors. The approved roadmap owns target state, +scope, and rationale. Suite integration is not part of this plan. + +## Execution Rules + +- Execute stages in numeric order; reread each stage's required policies + immediately before starting. Check applicable repository instructions too. +- Confirm the owning repository and working-tree state before editing. + Preserve unrelated changes. Validate and commit each completed unit using a + plain English message. Do not push or publish. +- Keep the existing production task flow: account/directories, source checkout, + conditional build, metadata, and a single configuration template. No new + tasks, facts, probes, credential lookups, aliases, or migration machinery are + needed. Trust Ansible's modules and upstream application semantics. +- Stop for an actual unresolved product decision; report genuine validation + limitations without inventing stronger acceptance gates or expanding scope. + +## Starting State and Repository Scope + +- Features 0018–0020 are complete. The Narratio role currently defaults to + v2.2.0; target v3.0.0 is fixed for this feature. +- Change only `ansible-role-narratio` and workspace planning/status documents. + Inspect `../narratio` read-only using tagged v3.0.0 documentation. Other roles, + suite, collection, consumer, and assets remain unchanged. Retain every + existing suite application/dependency pin, including Narratio v2.2.0. +- Use workspace `.venv/bin/ansible-playbook` and `.venv/bin/ansible-lint`. + Missing shell PATH entries do not mean a controller is unavailable. The + existing integration harness selects local tooling and installs its declared + collection dependency. +- Integration uses the local development host with sudo. Read the harness + before running: it modifies the managed local account/directories, source, + binary, and configuration, then reapplies baseline configuration on exit. + Confirm the local test installation is disposable. Do not deploy to a + controller or operational pipeline host. +- Source/dependency downloads are needed; WhisperX/provider access, real + credentials, downstream processing, and deployment are not. Use the existing + synthetic test credentials only for native configuration loading. + +## Stage 1: Migrate the Role and Existing Fixtures to the v3 Interface + +### Required Policy Review + +Read workspace `AGENTS.md`, `docs/orientation.md`, `docs/conventions.md`, +`docs/ansible.md`, `docs/testing.md`, and `docs/planning.md`. + +### Owning Repositories and Preconditions + +`ansible-role-narratio` owns implementation; workspace owns progress. Feature +0021 is approved. Read the role's defaults, argument specs, template, README, +and integration fixtures, plus tagged upstream configuration, release notes, +and Audita/Notarius integration contracts linked by the roadmap. + +### Required Changes + +1. Mark F0021 `in_progress` in workspace status when work starts. +2. Set `narratio_version: v3.0.0`. Preserve build, Go, account, filesystem, + required WhisperX URL, and all unchanged configuration interfaces. +3. In `defaults/main.yml`, `meta/argument_specs.yml`, and + `templates/pipeline.yml.j2`, add these optional null-default variables: + - `narratio_audita_credentials`: list with string elements → + `audita.credentials`. + - `narratio_audita_overlay`: string → `audita.overlay`. + - `narratio_audita_proposal_profile`: string → `audita.proposal_profile`. + - `narratio_audita_validation_profile`: string → `audita.validation_profile`. + - `narratio_notarius_credentials`: list with string elements → + `notarius.credentials`, replacing `narratio_notarius_llm_api_key`. + - `narratio_notarius_config_overlay`: string → `notarius.config_overlay`, + replacing `narratio_notarius_config_profile`. +4. Remove all nine retired Audita variables listed in the roadmap's Audita + Interface section from defaults, specs, and rendering. Retain modules, + binary, timeout, transcript description, config path, and output schema. +5. Update parent conditions and serialize with the existing filters. Use + `is not none`: null omits, `[]` and `""` render explicitly. These fields + belong in the existing root `audita`/`notarius` sections; do not add a + literal enclosing `pipeline` key. Do not inspect downstream configurations + or resolve secret-reference names inside Ansible. +6. Update the README variable list and affected examples to the new interface. + State the breaking rename/removal and direct old deployments to their + previous role pin. Detailed operator guidance follows in Stage 3. +7. Keep the existing integration harness usable in this commit: + - Remove `narratio_audita_report` and its assertion from the representative + fixture/verifier. + - Replace its singular Notarius credential with a one-element name list + using the existing synthetic reference. Replace the configuration-profile + variable, rendered-field assertion, verifier expectation, and empty-profile + harness case with their configuration-overlay equivalents. + - Retain other meaningful assertions and the existing synthetic-value log + check. Keep v2.1.0 selection/recovery baseline cases with new fields unset; + do not run the v3 representative configuration against the older binary. + +### Validation + +From the workspace root: + +```sh +bash -n ansible-role-narratio/tests/integration +ansible-role-narratio/tests/integration +git -C ansible-role-narratio diff --check +git diff --check +``` + +The harness includes syntax and lint. Require successful native baseline and +migrated representative validation, retained selected-version/supplied-Go +behavior, and clean repeated convergence. Review the diff for consistent public +interfaces and no production-task or suite changes. Do not test retired +variables with a rejection matrix; their removal is a documented interface +change, not a new Ansible guard. + +### Completion and Commit + +The role installs v3 and its existing behavioral suite works through the new +interface. Commit the role as `Adopt the Narratio v3 configuration interface`. +Commit workspace progress separately as `Record the Narratio upgrade in progress`. + +## Stage 2: Cover the New Selectors and Credential Lists + +### Required Policy Review + +Read workspace `AGENTS.md`, `docs/orientation.md`, `docs/conventions.md`, +`docs/ansible.md`, and `docs/testing.md`. + +### Owning Repository and Preconditions + +`ansible-role-narratio`. Stage 1 is committed and its integration suite passes. +Extend the existing fixtures and verifier; do not introduce a second framework. + +### Required Changes + +1. Make null-omission inputs explicit for the six new variables in the + baseline case. Verify those fields are omitted from the rendered YAML, + without asserting the complete default document. Keep baseline native + validation and the immediate clean repeat. These null inputs also remain + compatible with the older-version baseline cases. +2. Extend the representative case with explicitly named Audita overlay, + proposal profile, and validation profile, plus both credential lists. + Use two synthetic names for Audita and reuse one in the Notarius list so + the rendered result exercises a real list rather than a disguised scalar. + Suggested names are `NARRATIO_TEST_AUDITA_API_KEY` and the existing + `NARRATIO_TEST_NOTARIUS_API_KEY`. Assert supplied lists and selectors in the + managed YAML, preserving existing falsey/composition assertions. +3. Supply nonempty synthetic values through the test process environment and + pass those named values to the native validation command running as the + Narratio account. The managed YAML contains names only. Extend the existing + test-log check to those synthetic values without adding subprocess stubs, + provider calls, or a redaction test matrix. Narratio configuration validation + does not require creating the selected downstream overlay/profile assets. +4. Extend the existing empty-overlay case into one explicit-empty case: + override both credential lists to `[]` and all four downstream selector + strings to `""`. Parameterize only the affected verifier expectations so + this case checks actual empty lists/strings, not omitted fields. Native + validation should work without synthetic credential environment values for + this case. Add an immediate identical repeat with a zero-change recap. +5. Keep default version output informational. Retain explicit v2.1.0 selection, + supplied-Go/missing-binary recovery, and baseline/representative clean + repeats. Do not add assertions of current default values, module-owned + filesystem metadata, native rejection rules, or downstream execution. + +### Validation + +From the workspace root: + +```sh +bash -n ansible-role-narratio/tests/integration +ansible-role-narratio/tests/integration +git -C ansible-role-narratio diff --check +``` + +Require native acceptance and observed supplied values for baseline, named, +and explicit-empty configurations, plus clean immediate repeats. Validation +must stay offline with respect to application providers and downstream stages. + +### Completion and Commit + +The new role-owned rendering behavior and existing outcomes pass without +testing Narratio's internal forwarding implementation. Record commands/results. +Commit as `Test Narratio selectors and credential lists`. + +## Stage 3: Document Upgrade Ownership and Hand Off for Review + +### Required Policy Review + +Read workspace `AGENTS.md`, `docs/orientation.md`, `docs/conventions.md`, +`docs/ansible.md`, `docs/testing.md`, and `docs/planning.md`. + +### Owning Repositories and Preconditions + +`ansible-role-narratio` and workspace. Stages 1 and 2 are committed and pass. +This stage documents the accepted interface; it does not integrate the suite. + +### Required Changes + +1. Finish README upgrade guidance: identify removed/renamed role variables, + distinguish credential names from values, explain null versus explicit + empties, and explain independent downstream selectors. Audita requires + references for all configured backend credentials, not just selected ones. + Narratio resolves references at pipeline load, even for skipped stages. +2. Explain that Audita v2 owns its configuration, profiles, reporting, and + backend tuning. The role neither creates downstream files nor guesses + replacements for removed concurrency/model settings. Downstream binaries + must support selected runtime features; do not introduce version gates. +3. Document Narratio `--overlay` and selected `overlays/` files, preserving the + distinction from explicit imports and PromptKit LLM profiles. Explain the + Notarius `config_overlay` rename and defer suite testing-file migration. +4. Link the tagged v3.0.0 manifest migration procedure and summarize preserving + local progress/checkpoints and immutable snapshots. Explain named Notarius + selector extraction invalidation as native application behavior; no automatic + migration, cache cleanup, or blanket forced reruns belong in this role. +5. Record role commit references and validation evidence in the feature + roadmap, and concise progress in this plan, `docs/status.md`, and + `docs/roadmap/application-upgrade-sequence.md`. Keep F0021 `in_progress`, + awaiting independent review. Do not mark the feature done or describe its + revision as independently reviewed yet. + +### Validation + +Check documentation against the implemented interface and tagged upstream +release/configuration/operations contracts. From the workspace root: + +```sh +git -C ansible-role-narratio diff --check +git diff --check +scripts/status +``` + +Reuse Stage 2 behavioral evidence for documentation-only edits; rerun affected +checks if implementation changes. Confirm only the owning role and workspace +changed and no suite/collection/consumer pin advanced. + +### Completion and Commit + +Guidance is accurate and review evidence is recorded. Commit the role as +`Document the Narratio v3 upgrade`, then workspace tracking as +`Record Narratio upgrade validation`. + +## Final Validation and Handoff + +Report commits, commands and outcomes, any deviations or environment limits, +and final working-tree state. Explain any complexity beyond the unchanged +production task flow and the requirement necessitating it. Leave independent +review and closeout pending. Coordinated suite integration is next; do not +begin it, publish, push, or deploy from this plan. + +## Progress Notes + +Plan approved; implementation has not started. + +## Open Questions + +None. diff --git a/docs/status.md b/docs/status.md index 619ada2..1e41e9f 100644 --- a/docs/status.md +++ b/docs/status.md @@ -4,7 +4,8 @@ Last updated: 2026-09-28 Current milestone: Operational D&D pipeline deployment -Current focus: Plan standalone Narratio v3.0.0 adoption as the next unit of work. +Current focus: Implement Feature 0021's standalone Narratio v3.0.0 adoption. +The feature roadmap and staged implementation plan are approved. The suite remains unchanged. Features 0018 through 0020 are complete after independent review with no remediation findings. Follow the agreed @@ -44,6 +45,7 @@ is complete. | F0018 | Adopt Scriptorium v0.15.0 | Scriptorium role and workspace | done | Role commits `61dcafe`, `2fce529`, and `b174c8c` passed independent review with no findings; feature roadmap records validation; suite pins unchanged | | F0019 | Adopt Audita v2.0.1 | Audita role and workspace | done | Role `c3dd59f`, following `6137ff2` and `52f67fe`, passed independent review with no findings; [roadmap](roadmap/features/0019-audita-v2-adoption.md) records validation; suite pins unchanged | | F0020 | Adopt Notarius v0.8.1 | Notarius role and workspace | done | Role `6aef48b`, following `b1b922c` and `380b6a3`, passed independent review with no findings; [feature roadmap](roadmap/features/0020-notarius-v0-8-adoption.md) records evidence; suite pins unchanged | +| F0021 | Adopt Narratio v3.0.0 | Narratio role and workspace | ready | [Feature roadmap](roadmap/features/0021-narratio-v3-adoption.md) and [staged plan](roadmap/implementation.md) approved; suite integration deferred | | A0001 | Audit completed standalone-role test suites | workspace and standalone roles | done | Findings recorded in `docs/roadmap/test-suite-audit-findings.md` | | R0001 | Simplify completed standalone-role test suites | workspace and standalone roles | done | Role commits `e08c63c`, `d2d0b9d`, and `cfecf37` passed implementation and independent review |