diff --git a/docs/roadmap/features/0011-managed-pipeline-credential-delivery.md b/docs/roadmap/features/0011-managed-pipeline-credential-delivery.md index 04e8f22..18becfa 100644 --- a/docs/roadmap/features/0011-managed-pipeline-credential-delivery.md +++ b/docs/roadmap/features/0011-managed-pipeline-credential-delivery.md @@ -1,6 +1,6 @@ # Feature 0011: Managed Pipeline Credentials and Notarius Reuse -Status: ready +Status: done ## Summary @@ -78,19 +78,18 @@ retaining their existing independently overrideable variables. credential location. - Create that directory as the selected pipeline runtime user and group with mode `0750`. -- Add one public `dnd_pipeline_secrets` mapping. Keys are environment-variable - names. A non-null string value means that the named credential file must be - present with that content; null means that the specifically named managed - credential file must be absent. An omitted key is unmanaged. +- Add one public `dnd_pipeline_secrets` mapping from environment-variable names + to string file contents. Each supplied entry means that the named credential + file must be present with that content; an omitted key is unmanaged. - Default `dnd_pipeline_secrets` to an empty mapping so the role never invents credentials and remains usable with externally prepared process environment values. -- Validate selected keys as environment-variable names and reject empty - non-null values before constructing a destination path. -- Write non-null credential files as the selected runtime user and group with - mode `0640`, and remove only names explicitly assigned null. +- Validate selected keys as environment-variable names and values as strings + before constructing a destination path. +- Write supplied credential files as the selected runtime user and group with + mode `0640`. - Suppress credential values from Ansible task output during validation, - creation, updates, and removal. + creation, and updates. - Keep ordinary application configuration limited to non-secret credential reference names. - Coordinate Audita proposal and validation on the suite-default @@ -110,8 +109,8 @@ retaining their existing independently overrideable variables. Narratio's secrets bootstrap supplies the selected environment value and the existing Scriptorium adapter passes that named credential to its child. - Document the mapping interface, Ansible Vault inventory pattern, process - environment precedence, shared-group authorization boundary, explicit null - removal, application credential flow, and checkpoint-reuse policy. + environment precedence, shared-group authorization boundary, application + credential flow, and checkpoint-reuse policy. - Add focused integration coverage using distinctive synthetic values and native Narratio configuration loading. No default test contacts a model provider. @@ -167,21 +166,19 @@ The filenames are suite-selected reference names, never credential values. The selected runtime user and group remain configurable through the existing pipeline interface. -A consumer may keep encrypted values in inventory and select explicit removal -without embedding any value in ordinary configuration: +A consumer may keep encrypted values in inventory without embedding any value +in ordinary configuration: ```yaml dnd_pipeline_secrets: AUDITA_LLM_API_KEY: "{{ vault_dnd_audita_llm_api_key }}" OPENROUTER_API_KEY: "{{ vault_dnd_openrouter_api_key }}" - RETIRED_PROVIDER_KEY: null ``` -The role owns only these three named entries for this invocation. It converges -the first two to regular files, removes the third, and leaves every omitted -name alone. The actual `vault_dnd_*` values belong in encrypted inventory and -must not appear in examples, plans, test logs, commits, or ordinary generated -configuration. +The role owns only the supplied named entries for this invocation and leaves +every omitted name alone. The actual `vault_dnd_*` values belong in encrypted +inventory and must not appear in examples, plans, test logs, commits, or +ordinary generated configuration. When Narratio runs without `--secrets-dir`, it discovers the sibling directory and loads its immediate regular files before resolving the pipeline. An @@ -252,8 +249,9 @@ and its existing adapter forwards the selected profile credential. - Credential filenames must match the environment-variable grammar accepted by ConfigKit. Validation exists to prevent a caller-controlled key from becoming a path outside the secrets directory. -- Empty credential values are invalid. Null means explicit absence and is - distinct from an empty string. +- Credential values are written as supplied strings. ConfigKit and the typed + application reference own any nonempty-value requirement when a credential + is selected for use. - An omitted mapping key is outside the role's ownership. Exact-directory mirroring and deletion of unlisted files are intentionally excluded. - The shared runtime group is an authorization group. Its members may read the @@ -280,8 +278,8 @@ generic and does not own credential files or suite policy. The pipeline role needs one empty credential mapping, two Narratio Notarius policy variables, coordinated Audita reference defaults, and ordinary dependency wiring. Credential convergence consists of one directory task, one -validation loop, one copy loop for non-null entries, and one file loop for -explicit null removals. Both secret-bearing loops suppress their output. +validation loop, and one copy loop for supplied entries. Both secret-bearing +loops suppress their output. Documentation and focused native integration coverage complete the public interface. No state probe, handler, template assembly, manifest, wrapper, @@ -300,9 +298,10 @@ environment-precedence and redaction rules. A mapping is more ergonomic for inventory than a list of file objects because the native identity of each entry is already its environment-variable name. -Null provides explicit removal without making an empty mapping destructive or -requiring a filesystem scan. Omitted files remain available for consumers that -combine Ansible-managed credentials with another provisioning mechanism. +Omitted files remain unmanaged so consumers can combine Ansible-managed +credentials with another provisioning mechanism. Credential removal and +directory mirroring remain outside this feature until an operator-facing +lifecycle requirement is expressly selected. Per-user `0700` directories would follow Narratio's general recommendation but would require separate keys or an additional invocation convention for every @@ -327,12 +326,11 @@ application's documented ownership boundary. - The pipeline role creates `/usr/local/etc/narratio/secrets` with the selected runtime ownership and suite credential-directory mode. - `dnd_pipeline_secrets` defaults to an empty mapping and accepts valid - environment names with nonempty string or null values. -- Invalid names and empty non-null values fail before any destination outside - the declared secret directory can be constructed. -- Non-null entries converge to regular files with the selected runtime - ownership and mode `0640`; null entries converge to absence; omitted names - remain unmanaged. + environment names with string values. +- Invalid names and non-string values fail before any destination outside the + declared secret directory can be constructed. +- Supplied entries converge to regular files with the selected runtime + ownership and mode `0640`; omitted names remain unmanaged. - Credential values are suppressed from Ansible output and never appear in application YAML, generated examples, native effective-configuration output, test logs, or committed fixtures. @@ -350,12 +348,9 @@ application's documented ownership boundary. variables unset loads credentials through the conventional directory and succeeds at native offline configuration validation as the service account and one existing-group-member fixture. -- One representative explicit-null transition makes the selected synthetic - credential unavailable through the native bootstrap boundary without - inspecting module results or file metadata. - A distinctive synthetic secret never appears in captured integration output. -- The baseline and representative credential applications each report - `changed=0` and `failed=0` on an immediate identical repeat. +- The baseline credential application reports `changed=0` and `failed=0` on + an immediate identical repeat. - Syntax checking, production-profile lint, and the complete standalone Narratio and pipeline integration entry points pass. - No live provider, WhisperX service, storage backend, model call, or @@ -389,4 +384,6 @@ application's documented ownership boundary. not treat runtime-group membership as shared credential authorization. - Add explicit credential rotation orchestration and provider-side validity checks. +- Add an explicit credential-removal or directory-lifecycle contract if an + operator need is identified. - Add platform-specific mandatory-access-control policy for credential files. diff --git a/docs/roadmap/implementation.md b/docs/roadmap/implementation.md index 32d2503..28ea5ba 100644 --- a/docs/roadmap/implementation.md +++ b/docs/roadmap/implementation.md @@ -1,633 +1,8 @@ -# Implementation Plan 0011: Managed Pipeline Credentials and Notarius Reuse +# Implementation Plan -Status: ready +Status: pending -Target executor: GPT-5.6-Terra +Feature 0011 is complete. There is no active implementation plan. -Feature roadmap: -`docs/roadmap/features/0011-managed-pipeline-credential-delivery.md` - -Active plan location: `docs/roadmap/implementation.md` - -## Objective - -Upgrade the generic Narratio role to v2.1.0 and expose its native Notarius -credential and cache controls, then have the D&D pipeline role deliver shared -provider credentials through Narratio's conventional secrets directory, -coordinate the Audita and Notarius credential references, and request -compatible Notarius checkpoint reuse. - -The feature roadmap remains authoritative for scope, rationale, target end -state, and acceptance criteria. Decisions 0001 and 0002 remain authoritative -for direct Ansible convergence and the shared-runtime-group authorization -boundary. - -## Execution Rules - -- Execute stages in numeric order. -- Before each stage, confirm its preconditions and the working-tree state of - its owning repository. -- Before each stage, reread every document listed in that stage's Required - Policy Review. Do not rely on policy recalled from an earlier stage. -- Preserve unrelated and pre-existing changes. Stop if an owning repository is - unexpectedly dirty at a stage boundary. -- Keep `ansible-role-narratio` generic. It exposes Narratio's native optional - fields but does not select D&D credentials, manage secret files, or own - suite cache policy. -- Keep credential delivery and cross-application policy in - `ansible-role-dnd-pipeline`. Do not modify an application source repository - or move suite policy into the Audita, Notarius, or Scriptorium roles. -- Never place a real credential in source, documentation, examples, test - fixtures, command output, or commits. Tests may use distinctive synthetic - values and must prove that those values do not appear in captured output. -- Treat a `dnd_pipeline_secrets` key as untrusted path input. Validate the - complete mapping before any key is interpolated into a destination path. - Accept only environment-variable names matching - `^[A-Za-z_][A-Za-z0-9_]*$`; accept only nonempty strings or null as values. -- Apply `no_log: true` to every production task that evaluates, writes, or - removes mapping entries. Do not expose a loop label containing a credential - value. -- Use direct module-owned convergence: one fixed secrets-directory task, one - validation loop, one copy loop for non-null entries, and one file loop for - explicit null entries. Do not add preliminary `stat`, directory scans, - recursive ownership repair, checksums, handlers, wrappers, manifests, state - files, backups, or post-write validation tasks. -- An omitted credential key is unmanaged. Do not mirror the mapping onto the - directory, purge unlisted files, or infer that an empty mapping removes - anything. -- Keep ordinary application YAML secret-free. It may contain only credential - reference names. -- Leave Narratio's `notarius.chunk_cache` unset in the pipeline role. The - deployed Notarius root configuration remains the sole owner of ordinary - `auto` chunk-plan reuse, while Narratio owns its native forced-refresh - behavior. -- Test role-owned behavior through public role inputs and native application - commands. Do not test Ansible module guarantees, inspect module results, - assert task order, snapshot complete default documents, contact a live - provider, or execute the application pipeline. -- Run every stage's validation before committing it. Commit each completed - stage when it leaves its owning repository valid, using the plain-English - commit intent specified by that stage. -- Do not begin a dependent stage after an earlier stage fails validation. -- The pipeline integration suite installs role dependencies from canonical Git - origins. The Stage 1 Narratio-role commit must therefore be reachable from - its canonical origin before Stage 2 begins. If a fresh Galaxy installation - cannot resolve it, stop after Stage 1 and report the commit hash for - publication; do not substitute a branch, local role path, or weakened test. - -## Starting State and Preconditions - -- The workspace control repository contains the reviewed Feature 0011 roadmap - and this implementation plan in a clean planning commit on `main`. -- Features 0001 through 0010 and the standalone-role test-suite simplification - are complete. -- The implementation repositories are clean on `main` at these reviewed - starting commits: - - | Repository | Starting commit | - | --- | --- | - | `ansible-role-narratio` | `eeb1105590e638c1f46b42c98a5dbbce4ddeea86` | - | `ansible-role-dnd-pipeline` | `bbf01e107a5e4e70a2abd6c696bfdb9ccc38bd7e` | - -- The Narratio role matches `origin/main`. The pipeline role is intentionally - ahead of its canonical origin and must not be reset or replaced. -- `../narratio` is clean at immutable tag `v2.1.0`, commit - `c45e042462b77a8873b4215edc40e3840149db34`. -- Narratio v2.1.0 adds optional `notarius.llm_api_key`, `notarius.resume`, and - `notarius.chunk_cache` fields without requiring migration of existing v2 - configuration. Its conventional secrets bootstrap loads immediate regular - files from a sibling `secrets/` directory, gives an existing process - environment value precedence, and requires every configured typed reference - to resolve during configuration loading. -- Notarius v0.7.0 is already selected by the suite, records checkpoints under - the persistent FHS cache root, and leaves ordinary chunk-plan mode at - `auto`. -- PromptKit documents `OPENROUTER_API_KEY` as the credential name for the - OpenRouter backend selected by the suite's production Notarius profile. -- The workspace `.venv` provides Ansible Core 2.21.3 and ansible-lint 26.8.0. -- The local integration environment remains an accepted proxy for Ubuntu - 26.04, supports task-level privilege escalation, and provides ordinary POSIX - group and ACL behavior. - -## Repository Scope - -Repositories that change: - -- `ansible-role-narratio`: Narratio v2.1.0 selection, three nullable native - Notarius fields, generic documentation, and focused integration coverage. -- `ansible-role-dnd-pipeline`: compatible Narratio role and application pins, - credential delivery, Audita and Notarius reference coordination, checkpoint - reuse, consumer documentation, and focused integration coverage. -- `ansible-workspace-dnd`: implementation progress and review handoff only - after the role stages are complete. - -Repositories that may be inspected but must remain unchanged: - -- `../narratio`, which is the read-only v2.1.0 application authority. -- `../notarius`, `../audita`, `../seriatim`, `../scriptorium`, and PromptKit, - which may be consulted for current native contracts. -- `ansible-role-notarius`, `ansible-role-audita`, - `ansible-role-seriatim`, and `ansible-role-scriptorium`. -- `ansible-collection-dnd`, all asset repositories, `rakestraw-ansible`, and - every other repository in `workspace.yml`. - -## Stage 1: Upgrade the Generic Narratio Role - -### Objective - -Make the standalone role install Narratio v2.1.0 and expose the release's -three optional Notarius fields without adding suite-specific credential or -cache policy. - -### Required Policy Review - -Before beginning this stage, read: - -- `AGENTS.md` -- `docs/orientation.md` -- `docs/conventions.md` -- `docs/ansible.md` -- `docs/testing.md` -- `docs/roadmap/features/0011-managed-pipeline-credential-delivery.md` -- `docs/decisions/0001-minimal-module-driven-ansible.md` -- `../narratio/docs/releases/v2.1.0.md` -- `../narratio/docs/config.md` -- `../narratio/docs/integrations/notarius.md` -- `../narratio/examples/pipeline.full.annotated.yml` -- `ansible-role-narratio/defaults/main.yml` -- `ansible-role-narratio/meta/argument_specs.yml` -- `ansible-role-narratio/templates/pipeline.yml.j2` -- `ansible-role-narratio/tests/integration` -- `ansible-role-narratio/tests/representative.yml` -- `ansible-role-narratio/tests/verify-baseline.yml` -- `ansible-role-narratio/tests/verify-config.yml` -- `ansible-role-narratio/README.md` - -### Preconditions - -- `ansible-role-narratio` is clean at its stated starting commit. -- Its existing `tests/integration` entry point passes before modification. -- The upstream files listed above remain at the reviewed immutable v2.1.0 tag. - -### Owning Repository - -`ansible-role-narratio` - -### Required Changes - -1. Advance `narratio_version` from `v2.0.0` to `v2.1.0`. Do not change the - established source checkout, build condition, metadata convergence, or - application-account task flow. -2. Add these optional public variables beside the existing Notarius variables, - each defaulted to null: - `narratio_notarius_llm_api_key`, `narratio_notarius_resume`, and - `narratio_notarius_chunk_cache`. -3. Add matching optional argument-specification entries. Use `str` for the - credential reference and chunk-cache mode and `bool` for resume. Do not - duplicate Narratio's native enum or secret-resolution validation in the - role. -4. Extend the existing `notarius` template section and its parent-emission - condition. Render `llm_api_key`, `resume`, and `chunk_cache` only when their - corresponding variable `is not none`; preserve intentional `false` for - resume and serialize every value with the template's established filter. -5. Leave all three fields absent when null. Do not create a secrets directory, - read an environment variable, accept a raw credential value, or select a - Notarius provider in this standalone role. -6. Update the README's version, optional-variable list, native Notarius field - discussion, and validation summary. Explain that the credential field is - an environment-variable name and that credential delivery remains the - caller's responsibility. Describe resume and chunk-cache as native - application choices without assigning D&D defaults. -7. Update the integration suite to explicitly select v2.1.0 for the existing - version-selection assertion. Continue testing version selection rather than - asserting that v2.1.0 remains the default. -8. Extend the representative configuration case with a synthetic environment - reference, `resume: false` as the falsey-value case, and one valid explicit - chunk-cache mode. Supply a synthetic value for that named environment - variable only when running Narratio's native validation or safe inspection; - never put a credential value in managed YAML. -9. Observe the three representative fields and baseline null omission through - the repository's existing focused configuration checks. Retain native - `narratio config validate` coverage and immediate clean repeated - convergence. Do not test ConfigKit's resolution precedence or Narratio's - subprocess forwarding implementation. - -### Validation - -Run from `ansible-role-narratio`: - -```text -tests/integration -git diff --check -git status --short -``` - -Expected result: syntax and production-profile lint pass; an explicitly -selected v2.1.0 binary reports that version; baseline null values remain -omitted; representative credential-name, false resume, and chunk-cache values -load through Narratio's native configuration interface; immediate repeats -report `changed=0` and `failed=0`; and only intended role, test, and README -files are modified. - -### Completion Criteria - -- Defaults, argument specification, template, README, and tests expose one - coherent generic interface for the three v2.1.0 fields. -- The role owns no credential value, secrets directory, D&D provider choice, - or suite cache policy. -- Tests exercise the public configuration interface without snapshotting the - complete default document or re-testing application internals. -- The repository is valid and the stage is committed. -- The commit is reachable from the canonical Narratio-role origin before - Stage 2 begins. - -### Commit - -Commit intent: `Upgrade the Narratio role to version 2.1.0` - -## Stage 2: Implement Pipeline Credential Delivery and Reuse Policy - -### Objective - -Add the pipeline role's production credential-delivery tasks, coordinate its -Audita and Notarius references, and enable Narratio checkpoint reuse through -the reviewed Stage 1 interface. - -### Required Policy Review - -Before beginning this stage, read: - -- `AGENTS.md` -- `docs/orientation.md` -- `docs/conventions.md` -- `docs/ansible.md` -- `docs/testing.md` -- `docs/roadmap/features/0011-managed-pipeline-credential-delivery.md` -- `docs/decisions/0001-minimal-module-driven-ansible.md` -- `docs/decisions/0002-shared-readable-pipeline-filesystem-policy.md` -- the committed Stage 1 changes in `ansible-role-narratio` -- `../narratio/docs/releases/v2.1.0.md` -- `../narratio/docs/config.md` -- `../narratio/docs/integrations/audita.md` -- `../narratio/docs/integrations/notarius.md` -- `../narratio/docs/integrations/scriptorium.md` -- `ansible-role-dnd-pipeline/defaults/main.yml` -- `ansible-role-dnd-pipeline/meta/argument_specs.yml` -- `ansible-role-dnd-pipeline/meta/main.yml` -- `ansible-role-dnd-pipeline/requirements.yml` -- `ansible-role-dnd-pipeline/tasks/main.yml` -- `ansible-role-dnd-pipeline/tests/converge.yml` -- `ansible-role-dnd-pipeline/tests/integration` - -### Preconditions - -- Stage 1 is committed, validated, and clean. -- The Stage 1 commit is reachable from the canonical - `ansible-role-narratio` Git origin. Stop and report the hash if a fresh - Galaxy installation cannot resolve it. -- `ansible-role-dnd-pipeline` is clean at its stated starting commit or an - expected reviewed descendant. -- Its existing `tests/integration` entry point passes before modification. - -### Owning Repository - -`ansible-role-dnd-pipeline` - -### Required Changes - -1. Set `dnd_pipeline_narratio_version: v2.1.0` and pin the - `rakestraw.narratio` entry in `requirements.yml` to the exact Stage 1 - commit. Retain the canonical Git URL and all unrelated dependency pins. -2. Add `dnd_pipeline_secrets: {}` as a required, defaulted dictionary in the - public interface. Its keys are credential filenames/environment-variable - names; values are either nonempty strings to converge or null to remove the - exact named file. -3. Add required, defaulted - `dnd_pipeline_narratio_notarius_llm_api_key: OPENROUTER_API_KEY` as a string - and `dnd_pipeline_narratio_notarius_resume: true` as a boolean. Do not add a - suite variable for Narratio's optional chunk-cache override. -4. Change the suite default for - `dnd_pipeline_audita_llm_validation_api_key_env` from - `AUDITA_VALIDATION_LLM_API_KEY` to `AUDITA_LLM_API_KEY`. Retain the - proposal and validation fields as independently overrideable public - variables; do not add an equality assertion or promise split-key forwarding. -5. Pass `dnd_pipeline_audita_llm_proposal_api_key_env` through Narratio's - `narratio_audita_llm_api_key` interface. Pass the new Notarius reference and - resume variables through `narratio_notarius_llm_api_key` and - `narratio_notarius_resume`. Do not pass - `narratio_notarius_chunk_cache`; Notarius's root `auto` setting remains in - control for ordinary runs. -6. Before any task interpolates a mapping key into a path, validate every - `dnd_pipeline_secrets | dict2items` entry in one `ansible.builtin.assert` - loop. Require a string key matching `^[A-Za-z_][A-Za-z0-9_]*$` and a value - that is either null or a nonempty string. Apply `no_log: true` to the entire - validation task. Do not normalize, trim, rename, or infer keys or values. -7. Create `/usr/local/etc/narratio/secrets` with one ordinary - `ansible.builtin.file` task using the selected runtime user and group and - fixed mode `0750`. Do not recursively alter existing entries. -8. Converge non-null mapping entries with one `ansible.builtin.copy` loop to - `/usr/local/etc/narratio/secrets/`, using the selected runtime user and - group and fixed mode `0640`. Write only the supplied value; do not append - metadata, encode it, back it up, or reread it. Apply `no_log: true`. -9. Converge null entries to absence with one `ansible.builtin.file` loop over - the same mapping. Remove only the exact validated named file and apply - `no_log: true`. An empty mapping and omitted keys must leave existing - directory entries untouched. -10. Keep production convergence to those four operations: validation, - directory creation, non-null copy, and explicit-null removal. Do not add - `stat`, `find`, commands, handlers, environment files, shell profiles, - launchers, services, providers, rotation, or role-private state. -11. Adapt the existing integration driver's baseline arguments to supply - distinctive synthetic values for `AUDITA_LLM_API_KEY` and - `OPENROUTER_API_KEY` through `dnd_pipeline_secrets`, with both ambient - variables explicitly unset. Preserve isolated dependency installation and - cleanup restoration. -12. Extend the existing native Narratio effective-configuration check to - observe the two public credential reference names and `resume: true` while - confirming that Narratio's `chunk_cache` override is absent. Retain the - native Notarius check that its root chunk-plan mode is `auto`, the service - and existing-group-member validation paths, and the baseline clean repeat. - -### Validation - -Run from `ansible-role-dnd-pipeline`: - -```text -tests/integration -git diff --check -git status --short -``` - -Expected result: isolated role installation resolves the exact Stage 1 -commit; syntax and production-profile lint pass; the role converges the -synthetic credential mapping without displaying its values; Narratio natively -loads the conventional secrets directory as both the service account and the -existing-group-member fixture; its safe effective configuration names the -Audita and Notarius references and enables resume without a chunk-cache -override; Notarius retains `auto`; the baseline repeat reports `changed=0` and -`failed=0`; and only intended pipeline-role files are modified. - -### Completion Criteria - -- The suite installs Narratio v2.1.0 through the exact reviewed role revision. -- One public mapping owns only explicitly named credential files, including - explicit null removal, behind complete pre-path validation and secret-safe - task output. -- Audita's suite defaults use one typed reference, Notarius uses - `OPENROUTER_API_KEY`, and ordinary extraction requests checkpoint reuse. -- The pipeline does not duplicate Notarius's chunk-plan policy or add a second - credential-loading mechanism. -- The existing integration suite passes and the repository is committed. - -### Commit - -Commit intent: `Add managed credentials to the D&D pipeline` - -## Stage 3: Document and Exercise the Credential Contract - -### Objective - -Document the pipeline's credential and reuse policy and add the smallest -behavioral coverage for validation, ownership semantics, explicit removal, -redaction, authorization, and immediate idempotence. - -### Required Policy Review - -Before beginning this stage, read: - -- `AGENTS.md` -- `docs/orientation.md` -- `docs/conventions.md` -- `docs/ansible.md` -- `docs/testing.md` -- `docs/roadmap/features/0011-managed-pipeline-credential-delivery.md` -- `docs/decisions/0002-shared-readable-pipeline-filesystem-policy.md` -- the committed Stage 1 and Stage 2 changes -- `../narratio/docs/config.md` -- `../narratio/docs/cli.md` -- `../narratio/docs/integrations/notarius.md` -- `ansible-role-dnd-pipeline/README.md` -- `ansible-role-dnd-pipeline/tests/integration` -- `ansible-role-dnd-pipeline/tests/converge.yml` - -### Preconditions - -- Stages 1 and 2 are committed, validated, and clean. -- The pipeline's isolated dependency installation resolves the Stage 1 commit. -- The deployed baseline can be restored through the integration driver's - existing cleanup path. - -### Owning Repository - -`ansible-role-dnd-pipeline` - -### Required Changes - -1. Update the README's compatible Narratio version, Audita validation - credential default, public variable documentation, and obsolete statements - that API-key management and checkpoint reuse are deferred. -2. Document `dnd_pipeline_secrets` as an empty-by-default mapping with valid - environment-variable keys, nonempty string values for presence, null for - exact removal, and omission for unmanaged entries. Explain that the role - neither mirrors nor purges the directory. -3. Include an Ansible Vault inventory pattern that references encrypted - variables without showing a real credential. State that Vault is a - recommended consumer workflow, not functionality implemented by the role. -4. Document the fixed conventional path, `0750` directory and `0640` file - modes, selected runtime ownership, runtime-group authorization meaning, - process-environment precedence, and the absence of wrappers, services, or - parallel loaders. Make clear that ordinary configuration remains - world-readable and secret-free while the secrets subtree does not grant - world access. -5. Document the coordinated `AUDITA_LLM_API_KEY` default, the typed - `OPENROUTER_API_KEY` Notarius reference, Scriptorium's existing - profile-owned reference behavior, ordinary checkpoint resume, Notarius's - root-owned `auto` chunk cache, and Narratio's forced refresh behavior. -6. Add one representative expected-failure case for an invalid mapping key - and one for an empty non-null value. Require each convergence to fail before - any candidate destination is managed. Keep task output secret-safe, do not - assert a task name or other implementation trace, and do not add an - exhaustive validation matrix. -7. Exercise explicit-null removal with a uniquely named synthetic credential: - first manage it, select its name through the public Notarius reference - variable, and confirm native configuration loading succeeds. Then converge - the same mapping entry to null with the relevant ambient variable unset. - Observe unavailability through an expected failure from Narratio's native - configuration loader rather than by inspecting a module result or file - metadata. -8. Require an immediate identical repeat of the explicit-null state to report - `changed=0` and `failed=0`, then restore the baseline credential mapping so - cleanup and later tests remain deterministic. -9. Retain native configuration validation as the service account and the - existing-group-member fixture. Add at most one representative native - negative check showing that an identity outside the runtime group cannot - load the credential-backed pipeline; do not build a permission matrix or - assert numeric filesystem metadata. -10. Capture convergence and native inspection output and fail if either - distinctive synthetic credential value appears. Names may appear in safe - effective configuration; values must not. -11. Retain the existing baseline, testing-profile, selected-version, - caller-composed Notarius pipeline, shared-cache, immutable local asset, - and clean-repeat coverage. Do not contact an LLM, WhisperX, storage, or - publishing endpoint; do not execute extraction or test ConfigKit, - Narratio, Notarius, or trusted Ansible modules independently. - -### Validation - -Run from `ansible-role-dnd-pipeline`: - -```text -tests/integration -git diff --check -git status --short -``` - -Expected result: isolated dependency installation, syntax checking, and -production-profile lint pass; valid synthetic credentials load through the -conventional directory for authorized identities; invalid names and empty -values fail safely; explicit null makes only the selected credential -unavailable through native loading and converges cleanly on repeat; no -synthetic value appears in captured output; all existing representative paths -continue to pass; and documentation matches the public interface. - -### Completion Criteria - -- Consumer documentation completely describes the mapping, Vault usage, - authorization boundary, credential flows, cache policy, and exclusions. -- Tests protect role-owned validation, explicit ownership semantics, - redaction, authorized use, and clean repetition without testing module or - application internals. -- The default suite and representative explicit-null state each converge - cleanly on immediate repeat. -- The full integration suite passes and the repository is committed. - -### Commit - -Commit intent: `Document and test pipeline credential delivery` - -## Stage 4: Record the Implementation for Review - -### Objective - -Record the completed implementation and validation evidence in the workspace -without declaring the feature complete before independent review. - -### Required Policy Review - -Before beginning this stage, read: - -- `AGENTS.md` -- `docs/orientation.md` -- `docs/conventions.md` -- `docs/planning.md` -- `docs/roadmap/features/0011-managed-pipeline-credential-delivery.md` -- `docs/roadmap/implementation.md` -- `docs/status.md` - -### Preconditions - -- Stages 1 through 3 are committed and their owning repositories are clean. -- Both complete integration entry points pass at the committed revisions. -- The workspace control repository is clean at the planning commit. - -### Owning Repository - -`ansible-workspace-dnd` - -### Required Changes - -1. Update this plan's Progress Notes with the three implementation commit - hashes and concise validation results. -2. Update `docs/status.md` so Feature 0011 is `in_progress`, its evidence names - the implementation commits, and the current focus is independent review of - the completed implementation. -3. Keep the feature roadmap `ready`; do not mark Feature 0011 `done`, clean the - durable roadmap, or create or revise `docs/roadmap/remediation.md`. Those - actions belong to independent review and closeout. -4. Do not change implementation repositories or unrelated workspace policy in - this stage. - -### Validation - -Run from the workspace root: - -```text -git diff --check -./scripts/status -git status --short -``` - -Expected result: the status report identifies only the intended workspace -tracking edits before commit; both implementation repositories are clean; and -the dashboard and progress notes point to the exact completed stage commits. - -### Completion Criteria - -- Implementation and validation evidence is traceable across repositories. -- Feature 0011 remains awaiting independent review rather than being marked - complete. -- The workspace control repository is valid and the stage is committed. - -### Commit - -Commit intent: `Record the pipeline credential implementation` - -## Final Validation and Handoff - -After Stage 4, rerun both owning repositories' complete integration entry -points: - -```text -cd ansible-role-narratio && tests/integration -cd ../ansible-role-dnd-pipeline && tests/integration -``` - -Then, from the workspace root, run: - -```text -git diff --check -./scripts/status -``` - -The final handoff must: - -- identify the Stage 1 through Stage 4 commit hashes and confirm every changed - repository has a clean working tree; -- confirm syntax and production-profile lint pass in both changed roles; -- confirm the standalone role installs an explicitly selected Narratio v2.1.0, - exposes all three nullable native fields, validates representative values, - and immediately converges cleanly; -- confirm the pipeline installs the exact immutable Narratio-role commit, - loads managed synthetic credentials for authorized identities, rejects bad - mapping entries before path use, removes an explicitly null entry, suppresses - credential values, requests Notarius checkpoint reuse, leaves ordinary - chunk-plan mode with Notarius, and immediately converges cleanly; -- confirm no upstream application, asset, collection, unrelated standalone - role, real credential, external service, or real human account was modified; -- summarize any deviation, blocker, or deferred work, including any pause - required to publish the Stage 1 dependency commit; -- report any production complexity beyond the three generic Narratio fields, - suite variables and dependency wiring, and the four direct credential tasks, - together with the acceptance criterion that required it; and -- leave Feature 0011 `in_progress` for independent review rather than marking - it `done`. - -## Progress Notes - -- Stage 1 complete: `292d8669e7aad5cfe80b0c469d22b6177e8cf0f1` - (`Upgrade the Narratio role to version 2.1.0`). The Narratio integration - suite passed syntax and production-profile lint, explicit v2.1.0 selection, - native representative validation, redaction of the synthetic test value, and - immediate clean baseline and representative repeats. The commit is published - on the canonical Narratio-role origin. -- Stage 2 complete: `28a4d2762fbd6073fd7c8f14ead0c88f6e64928c` - (`Add managed credentials to the D&D pipeline`). The pipeline integration - suite passed isolated dependency installation, syntax and production-profile - lint, native configuration checks, secret-safe convergence, and all existing - clean-repeat coverage. -- Stage 3 complete: `8ff07618202fc4b15dd6a6c2c79c16328e028795` - (`Document and test pipeline credential delivery`). The pipeline integration - suite passed focused invalid-input, removal, authorization, redaction, and - idempotence coverage together with all retained regression paths. -- Stage 4 complete: workspace tracking now records implementation evidence and - leaves Feature 0011 `in_progress` for independent review. +Select the next unit of work and agree on its numbered feature roadmap before +replacing this notice with a staged implementation plan. diff --git a/docs/roadmap/mvp.md b/docs/roadmap/mvp.md index 2f8316e..c2c99c4 100644 --- a/docs/roadmap/mvp.md +++ b/docs/roadmap/mvp.md @@ -263,11 +263,12 @@ application-role releases in one source repository. general single-user `0700` and `0600` recommendation while retaining no world access. Ordinary configuration contains only credential reference names. -- The pipeline role exposes one empty-by-default credential mapping. Non-null - entries manage files named for valid environment variables; null explicitly - removes the named file; omitted names remain unmanaged. Credential values +- The pipeline role exposes one empty-by-default credential mapping from valid + environment-variable names to string file contents. Supplied entries manage + the corresponding files; omitted names remain unmanaged. Credential values are expected to come from encrypted inventory such as Ansible Vault and are - suppressed from task output. + suppressed from task output. Credential removal and directory mirroring are + outside the MVP. - Narratio's native ConfigKit secrets bootstrap owns loading these files and process-environment precedence. The suite does not add a wrapper, shell profile, service environment file, or parallel credential loader. diff --git a/docs/roadmap/remediation.md b/docs/roadmap/remediation.md index 1f4f620..e160298 100644 --- a/docs/roadmap/remediation.md +++ b/docs/roadmap/remediation.md @@ -1,112 +1,149 @@ -# Feature 0010 Remediation Review +# Feature 0011 Remediation Review Status: resolved Reviewed implementation commits: -- `14a38e0 Upgrade the Notarius role to version 0.7.0` -- `90893ae Integrate Notarius extraction into the D&D pipeline` -- `7fd7b24 Document and test the D&D Notarius integration` +- `292d866 Upgrade the Narratio role to version 2.1.0` +- `28a4d27 Add managed credentials to the D&D pipeline` +- `8ff0761 Document and test pipeline credential delivery` ## Review Summary -Feature 0010 implements the intended production architecture. The standalone -Notarius role supports ConfigKit imports without owning imported fragments. The -pipeline role owns the complete D&D extraction topology, renders it as a -ConfigKit fragment, selects it from the root Notarius configuration, connects -Narratio to that pipeline and its artifact paths, and manages a shared FHS -cache with the pipeline's group-access policy. The complete standalone and -pipeline integration suites pass, including native application validation and -immediate clean repeats. +Feature 0011 implements the principal target architecture. The standalone +Narratio role now installs v2.1.0 and generically exposes its three native +Notarius controls. The pipeline pins that role revision, manages a restricted +conventional secrets directory, coordinates the Audita and Notarius credential +references, and enables ordinary Notarius checkpoint reuse without duplicating +the root-owned chunk-cache policy. Both complete integration entry points pass, +including native application validation and immediate clean repeats. -No production behavior gap was found. Two low-severity issues should be -addressed: the configurable pipeline-ID boundary is not observed through -Narratio in the integration suite, and the ACL task names no longer describe -all of the roots they manage. +Two issues remain. First, the feature roadmap and implementation plan +strengthened the requested credential-delivery interface with deletion and +empty-value policies that were not required. The implementation consequently +contains an extra production task and substantial test and documentation +machinery for those invented semantics. Second, the pipeline README does not +explain the Scriptorium credential flow that the feature roadmap expressly +requires it to document. -## Finding 1: Alternate Pipeline Selection Is Not Observed Through Narratio +## Finding 1: The Credential Mapping Adds Unrequested Lifecycle Semantics -Severity: low +Severity: moderate -`ansible-role-dnd-pipeline/tests/integration:172-187` composes two alternate -Notarius pipeline maps and selects `archive-session` through -`dnd_pipeline_narratio_notarius_pipeline_id`. The test then validates only the -Notarius configuration and cache-mode override. It does not inspect Narratio's -effective configuration after the alternate convergence. +The user requested an inventory-friendly way to deliver credential values to +Narratio's conventional secrets directory. The roadmap subsequently added two +stronger behaviors without an express requirement: null means delete an exact +file, and an empty string must be rejected by the role. Those choices were +then made mandatory by the implementation plan. -The production wiring in `ansible-role-dnd-pipeline/meta/main.yml:45-50` is -correct, but a future regression that stops forwarding the selected pipeline -ID to Narratio could pass the current test. This is a focused behavioral -coverage gap at a feature-owned integration boundary, not a request for a -snapshot or implementation-detail assertion. +The resulting production implementation appears in +`ansible-role-dnd-pipeline/tasks/main.yml:10-53`. It validates the stronger +value policy, conditionally copies non-null entries, and uses a separate file +loop to delete null entries. The public contract and example are repeated in +`ansible-role-dnd-pipeline/README.md:275-291`, while +`ansible-role-dnd-pipeline/tests/integration:167-194` adds an empty-value +failure case and a multi-convergence deletion lifecycle. + +The environment-variable-name validation is justified because caller input +becomes a destination path. The two value semantics are different: they add +credential lifecycle and application-validity guarantees that the role was +not asked to own. ConfigKit accepts an empty immediate credential file and +performs its own nonempty check only when a configured typed reference resolves +to that value. A simple delivery role should write the caller-supplied string +and trust the native application boundary. Credential deletion can be added +later if an operator-facing removal contract is expressly requested. ### Required Remediation -- After the alternate convergence, use Narratio's native configuration command - with a deployed campaign and inspect the effective selected pipeline ID. -- Assert that Narratio observes `archive-session`. -- Retain the existing native Notarius checks, cache-mode check, and immediate - clean repeat. -- Do not run model-backed extraction or snapshot the complete generated - configuration. +- Narrow `dnd_pipeline_secrets` to an empty-by-default mapping from valid + environment-variable names to string file contents. +- Retain complete key validation before path construction and retain + `no_log: true` for every task that evaluates or writes credential values. +- Validate that values are strings, but do not add a nonempty policy that is + stricter than the native bootstrap and typed-reference boundary. +- Converge every supplied entry directly with one `ansible.builtin.copy` loop. + Remove the null condition and the separate deletion task. +- Treat omitted keys as unmanaged. Do not invent a replacement deletion, + purge, directory-mirroring, or credential-validation mechanism. +- Remove the empty-value and explicit-null lifecycle cases from the integration + suite. Retain the valid-name boundary case, representative credential + delivery, redaction checks, authorized native loading, and immediate clean + repeat. +- Correct the Feature 0011 roadmap, the settled MVP text, and the pipeline + README so they describe the narrower interface and no longer present the + strengthened semantics as user decisions. -## Finding 2: ACL Task Names Exclude the Notarius Cache They Manage +## Finding 2: Scriptorium Credential Delivery Is Not Documented for Consumers Severity: low -`ansible-role-dnd-pipeline/tasks/main.yml:19` and -`ansible-role-dnd-pipeline/tasks/main.yml:39` describe their ACL loops as -managing "Narratio writable roots." Both loops now also manage the Notarius -cache root. The task behavior is correct, but the execution log is misleading -about its scope. +The Feature 0011 roadmap requires documentation of all application credential +flows, including Scriptorium's existing profile-owned `api_key_env` selection. +`ansible-role-dnd-pipeline/README.md:275-291` explains the generic mapping and +the earlier Notarius section explains its typed reference, but the README never +tells an operator how a selected Scriptorium profile obtains its credential. + +Production behavior is already correct: Narratio bootstraps the named file into +its environment and its existing Scriptorium adapter forwards the selected +profile credential. This is a documentation gap, not a request for another +variable, task, or integration test. ### Required Remediation -- Rename both ACL tasks to describe the shared pipeline writable roots. -- Do not otherwise change the direct desired-state ACL implementation or its - task flow. +- Add a concise consumer-facing explanation that a Scriptorium profile remains + the owner of its `api_key_env` name. +- Explain that an operator can add a matching filename/value entry to + `dnd_pipeline_secrets`; Narratio's bootstrap supplies it and the existing + adapter forwards only the profile-selected credential. +- Do not add a suite-specific Scriptorium credential variable, inspect profile + files in Ansible, duplicate the application adapter, or add tests of + Narratio/Scriptorium internals. ## Resolution -Resolved in pipeline-role commit `bbf01e1`. The alternate-map integration case -now uses Narratio's native `config show` command to observe the selected -`archive-session` pipeline ID before its immediate clean repeat. The two ACL -task names now describe shared pipeline writable roots without changing their -module arguments, loops, or task flow. +Resolved in pipeline-role commit `de1b191`. The credential mapping now accepts +valid environment-variable names with string contents, validates the complete +mapping before path construction, and converges every supplied entry through +one secret-safe `ansible.builtin.copy` loop. The unrequested empty-value policy, +null deletion task, and associated lifecycle tests were removed. Omitted names +remain unmanaged. + +The pipeline README now explains how a Scriptorium profile's application-owned +`api_key_env` name selects a matching `dnd_pipeline_secrets` entry without +adding another suite variable or duplicating adapter behavior. The durable +Feature 0011 roadmap and MVP policy were corrected to record the narrower +operator-facing contract. ## Required Validation -After remediation, run: +Remediation validation: ```text cd ansible-role-dnd-pipeline && tests/integration git diff --check ``` -The expected result is successful native configuration validation for the -base, testing, and alternate profiles; observation of `archive-session` through -Narratio; successful shared-cache access; and immediate repeats with -`changed=0` and `failed=0`. +The result was successful syntax and lint validation; synthetic +credential delivery through the conventional directory; native Narratio +configuration loading for authorized identities; absence of synthetic values +from captured output; preservation of Notarius resume and root-owned `auto` +chunk caching; and immediate repeats with `changed=0` and `failed=0`. ## Review Validation Evidence -The independent review inspected the full implementation diffs and confirmed -that the dependency pins resolve to the intended role commits. A semantic -comparison found that the managed D&D pipeline topology matches the Notarius -v0.7.0 example after removing only the external reference paths that belong to -Narratio. The Narratio output map matches its upstream annotated example, and -the intentional omission of the optional spell catalog agrees with the feature -roadmap. - -Both `ansible-role-notarius/tests/integration` and -`ansible-role-dnd-pipeline/tests/integration` passed. Post-suite checks also -confirmed the expected Seriatim and Notarius versions, native Notarius pipeline -validation, native Narratio validation for the base and testing profiles, and -a clean repeat. `git diff --check` passed, and all repositories were clean -before this review document was replaced. +The review inspected the exact implementation diffs and confirmed that the +Narratio dependency pin resolves to the reviewed `292d866` revision. The +standalone Narratio `tests/integration` entry point passed with v2.1.0 native +configuration validation and clean repeats. The complete pipeline +`tests/integration` entry point also passed, including isolated dependency +installation, syntax checking, production-profile lint, authorized and denied +native loading, redaction checks, profile and version selections, and clean +repeats. No live model, transcription, storage, or publishing endpoint was +contacted. After remediation, the complete pipeline integration entry point passed again. -It observed `archive-session` through Narratio, validated the alternate -Notarius pipeline, and reported `changed=0` and `failed=0` on the identical -repeat. Cleanup restored Seriatim v1.5.0 and the default `dnd-session` -configuration. `git diff --check` also passed. +It exercised isolated dependency installation, the invalid-name boundary, +synthetic credential delivery and redaction, authorized native loading, +Notarius resume and root-owned `auto` chunk caching, the retained profile and +version selections, and immediate clean repeats. `git diff --check` also +passed before the remediation commit. diff --git a/docs/status.md b/docs/status.md index d927de5..f3da8ce 100644 --- a/docs/status.md +++ b/docs/status.md @@ -2,9 +2,9 @@ Last updated: 2026-09-16 -Current milestone: M1-03 suite-level pipeline role +Current milestone: M1-04 collection assembly -Current focus: Independent review of the completed Feature 0011 implementation. +Current focus: Select the next unit of work; collection assembly is proposed. | ID | Deliverable | Owner | State | Evidence or next step | | --- | --- | --- | --- | --- | @@ -15,7 +15,7 @@ Current focus: Independent review of the completed Feature 0011 implementation. | M0-05 | Suite-level role repository | ansible-role-dnd-pipeline | done | Repository created and added to `workspace.yml` | | M1-01 | Remaining MVP design decisions | workspace | done | Feature 0003 roadmap fixes the Notarius configuration interface | | M1-02 | Standalone application roles | role repositories | done | All five standalone roles implemented and independently reviewed | -| M1-03 | Suite-level pipeline role | ansible-role-dnd-pipeline | in_progress | Execute the ready Feature 0011 implementation plan | +| M1-03 | Suite-level pipeline role | ansible-role-dnd-pipeline | done | Features 0006 through 0011 completed through remediation commit `de1b191` | | M1-04 | Collection assembly | ansible-collection-dnd | proposed | Select and implement the pipeline-role incorporation mechanism | | F0001 | Install Seriatim from pinned source | ansible-role-seriatim | done | Implementation, integration coverage, and final remediation completed through `fb8dc41` | | F0002 | Install and configure Audita | ansible-role-audita | done | Field-specific configuration completed in `e61f02f` and `847c4c3`; full suite passed | @@ -27,7 +27,7 @@ Current focus: Independent review of the completed Feature 0011 implementation. | F0008 | Define opinionated D&D processing configuration | ansible-role-dnd-pipeline | done | Implemented and independently reviewed through remediation commit `3e253d7` | | F0009 | Establish shared pipeline filesystem and execution | configuration-bearing application roles and ansible-role-dnd-pipeline | done | Implemented and independently reviewed through remediation commit `b15fe8d` | | F0010 | Integrate D&D Notarius extraction | ansible-role-notarius and ansible-role-dnd-pipeline | done | Implemented and independently reviewed through remediation commit `bbf01e1` | -| F0011 | Deliver pipeline credentials and Notarius reuse | ansible-role-narratio and ansible-role-dnd-pipeline | in_progress | Implementation commits `292d866`, `28a4d27`, and `8ff0761`; awaiting independent review | +| F0011 | Deliver pipeline credentials and Notarius reuse | ansible-role-narratio and ansible-role-dnd-pipeline | done | Implemented and independently reviewed through remediation commit `de1b191` | | A0001 | Audit completed standalone-role test suites | workspace and standalone roles | done | Findings recorded in `docs/roadmap/test-suite-audit-findings.md` | | R0001 | Simplify completed standalone-role test suites | workspace and standalone roles | done | Role commits `e08c63c`, `d2d0b9d`, and `cfecf37` passed implementation and independent review |