Resume D&D storage and publishing plan

docs/roadmap/implementation.md

@@ -14,10 +14,12 @@ Active plan location: `docs/roadmap/implementation.md`
 Configure the authoritative D&D pipeline role to use the selected Backblaze
 S3-compatible archive, typed object-storage credential references, maximum S3
 audio caching, immutable run publication, selected transcript and D&D artifact
-outputs, and cleanup of successfully published workspace state. Validate the
-configuration through Narratio's native interface, import the exact reviewed
-pipeline commit into the collection from public Gitea, and leave the result
-ready for the real-host deployment in Feature 0016.
+outputs, and cleanup of successfully published workspace state. Advance the
+pipeline to Narratio v2.1.1 so explicit publish requiredness composes with
+native destination derivation. Validate the configuration through Narratio's
+native interface, import the exact reviewed pipeline commit into the collection
+from public Gitea, and leave the result ready for the real-host deployment in
+Feature 0016.
 
 The Feature 0015 roadmap remains authoritative for scope, rationale, target
 state, and acceptance criteria. This plan does not authorize a second storage
@@ -69,16 +71,22 @@ lane, or changes to the generic Narratio role.
 - Features 0009, 0011, 0012, 0013, and 0014 are complete.
 - The approved Feature 0015 roadmap and this implementation plan are committed
   in the workspace repository before the sprint begins.
-- `ansible-role-dnd-pipeline` starts clean on `main` at
-  `c13f726a0a846180a1aa95d110b8bbc801edaae5`, equal to `origin/main`.
+- `ansible-role-dnd-pipeline` remains based on
+  `c13f726a0a846180a1aa95d110b8bbc801edaae5`, equal to `origin/main`, with the
+  interrupted Stage 1 edits preserved and uncommitted in `README.md`,
+  `defaults/main.yml`, `meta/argument_specs.yml`, `meta/main.yml`, and
+  `tests/integration`. Review and complete those edits; do not discard them.
 - `ansible-collection-dnd` starts clean on `main` at
   `c0b0ebd13e92199f3d94e138c6cc50d1ece5dadd`, equal to `origin/main`.
 - `ansible-role-narratio` is clean at
   `292d8669e7aad5cfe80b0c469d22b6177e8cf0f1`, equal to `origin/main`, and its
-  `v0.1.0` role release contains the Narratio v2.1.0 field-specific interface
-  required by this feature.
+  `v0.1.0` role release contains the field-specific interface required by this
+  feature and can install Narratio v2.1.1 through `narratio_version`.
 - The sibling Narratio application repository is clean at public release
-  `v2.1.0`. It is a read-only contract source during this sprint.
+  `v2.1.1`, commit `442469b9f57e65ce17f09059609dd1a37cf8c3b6`. That
+  backward-compatible release corrects destination derivation with explicit
+  publish requiredness and is a read-only contract source during the resumed
+  sprint.
 - The workspace `.venv` supplies the tested `ansible-playbook`,
   `ansible-galaxy`, and `ansible-lint` commands used by the repository test
   drivers.
@@ -99,7 +107,7 @@ lane, or changes to the generic Narratio role.
   collection snapshot.
 - `ansible-workspace-dnd` records the implementation handoff after the two
   implementation repositories are complete.
-- `ansible-role-narratio`, all other standalone roles, application sources,
+- `ansible-role-narratio`, all standalone roles, application sources,
   asset repositories, and `rakestraw-ansible` may be inspected but must remain
   unchanged.
 
@@ -131,13 +139,17 @@ Before beginning this stage, read:
 
 ### Preconditions
 
-- The workspace planning commit containing this plan is present.
-- `ansible-role-dnd-pipeline` is clean at the starting revision, is not behind
-  or diverged from `origin/main`, and contains no unrelated changes.
+- The workspace planning commit containing the original approved plan and this
+  resumed-plan revision are present.
+- `ansible-role-dnd-pipeline` is based on the starting revision, is not behind
+  or diverged from `origin/main`, and its only working-tree changes are the
+  preserved interrupted Stage 1 edits identified in Starting State and
+  Preconditions.
 - The current `rakestraw.narratio` dependency mapping and existing integration
   driver still match the structures reviewed while preparing this plan.
-- Narratio v2.1.0 still accepts the roadmap-selected storage, publish, cleanup,
-  cache, and typed credential-reference fields.
+- Narratio v2.1.1 remains publicly available and accepts the roadmap-selected
+  storage, publish, cleanup, cache, and typed credential-reference fields,
+  including rules with explicit `required` and omitted derivable `dest`.
 
 ### Owning Repository
 
@@ -145,7 +157,11 @@ Before beginning this stage, read:
 
 ### Required Changes
 
-1. Add the following exact suite defaults to `defaults/main.yml` near the
+1. Advance the existing application default to
+   `dnd_pipeline_narratio_version: v2.1.1`. Do not change the standalone
+   Narratio role release pin: its existing public interface can install this
+   compatible application patch release.
+2. Add the following exact suite defaults to `defaults/main.yml` near the
    existing Narratio policy variables:
 
    ```yaml
@@ -181,7 +197,7 @@ Before beginning this stage, read:
    Keep every `dest` omitted. Narratio derives transcript destinations from
    built-in sources and artifact destinations from the already managed
    artifact output paths.
-2. Add matching public entries to `meta/argument_specs.yml`. Use `type: bool`
+3. Add matching public entries to `meta/argument_specs.yml`. Use `type: bool`
    for the cleanup, path-style, cache, enabled, and upload settings; `type: str`
    for the backend and S3 string fields; and `type: list` for the complete
    publish output list. The S3 child fields, including both credential
@@ -190,55 +206,56 @@ Before beginning this stage, read:
    pair to `null` when selecting the AWS SDK provider chain. The remaining
    concrete suite-policy fields follow the role's existing required/defaulted
    argument-spec convention.
-3. In the `rakestraw.narratio` dependency entry in `meta/main.yml`, map every
+4. In the `rakestraw.narratio` dependency entry in `meta/main.yml`, map every
    new suite variable directly to its same-purpose standalone variable:
    `narratio_workspace_cleanup_after_publish`, the eight
    `narratio_storage_*` fields, `narratio_cache_s3_audio`,
    `narratio_publish_enabled`, `narratio_publish_upload_run`, and
    `narratio_publish_outputs`.
-4. Do not map `narratio_spool_delete_audio_after_publish`. Do not map
+5. Do not map `narratio_spool_delete_audio_after_publish`. Do not map
    `narratio_notarius_chunk_cache`. Do not add a new import, fragment,
    template, task, handler, assertion, or configuration-validation command.
-5. Revise the README's variable reference and processing-policy sections to
+6. Revise the README's application-version reference, upstream links, variable
+   reference, and processing-policy sections for Narratio v2.1.1 and
    document the selected S3 destination, paired nullable credential
    references, complete-list publish semantics, native destination derivation,
    maximum S3 audio caching, successful-publication workspace cleanup, and
    deliberate omission of spool deletion.
-6. Extend the README's managed-credential example to show
+7. Extend the README's managed-credential example to show
    `OBJECT_STORAGE_KEY_ID` and `OBJECT_STORAGE_KEY` mapped from encrypted
    inventory beside the existing Audita and OpenRouter entries. Explain that
    values remain outside ordinary configuration and that setting both S3
    credential references to `null` selects Narratio's ambient AWS SDK provider
    chain.
-7. Replace the README's statement that storage and publishing are deferred
+8. Replace the README's statement that storage and publishing are deferred
    with concise operator-facing descriptions and public upstream links for S3
    session audio, publish, restore, publication locks, status, and cleanup.
    Explain that accepted Notarius lanes remain
    `narratio.extraction.<output_key>` analyze inputs independently of whether
    they have stable publish rules.
-8. Adapt the existing `tests/integration` baseline rather than adding a new
+9. Adapt the existing `tests/integration` baseline rather than adding a new
    scenario. Add two synthetic object-storage values to
    `dnd_pipeline_secrets`; explicitly unset `OBJECT_STORAGE_KEY_ID` and
    `OBJECT_STORAGE_KEY` alongside the existing provider variables when
    invoking Ansible; and explicitly pass representative non-default S3,
    cleanup, cache, and publish values through the new public variables.
-9. Use representative values that exercise the interface without freezing the
+10. Use representative values that exercise the interface without freezing the
    defaults: for example an `.invalid` HTTPS endpoint and fixture bucket,
    prefix, and region; intentional `false` values for cleanup, path style, and
    S3 audio caching; the two configured credential reference names; enabled
    run publication; and a short complete publish list containing one built-in
    transcript source and `narratio.artifact.session_recap` with omitted
    destinations.
-10. Extend the existing secret-safe effective-configuration inspection only
+11. Extend the existing secret-safe effective-configuration inspection only
     enough to demonstrate the explicitly supplied representative storage
     fields, typed credential names, falsey cache/cleanup choices, enabled
     publication, and Narratio-derived destinations. Continue to use
     `narratio config validate` as the native validity boundary. Do not inspect
     role task results, call S3, or reproduce Narratio's validation rules.
-11. Extend the current non-disclosure check to cover both synthetic
+12. Extend the current non-disclosure check to cover both synthetic
     object-storage values. Preserve the complete existing integration matrix,
     cleanup behavior, and immediate clean-repeat assertions.
-12. Run the complete repository validation. Commit the valid result with the
+13. Run the complete repository validation. Commit the valid result with the
     stated plain-English message, push `main` to public Gitea, and record the
     full pushed commit hash for Stage 3.
 
@@ -252,15 +269,17 @@ Run from the workspace root:
 
 Expected result: syntax checking and production-profile lint pass; all five
 applications remain valid; the explicitly supplied S3 and publish values
-resolve through Narratio's native configuration interface; only reference
-names appear in effective output; no synthetic credential value appears in
-captured output; no external S3 or model request occurs; and the immediate
-identical repeat reports `changed=0` and `failed=0`.
+resolve through Narratio v2.1.1's native configuration interface; explicit
+publish requiredness and omitted destinations produce the derived destinations;
+only reference names appear in effective output; no synthetic credential value
+appears in captured output; no external S3 or model request occurs; and the
+immediate identical repeat reports `changed=0` and `failed=0`.
 
 ### Completion Criteria
 
-- Defaults, argument specifications, dependency mappings, and documentation
-  implement exactly the Feature 0015 contract.
+- The pipeline selects Narratio v2.1.1, and defaults, argument specifications,
+  dependency mappings, and documentation implement exactly the Feature 0015
+  contract.
 - No production task, template, fragment, validation command, or second secret
   path was added.
 - The representative integration scenario exercises public inputs rather than
@@ -400,7 +419,8 @@ Before beginning this stage, read:
    dependency mappings, README, and detailed integration fixture.
 3. Do not modify `galaxy.yml`, `requirements.yml`, release documentation, or
    the published `v0.1.0` artifacts. Feature 0015 changes no standalone role or
-   native collection dependency and publishes no release.
+   native collection dependency and publishes no new release; the pipeline's
+   application-version variable carries the Narratio v2.1.1 selection.
 4. Run the complete collection integration suite against the built and
    installed artifact. Do not add collection-level assertions for the detailed
    S3 configuration; native validation and clean convergence remain the
@@ -520,8 +540,9 @@ Commit intent: `Record D&D storage and publishing implementation`
   unexpected branch divergence or untracked artifact.
 - Report the full pipeline commit, collection fixture commit, collection
   subtree merge and squash commits, and workspace handoff commit.
-- Confirm that `ansible-role-narratio`, all other standalone roles,
-  applications, assets, and `rakestraw-ansible` remained unchanged.
+- Confirm that `ansible-role-narratio`, all standalone roles, applications,
+  assets, and `rakestraw-ansible` remained unchanged during the resumed sprint;
+  Narratio v2.1.1 was an already released prerequisite.
 - Summarize any deviation from this plan, unresolved finding, or complexity
   added beyond direct variable mapping and identify the acceptance criterion
   that required it.
@@ -530,4 +551,12 @@ Commit intent: `Record D&D storage and publishing implementation`
 
 ## Progress Notes
 
-- Planning complete; implementation has not begun.
+- The original plan was approved and Stage 1 began against Narratio v2.1.0.
+- Native validation exposed Narratio's publish-output atomic-sequence
+  bookkeeping defect; the implementation agent correctly stopped with its
+  partial pipeline edits uncommitted.
+- Narratio v2.1.1 at `442469b9f57e65ce17f09059609dd1a37cf8c3b6`
+  corrected the defect and added focused regression coverage without changing
+  the public configuration contract.
+- Resume Stage 1 from the preserved pipeline edits, advance the application pin
+  to v2.1.1, rerun the complete validation, and then continue in stage order.