eric/ansible-workspace-dnd
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Resume D&D storage and publishing plan

Browse Source
This commit is contained in:
Eric Rakestraw
2026-09-21 03:20:03 +00:00
parent e262b61292
commit a95a0ea280
3 changed files with 115 additions and 54 deletions
Download Patch File Download Diff File Expand all files Collapse all files

62
docs/roadmap/features/0015-dnd-s3-storage-and-publishing.md
View File

@@ -11,7 +11,7 @@ object-storage credentials delivered through its existing secrets directory,
publish the useful transcript and D&D artifact outputs, and retain maximum S3
audio caching.
The standalone Narratio role already exposes Narratio v2.1.0's complete
The standalone Narratio role already exposes Narratio v2.1.1's complete
configuration surface for this behavior. This feature supplies D&D suite
policy through that existing interface; it does not add another storage
implementation or credential mechanism.
@@ -25,6 +25,13 @@ publishing, and end-to-end external execution as deferred. The resulting suite
can validate configuration and process local state, but it does not select the
remote archive or publish the outputs needed for an operational deployment.
Narratio v2.1.0 also rejected a valid publish list when a rule explicitly set
`required` while omitting a derivable `dest`. Narratio v2.1.1 corrects that
resolution bookkeeping defect without changing the publish schema or
destination rules. This feature therefore advances the pipeline application
pin to v2.1.1 and retains explicit suite-owned requiredness together with
native destination derivation.
The prior test-machine configuration records the intended Backblaze B2
endpoint, bucket, prefix, credential names, publication set, and post-publish
workspace cleanup. Narratio's current contract has since replaced the legacy
@@ -53,6 +60,8 @@ Notarius, so S3 needs only to join the same path.
## Scope
- Advance the suite's Narratio application version from `v2.1.0` to the
backward-compatible `v2.1.1` bugfix release.
- Add suite-owned pipeline variables for the selected Narratio workspace,
S3, cache, and publish settings.
- Select the S3 backend with these D&D deployment defaults:
@@ -137,6 +146,9 @@ Notarius, so S3 needs only to join the same path.
- Adding notification providers. Narratio v2.1 supports only its existing
`noop` notification mode.
- Releasing new role or collection versions as part of this implementation.
- Modifying Narratio, ConfigKit, or the standalone Narratio role. Narratio
v2.1.1 is an already published prerequisite, and the existing generic role
can install the selected application tag through `narratio_version`.
## Target End State
@@ -144,6 +156,7 @@ The standalone pipeline role exposes these suite variables with the selected
defaults:
```yaml
dnd_pipeline_narratio_version: v2.1.1
dnd_pipeline_narratio_workspace_cleanup_after_publish: true
dnd_pipeline_narratio_storage_backend: s3
dnd_pipeline_narratio_storage_s3_bucket: rakestraw-assets
@@ -223,14 +236,16 @@ remove, rename, or constrain this future composition path.
defaults, dependency mappings, consumer documentation, and representative
integration scenario.
- `ansible-role-narratio` remains the generic owner of installation and native
pipeline rendering. Its reviewed v2.1.0 interface already covers every
pipeline rendering. Its reviewed `v0.1.0` role interface is compatible with
Narratio v2.1.1 and already covers every
configuration field required by this feature and should remain unchanged.
- `ansible-collection-dnd` owns the faithful pipeline subtree import, the
unified consumer dependency manifest, and artifact-boundary validation.
- `ansible-workspace-dnd` owns this roadmap, the later implementation plan,
status, and independent review record.
- `narratio` is a read-only source of storage, publication, credential,
caching, cleanup, restore, and CLI contracts.
- `narratio` v2.1.1 is the released compatibility prerequisite and a read-only
source of storage, publication, credential, caching, cleanup, restore, and
CLI contracts during this implementation.
- `rakestraw-ansible` remains unchanged until Feature 0016 consumes this
interface on `pipeline-dnd`.
@@ -254,6 +269,10 @@ remove, rename, or constrain this future composition path.
- Publish destinations remain omitted where Narratio can derive them. The
configured source identity and artifact output path remain the single source
of truth.
- Narratio v2.1.1 is the minimum application version for this feature because
it preserves destination derivation when `required` is explicit. Do not work
around older behavior by duplicating destinations or omitting deliberate
requiredness.
- Workspace cleanup is gated by Narratio's own successful-publication
evidence. Ansible does not independently inspect remote state or remove
runtime content.
@@ -273,11 +292,12 @@ remove, rename, or constrain this future composition path.
## Simplest Viable Implementation
Add the selected variables to the pipeline defaults and argument specification,
then map them into the existing `rakestraw.narratio` dependency. Update the
pipeline README to document the policy, credential names, complete-list
replacement semantics, and native capabilities now available. Do not add a
template, task, handler, fact, probe, or storage-specific validation layer.
Advance `dnd_pipeline_narratio_version` to v2.1.1. Add the selected variables to
the pipeline defaults and argument specification, then map them into the
existing `rakestraw.narratio` dependency. Update the pipeline README to document
the policy, credential names, complete-list replacement semantics, and native
capabilities now available. Do not add a template, task, handler, fact, probe,
or storage-specific validation layer.
In the existing pipeline integration driver, extend the synthetic secrets map
with object-storage values and explicitly supply representative S3 and publish
@@ -313,6 +333,12 @@ visible locally, but it would duplicate paths already owned by Narratio's
built-in sources and the configured artifact definitions. Native destination
derivation keeps path coordination automatic.
Omitting explicit `required: true` values would avoid the v2.1.0 failure only
by relying on an unrelated defaulting path. Changing the generic role's YAML
format would not repair the underlying sequence-resolution mismatch. Narratio
v2.1.1 fixes the owning application boundary, allowing the suite to state its
requiredness policy directly without duplicating destinations.
Deleting spool audio after publication would minimize transient disk usage,
but no such retention policy has been selected, and local-audio sessions may
depend on that content. Leaving the field omitted preserves Narratio's default
@@ -330,6 +356,8 @@ operational workflow.
- The pipeline role declares and documents every suite variable shown in the
target state, with matching argument-spec entries and direct Narratio
dependency mappings.
- The pipeline installs Narratio v2.1.1, whose released configuration loader
supports explicit requiredness together with derived publish destinations.
- The two credential-reference variables accept paired `null` overrides so a
caller can deliberately select Narratio's ambient AWS SDK provider chain.
- The default effective Narratio configuration selects the specified
@@ -346,8 +374,9 @@ operational workflow.
credential value.
- No new production task, template, handler, fact, state file, wrapper, or
storage-validation command is introduced.
- The standalone Narratio role remains unchanged because its v2.1.0 interface
already renders the required native fields and preserves null omission.
- The standalone Narratio role remains unchanged because its `v0.1.0` role
interface can install Narratio v2.1.1, renders the required native fields,
and preserves null omission.
- The existing pipeline integration scenario explicitly supplies
representative S3 and publish inputs, resolves synthetic values for all four
typed credentials, passes Narratio's native configuration validation, and
@@ -363,7 +392,9 @@ operational workflow.
validation, and clean-repeat suite passes without a duplicate S3 behavior
test.
- No real object-storage request, paid model request, application repository
change, asset change, release tag, or published artifact is required.
change during the remaining implementation, asset change, new release tag,
or published artifact is required. The existing Narratio v2.1.1 release is a
prerequisite rather than an implementation deliverable.
- Pipeline documentation links operators to Narratio's canonical procedures
for S3 session audio, publish, restore, locks, status, and cleanup and no
longer describes storage and publishing as deferred.
@@ -373,9 +404,10 @@ operational workflow.
## Dependencies and Follow-up Work
Features 0009, 0011, 0012, 0013, and 0014 are prerequisites. In particular,
Feature 0011 supplies the credential directory and Feature 0014 supplies the
human-operator access needed for the eventual live deployment.
Features 0009, 0011, 0012, 0013, and 0014 and the public Narratio v2.1.1 bugfix
release are prerequisites. In particular, Feature 0011 supplies the credential
directory and Feature 0014 supplies the human-operator access needed for the
eventual live deployment.
[Feature 0016](0016-deploy-dnd-pipeline-to-pipeline-dnd.md) follows this work.
It will put the four actual values in encrypted host inventory, pin the exact

103
docs/roadmap/implementation.md
View File

@@ -14,10 +14,12 @@ Active plan location: `docs/roadmap/implementation.md`
Configure the authoritative D&D pipeline role to use the selected Backblaze
S3-compatible archive, typed object-storage credential references, maximum S3
audio caching, immutable run publication, selected transcript and D&D artifact
outputs, and cleanup of successfully published workspace state. Validate the
configuration through Narratio's native interface, import the exact reviewed
pipeline commit into the collection from public Gitea, and leave the result
ready for the real-host deployment in Feature 0016.
outputs, and cleanup of successfully published workspace state. Advance the
pipeline to Narratio v2.1.1 so explicit publish requiredness composes with
native destination derivation. Validate the configuration through Narratio's
native interface, import the exact reviewed pipeline commit into the collection
from public Gitea, and leave the result ready for the real-host deployment in
Feature 0016.
The Feature 0015 roadmap remains authoritative for scope, rationale, target
state, and acceptance criteria. This plan does not authorize a second storage
@@ -69,16 +71,22 @@ lane, or changes to the generic Narratio role.
- Features 0009, 0011, 0012, 0013, and 0014 are complete.
- The approved Feature 0015 roadmap and this implementation plan are committed
in the workspace repository before the sprint begins.
- `ansible-role-dnd-pipeline` starts clean on `main` at
`c13f726a0a846180a1aa95d110b8bbc801edaae5`, equal to `origin/main`.
- `ansible-role-dnd-pipeline` remains based on
`c13f726a0a846180a1aa95d110b8bbc801edaae5`, equal to `origin/main`, with the
interrupted Stage 1 edits preserved and uncommitted in `README.md`,
`defaults/main.yml`, `meta/argument_specs.yml`, `meta/main.yml`, and
`tests/integration`. Review and complete those edits; do not discard them.
- `ansible-collection-dnd` starts clean on `main` at
`c0b0ebd13e92199f3d94e138c6cc50d1ece5dadd`, equal to `origin/main`.
- `ansible-role-narratio` is clean at
`292d8669e7aad5cfe80b0c469d22b6177e8cf0f1`, equal to `origin/main`, and its
`v0.1.0` role release contains the Narratio v2.1.0 field-specific interface
required by this feature.
`v0.1.0` role release contains the field-specific interface required by this
feature and can install Narratio v2.1.1 through `narratio_version`.
- The sibling Narratio application repository is clean at public release
`v2.1.0`. It is a read-only contract source during this sprint.
`v2.1.1`, commit `442469b9f57e65ce17f09059609dd1a37cf8c3b6`. That
backward-compatible release corrects destination derivation with explicit
publish requiredness and is a read-only contract source during the resumed
sprint.
- The workspace `.venv` supplies the tested `ansible-playbook`,
`ansible-galaxy`, and `ansible-lint` commands used by the repository test
drivers.
@@ -99,7 +107,7 @@ lane, or changes to the generic Narratio role.
collection snapshot.
- `ansible-workspace-dnd` records the implementation handoff after the two
implementation repositories are complete.
- `ansible-role-narratio`, all other standalone roles, application sources,
- `ansible-role-narratio`, all standalone roles, application sources,
asset repositories, and `rakestraw-ansible` may be inspected but must remain
unchanged.
@@ -131,13 +139,17 @@ Before beginning this stage, read:
### Preconditions
- The workspace planning commit containing this plan is present.
- `ansible-role-dnd-pipeline` is clean at the starting revision, is not behind
or diverged from `origin/main`, and contains no unrelated changes.
- The workspace planning commit containing the original approved plan and this
resumed-plan revision are present.
- `ansible-role-dnd-pipeline` is based on the starting revision, is not behind
or diverged from `origin/main`, and its only working-tree changes are the
preserved interrupted Stage 1 edits identified in Starting State and
Preconditions.
- The current `rakestraw.narratio` dependency mapping and existing integration
driver still match the structures reviewed while preparing this plan.
- Narratio v2.1.0 still accepts the roadmap-selected storage, publish, cleanup,
cache, and typed credential-reference fields.
- Narratio v2.1.1 remains publicly available and accepts the roadmap-selected
storage, publish, cleanup, cache, and typed credential-reference fields,
including rules with explicit `required` and omitted derivable `dest`.
### Owning Repository
@@ -145,7 +157,11 @@ Before beginning this stage, read:
### Required Changes
1. Add the following exact suite defaults to `defaults/main.yml` near the
1. Advance the existing application default to
`dnd_pipeline_narratio_version: v2.1.1`. Do not change the standalone
Narratio role release pin: its existing public interface can install this
compatible application patch release.
2. Add the following exact suite defaults to `defaults/main.yml` near the
existing Narratio policy variables:
```yaml
@@ -181,7 +197,7 @@ Before beginning this stage, read:
Keep every `dest` omitted. Narratio derives transcript destinations from
built-in sources and artifact destinations from the already managed
artifact output paths.
2. Add matching public entries to `meta/argument_specs.yml`. Use `type: bool`
3. Add matching public entries to `meta/argument_specs.yml`. Use `type: bool`
for the cleanup, path-style, cache, enabled, and upload settings; `type: str`
for the backend and S3 string fields; and `type: list` for the complete
publish output list. The S3 child fields, including both credential
@@ -190,55 +206,56 @@ Before beginning this stage, read:
pair to `null` when selecting the AWS SDK provider chain. The remaining
concrete suite-policy fields follow the role's existing required/defaulted
argument-spec convention.
3. In the `rakestraw.narratio` dependency entry in `meta/main.yml`, map every
4. In the `rakestraw.narratio` dependency entry in `meta/main.yml`, map every
new suite variable directly to its same-purpose standalone variable:
`narratio_workspace_cleanup_after_publish`, the eight
`narratio_storage_*` fields, `narratio_cache_s3_audio`,
`narratio_publish_enabled`, `narratio_publish_upload_run`, and
`narratio_publish_outputs`.
4. Do not map `narratio_spool_delete_audio_after_publish`. Do not map
5. Do not map `narratio_spool_delete_audio_after_publish`. Do not map
`narratio_notarius_chunk_cache`. Do not add a new import, fragment,
template, task, handler, assertion, or configuration-validation command.
5. Revise the README's variable reference and processing-policy sections to
6. Revise the README's application-version reference, upstream links, variable
reference, and processing-policy sections for Narratio v2.1.1 and
document the selected S3 destination, paired nullable credential
references, complete-list publish semantics, native destination derivation,
maximum S3 audio caching, successful-publication workspace cleanup, and
deliberate omission of spool deletion.
6. Extend the README's managed-credential example to show
7. Extend the README's managed-credential example to show
`OBJECT_STORAGE_KEY_ID` and `OBJECT_STORAGE_KEY` mapped from encrypted
inventory beside the existing Audita and OpenRouter entries. Explain that
values remain outside ordinary configuration and that setting both S3
credential references to `null` selects Narratio's ambient AWS SDK provider
chain.
7. Replace the README's statement that storage and publishing are deferred
8. Replace the README's statement that storage and publishing are deferred
with concise operator-facing descriptions and public upstream links for S3
session audio, publish, restore, publication locks, status, and cleanup.
Explain that accepted Notarius lanes remain
`narratio.extraction.<output_key>` analyze inputs independently of whether
they have stable publish rules.
8. Adapt the existing `tests/integration` baseline rather than adding a new
9. Adapt the existing `tests/integration` baseline rather than adding a new
scenario. Add two synthetic object-storage values to
`dnd_pipeline_secrets`; explicitly unset `OBJECT_STORAGE_KEY_ID` and
`OBJECT_STORAGE_KEY` alongside the existing provider variables when
invoking Ansible; and explicitly pass representative non-default S3,
cleanup, cache, and publish values through the new public variables.
9. Use representative values that exercise the interface without freezing the
10. Use representative values that exercise the interface without freezing the
defaults: for example an `.invalid` HTTPS endpoint and fixture bucket,
prefix, and region; intentional `false` values for cleanup, path style, and
S3 audio caching; the two configured credential reference names; enabled
run publication; and a short complete publish list containing one built-in
transcript source and `narratio.artifact.session_recap` with omitted
destinations.
10. Extend the existing secret-safe effective-configuration inspection only
11. Extend the existing secret-safe effective-configuration inspection only
enough to demonstrate the explicitly supplied representative storage
fields, typed credential names, falsey cache/cleanup choices, enabled
publication, and Narratio-derived destinations. Continue to use
`narratio config validate` as the native validity boundary. Do not inspect
role task results, call S3, or reproduce Narratio's validation rules.
11. Extend the current non-disclosure check to cover both synthetic
12. Extend the current non-disclosure check to cover both synthetic
object-storage values. Preserve the complete existing integration matrix,
cleanup behavior, and immediate clean-repeat assertions.
12. Run the complete repository validation. Commit the valid result with the
13. Run the complete repository validation. Commit the valid result with the
stated plain-English message, push `main` to public Gitea, and record the
full pushed commit hash for Stage 3.
@@ -252,15 +269,17 @@ Run from the workspace root:
Expected result: syntax checking and production-profile lint pass; all five
applications remain valid; the explicitly supplied S3 and publish values
resolve through Narratio's native configuration interface; only reference
names appear in effective output; no synthetic credential value appears in
captured output; no external S3 or model request occurs; and the immediate
identical repeat reports `changed=0` and `failed=0`.
resolve through Narratio v2.1.1's native configuration interface; explicit
publish requiredness and omitted destinations produce the derived destinations;
only reference names appear in effective output; no synthetic credential value
appears in captured output; no external S3 or model request occurs; and the
immediate identical repeat reports `changed=0` and `failed=0`.
### Completion Criteria
- Defaults, argument specifications, dependency mappings, and documentation
implement exactly the Feature 0015 contract.
- The pipeline selects Narratio v2.1.1, and defaults, argument specifications,
dependency mappings, and documentation implement exactly the Feature 0015
contract.
- No production task, template, fragment, validation command, or second secret
path was added.
- The representative integration scenario exercises public inputs rather than
@@ -400,7 +419,8 @@ Before beginning this stage, read:
dependency mappings, README, and detailed integration fixture.
3. Do not modify `galaxy.yml`, `requirements.yml`, release documentation, or
the published `v0.1.0` artifacts. Feature 0015 changes no standalone role or
native collection dependency and publishes no release.
native collection dependency and publishes no new release; the pipeline's
application-version variable carries the Narratio v2.1.1 selection.
4. Run the complete collection integration suite against the built and
installed artifact. Do not add collection-level assertions for the detailed
S3 configuration; native validation and clean convergence remain the
@@ -520,8 +540,9 @@ Commit intent: `Record D&D storage and publishing implementation`
unexpected branch divergence or untracked artifact.
- Report the full pipeline commit, collection fixture commit, collection
subtree merge and squash commits, and workspace handoff commit.
- Confirm that `ansible-role-narratio`, all other standalone roles,
applications, assets, and `rakestraw-ansible` remained unchanged.
- Confirm that `ansible-role-narratio`, all standalone roles, applications,
assets, and `rakestraw-ansible` remained unchanged during the resumed sprint;
Narratio v2.1.1 was an already released prerequisite.
- Summarize any deviation from this plan, unresolved finding, or complexity
added beyond direct variable mapping and identify the acceptance criterion
that required it.
@@ -530,4 +551,12 @@ Commit intent: `Record D&D storage and publishing implementation`
## Progress Notes
- Planning complete; implementation has not begun.
- The original plan was approved and Stage 1 began against Narratio v2.1.0.
- Native validation exposed Narratio's publish-output atomic-sequence
bookkeeping defect; the implementation agent correctly stopped with its
partial pipeline edits uncommitted.
- Narratio v2.1.1 at `442469b9f57e65ce17f09059609dd1a37cf8c3b6`
corrected the defect and added focused regression coverage without changing
the public configuration contract.
- Resume Stage 1 from the preserved pipeline edits, advance the application pin
to v2.1.1, rerun the complete validation, and then continue in stage order.

4
docs/status.md
View File

@@ -4,7 +4,7 @@ Last updated: 2026-09-21
Current milestone: Operational D&D pipeline deployment
Current focus: Review the Feature 0015 staged implementation plan.
Current focus: Resume Feature 0015 implementation against Narratio v2.1.1.
| ID | Deliverable | Owner | State | Evidence or next step |
| --- | --- | --- | --- | --- |
@@ -31,7 +31,7 @@ Current focus: Review the Feature 0015 staged implementation plan.
| F0012 | Assemble the D&D collection | ansible-collection-dnd and ansible-role-dnd-pipeline | done | Pipeline `600eb21` assembled and validated through collection remediation commit `da67b34` |
| F0013 | Publish the first coordinated suite release | all Ansible repositories and workspace | done | Public [`rakestraw.dnd` 0.1.0 release](https://gitea.maximumdirect.net/eric/ansible-collection-dnd/releases/tag/v0.1.0); documentation remediation completed in pipeline `78afffe` and collection `5e695f3` |
| F0014 | Enroll existing pipeline operators | ansible-role-dnd-pipeline, ansible-collection-dnd, and workspace | done | Pipeline `c13f726` and collection import `c0b0ebd` passed their complete integration suites and independent review with no remediation findings |
| F0015 | Enable D&D S3 storage and publishing | ansible-role-dnd-pipeline, ansible-collection-dnd, and workspace | ready | [Roadmap](roadmap/features/0015-dnd-s3-storage-and-publishing.md) approved; prepare the staged implementation plan |
| F0015 | Enable D&D S3 storage and publishing | ansible-role-dnd-pipeline, ansible-collection-dnd, and workspace | ready | [Roadmap](roadmap/features/0015-dnd-s3-storage-and-publishing.md) and staged plan updated for released [Narratio v2.1.1](https://gitea.maximumdirect.net/eric/narratio/releases/tag/v2.1.1); resume Stage 1 from preserved pipeline edits |
| F0016 | Deploy the D&D pipeline to `pipeline-dnd` | rakestraw-ansible and workspace | proposed | [Roadmap](roadmap/features/0016-deploy-dnd-pipeline-to-pipeline-dnd.md) drafted; Feature 0015 is a prerequisite |
| A0001 | Audit completed standalone-role test suites | workspace and standalone roles | done | Findings recorded in `docs/roadmap/test-suite-audit-findings.md` |
| R0001 | Simplify completed standalone-role test suites | workspace and standalone roles | done | Role commits `e08c63c`, `d2d0b9d`, and `cfecf37` passed implementation and independent review |