Plan D&D S3 storage and publishing
This commit is contained in:
@@ -1,6 +1,6 @@
|
||||
# Feature 0014: Enroll Existing Pipeline Operators
|
||||
|
||||
Status: in_progress
|
||||
Status: done
|
||||
|
||||
## Summary
|
||||
|
||||
|
||||
392
docs/roadmap/features/0015-dnd-s3-storage-and-publishing.md
Normal file
392
docs/roadmap/features/0015-dnd-s3-storage-and-publishing.md
Normal file
@@ -0,0 +1,392 @@
|
||||
# Feature 0015: Enable D&D S3 Storage and Publishing
|
||||
|
||||
Status: ready
|
||||
|
||||
## Summary
|
||||
|
||||
Make the D&D pipeline operational with Narratio's current S3 storage,
|
||||
publication, caching, cleanup, and restore capabilities. The pipeline role
|
||||
will select the established Backblaze B2 destination, reference two typed
|
||||
object-storage credentials delivered through its existing secrets directory,
|
||||
publish the useful transcript and D&D artifact outputs, and retain maximum S3
|
||||
audio caching.
|
||||
|
||||
The standalone Narratio role already exposes Narratio v2.1.0's complete
|
||||
configuration surface for this behavior. This feature supplies D&D suite
|
||||
policy through that existing interface; it does not add another storage
|
||||
implementation or credential mechanism.
|
||||
|
||||
## Problem or Opportunity
|
||||
|
||||
The pipeline role currently installs Narratio v2.1.0 and configures the full
|
||||
processing graph, but deliberately leaves Narratio storage and publishing at
|
||||
their application-level defaults. Its README consequently describes storage,
|
||||
publishing, and end-to-end external execution as deferred. The resulting suite
|
||||
can validate configuration and process local state, but it does not select the
|
||||
remote archive or publish the outputs needed for an operational deployment.
|
||||
|
||||
The prior test-machine configuration records the intended Backblaze B2
|
||||
endpoint, bucket, prefix, credential names, publication set, and post-publish
|
||||
workspace cleanup. Narratio's current contract has since replaced the legacy
|
||||
credential field names and secrets configuration with ConfigKit typed
|
||||
references and a conventional sibling `secrets/` directory. The Ansible suite
|
||||
already implements that new credential-delivery boundary for Audita and
|
||||
Notarius, so S3 needs only to join the same path.
|
||||
|
||||
## Governing Policies
|
||||
|
||||
- [The Ansible house style](../../ansible.md) requires suite policy to flow
|
||||
through the standalone role's field-specific interface and optional fields
|
||||
to remain application-owned unless the suite deliberately selects them.
|
||||
- [The testing house style](../../testing.md) requires representative
|
||||
public-interface and native-application validation without duplicating AWS
|
||||
SDK, ConfigKit, or Ansible module behavior.
|
||||
- [The MVP roadmap](../mvp.md) makes the pipeline role responsible for
|
||||
cross-application configuration and end-to-end validation.
|
||||
- [Feature 0011](0011-managed-pipeline-credential-delivery.md) establishes the
|
||||
managed Narratio secrets directory and typed credential-forwarding policy.
|
||||
- [Feature 0012](0012-collection-assembly.md) makes the standalone pipeline
|
||||
repository authoritative and imports reviewed changes through public
|
||||
Gitea.
|
||||
- [Feature 0016](0016-deploy-dnd-pipeline-to-pipeline-dnd.md) consumes this
|
||||
feature's reviewed interface for the first real-host deployment.
|
||||
|
||||
## Scope
|
||||
|
||||
- Add suite-owned pipeline variables for the selected Narratio workspace,
|
||||
S3, cache, and publish settings.
|
||||
- Select the S3 backend with these D&D deployment defaults:
|
||||
|
||||
| Setting | Suite default |
|
||||
| --- | --- |
|
||||
| Endpoint | `https://s3.us-east-005.backblazeb2.com` |
|
||||
| Bucket | `rakestraw-assets` |
|
||||
| Root prefix | `dnd` |
|
||||
| Region | `us-east-1` |
|
||||
| Force path style | `true` |
|
||||
| Access-key reference | `OBJECT_STORAGE_KEY_ID` |
|
||||
| Secret-key reference | `OBJECT_STORAGE_KEY` |
|
||||
|
||||
- Pass the two credential reference names to the standalone Narratio role.
|
||||
Their values use the existing `dnd_pipeline_secrets` map and managed
|
||||
`/usr/local/etc/narratio/secrets/` directory.
|
||||
- Enable S3 audio caching explicitly so the suite retains Narratio's maximum
|
||||
available audio reuse.
|
||||
- Enable run publication and publish these complete native output rules:
|
||||
|
||||
| Source | Required |
|
||||
| --- | --- |
|
||||
| `narratio.transcript.final` | `true` |
|
||||
| `narratio.transcript.final_trimmed` | `true` |
|
||||
| `narratio.transcript.final_markdown` | `true` |
|
||||
| `narratio.transcript.final_trimmed_markdown` | `true` |
|
||||
| `narratio.artifact.session_recap` | `true` |
|
||||
| `narratio.artifact.hostile_rumors` | `true` |
|
||||
| `narratio.artifact.deranged_rumors` | `false` |
|
||||
|
||||
- Let Narratio derive each destination from the built-in transcript path or
|
||||
configured artifact output path rather than duplicating those paths in the
|
||||
publish list.
|
||||
- Enable workspace cleanup only after Narratio records successful run
|
||||
publication. Leave spool-audio deletion under Narratio's existing default
|
||||
because the suite has selected no policy requiring source-audio removal.
|
||||
- Keep the production and `testing` profile storage and publishing behavior
|
||||
identical. The existing overlay remains limited to model-profile selection.
|
||||
- Document the four credentials normally required by the operational suite:
|
||||
`AUDITA_LLM_API_KEY`, `OPENROUTER_API_KEY`,
|
||||
`OBJECT_STORAGE_KEY_ID`, and `OBJECT_STORAGE_KEY`.
|
||||
- Document the operator-visible publication, restore, S3-session-audio, lock,
|
||||
caching, and cleanup capabilities enabled by the selected configuration,
|
||||
while linking Narratio's canonical CLI and operations documentation for
|
||||
command details.
|
||||
- Preserve every accepted Notarius lane as its existing
|
||||
`narratio.extraction.<output_key>` source. Future Narratio artifact
|
||||
definitions can consume those sources during analyze without first making
|
||||
the extraction lanes separate published outputs.
|
||||
- Adapt the existing pipeline integration scenario to resolve synthetic S3
|
||||
credentials and validate one explicitly supplied representative storage and
|
||||
publishing configuration through Narratio's native interface without
|
||||
contacting object storage.
|
||||
- Import the reviewed public pipeline commit into the collection through the
|
||||
established squashed subtree workflow, adapt the collection's artifact-level
|
||||
fixture to supply the added credential names, and validate the assembled
|
||||
artifact.
|
||||
|
||||
## Non-goals
|
||||
|
||||
- Implementing an S3 client, publication engine, remote commit format,
|
||||
restore mechanism, publish-lock mechanism, or audio cache in Ansible.
|
||||
Narratio owns all of those behaviors.
|
||||
- Placing credential values in role defaults, templates, documentation,
|
||||
tests, ordinary inventory, or the generated pipeline configuration.
|
||||
- Adding a second secret-delivery mechanism, environment wrapper, service
|
||||
unit, credential synchronizer, or AWS configuration-file manager.
|
||||
- Managing Backblaze buckets, object lifecycle policy, application keys,
|
||||
retention policy, versioning, encryption, quotas, or network access.
|
||||
- Managing per-session `session.yml` files or selecting local versus S3 audio
|
||||
input for an individual session.
|
||||
- Running a paid or networked publication test from a repository-local test
|
||||
suite. Feature 0016 owns the representative live end-to-end exercise.
|
||||
- Publishing every Notarius extraction lane as a named current output. The
|
||||
lanes remain available as analyze inputs independently of publication; stable
|
||||
remote destinations require an explicit operator need.
|
||||
- Setting Narratio's optional `notarius.chunk_cache` override. The managed
|
||||
Notarius configuration remains the single owner of its ordinary `auto`
|
||||
cache policy, and Narratio v2.1 already requests refresh for forced
|
||||
extraction.
|
||||
- Adding notification providers. Narratio v2.1 supports only its existing
|
||||
`noop` notification mode.
|
||||
- Releasing new role or collection versions as part of this implementation.
|
||||
|
||||
## Target End State
|
||||
|
||||
The standalone pipeline role exposes these suite variables with the selected
|
||||
defaults:
|
||||
|
||||
```yaml
|
||||
dnd_pipeline_narratio_workspace_cleanup_after_publish: true
|
||||
dnd_pipeline_narratio_storage_backend: s3
|
||||
dnd_pipeline_narratio_storage_s3_bucket: rakestraw-assets
|
||||
dnd_pipeline_narratio_storage_s3_root_prefix: dnd
|
||||
dnd_pipeline_narratio_storage_s3_region: us-east-1
|
||||
dnd_pipeline_narratio_storage_s3_endpoint: >-
|
||||
https://s3.us-east-005.backblazeb2.com
|
||||
dnd_pipeline_narratio_storage_s3_force_path_style: true
|
||||
dnd_pipeline_narratio_storage_s3_access_key_id: OBJECT_STORAGE_KEY_ID
|
||||
dnd_pipeline_narratio_storage_s3_secret_access_key: OBJECT_STORAGE_KEY
|
||||
dnd_pipeline_narratio_cache_s3_audio: true
|
||||
dnd_pipeline_narratio_publish_enabled: true
|
||||
dnd_pipeline_narratio_publish_upload_run: true
|
||||
dnd_pipeline_narratio_publish_outputs:
|
||||
- source: narratio.transcript.final
|
||||
required: true
|
||||
- source: narratio.transcript.final_trimmed
|
||||
required: true
|
||||
- source: narratio.transcript.final_markdown
|
||||
required: true
|
||||
- source: narratio.transcript.final_trimmed_markdown
|
||||
required: true
|
||||
- source: narratio.artifact.session_recap
|
||||
required: true
|
||||
- source: narratio.artifact.hostile_rumors
|
||||
required: true
|
||||
- source: narratio.artifact.deranged_rumors
|
||||
required: false
|
||||
```
|
||||
|
||||
Each value is declared in the pipeline argument specification and passed
|
||||
directly to its corresponding `narratio_*` variable in the pipeline role's
|
||||
Narratio dependency. Narratio's existing root template renders the native
|
||||
configuration. No new pipeline fragment or production task is required.
|
||||
|
||||
The ordinary generated configuration contains only the credential names:
|
||||
|
||||
```yaml
|
||||
storage:
|
||||
backend: s3
|
||||
s3:
|
||||
access_key_id: OBJECT_STORAGE_KEY_ID
|
||||
secret_access_key: OBJECT_STORAGE_KEY
|
||||
```
|
||||
|
||||
Consumers supply the two values beside the existing model-provider values
|
||||
through encrypted inventory:
|
||||
|
||||
```yaml
|
||||
dnd_pipeline_secrets:
|
||||
AUDITA_LLM_API_KEY: "{{ vault_dnd_audita_llm_api_key }}"
|
||||
OPENROUTER_API_KEY: "{{ vault_dnd_openrouter_api_key }}"
|
||||
OBJECT_STORAGE_KEY_ID: "{{ vault_dnd_object_storage_key_id }}"
|
||||
OBJECT_STORAGE_KEY: "{{ vault_dnd_object_storage_key }}"
|
||||
```
|
||||
|
||||
The example defines references only; no roadmap, role, or test contains real
|
||||
credential values. The existing credential task writes the selected entries
|
||||
with the suite's established group-authorized secret modes. Omitted map keys
|
||||
remain unmanaged, as defined by Feature 0011.
|
||||
|
||||
With valid credentials, Narratio can accept either local or S3 audio selected
|
||||
by each session file, cache S3 audio beneath `/var/cache/narratio`, publish an
|
||||
immutable run and the selected stable outputs, use remote publication locks,
|
||||
restore committed state, and clean the published workspace. These are native
|
||||
Narratio commands and lifecycle rules rather than additional Ansible tasks.
|
||||
|
||||
The ten accepted Notarius outputs continue to enter Narratio's artifact
|
||||
catalog as `narratio.extraction.*` sources. Analyze-stage Scriptorium artifacts
|
||||
may reference those sources directly in their input mappings. Feature 0015
|
||||
does not yet add such artifact definitions, but its publication choices do not
|
||||
remove, rename, or constrain this future composition path.
|
||||
|
||||
## Repository Ownership
|
||||
|
||||
- `ansible-role-dnd-pipeline` owns the D&D suite variables, opinionated
|
||||
defaults, dependency mappings, consumer documentation, and representative
|
||||
integration scenario.
|
||||
- `ansible-role-narratio` remains the generic owner of installation and native
|
||||
pipeline rendering. Its reviewed v2.1.0 interface already covers every
|
||||
configuration field required by this feature and should remain unchanged.
|
||||
- `ansible-collection-dnd` owns the faithful pipeline subtree import, the
|
||||
unified consumer dependency manifest, and artifact-boundary validation.
|
||||
- `ansible-workspace-dnd` owns this roadmap, the later implementation plan,
|
||||
status, and independent review record.
|
||||
- `narratio` is a read-only source of storage, publication, credential,
|
||||
caching, cleanup, restore, and CLI contracts.
|
||||
- `rakestraw-ansible` remains unchanged until Feature 0016 consumes this
|
||||
interface on `pipeline-dnd`.
|
||||
|
||||
## Constraints and Settled Decisions
|
||||
|
||||
- The S3 endpoint, bucket, prefix, region, and path-style values are deliberate
|
||||
D&D suite defaults. They are ordinary variables and remain overrideable by
|
||||
inventory.
|
||||
- Credential references are also policy defaults, but credential values have
|
||||
no defaults. Operators must supply matching entries through
|
||||
`dnd_pipeline_secrets` or another environment source supported by Narratio.
|
||||
- The two credential-reference variables are nullable as a pair. Setting both
|
||||
to `null` omits the references and preserves Narratio's AWS SDK provider-chain
|
||||
path; configuring only one remains invalid under Narratio's native contract.
|
||||
- The role does not assert that the secrets map contains particular keys.
|
||||
Narratio's native loader owns typed-reference resolution, and external
|
||||
environment values remain a supported higher-precedence source.
|
||||
- `dnd_pipeline_narratio_publish_outputs` is one complete native list. A caller
|
||||
replacing it assumes responsibility for the entire selected publication
|
||||
set; the role does not merge rules.
|
||||
- Publish destinations remain omitted where Narratio can derive them. The
|
||||
configured source identity and artifact output path remain the single source
|
||||
of truth.
|
||||
- Workspace cleanup is gated by Narratio's own successful-publication
|
||||
evidence. Ansible does not independently inspect remote state or remove
|
||||
runtime content.
|
||||
- S3 audio caching is enabled, and ordinary cleanup preserves the cache.
|
||||
Explicit operator cache clearing remains a Narratio CLI operation.
|
||||
- The testing profile does not silently disable Notarius, storage,
|
||||
publication, cleanup, or credentials. It continues to differ only in the
|
||||
already selected inexpensive Scriptorium model profile.
|
||||
- S3 storage also enables Narratio's native restore and publication-lock
|
||||
commands without another configuration surface. Ansible documentation may
|
||||
describe availability but must not duplicate Narratio's detailed operating
|
||||
manual.
|
||||
- Notarius extraction-source availability and remote publication are separate
|
||||
concerns. Analyze consumes `narratio.extraction.*` sources from the accepted
|
||||
extraction catalog; `publish.outputs` selects only which sources also receive
|
||||
stable published destinations.
|
||||
|
||||
## Simplest Viable Implementation
|
||||
|
||||
Add the selected variables to the pipeline defaults and argument specification,
|
||||
then map them into the existing `rakestraw.narratio` dependency. Update the
|
||||
pipeline README to document the policy, credential names, complete-list
|
||||
replacement semantics, and native capabilities now available. Do not add a
|
||||
template, task, handler, fact, probe, or storage-specific validation layer.
|
||||
|
||||
In the existing pipeline integration driver, extend the synthetic secrets map
|
||||
with object-storage values and explicitly supply representative S3 and publish
|
||||
variables. Use `narratio config validate` and its secret-safe effective
|
||||
configuration output to prove that the public inputs compose into a valid
|
||||
configuration. Retain the immediate clean repeat and secret non-disclosure
|
||||
checks. Do not call a real S3 service or assert that the representative values
|
||||
equal current role defaults.
|
||||
|
||||
Push the reviewed pipeline commit, import that exact public revision into the
|
||||
collection through `git subtree pull --squash`, and extend only the
|
||||
collection-level fixture inputs needed for native validation. The collection
|
||||
suite should continue to prove artifact installation and one representative
|
||||
assembled convergence rather than duplicate the detailed pipeline matrix.
|
||||
|
||||
## Rationale and Alternatives
|
||||
|
||||
Implementing storage behavior in Ansible would duplicate a mature native
|
||||
application boundary. Narratio already owns credential resolution, AWS SDK
|
||||
provider construction, immutable commits, verification, locks, restore,
|
||||
caching, and cleanup. Passing declarative configuration through the generic
|
||||
role is both smaller and more reliable.
|
||||
|
||||
Using the AWS SDK's ambient provider chain would avoid named secret files, but
|
||||
the initial deployment has explicit Backblaze application credentials and the
|
||||
suite already has a suitable managed typed-secret path. Keeping the two
|
||||
credential-reference variables nullable preserves the ambient-provider
|
||||
alternative: consumers can set both suite variables to `null` when their
|
||||
deployment uses another provider.
|
||||
|
||||
Copying destinations into every publish rule would make the output layout more
|
||||
visible locally, but it would duplicate paths already owned by Narratio's
|
||||
built-in sources and the configured artifact definitions. Native destination
|
||||
derivation keeps path coordination automatic.
|
||||
|
||||
Deleting spool audio after publication would minimize transient disk usage,
|
||||
but no such retention policy has been selected, and local-audio sessions may
|
||||
depend on that content. Leaving the field omitted preserves Narratio's default
|
||||
while enabling the explicitly requested S3 audio cache.
|
||||
|
||||
Publishing all ten Notarius extraction lanes would create stable remote output
|
||||
contracts that have not been requested. It is also unnecessary for the stated
|
||||
future use: analyze-stage artifacts can consume `narratio.extraction.*`
|
||||
directly. The immutable published run retains the processing result, while the
|
||||
selected transcript and human-facing artifact outputs match the known
|
||||
operational workflow.
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- The pipeline role declares and documents every suite variable shown in the
|
||||
target state, with matching argument-spec entries and direct Narratio
|
||||
dependency mappings.
|
||||
- The two credential-reference variables accept paired `null` overrides so a
|
||||
caller can deliberately select Narratio's ambient AWS SDK provider chain.
|
||||
- The default effective Narratio configuration selects the specified
|
||||
Backblaze-compatible S3 destination, typed credential names, S3 audio cache,
|
||||
successful-publication workspace cleanup, run upload, and seven output
|
||||
rules.
|
||||
- The publish list relies on Narratio's native destination derivation and
|
||||
retains `deranged_rumors` as optional while the other six outputs are
|
||||
required.
|
||||
- Spool-audio deletion remains unset by the pipeline role, and the testing
|
||||
profile introduces no storage or publication override.
|
||||
- `dnd_pipeline_secrets` remains a generic empty-by-default mapping; the role
|
||||
documents all four expected operational entries without embedding any
|
||||
credential value.
|
||||
- No new production task, template, handler, fact, state file, wrapper, or
|
||||
storage-validation command is introduced.
|
||||
- The standalone Narratio role remains unchanged because its v2.1.0 interface
|
||||
already renders the required native fields and preserves null omission.
|
||||
- The existing pipeline integration scenario explicitly supplies
|
||||
representative S3 and publish inputs, resolves synthetic values for all four
|
||||
typed credentials, passes Narratio's native configuration validation, and
|
||||
exposes no synthetic value in captured output.
|
||||
- The immediate identical second pipeline application reports `changed=0`
|
||||
and `failed=0`; syntax checking, production-profile lint, the complete
|
||||
standalone integration suite, and `git diff --check` pass.
|
||||
- The exact reviewed pipeline commit is pushed to public Gitea before the
|
||||
collection imports it through the documented remote-only squashed subtree
|
||||
workflow.
|
||||
- The collection fixture supplies the added synthetic credential names, and
|
||||
its existing build, isolated installation, FQCN resolution, native
|
||||
validation, and clean-repeat suite passes without a duplicate S3 behavior
|
||||
test.
|
||||
- No real object-storage request, paid model request, application repository
|
||||
change, asset change, release tag, or published artifact is required.
|
||||
- Pipeline documentation links operators to Narratio's canonical procedures
|
||||
for S3 session audio, publish, restore, locks, status, and cleanup and no
|
||||
longer describes storage and publishing as deferred.
|
||||
- Documentation distinguishes accepted Notarius extraction sources from
|
||||
published outputs and confirms that future analyze-stage artifacts may use
|
||||
`narratio.extraction.<output_key>` inputs without publishing those lanes.
|
||||
|
||||
## Dependencies and Follow-up Work
|
||||
|
||||
Features 0009, 0011, 0012, 0013, and 0014 are prerequisites. In particular,
|
||||
Feature 0011 supplies the credential directory and Feature 0014 supplies the
|
||||
human-operator access needed for the eventual live deployment.
|
||||
|
||||
[Feature 0016](0016-deploy-dnd-pipeline-to-pipeline-dnd.md) follows this work.
|
||||
It will put the four actual values in encrypted host inventory, pin the exact
|
||||
reviewed suite revision in `rakestraw-ansible`, converge `pipeline-dnd`, and
|
||||
exercise one real session through remote publication. Bucket lifecycle policy,
|
||||
backup policy, credential rotation, additional publish outputs, automated
|
||||
scheduling, and cleanup of the proof-of-concept deployment remain separate
|
||||
future work.
|
||||
|
||||
A later artifact-development feature may add D&D-specific Narratio artifact
|
||||
definitions whose Scriptorium input mappings consume selected
|
||||
`narratio.extraction.*` sources. That feature should define each artifact's
|
||||
purpose, inputs, prompt, output contract, and publication choice; it does not
|
||||
require a storage-interface change.
|
||||
@@ -0,0 +1,278 @@
|
||||
# Feature 0016: Deploy the D&D Pipeline to `pipeline-dnd`
|
||||
|
||||
Status: proposed
|
||||
|
||||
## Summary
|
||||
|
||||
Add a minimal playbook and host inventory to `rakestraw-ansible` that apply
|
||||
`rakestraw.dnd.pipeline` to the existing `pipeline-dnd` host. Configure the
|
||||
host through `host_vars`, enroll `eric` as the initial human operator, keep
|
||||
credential values in Ansible Vault, and prove the complete pipeline through a
|
||||
real repeatable deployment and representative D&D session.
|
||||
|
||||
[Feature 0015](0015-dnd-s3-storage-and-publishing.md) must first provide the
|
||||
storage and publishing behavior needed for an operational end-to-end pipeline.
|
||||
Feature 0016 consumes that reviewed interface; it does not invent a temporary
|
||||
storage contract.
|
||||
|
||||
## Problem or Opportunity
|
||||
|
||||
The collection and its application roles are implemented and installable, but
|
||||
the operational Ansible repository does not yet have a playbook for deploying
|
||||
the new collection architecture. Its existing D&D playbook targets a retired
|
||||
inventory name and invokes the proof-of-concept local `pipeline_dnd` role.
|
||||
|
||||
The current `pipeline-dnd` host inventory already records the connection
|
||||
identity and useful legacy Narratio storage values. A production-shaped first
|
||||
deployment now needs to express the new role's public interface, deliver
|
||||
secrets safely, reconcile the controller's dependency manifest, and validate
|
||||
the actual host rather than extending the role repositories with another
|
||||
synthetic scenario.
|
||||
|
||||
## Scope
|
||||
|
||||
- Add `rakestraw-ansible/playbooks/pipeline/dnd.yml`, targeting the existing
|
||||
`pipeline-dnd` inventory host and applying only
|
||||
`rakestraw.dnd.pipeline`.
|
||||
- Keep the playbook limited to its host selection and role list. Put all
|
||||
deployment choices in `inventory/host_vars/pipeline-dnd/`.
|
||||
- Configure the external WhisperX transcription endpoint as
|
||||
`https://transcription.ai.rakestrawhome.com/transcribe`.
|
||||
- Enroll the existing `eric` account through
|
||||
`dnd_pipeline_operator_users`.
|
||||
- Map the role's `dnd_pipeline_secrets` interface to host-specific Vault
|
||||
variables. Store credential values only in an Ansible Vault-encrypted host
|
||||
variables file.
|
||||
- Carry the storage and publishing settings selected by Feature 0015 through
|
||||
its final public pipeline interface. Preserve the operational intent of the
|
||||
existing Backblaze B2 endpoint, bucket, prefix, region, and path-style
|
||||
settings where they remain applicable.
|
||||
- Reconcile existing legacy Narratio host variables so they cannot silently
|
||||
override the suite policy selected by the new pipeline role.
|
||||
- Update `rakestraw-ansible/requirements.yml` so a clean controller can install
|
||||
the reviewed collection snapshot, all five standalone role dependencies,
|
||||
and required native collections from one manifest.
|
||||
- Validate controller dependency resolution, playbook syntax, initial
|
||||
convergence, immediate clean convergence, operator access, native Narratio
|
||||
configuration, and one representative end-to-end pipeline run on the real
|
||||
host.
|
||||
- Document the operational invocation, Vault prerequisite, initial operator,
|
||||
and expected validation path in `rakestraw-ansible`.
|
||||
|
||||
## Non-goals
|
||||
|
||||
- Implementing storage or publishing behavior. Feature 0015 owns that
|
||||
prerequisite.
|
||||
- Changing application or role behavior merely to accommodate this
|
||||
deployment.
|
||||
- Replacing the existing inventory alias with `pipeline.dnd`, changing DNS,
|
||||
or moving connection variables out of the current host inventory.
|
||||
- Adding a wrapper role, pre-task layer, service, timer, scheduler, web UI, or
|
||||
unattended session-ingestion mechanism.
|
||||
- Creating or managing the `eric` account, its login policy, SSH access, or
|
||||
sudo policy.
|
||||
- Adding another production host, high availability, backup policy, or
|
||||
disaster-recovery automation.
|
||||
- Removing the proof-of-concept local `pipeline_dnd` role or its historical
|
||||
playbook before the new deployment has demonstrated operational parity.
|
||||
- Publishing a new coordinated suite release solely as part of this
|
||||
deployment feature.
|
||||
|
||||
## Target End State
|
||||
|
||||
`rakestraw-ansible/playbooks/pipeline/dnd.yml` is an intentionally small
|
||||
consumer playbook equivalent to:
|
||||
|
||||
```yaml
|
||||
---
|
||||
- hosts:
|
||||
- pipeline-dnd
|
||||
roles:
|
||||
- rakestraw.dnd.pipeline
|
||||
...
|
||||
```
|
||||
|
||||
The existing inventory selects the host's connection identity. A dedicated
|
||||
host-variable file selects the pipeline's non-secret public inputs, including
|
||||
the WhisperX endpoint, the initial operator list, and the storage and
|
||||
publishing values exposed by Feature 0015. It maps runtime credential names to
|
||||
Vault-backed variables without embedding credential values in the playbook or
|
||||
ordinary plaintext inventory, for example:
|
||||
|
||||
```yaml
|
||||
dnd_pipeline_operator_users:
|
||||
- eric
|
||||
|
||||
dnd_pipeline_whisperx_transcribe_url: >-
|
||||
https://transcription.ai.rakestrawhome.com/transcribe
|
||||
|
||||
dnd_pipeline_secrets:
|
||||
AUDITA_LLM_API_KEY: "{{ vault_dnd_audita_llm_api_key }}"
|
||||
OPENROUTER_API_KEY: "{{ vault_dnd_openrouter_api_key }}"
|
||||
OBJECT_STORAGE_KEY_ID: "{{ vault_dnd_object_storage_key_id }}"
|
||||
OBJECT_STORAGE_KEY: "{{ vault_dnd_object_storage_key }}"
|
||||
```
|
||||
|
||||
The corresponding credential variables reside in a tracked file encrypted in
|
||||
full with Ansible Vault. The Vault password or identity mechanism remains
|
||||
outside Git and outside this feature's managed content.
|
||||
|
||||
The controller dependency manifest selects one reviewed, publicly reachable
|
||||
suite snapshot containing Features 0014 and 0015. It also installs that
|
||||
snapshot's exact standalone role and native collection dependencies. A clean
|
||||
controller therefore does not depend on sibling development checkouts or
|
||||
previously installed roles.
|
||||
|
||||
On `pipeline-dnd`, the role converges the full application suite and its D&D
|
||||
configuration. The `narratio` service identity can run the pipeline, and
|
||||
`eric` can run it after starting a login session that observes the granted
|
||||
supplementary group membership. A representative session can be transcribed,
|
||||
processed, stored, and published through the configured production path. An
|
||||
immediate identical Ansible run reports no changes.
|
||||
|
||||
## Repository Ownership
|
||||
|
||||
- `rakestraw-ansible` owns the consumer playbook, host inventory, encrypted
|
||||
host credentials, controller dependency manifest, and deployment runbook.
|
||||
- `ansible-role-dnd-pipeline` owns the suite's public variables and the
|
||||
opinionated cross-application behavior consumed by the deployment.
|
||||
- `ansible-collection-dnd` owns the assembled `rakestraw.dnd.pipeline` role
|
||||
snapshot and consumer-facing compatibility manifest.
|
||||
- The standalone application roles continue to own installation and native
|
||||
application configuration. Feature 0016 consumes their reviewed releases or
|
||||
commits without reimplementing their behavior.
|
||||
- `ansible-workspace-dnd` owns this roadmap, the later implementation plan,
|
||||
progress tracking, and independent review record.
|
||||
- Application and asset repositories are read-only sources of runtime
|
||||
contracts and deployed content for this feature.
|
||||
|
||||
## Constraints and Settled Decisions
|
||||
|
||||
- Feature 0015 is a prerequisite and must settle, implement, and review the
|
||||
storage and publishing interface before Feature 0016 becomes ready for an
|
||||
implementation sprint.
|
||||
- The target is the inventory alias `pipeline-dnd` as currently defined in
|
||||
`rakestraw-ansible`, not a new `pipeline.dnd` alias.
|
||||
- The playbook follows the repository's established style: hosts and roles
|
||||
only. Facts remain at Ansible's normal default, privilege escalation remains
|
||||
task-owned by the roles, and deployment variables belong in host inventory.
|
||||
- `eric` is the initial human operator and must already exist on the managed
|
||||
host. The pipeline role grants only additive runtime-group membership.
|
||||
- API credentials use the existing typed `dnd_pipeline_secrets` interface and
|
||||
Ansible Vault. Plaintext secret values must not enter Git, command lines,
|
||||
logs, examples, or ordinary variable files.
|
||||
- The public Gitea repositories are the sole dependency sources. The
|
||||
deployment must not reference this workspace's sibling checkout paths.
|
||||
- The dependency manifest must select an immutable release when Feature 0015
|
||||
produces one. Otherwise, it must pin the exact reviewed public Git commit;
|
||||
it must not track an unbounded branch for the production deployment.
|
||||
- The collection artifact does not itself install arbitrary Git-hosted role
|
||||
dependencies. The operational manifest must therefore include the five
|
||||
standalone roles as well as the collection and its native collection
|
||||
dependencies.
|
||||
- The legacy host variables must be compared with the final Feature 0015
|
||||
interface. Applicable values move to the new suite configuration, while
|
||||
obsolete or conflicting variables are removed rather than left to win by
|
||||
inventory precedence.
|
||||
- The initial real-host run is an operational acceptance exercise. It should
|
||||
test observable pipeline behavior and convergence, not module task traces or
|
||||
the current text of defaults.
|
||||
|
||||
## Simplest Viable Implementation
|
||||
|
||||
After Feature 0015 is reviewed, select its exact public collection revision
|
||||
and use the associated compatibility manifest to update the controller's
|
||||
single root requirements file. Add the minimal playbook and one ordinary
|
||||
host-variable file for public inputs. Add one fully encrypted host Vault file
|
||||
for the two credential values, populated by the operator without exposing its
|
||||
plaintext during implementation or review.
|
||||
|
||||
Install or update dependencies from the root manifest, confirm that Ansible
|
||||
resolves `rakestraw.dnd.pipeline`, syntax-check the playbook, and converge only
|
||||
`pipeline-dnd`. Start a fresh operator login after enrollment, validate the
|
||||
native Narratio configuration under the intended runtime identity, rerun the
|
||||
same play immediately to confirm clean convergence, and process one selected
|
||||
session through the complete Feature 0015 publishing path. Record concise
|
||||
operational instructions and results; do not add a second test harness around
|
||||
the real deployment.
|
||||
|
||||
## Rationale and Alternatives
|
||||
|
||||
Keeping the playbook nearly declarative makes inventory the durable record of
|
||||
what differs about this host and allows ordinary Ansible precedence to remain
|
||||
the customization mechanism. Embedding endpoint, operator, storage, or secret
|
||||
mapping variables in the playbook would make a one-host play harder to reuse
|
||||
and would diverge from the established `rakestraw-ansible` style.
|
||||
|
||||
A local wrapper role could hide the collection interface, but it would create
|
||||
another layer with no independent behavior. Applying the collection role by
|
||||
FQCN proves the same consumer path that future hosts should use.
|
||||
|
||||
Tracking collection `main` would be convenient during development, but it
|
||||
would make a later controller bootstrap select an unreviewed compatibility
|
||||
set. An exact public commit retains development flexibility until the next
|
||||
release while still making this deployment reproducible.
|
||||
|
||||
Running the real-host deployment before Feature 0015 would establish only an
|
||||
installation demonstration: the pipeline still could not complete its
|
||||
required storage and publishing path. Making Feature 0015 an explicit
|
||||
prerequisite avoids temporary inventory variables and gives the acceptance
|
||||
run a meaningful end-to-end boundary.
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- Feature 0015 is complete and independently reviewed before Feature 0016
|
||||
implementation begins.
|
||||
- `playbooks/pipeline/dnd.yml` targets only `pipeline-dnd`, applies only
|
||||
`rakestraw.dnd.pipeline`, and contains no deployment variables or procedural
|
||||
setup.
|
||||
- Host inventory sets the confirmed WhisperX endpoint, enrolls only `eric`,
|
||||
and supplies the final Feature 0015 storage and publishing inputs.
|
||||
- `dnd_pipeline_secrets` maps the required typed runtime keys to host Vault
|
||||
variables, and the actual values exist only in a valid Ansible
|
||||
Vault-encrypted file.
|
||||
- No plaintext credential value is present in any tracked revision, diff,
|
||||
command example, validation output, or deployment documentation.
|
||||
- Legacy Narratio storage values are either represented through the current
|
||||
pipeline interface or deliberately retired; no stale high-precedence host
|
||||
variable can override the selected suite configuration accidentally.
|
||||
- The root requirements manifest preserves unrelated existing dependencies
|
||||
and installs the exact reviewed public collection snapshot, its five
|
||||
standalone role dependencies, and its required native collections without
|
||||
using local repository paths.
|
||||
- On a controller bootstrapped from that manifest, `ansible-doc` resolves
|
||||
`rakestraw.dnd.pipeline` and each role dependency, and the new playbook
|
||||
passes syntax checking.
|
||||
- The initial limited playbook run against `pipeline-dnd` completes
|
||||
successfully using the encrypted inventory and ordinary sudo-capable remote
|
||||
account.
|
||||
- The immediate identical second run reports `changed=0` and `failed=0`.
|
||||
- Narratio's native configuration validation succeeds under the intended
|
||||
runtime identity, and a fresh session for `eric` can access and invoke the
|
||||
shared pipeline through runtime-group membership.
|
||||
- One operator-selected D&D session completes transcription, processing,
|
||||
storage, and publishing, and its expected published output is observable at
|
||||
the configured destination.
|
||||
- The operational documentation explains dependency installation, Vault use,
|
||||
deployment invocation, the login-session membership caveat, convergence
|
||||
verification, and representative pipeline invocation without disclosing
|
||||
credentials.
|
||||
- The historical proof-of-concept role and playbook remain available during
|
||||
the initial rollout; their later retirement requires a separate explicit
|
||||
decision.
|
||||
|
||||
## Dependencies and Follow-up Work
|
||||
|
||||
[Feature 0015](0015-dnd-s3-storage-and-publishing.md) is the immediate next unit
|
||||
of work and a hard prerequisite. It selects the pipeline interface,
|
||||
credentials, and observable publishing boundary that Feature 0016 will deploy.
|
||||
|
||||
Features 0013 and 0014 provide the public collection installation path and
|
||||
existing-operator enrollment required here. Features 0009 and 0011 provide
|
||||
the shared filesystem and managed credential-delivery policies.
|
||||
|
||||
After the real deployment demonstrates parity, a later cleanup feature may
|
||||
retire the proof-of-concept `pipeline_dnd` role, its old playbook, and any
|
||||
obsolete inventory values. Service scheduling, unattended ingestion,
|
||||
additional human operators, backup policy, and additional hosts also remain
|
||||
separate work.
|
||||
@@ -1,26 +1,29 @@
|
||||
# Implementation Plan 0014: Enroll Existing Pipeline Operators
|
||||
# Implementation Plan 0015: Enable D&D S3 Storage and Publishing
|
||||
|
||||
Status: ready
|
||||
|
||||
Target executor: GPT-5.6-Terra
|
||||
|
||||
Feature roadmap:
|
||||
`docs/roadmap/features/0014-existing-pipeline-operator-enrollment.md`
|
||||
`docs/roadmap/features/0015-dnd-s3-storage-and-publishing.md`
|
||||
|
||||
Active plan location: `docs/roadmap/implementation.md`
|
||||
|
||||
## Objective
|
||||
|
||||
Add one empty-by-default pipeline interface that enrolls explicitly selected
|
||||
existing human accounts in the suite's runtime group. Implement the behavior
|
||||
with one account-existence lookup and one additive membership task, adapt the
|
||||
existing operator integration scenario, and import the reviewed result into
|
||||
the collection from public Gitea.
|
||||
Configure the authoritative D&D pipeline role to use the selected Backblaze
|
||||
S3-compatible archive, typed object-storage credential references, maximum S3
|
||||
audio caching, immutable run publication, selected transcript and D&D artifact
|
||||
outputs, and cleanup of successfully published workspace state. Validate the
|
||||
configuration through Narratio's native interface, import the exact reviewed
|
||||
pipeline commit into the collection from public Gitea, and leave the result
|
||||
ready for the real-host deployment in Feature 0016.
|
||||
|
||||
The Feature 0014 roadmap remains authoritative for scope, rationale, target
|
||||
state, and acceptance criteria. This plan does not authorize human-account
|
||||
creation or removal, exact-list reconciliation, filesystem-policy changes, or
|
||||
a new release.
|
||||
The Feature 0015 roadmap remains authoritative for scope, rationale, target
|
||||
state, and acceptance criteria. This plan does not authorize a second storage
|
||||
implementation, real object-storage calls from repository tests, per-session
|
||||
configuration management, stable publication of every Notarius extraction
|
||||
lane, or changes to the generic Narratio role.
|
||||
|
||||
## Execution Rules
|
||||
|
||||
@@ -33,37 +36,49 @@ a new release.
|
||||
worktree changes or branch divergence rather than overwriting them.
|
||||
- Do not make unresolved product or architecture decisions during execution;
|
||||
stop and report the decision required.
|
||||
- Keep the production behavior to the roadmap's exact direct task flow: one
|
||||
`ansible.builtin.getent` loop followed by one `ansible.builtin.user` loop.
|
||||
- Do not add an assertion, registered fact, handler, command, shell operation,
|
||||
explicit empty-list condition, membership comparison, post-change query,
|
||||
sidecar state, or account abstraction.
|
||||
- Trust `getent` to fail for a missing key and `user` to converge additive
|
||||
membership. Do not reproduce either module's checks.
|
||||
- Test the public behavior by adapting the existing temporary-operator
|
||||
scenario. Do not add a missing-user test, inspect `/etc/passwd` or
|
||||
`/etc/group`, assert task results, or duplicate the detailed test in the
|
||||
collection.
|
||||
- Run each stage's validation before committing it. Do not proceed after a
|
||||
validation failure.
|
||||
- Implement storage and publication entirely through the standalone Narratio
|
||||
role's existing field-specific variables. Do not add a production task,
|
||||
template, handler, fact, probe, command, wrapper, state file, or validation
|
||||
layer to the pipeline role.
|
||||
- Keep `dnd_pipeline_secrets` as the existing generic empty-by-default map.
|
||||
Add no real, generated, placeholder, or fallback credential value to role
|
||||
defaults, templates, documentation, tests, logs, or ordinary inventory.
|
||||
- Keep the S3 access-key and secret-key reference variables nullable as a
|
||||
pair. Do not add an Ansible assertion for pairing or secret presence;
|
||||
Narratio owns native configuration validation and AWS provider selection.
|
||||
- Do not configure `narratio_spool_delete_audio_after_publish` or
|
||||
`narratio_notarius_chunk_cache`. Preserve the roadmap's single-owner
|
||||
decisions for spool retention and Notarius cache policy.
|
||||
- Treat accepted Notarius outputs and published outputs as separate
|
||||
interfaces. Preserve `narratio.extraction.<output_key>` sources for future
|
||||
analyze inputs without adding publication rules or new artifacts for them.
|
||||
- Exercise explicitly supplied representative values in the detailed pipeline
|
||||
integration scenario. Do not assert that those values are current defaults
|
||||
or duplicate Narratio's own storage and publication test matrix.
|
||||
- Do not make a real S3 request or paid model request during repository-local
|
||||
validation.
|
||||
- Run every stage's validation before completing the stage. Do not proceed
|
||||
after a validation failure.
|
||||
- Commit each completed stage with the stated plain-English intent. Push the
|
||||
authoritative pipeline commit before importing it into the collection.
|
||||
- Use only the public Gitea pipeline URL for the subtree update; do not import
|
||||
- Use only the public Gitea pipeline URL for the subtree update. Do not import
|
||||
from the local sibling repository.
|
||||
|
||||
## Starting State and Preconditions
|
||||
|
||||
- Features 0009, 0011, 0012, and 0013 are complete.
|
||||
- This roadmap and implementation plan have been reviewed and committed in the
|
||||
workspace repository before execution begins.
|
||||
- The completed Feature 0013 closeout commits `eda361f` in the workspace and
|
||||
`5e695f3` in the collection are pushed to their respective `origin/main`
|
||||
branches before implementation begins. Preserve the Git-generated subtree
|
||||
squash commit that accompanies `5e695f3`.
|
||||
- `ansible-role-dnd-pipeline` starts clean on `main` at `78afffe`, equal to
|
||||
`origin/main`.
|
||||
- `ansible-collection-dnd` starts clean on `main` at `5e695f3`, equal to
|
||||
`origin/main` after the prerequisite push.
|
||||
- Features 0009, 0011, 0012, 0013, and 0014 are complete.
|
||||
- The approved Feature 0015 roadmap and this implementation plan are committed
|
||||
in the workspace repository before the sprint begins.
|
||||
- `ansible-role-dnd-pipeline` starts clean on `main` at
|
||||
`c13f726a0a846180a1aa95d110b8bbc801edaae5`, equal to `origin/main`.
|
||||
- `ansible-collection-dnd` starts clean on `main` at
|
||||
`c0b0ebd13e92199f3d94e138c6cc50d1ece5dadd`, equal to `origin/main`.
|
||||
- `ansible-role-narratio` is clean at
|
||||
`292d8669e7aad5cfe80b0c469d22b6177e8cf0f1`, equal to `origin/main`, and its
|
||||
`v0.1.0` role release contains the Narratio v2.1.0 field-specific interface
|
||||
required by this feature.
|
||||
- The sibling Narratio application repository is clean at public release
|
||||
`v2.1.0`. It is a read-only contract source during this sprint.
|
||||
- The workspace `.venv` supplies the tested `ansible-playbook`,
|
||||
`ansible-galaxy`, and `ansible-lint` commands used by the repository test
|
||||
drivers.
|
||||
@@ -71,27 +86,31 @@ a new release.
|
||||
test-time sudo is available for the existing integration fixtures.
|
||||
- Git credentials can push the pipeline and collection commits to their public
|
||||
Gitea origins.
|
||||
- No actual Backblaze or model-provider credential is required or available to
|
||||
the implementation. Tests use conspicuously synthetic values only.
|
||||
|
||||
## Repository Scope
|
||||
|
||||
- `ansible-role-dnd-pipeline` owns the variable, argument specification,
|
||||
account lookup, additive enrollment task, README, and detailed integration
|
||||
behavior.
|
||||
- `ansible-collection-dnd` receives the reviewed pipeline commit only through
|
||||
its established squashed subtree and validates the assembled artifact.
|
||||
- `ansible-workspace-dnd` records implementation progress and the review
|
||||
handoff after the two implementation repositories are complete.
|
||||
- All standalone application-role repositories, application source
|
||||
repositories, asset repositories, and `rakestraw-ansible` are out of scope
|
||||
and must remain unchanged.
|
||||
- `ansible-role-dnd-pipeline` owns the suite defaults, argument specification,
|
||||
Narratio dependency mappings, consumer documentation, and detailed
|
||||
representative integration behavior.
|
||||
- `ansible-collection-dnd` owns the artifact-level fixture update, faithful
|
||||
squashed subtree import, assembled-artifact validation, and resulting public
|
||||
collection snapshot.
|
||||
- `ansible-workspace-dnd` records the implementation handoff after the two
|
||||
implementation repositories are complete.
|
||||
- `ansible-role-narratio`, all other standalone roles, application sources,
|
||||
asset repositories, and `rakestraw-ansible` may be inspected but must remain
|
||||
unchanged.
|
||||
|
||||
## Stage 1: Implement Existing-Operator Enrollment
|
||||
## Stage 1: Implement D&D Storage and Publication Policy
|
||||
|
||||
### Objective
|
||||
|
||||
Give the standalone pipeline role its minimal public interface for enrolling
|
||||
existing human accounts and prove the behavior through the existing operator
|
||||
integration scenario.
|
||||
Expose the approved suite-level S3 and publishing inputs through the
|
||||
authoritative pipeline role, map them directly into Narratio, document the
|
||||
operator contract, and prove representative composition without networked
|
||||
storage access.
|
||||
|
||||
### Required Policy Review
|
||||
|
||||
@@ -103,17 +122,22 @@ Before beginning this stage, read:
|
||||
- `docs/ansible.md`
|
||||
- `docs/testing.md`
|
||||
- `docs/planning.md`
|
||||
- `docs/roadmap/features/0014-existing-pipeline-operator-enrollment.md`
|
||||
- `docs/roadmap/features/0015-dnd-s3-storage-and-publishing.md`
|
||||
- `ansible-role-dnd-pipeline/README.md`
|
||||
- `../narratio/AGENTS.md`
|
||||
- `../narratio/docs/development.md`
|
||||
- `../narratio/docs/config.md`
|
||||
- `../narratio/docs/operations.md`
|
||||
|
||||
### Preconditions
|
||||
|
||||
- The workspace planning commit is present.
|
||||
- `ansible-role-dnd-pipeline` is clean at `78afffe` on `main`, is not behind
|
||||
- The workspace planning commit containing this plan is present.
|
||||
- `ansible-role-dnd-pipeline` is clean at the starting revision, is not behind
|
||||
or diverged from `origin/main`, and contains no unrelated changes.
|
||||
- Its Narratio dependency still creates the selected runtime group before the
|
||||
pipeline role's ordinary tasks execute.
|
||||
- The existing integration driver still contains the temporary operator and
|
||||
representative group-access scenario described by the roadmap.
|
||||
- The current `rakestraw.narratio` dependency mapping and existing integration
|
||||
driver still match the structures reviewed while preparing this plan.
|
||||
- Narratio v2.1.0 still accepts the roadmap-selected storage, publish, cleanup,
|
||||
cache, and typed credential-reference fields.
|
||||
|
||||
### Owning Repository
|
||||
|
||||
@@ -121,38 +145,102 @@ Before beginning this stage, read:
|
||||
|
||||
### Required Changes
|
||||
|
||||
1. Add `dnd_pipeline_operator_users: []` beside the existing runtime identity
|
||||
defaults in `defaults/main.yml`.
|
||||
2. Add the matching `meta/argument_specs.yml` entry with `type: list`,
|
||||
`elements: str`, and the same required/default-backed convention used by
|
||||
the role's other public variables.
|
||||
3. Add one unprivileged `ansible.builtin.getent` task near the beginning of
|
||||
`tasks/main.yml`. Loop directly over `dnd_pipeline_operator_users`, use the
|
||||
`passwd` database and each username as `key`, and rely on the module's
|
||||
default missing-key failure. Do not register its result or add a separate
|
||||
assertion.
|
||||
4. Immediately follow it with one `become: true` `ansible.builtin.user` task
|
||||
over the same list. Set `name` to the selected account,
|
||||
`groups` to `dnd_pipeline_runtime_group`, `append: true`, and
|
||||
`state: present`. Do not set or manage any other account property.
|
||||
5. Revise the README variable reference and shared-execution section. Replace
|
||||
the consumer-owned membership example and deferred wording with the new
|
||||
interface, existing-account prerequisite, additive non-removal behavior,
|
||||
authorization consequence, and new-login-session caveat.
|
||||
6. Adapt `tests/integration` rather than adding another scenario. Create its
|
||||
temporary operator before the initial role application without membership
|
||||
in the runtime group, pass that username through
|
||||
`dnd_pipeline_operator_users` for the baseline and repeated applications,
|
||||
and retain the existing native configuration, secret-boundary, and shared
|
||||
cache exercises under that identity. Reorder cleanup as necessary so any
|
||||
final baseline convergence occurs while the selected fixture account still
|
||||
exists, then remove the fixture account.
|
||||
7. Retain the existing clean-repeat assertion. Do not add a negative
|
||||
missing-user case or assertions about module internals, task traces, the
|
||||
current default value, or raw account database entries.
|
||||
8. Run the complete existing repository validation, commit the result, push
|
||||
`main` to the public pipeline origin, and record the full pushed commit for
|
||||
Stage 2.
|
||||
1. Add the following exact suite defaults to `defaults/main.yml` near the
|
||||
existing Narratio policy variables:
|
||||
|
||||
```yaml
|
||||
dnd_pipeline_narratio_workspace_cleanup_after_publish: true
|
||||
dnd_pipeline_narratio_storage_backend: s3
|
||||
dnd_pipeline_narratio_storage_s3_bucket: rakestraw-assets
|
||||
dnd_pipeline_narratio_storage_s3_root_prefix: dnd
|
||||
dnd_pipeline_narratio_storage_s3_region: us-east-1
|
||||
dnd_pipeline_narratio_storage_s3_endpoint: https://s3.us-east-005.backblazeb2.com
|
||||
dnd_pipeline_narratio_storage_s3_force_path_style: true
|
||||
dnd_pipeline_narratio_storage_s3_access_key_id: OBJECT_STORAGE_KEY_ID
|
||||
dnd_pipeline_narratio_storage_s3_secret_access_key: OBJECT_STORAGE_KEY
|
||||
dnd_pipeline_narratio_cache_s3_audio: true
|
||||
dnd_pipeline_narratio_publish_enabled: true
|
||||
dnd_pipeline_narratio_publish_upload_run: true
|
||||
dnd_pipeline_narratio_publish_outputs:
|
||||
- source: narratio.transcript.final
|
||||
required: true
|
||||
- source: narratio.transcript.final_trimmed
|
||||
required: true
|
||||
- source: narratio.transcript.final_markdown
|
||||
required: true
|
||||
- source: narratio.transcript.final_trimmed_markdown
|
||||
required: true
|
||||
- source: narratio.artifact.session_recap
|
||||
required: true
|
||||
- source: narratio.artifact.hostile_rumors
|
||||
required: true
|
||||
- source: narratio.artifact.deranged_rumors
|
||||
required: false
|
||||
```
|
||||
|
||||
Keep every `dest` omitted. Narratio derives transcript destinations from
|
||||
built-in sources and artifact destinations from the already managed
|
||||
artifact output paths.
|
||||
2. Add matching public entries to `meta/argument_specs.yml`. Use `type: bool`
|
||||
for the cleanup, path-style, cache, enabled, and upload settings; `type: str`
|
||||
for the backend and S3 string fields; and `type: list` for the complete
|
||||
publish output list. The S3 child fields, including both credential
|
||||
references, must be non-required so callers may set every S3 child to
|
||||
`null` when deliberately selecting the local backend or set the credential
|
||||
pair to `null` when selecting the AWS SDK provider chain. The remaining
|
||||
concrete suite-policy fields follow the role's existing required/defaulted
|
||||
argument-spec convention.
|
||||
3. In the `rakestraw.narratio` dependency entry in `meta/main.yml`, map every
|
||||
new suite variable directly to its same-purpose standalone variable:
|
||||
`narratio_workspace_cleanup_after_publish`, the eight
|
||||
`narratio_storage_*` fields, `narratio_cache_s3_audio`,
|
||||
`narratio_publish_enabled`, `narratio_publish_upload_run`, and
|
||||
`narratio_publish_outputs`.
|
||||
4. Do not map `narratio_spool_delete_audio_after_publish`. Do not map
|
||||
`narratio_notarius_chunk_cache`. Do not add a new import, fragment,
|
||||
template, task, handler, assertion, or configuration-validation command.
|
||||
5. Revise the README's variable reference and processing-policy sections to
|
||||
document the selected S3 destination, paired nullable credential
|
||||
references, complete-list publish semantics, native destination derivation,
|
||||
maximum S3 audio caching, successful-publication workspace cleanup, and
|
||||
deliberate omission of spool deletion.
|
||||
6. Extend the README's managed-credential example to show
|
||||
`OBJECT_STORAGE_KEY_ID` and `OBJECT_STORAGE_KEY` mapped from encrypted
|
||||
inventory beside the existing Audita and OpenRouter entries. Explain that
|
||||
values remain outside ordinary configuration and that setting both S3
|
||||
credential references to `null` selects Narratio's ambient AWS SDK provider
|
||||
chain.
|
||||
7. Replace the README's statement that storage and publishing are deferred
|
||||
with concise operator-facing descriptions and public upstream links for S3
|
||||
session audio, publish, restore, publication locks, status, and cleanup.
|
||||
Explain that accepted Notarius lanes remain
|
||||
`narratio.extraction.<output_key>` analyze inputs independently of whether
|
||||
they have stable publish rules.
|
||||
8. Adapt the existing `tests/integration` baseline rather than adding a new
|
||||
scenario. Add two synthetic object-storage values to
|
||||
`dnd_pipeline_secrets`; explicitly unset `OBJECT_STORAGE_KEY_ID` and
|
||||
`OBJECT_STORAGE_KEY` alongside the existing provider variables when
|
||||
invoking Ansible; and explicitly pass representative non-default S3,
|
||||
cleanup, cache, and publish values through the new public variables.
|
||||
9. Use representative values that exercise the interface without freezing the
|
||||
defaults: for example an `.invalid` HTTPS endpoint and fixture bucket,
|
||||
prefix, and region; intentional `false` values for cleanup, path style, and
|
||||
S3 audio caching; the two configured credential reference names; enabled
|
||||
run publication; and a short complete publish list containing one built-in
|
||||
transcript source and `narratio.artifact.session_recap` with omitted
|
||||
destinations.
|
||||
10. Extend the existing secret-safe effective-configuration inspection only
|
||||
enough to demonstrate the explicitly supplied representative storage
|
||||
fields, typed credential names, falsey cache/cleanup choices, enabled
|
||||
publication, and Narratio-derived destinations. Continue to use
|
||||
`narratio config validate` as the native validity boundary. Do not inspect
|
||||
role task results, call S3, or reproduce Narratio's validation rules.
|
||||
11. Extend the current non-disclosure check to cover both synthetic
|
||||
object-storage values. Preserve the complete existing integration matrix,
|
||||
cleanup behavior, and immediate clean-repeat assertions.
|
||||
12. Run the complete repository validation. Commit the valid result with the
|
||||
stated plain-English message, push `main` to public Gitea, and record the
|
||||
full pushed commit hash for Stage 3.
|
||||
|
||||
### Validation
|
||||
|
||||
@@ -162,34 +250,36 @@ Run from the workspace root:
|
||||
(cd ansible-role-dnd-pipeline && tests/integration && git diff --check)
|
||||
```
|
||||
|
||||
Expected result: syntax and production-profile lint pass; the temporary
|
||||
existing account is enrolled through the public role variable and completes
|
||||
the existing representative operator-access scenario; native application
|
||||
checks pass; the immediate identical role application reports `changed=0`
|
||||
and `failed=0`; and no credential value appears in test output.
|
||||
Expected result: syntax checking and production-profile lint pass; all five
|
||||
applications remain valid; the explicitly supplied S3 and publish values
|
||||
resolve through Narratio's native configuration interface; only reference
|
||||
names appear in effective output; no synthetic credential value appears in
|
||||
captured output; no external S3 or model request occurs; and the immediate
|
||||
identical repeat reports `changed=0` and `failed=0`.
|
||||
|
||||
### Completion Criteria
|
||||
|
||||
- The public interface, argument specification, two direct tasks, and README
|
||||
implement exactly the roadmap contract.
|
||||
- The existing operator scenario uses the role interface rather than fixture
|
||||
group enrollment.
|
||||
- Defaults, argument specifications, dependency mappings, and documentation
|
||||
implement exactly the Feature 0015 contract.
|
||||
- No production task, template, fragment, validation command, or second secret
|
||||
path was added.
|
||||
- The representative integration scenario exercises public inputs rather than
|
||||
snapshotting defaults and retains the existing clean-repeat boundary.
|
||||
- The full standalone integration suite and `git diff --check` pass.
|
||||
- The role repository is clean after a plain-English commit.
|
||||
- The pipeline repository is clean after the plain-English commit.
|
||||
- The exact commit is present on public `origin/main` and its full hash is
|
||||
available for the collection import.
|
||||
|
||||
### Commit
|
||||
|
||||
Commit intent: `Add optional pipeline operator enrollment`
|
||||
Commit intent: `Enable D&D storage and publishing`
|
||||
|
||||
## Stage 2: Import the Reviewed Pipeline Change
|
||||
## Stage 2: Prepare the Collection Credential Fixture
|
||||
|
||||
### Objective
|
||||
|
||||
Expose the reviewed operator interface through `rakestraw.dnd.pipeline` while
|
||||
preserving the authoritative-standalone-repository and collection-artifact
|
||||
boundaries.
|
||||
Make the collection's artifact-level fixture compatible with the forthcoming
|
||||
pipeline snapshot while its currently bundled role remains valid.
|
||||
|
||||
### Required Policy Review
|
||||
|
||||
@@ -201,17 +291,93 @@ Before beginning this stage, read:
|
||||
- `docs/ansible.md`
|
||||
- `docs/testing.md`
|
||||
- `docs/planning.md`
|
||||
- `docs/roadmap/features/0014-existing-pipeline-operator-enrollment.md`
|
||||
- `docs/roadmap/features/0015-dnd-s3-storage-and-publishing.md`
|
||||
- `ansible-collection-dnd/README.md`
|
||||
|
||||
### Preconditions
|
||||
|
||||
- Stage 1 is complete and its exact commit is publicly reachable from
|
||||
`https://gitea.maximumdirect.net/eric/ansible-role-dnd-pipeline.git`.
|
||||
- `ansible-collection-dnd` is clean at `5e695f3` on `main`, is not behind or
|
||||
- Stage 1 is complete, validated, committed, and pushed.
|
||||
- `ansible-collection-dnd` is clean at the starting revision, is not behind or
|
||||
diverged from `origin/main`, and contains no unrelated changes.
|
||||
- The collection's current `roles/pipeline` subtree still corresponds to the
|
||||
authoritative pipeline history through `78afffe`.
|
||||
- Its top-level `tests/integration` fixture still supplies the two existing
|
||||
synthetic provider credentials through `dnd_pipeline_secrets`.
|
||||
|
||||
### Owning Repository
|
||||
|
||||
`ansible-collection-dnd`
|
||||
|
||||
### Required Changes
|
||||
|
||||
1. Update only the collection-level `tests/integration` fixture outside
|
||||
`roles/pipeline`. Add conspicuously synthetic values for
|
||||
`OBJECT_STORAGE_KEY_ID` and `OBJECT_STORAGE_KEY` to its existing
|
||||
`dnd_pipeline_secrets` input.
|
||||
2. Explicitly unset those two environment variables beside the existing
|
||||
Audita and OpenRouter variables when the fixture runs Ansible so ambient
|
||||
controller state cannot replace the managed synthetic files.
|
||||
3. Do not add storage assertions or duplicate the standalone pipeline's
|
||||
detailed representative test. The collection fixture owns only enough
|
||||
input to validate the assembled artifact after Stage 3.
|
||||
4. Do not edit anything under `roles/pipeline`, alter dependency manifests,
|
||||
or change collection release metadata.
|
||||
5. Check the fixture's shell syntax and whitespace, then commit the compatible
|
||||
preparation with the stated message. Do not push separately unless needed;
|
||||
Stage 3 will push both collection commits after full artifact validation.
|
||||
|
||||
### Validation
|
||||
|
||||
Run from the workspace root:
|
||||
|
||||
```text
|
||||
(cd ansible-collection-dnd && bash -n tests/integration && git diff --check)
|
||||
```
|
||||
|
||||
Expected result: the collection integration driver remains valid shell; the
|
||||
diff changes only its generic managed-secret input and environment scrub; and
|
||||
no subtree, dependency, or release file changed. Full semantic validation
|
||||
occurs once against the imported snapshot in Stage 3.
|
||||
|
||||
### Completion Criteria
|
||||
|
||||
- The artifact-level fixture provides all four synthetic operational
|
||||
credential names without exposing real values.
|
||||
- No detailed storage behavior was duplicated in the collection.
|
||||
- Only the collection-owned fixture changed.
|
||||
- Shell syntax and `git diff --check` pass.
|
||||
- The collection repository is clean after the plain-English commit.
|
||||
|
||||
### Commit
|
||||
|
||||
Commit intent: `Prepare collection storage credentials`
|
||||
|
||||
## Stage 3: Import and Validate the Reviewed Pipeline Snapshot
|
||||
|
||||
### Objective
|
||||
|
||||
Expose the reviewed Feature 0015 behavior through `rakestraw.dnd.pipeline`
|
||||
while preserving the public-remote subtree and collection-artifact boundaries.
|
||||
|
||||
### Required Policy Review
|
||||
|
||||
Before beginning this stage, read:
|
||||
|
||||
- `AGENTS.md`
|
||||
- `docs/orientation.md`
|
||||
- `docs/conventions.md`
|
||||
- `docs/ansible.md`
|
||||
- `docs/testing.md`
|
||||
- `docs/planning.md`
|
||||
- `docs/roadmap/features/0015-dnd-s3-storage-and-publishing.md`
|
||||
- `ansible-collection-dnd/README.md`
|
||||
|
||||
### Preconditions
|
||||
|
||||
- Stage 1's exact pipeline commit is publicly reachable from
|
||||
`https://gitea.maximumdirect.net/eric/ansible-role-dnd-pipeline.git`.
|
||||
- Stage 2 is complete and committed in `ansible-collection-dnd`.
|
||||
- The collection is clean at the Stage 2 commit and its bundled pipeline
|
||||
subtree still corresponds to the authoritative pipeline history through
|
||||
`c13f726a0a846180a1aa95d110b8bbc801edaae5`.
|
||||
|
||||
### Owning Repository
|
||||
|
||||
@@ -220,27 +386,28 @@ Before beginning this stage, read:
|
||||
### Required Changes
|
||||
|
||||
1. Set a shell variable to the full pushed Stage 1 commit hash and run the
|
||||
documented remote-only update from the collection repository:
|
||||
documented remote-only subtree update from the collection repository:
|
||||
|
||||
```text
|
||||
pipeline_revision=<FULL_PUSHED_STAGE_1_COMMIT>
|
||||
git subtree pull --prefix=roles/pipeline \
|
||||
https://gitea.maximumdirect.net/eric/ansible-role-dnd-pipeline.git \
|
||||
"$pipeline_revision" --squash -m "Import pipeline operator enrollment"
|
||||
"$pipeline_revision" --squash -m "Import D&D storage and publishing"
|
||||
```
|
||||
|
||||
2. Do not edit role behavior directly beneath `roles/pipeline`. The subtree
|
||||
import should carry the new default, argument specification, tasks, README,
|
||||
and standalone tests faithfully.
|
||||
3. Do not change `galaxy.yml`, dependency manifests, collection-level tests,
|
||||
or release documentation. No dependency or published version changes in
|
||||
this feature, and the detailed behavior remains covered by Stage 1.
|
||||
4. Run the existing collection integration suite against the built and
|
||||
installed artifact. Commit no additional files after the subtree command
|
||||
unless validation exposes a genuine collection-boundary issue within this
|
||||
feature's scope.
|
||||
5. Push the resulting collection history to `origin/main` and record the
|
||||
resulting merge commit and Git-generated squashed subtree commit.
|
||||
2. Do not edit role behavior directly beneath `roles/pipeline`. The import
|
||||
must faithfully carry the reviewed defaults, argument specification,
|
||||
dependency mappings, README, and detailed integration fixture.
|
||||
3. Do not modify `galaxy.yml`, `requirements.yml`, release documentation, or
|
||||
the published `v0.1.0` artifacts. Feature 0015 changes no standalone role or
|
||||
native collection dependency and publishes no release.
|
||||
4. Run the complete collection integration suite against the built and
|
||||
installed artifact. Do not add collection-level assertions for the detailed
|
||||
S3 configuration; native validation and clean convergence remain the
|
||||
collection boundary.
|
||||
5. If validation succeeds, push the Stage 2 fixture commit, the subtree merge
|
||||
commit, and its Git-generated squash commit to `origin/main`. Record the
|
||||
resulting commit hashes for Stage 4.
|
||||
|
||||
### Validation
|
||||
|
||||
@@ -250,33 +417,34 @@ Run from the workspace root:
|
||||
(cd ansible-collection-dnd && tests/integration && git diff --check)
|
||||
```
|
||||
|
||||
Expected result: the collection builds; the archive contains the updated
|
||||
pipeline role while retaining its development-file exclusions; isolated
|
||||
dependency and artifact installation succeeds; the FQCN resolves; native
|
||||
representative application checks pass; and the immediate repeat reports
|
||||
`changed=0` and `failed=0`.
|
||||
Expected result: the collection builds; its archive contains the updated
|
||||
pipeline role while retaining development-file exclusions; isolated
|
||||
dependency and artifact installation succeeds; the FQCN resolves; Narratio
|
||||
loads its typed storage references from the synthetic credential files without
|
||||
contacting S3; native representative application checks pass; and the
|
||||
immediate repeat reports `changed=0` and `failed=0`.
|
||||
|
||||
### Completion Criteria
|
||||
|
||||
- Git history records a squashed subtree import from the exact public Stage 1
|
||||
commit with the stated plain-English merge message.
|
||||
- Git history records the exact public Stage 1 revision through the documented
|
||||
squashed subtree workflow and stated merge message.
|
||||
- No pipeline behavior was authored directly in the collection.
|
||||
- The existing collection suite and `git diff --check` pass without an
|
||||
unnecessary duplicate operator test.
|
||||
- The collection is clean and its new history is present on `origin/main`.
|
||||
- The complete collection suite and `git diff --check` pass.
|
||||
- The collection repository is clean and its Stage 2 and Stage 3 history is
|
||||
present on public `origin/main`.
|
||||
|
||||
### Commit
|
||||
|
||||
Commit intent: `Import pipeline operator enrollment`
|
||||
Commit intent: `Import D&D storage and publishing`
|
||||
|
||||
The subtree command creates the stage's merge commit and its generated squash
|
||||
The subtree command creates this stage's merge commit and its generated squash
|
||||
commit; do not add an empty or redundant manual commit.
|
||||
|
||||
## Stage 3: Record the Implementation Handoff
|
||||
## Stage 4: Record the Implementation Handoff
|
||||
|
||||
### Objective
|
||||
|
||||
Record the completed implementation evidence without marking Feature 0014
|
||||
Record the completed implementation evidence without marking Feature 0015
|
||||
done before independent review.
|
||||
|
||||
### Required Policy Review
|
||||
@@ -287,14 +455,15 @@ Before beginning this stage, read:
|
||||
- `docs/orientation.md`
|
||||
- `docs/conventions.md`
|
||||
- `docs/planning.md`
|
||||
- `docs/roadmap/features/0014-existing-pipeline-operator-enrollment.md`
|
||||
- `docs/roadmap/features/0015-dnd-s3-storage-and-publishing.md`
|
||||
|
||||
### Preconditions
|
||||
|
||||
- Stages 1 and 2 are complete, committed, pushed, and validated.
|
||||
- Stages 1 through 3 are complete, committed, pushed where required, and
|
||||
validated.
|
||||
- Their full commit hashes and validation results are available.
|
||||
- The workspace repository is clean apart from the expected progress-record
|
||||
edits for this stage and is not behind or diverged from `origin/main`.
|
||||
- The workspace repository is clean at the planning commit, is not behind or
|
||||
diverged from `origin/main`, and contains no unrelated changes.
|
||||
|
||||
### Owning Repository
|
||||
|
||||
@@ -302,15 +471,17 @@ Before beginning this stage, read:
|
||||
|
||||
### Required Changes
|
||||
|
||||
1. Change Feature 0014's roadmap status and status-table state to
|
||||
1. Change Feature 0015's roadmap status and status-table state to
|
||||
`in_progress`, indicating implementation is complete but independent review
|
||||
remains outstanding.
|
||||
2. Update the workspace current focus and Feature 0014 evidence with the exact
|
||||
pipeline and collection commit hashes and the validation completed.
|
||||
3. Replace this plan's Progress Notes with a concise record of the three stage
|
||||
2. Update the workspace current focus and Feature 0015 evidence with the exact
|
||||
pipeline commit, collection fixture commit, collection subtree merge and
|
||||
squash commits, and validation completed.
|
||||
3. Replace this plan's Progress Notes with a concise record of the four stage
|
||||
states and their commits. Do not copy command logs into the repository.
|
||||
4. Do not mark the feature `done`, remove the active plan, or create
|
||||
`docs/roadmap/remediation.md`; those are review and closeout decisions.
|
||||
4. Do not mark the feature `done`, remove this active plan, change Feature
|
||||
0016's state, or create `docs/roadmap/remediation.md`; those are later review
|
||||
and closeout decisions.
|
||||
5. Commit the workspace handoff record with the stated plain-English message.
|
||||
|
||||
### Validation
|
||||
@@ -324,49 +495,39 @@ git diff --check
|
||||
|
||||
Expected result: every repository is clean; the pipeline and collection are
|
||||
not ahead of or behind their origins; only the workspace handoff commit may be
|
||||
ahead of `origin/main`; and the dashboard points to the exact implementation
|
||||
ahead of `origin/main`; and the dashboard points to exact implementation
|
||||
evidence while retaining independent review as the next step.
|
||||
|
||||
### Completion Criteria
|
||||
|
||||
- Workspace status accurately identifies Feature 0014 as implemented and
|
||||
awaiting review.
|
||||
- The roadmap remains the durable target-state record and is not polluted with
|
||||
execution logs.
|
||||
- Workspace status accurately identifies Feature 0015 as implemented and
|
||||
awaiting independent review.
|
||||
- The roadmap remains the durable target-state record and contains no command
|
||||
log or transient execution notes.
|
||||
- The active plan records concise implementation evidence and contains no
|
||||
false completion claim.
|
||||
- The workspace repository is clean after the handoff commit.
|
||||
|
||||
### Commit
|
||||
|
||||
Commit intent: `Record pipeline operator enrollment implementation`
|
||||
Commit intent: `Record D&D storage and publishing implementation`
|
||||
|
||||
## Final Validation and Handoff
|
||||
|
||||
- Confirm Stage 1's complete standalone integration result and Stage 2's
|
||||
complete collection integration result.
|
||||
- Confirm Stage 1's complete standalone pipeline integration result and Stage
|
||||
3's complete collection integration result.
|
||||
- Run `./scripts/status` and confirm every repository is clean, with no
|
||||
unexpected branch divergence or untracked artifact.
|
||||
- Report the full pipeline, collection merge, collection subtree-squash, and
|
||||
workspace handoff commit hashes.
|
||||
- Summarize any deviation from this plan, unresolved finding, or added
|
||||
implementation complexity and identify the acceptance criterion that
|
||||
required it.
|
||||
- Do not mark Feature 0014 `done`; implementation results require independent
|
||||
- Report the full pipeline commit, collection fixture commit, collection
|
||||
subtree merge and squash commits, and workspace handoff commit.
|
||||
- Confirm that `ansible-role-narratio`, all other standalone roles,
|
||||
applications, assets, and `rakestraw-ansible` remained unchanged.
|
||||
- Summarize any deviation from this plan, unresolved finding, or complexity
|
||||
added beyond direct variable mapping and identify the acceptance criterion
|
||||
that required it.
|
||||
- Do not mark Feature 0015 `done`; implementation results require independent
|
||||
review first.
|
||||
|
||||
## Progress Notes
|
||||
|
||||
- Stage 1 complete: `ansible-role-dnd-pipeline`
|
||||
`c13f726a0a846180a1aa95d110b8bbc801edaae5` implements optional enrollment
|
||||
of existing operators; the complete standalone integration suite passed and
|
||||
the commit is public on `origin/main`.
|
||||
- Stage 2 complete: `ansible-collection-dnd` merge
|
||||
`c0b0ebd13e92199f3d94e138c6cc50d1ece5dadd` imports the exact public pipeline
|
||||
change through squash `d292297b6ad179e4664c01ad07cddaefff12f769`;
|
||||
the complete collection integration suite passed and the history is public
|
||||
on `origin/main`.
|
||||
- Stage 3 complete: `ansible-workspace-dnd`
|
||||
`bf81461de60001a0d8050e560383c6ddf533209f` records the implementation
|
||||
evidence while Feature 0014 remains `in_progress` for independent review.
|
||||
- Deviations: none.
|
||||
- Planning complete; implementation has not begun.
|
||||
|
||||
@@ -370,6 +370,15 @@ application-role releases in one source repository.
|
||||
- The Narratio `testing` profile keeps the Notarius stage enabled and inherits
|
||||
the same D&D pipeline and extraction profile as production until a distinct
|
||||
testing extraction policy is selected.
|
||||
- The D&D suite selects its Backblaze-compatible S3 archive, typed
|
||||
`OBJECT_STORAGE_KEY_ID` and `OBJECT_STORAGE_KEY` references, maximum S3 audio
|
||||
caching, run publication, human-facing transcript and artifact outputs, and
|
||||
cleanup of successfully published workspace state. Feature 0015 owns the
|
||||
exact overrideable defaults and publication set.
|
||||
- Accepted Notarius lane outputs remain addressable inside Narratio as
|
||||
`narratio.extraction.<output_key>` sources. Future analyze-stage Scriptorium
|
||||
artifacts may consume those sources directly; publishing an extraction lane
|
||||
to a stable remote destination is an independent policy choice.
|
||||
|
||||
### Execution model
|
||||
|
||||
@@ -389,7 +398,7 @@ later design decision changes them:
|
||||
- `/usr/local/etc/notarius/pipelines` for suite-owned imported Notarius
|
||||
pipeline definitions
|
||||
- `/usr/local/etc/scriptorium/config.yml` for Scriptorium configuration
|
||||
- `/usr/local/etc/narratio/secrets` for future restricted credential material
|
||||
- `/usr/local/etc/narratio/secrets` for suite-managed restricted credentials
|
||||
- `/usr/local/share/narratio/campaigns` for version-controlled campaign data
|
||||
- `/usr/local/share/narratio/prompt-library` for version-controlled prompts
|
||||
and schemas
|
||||
|
||||
@@ -1,10 +1,10 @@
|
||||
# Workspace Status
|
||||
|
||||
Last updated: 2026-09-18
|
||||
Last updated: 2026-09-21
|
||||
|
||||
Current milestone: Existing pipeline operator enrollment
|
||||
Current milestone: Operational D&D pipeline deployment
|
||||
|
||||
Current focus: Independently review the Feature 0014 implementation.
|
||||
Current focus: Review the Feature 0015 staged implementation plan.
|
||||
|
||||
| ID | Deliverable | Owner | State | Evidence or next step |
|
||||
| --- | --- | --- | --- | --- |
|
||||
@@ -30,7 +30,9 @@ Current focus: Independently review the Feature 0014 implementation.
|
||||
| F0011 | Deliver pipeline credentials and Notarius reuse | ansible-role-narratio and ansible-role-dnd-pipeline | done | Implemented and independently reviewed through remediation commit `de1b191` |
|
||||
| F0012 | Assemble the D&D collection | ansible-collection-dnd and ansible-role-dnd-pipeline | done | Pipeline `600eb21` assembled and validated through collection remediation commit `da67b34` |
|
||||
| F0013 | Publish the first coordinated suite release | all Ansible repositories and workspace | done | Public [`rakestraw.dnd` 0.1.0 release](https://gitea.maximumdirect.net/eric/ansible-collection-dnd/releases/tag/v0.1.0); documentation remediation completed in pipeline `78afffe` and collection `5e695f3` |
|
||||
| F0014 | Enroll existing pipeline operators | ansible-role-dnd-pipeline, ansible-collection-dnd, and workspace | in_progress | Review pipeline `c13f726a0a846180a1aa95d110b8bbc801edaae5` and collection `c0b0ebd13e92199f3d94e138c6cc50d1ece5dadd`; both complete integration suites passed |
|
||||
| F0014 | Enroll existing pipeline operators | ansible-role-dnd-pipeline, ansible-collection-dnd, and workspace | done | Pipeline `c13f726` and collection import `c0b0ebd` passed their complete integration suites and independent review with no remediation findings |
|
||||
| F0015 | Enable D&D S3 storage and publishing | ansible-role-dnd-pipeline, ansible-collection-dnd, and workspace | ready | [Roadmap](roadmap/features/0015-dnd-s3-storage-and-publishing.md) approved; prepare the staged implementation plan |
|
||||
| F0016 | Deploy the D&D pipeline to `pipeline-dnd` | rakestraw-ansible and workspace | proposed | [Roadmap](roadmap/features/0016-deploy-dnd-pipeline-to-pipeline-dnd.md) drafted; Feature 0015 is a prerequisite |
|
||||
| A0001 | Audit completed standalone-role test suites | workspace and standalone roles | done | Findings recorded in `docs/roadmap/test-suite-audit-findings.md` |
|
||||
| R0001 | Simplify completed standalone-role test suites | workspace and standalone roles | done | Role commits `e08c63c`, `d2d0b9d`, and `cfecf37` passed implementation and independent review |
|
||||
|
||||
|
||||
Reference in New Issue
Block a user