Compare commits
15 Commits
ec69f5a03b
...
a231b57294
| Author | SHA1 | Date | |
|---|---|---|---|
| a231b57294 | |||
| 1d1f34fc8b | |||
| f9901f17d1 | |||
| f8fe8dd0c6 | |||
| e482ea4b01 | |||
| bbb7acc315 | |||
| d0c776b47e | |||
| 45599bb020 | |||
| bb0f8cf84b | |||
| 76c10f79f5 | |||
| f399be34cd | |||
| 2feb622937 | |||
| 974b57e33e | |||
| a95a0ea280 | |||
| e262b61292 |
@@ -1,6 +1,6 @@
|
||||
# Feature 0014: Enroll Existing Pipeline Operators
|
||||
|
||||
Status: in_progress
|
||||
Status: done
|
||||
|
||||
## Summary
|
||||
|
||||
|
||||
424
docs/roadmap/features/0015-dnd-s3-storage-and-publishing.md
Normal file
424
docs/roadmap/features/0015-dnd-s3-storage-and-publishing.md
Normal file
@@ -0,0 +1,424 @@
|
||||
# Feature 0015: Enable D&D S3 Storage and Publishing
|
||||
|
||||
Status: done
|
||||
|
||||
## Summary
|
||||
|
||||
Make the D&D pipeline operational with Narratio's current S3 storage,
|
||||
publication, caching, cleanup, and restore capabilities. The pipeline role
|
||||
will select the established Backblaze B2 destination, reference two typed
|
||||
object-storage credentials delivered through its existing secrets directory,
|
||||
publish the useful transcript and D&D artifact outputs, and retain maximum S3
|
||||
audio caching.
|
||||
|
||||
The standalone Narratio role already exposes Narratio v2.1.1's complete
|
||||
configuration surface for this behavior. This feature supplies D&D suite
|
||||
policy through that existing interface; it does not add another storage
|
||||
implementation or credential mechanism.
|
||||
|
||||
## Problem or Opportunity
|
||||
|
||||
The pipeline role currently installs Narratio v2.1.0 and configures the full
|
||||
processing graph, but deliberately leaves Narratio storage and publishing at
|
||||
their application-level defaults. Its README consequently describes storage,
|
||||
publishing, and end-to-end external execution as deferred. The resulting suite
|
||||
can validate configuration and process local state, but it does not select the
|
||||
remote archive or publish the outputs needed for an operational deployment.
|
||||
|
||||
Narratio v2.1.0 also rejected a valid publish list when a rule explicitly set
|
||||
`required` while omitting a derivable `dest`. Narratio v2.1.1 corrects that
|
||||
resolution bookkeeping defect without changing the publish schema or
|
||||
destination rules. This feature therefore advances the pipeline application
|
||||
pin to v2.1.1 and retains explicit suite-owned requiredness together with
|
||||
native destination derivation.
|
||||
|
||||
The prior test-machine configuration records the intended Backblaze B2
|
||||
endpoint, bucket, prefix, credential names, publication set, and post-publish
|
||||
workspace cleanup. Narratio's current contract has since replaced the legacy
|
||||
credential field names and secrets configuration with ConfigKit typed
|
||||
references and a conventional sibling `secrets/` directory. The Ansible suite
|
||||
already implements that new credential-delivery boundary for Audita and
|
||||
Notarius, so S3 needs only to join the same path.
|
||||
|
||||
## Governing Policies
|
||||
|
||||
- [The Ansible house style](../../ansible.md) requires suite policy to flow
|
||||
through the standalone role's field-specific interface and optional fields
|
||||
to remain application-owned unless the suite deliberately selects them.
|
||||
- [The testing house style](../../testing.md) requires representative
|
||||
public-interface and native-application validation without duplicating AWS
|
||||
SDK, ConfigKit, or Ansible module behavior.
|
||||
- [The MVP roadmap](../mvp.md) makes the pipeline role responsible for
|
||||
cross-application configuration and end-to-end validation.
|
||||
- [Feature 0011](0011-managed-pipeline-credential-delivery.md) establishes the
|
||||
managed Narratio secrets directory and typed credential-forwarding policy.
|
||||
- [Feature 0012](0012-collection-assembly.md) makes the standalone pipeline
|
||||
repository authoritative and imports reviewed changes through public
|
||||
Gitea.
|
||||
- [Feature 0016](0016-deploy-dnd-pipeline-to-pipeline-dnd.md) consumes this
|
||||
feature's reviewed interface for the first real-host deployment.
|
||||
|
||||
## Scope
|
||||
|
||||
- Advance the suite's Narratio application version from `v2.1.0` to the
|
||||
backward-compatible `v2.1.1` bugfix release.
|
||||
- Add suite-owned pipeline variables for the selected Narratio workspace,
|
||||
S3, cache, and publish settings.
|
||||
- Select the S3 backend with these D&D deployment defaults:
|
||||
|
||||
| Setting | Suite default |
|
||||
| --- | --- |
|
||||
| Endpoint | `https://s3.us-east-005.backblazeb2.com` |
|
||||
| Bucket | `rakestraw-assets` |
|
||||
| Root prefix | `dnd` |
|
||||
| Region | `us-east-1` |
|
||||
| Force path style | `true` |
|
||||
| Access-key reference | `OBJECT_STORAGE_KEY_ID` |
|
||||
| Secret-key reference | `OBJECT_STORAGE_KEY` |
|
||||
|
||||
- Pass the two credential reference names to the standalone Narratio role.
|
||||
Their values use the existing `dnd_pipeline_secrets` map and managed
|
||||
`/usr/local/etc/narratio/secrets/` directory.
|
||||
- Enable S3 audio caching explicitly so the suite retains Narratio's maximum
|
||||
available audio reuse.
|
||||
- Enable run publication and publish these complete native output rules:
|
||||
|
||||
| Source | Required |
|
||||
| --- | --- |
|
||||
| `narratio.transcript.final` | `true` |
|
||||
| `narratio.transcript.final_trimmed` | `true` |
|
||||
| `narratio.transcript.final_markdown` | `true` |
|
||||
| `narratio.transcript.final_trimmed_markdown` | `true` |
|
||||
| `narratio.artifact.session_recap` | `true` |
|
||||
| `narratio.artifact.hostile_rumors` | `true` |
|
||||
| `narratio.artifact.deranged_rumors` | `false` |
|
||||
|
||||
- Let Narratio derive each destination from the built-in transcript path or
|
||||
configured artifact output path rather than duplicating those paths in the
|
||||
publish list.
|
||||
- Enable workspace cleanup only after Narratio records successful run
|
||||
publication. Leave spool-audio deletion under Narratio's existing default
|
||||
because the suite has selected no policy requiring source-audio removal.
|
||||
- Keep the production and `testing` profile storage and publishing behavior
|
||||
identical. The existing overlay remains limited to model-profile selection.
|
||||
- Document the four credentials normally required by the operational suite:
|
||||
`AUDITA_LLM_API_KEY`, `OPENROUTER_API_KEY`,
|
||||
`OBJECT_STORAGE_KEY_ID`, and `OBJECT_STORAGE_KEY`.
|
||||
- Document the operator-visible publication, restore, S3-session-audio, lock,
|
||||
caching, and cleanup capabilities enabled by the selected configuration,
|
||||
while linking Narratio's canonical CLI and operations documentation for
|
||||
command details.
|
||||
- Preserve every accepted Notarius lane as its existing
|
||||
`narratio.extraction.<output_key>` source. Future Narratio artifact
|
||||
definitions can consume those sources during analyze without first making
|
||||
the extraction lanes separate published outputs.
|
||||
- Adapt the existing pipeline integration scenario to resolve synthetic S3
|
||||
credentials and validate one explicitly supplied representative storage and
|
||||
publishing configuration through Narratio's native interface without
|
||||
contacting object storage.
|
||||
- Import the reviewed public pipeline commit into the collection through the
|
||||
established squashed subtree workflow, adapt the collection's artifact-level
|
||||
fixture to supply the added credential names, and validate the assembled
|
||||
artifact.
|
||||
|
||||
## Non-goals
|
||||
|
||||
- Implementing an S3 client, publication engine, remote commit format,
|
||||
restore mechanism, publish-lock mechanism, or audio cache in Ansible.
|
||||
Narratio owns all of those behaviors.
|
||||
- Placing credential values in role defaults, templates, documentation,
|
||||
tests, ordinary inventory, or the generated pipeline configuration.
|
||||
- Adding a second secret-delivery mechanism, environment wrapper, service
|
||||
unit, credential synchronizer, or AWS configuration-file manager.
|
||||
- Managing Backblaze buckets, object lifecycle policy, application keys,
|
||||
retention policy, versioning, encryption, quotas, or network access.
|
||||
- Managing per-session `session.yml` files or selecting local versus S3 audio
|
||||
input for an individual session.
|
||||
- Running a paid or networked publication test from a repository-local test
|
||||
suite. Feature 0016 owns the representative live end-to-end exercise.
|
||||
- Publishing every Notarius extraction lane as a named current output. The
|
||||
lanes remain available as analyze inputs independently of publication; stable
|
||||
remote destinations require an explicit operator need.
|
||||
- Setting Narratio's optional `notarius.chunk_cache` override. The managed
|
||||
Notarius configuration remains the single owner of its ordinary `auto`
|
||||
cache policy, and Narratio v2.1 already requests refresh for forced
|
||||
extraction.
|
||||
- Adding notification providers. Narratio v2.1 supports only its existing
|
||||
`noop` notification mode.
|
||||
- Releasing new role or collection versions as part of this implementation.
|
||||
- Modifying Narratio, ConfigKit, or the standalone Narratio role. Narratio
|
||||
v2.1.1 is an already published prerequisite, and the existing generic role
|
||||
can install the selected application tag through `narratio_version`.
|
||||
|
||||
## Target End State
|
||||
|
||||
The standalone pipeline role exposes these suite variables with the selected
|
||||
defaults:
|
||||
|
||||
```yaml
|
||||
dnd_pipeline_narratio_version: v2.1.1
|
||||
dnd_pipeline_narratio_workspace_cleanup_after_publish: true
|
||||
dnd_pipeline_narratio_storage_backend: s3
|
||||
dnd_pipeline_narratio_storage_s3_bucket: rakestraw-assets
|
||||
dnd_pipeline_narratio_storage_s3_root_prefix: dnd
|
||||
dnd_pipeline_narratio_storage_s3_region: us-east-1
|
||||
dnd_pipeline_narratio_storage_s3_endpoint: >-
|
||||
https://s3.us-east-005.backblazeb2.com
|
||||
dnd_pipeline_narratio_storage_s3_force_path_style: true
|
||||
dnd_pipeline_narratio_storage_s3_access_key_id: OBJECT_STORAGE_KEY_ID
|
||||
dnd_pipeline_narratio_storage_s3_secret_access_key: OBJECT_STORAGE_KEY
|
||||
dnd_pipeline_narratio_cache_s3_audio: true
|
||||
dnd_pipeline_narratio_publish_enabled: true
|
||||
dnd_pipeline_narratio_publish_upload_run: true
|
||||
dnd_pipeline_narratio_publish_outputs:
|
||||
- source: narratio.transcript.final
|
||||
required: true
|
||||
- source: narratio.transcript.final_trimmed
|
||||
required: true
|
||||
- source: narratio.transcript.final_markdown
|
||||
required: true
|
||||
- source: narratio.transcript.final_trimmed_markdown
|
||||
required: true
|
||||
- source: narratio.artifact.session_recap
|
||||
required: true
|
||||
- source: narratio.artifact.hostile_rumors
|
||||
required: true
|
||||
- source: narratio.artifact.deranged_rumors
|
||||
required: false
|
||||
```
|
||||
|
||||
Each value is declared in the pipeline argument specification and passed
|
||||
directly to its corresponding `narratio_*` variable in the pipeline role's
|
||||
Narratio dependency. Narratio's existing root template renders the native
|
||||
configuration. No new pipeline fragment or production task is required.
|
||||
|
||||
The ordinary generated configuration contains only the credential names:
|
||||
|
||||
```yaml
|
||||
storage:
|
||||
backend: s3
|
||||
s3:
|
||||
access_key_id: OBJECT_STORAGE_KEY_ID
|
||||
secret_access_key: OBJECT_STORAGE_KEY
|
||||
```
|
||||
|
||||
Consumers supply the two values beside the existing model-provider values
|
||||
through encrypted inventory:
|
||||
|
||||
```yaml
|
||||
dnd_pipeline_secrets:
|
||||
AUDITA_LLM_API_KEY: "{{ vault_dnd_audita_llm_api_key }}"
|
||||
OPENROUTER_API_KEY: "{{ vault_dnd_openrouter_api_key }}"
|
||||
OBJECT_STORAGE_KEY_ID: "{{ vault_dnd_object_storage_key_id }}"
|
||||
OBJECT_STORAGE_KEY: "{{ vault_dnd_object_storage_key }}"
|
||||
```
|
||||
|
||||
The example defines references only; no roadmap, role, or test contains real
|
||||
credential values. The existing credential task writes the selected entries
|
||||
with the suite's established group-authorized secret modes. Omitted map keys
|
||||
remain unmanaged, as defined by Feature 0011.
|
||||
|
||||
With valid credentials, Narratio can accept either local or S3 audio selected
|
||||
by each session file, cache S3 audio beneath `/var/cache/narratio`, publish an
|
||||
immutable run and the selected stable outputs, use remote publication locks,
|
||||
restore committed state, and clean the published workspace. These are native
|
||||
Narratio commands and lifecycle rules rather than additional Ansible tasks.
|
||||
|
||||
The ten accepted Notarius outputs continue to enter Narratio's artifact
|
||||
catalog as `narratio.extraction.*` sources. Analyze-stage Scriptorium artifacts
|
||||
may reference those sources directly in their input mappings. Feature 0015
|
||||
does not yet add such artifact definitions, but its publication choices do not
|
||||
remove, rename, or constrain this future composition path.
|
||||
|
||||
## Repository Ownership
|
||||
|
||||
- `ansible-role-dnd-pipeline` owns the D&D suite variables, opinionated
|
||||
defaults, dependency mappings, consumer documentation, and representative
|
||||
integration scenario.
|
||||
- `ansible-role-narratio` remains the generic owner of installation and native
|
||||
pipeline rendering. Its reviewed `v0.1.0` role interface is compatible with
|
||||
Narratio v2.1.1 and already covers every
|
||||
configuration field required by this feature and should remain unchanged.
|
||||
- `ansible-collection-dnd` owns the faithful pipeline subtree import, the
|
||||
unified consumer dependency manifest, and artifact-boundary validation.
|
||||
- `ansible-workspace-dnd` owns this roadmap, the later implementation plan,
|
||||
status, and independent review record.
|
||||
- `narratio` v2.1.1 is the released compatibility prerequisite and a read-only
|
||||
source of storage, publication, credential, caching, cleanup, restore, and
|
||||
CLI contracts during this implementation.
|
||||
- `rakestraw-ansible` remains unchanged until Feature 0016 consumes this
|
||||
interface on `pipeline-dnd`.
|
||||
|
||||
## Constraints and Settled Decisions
|
||||
|
||||
- The S3 endpoint, bucket, prefix, region, and path-style values are deliberate
|
||||
D&D suite defaults. They are ordinary variables and remain overrideable by
|
||||
inventory.
|
||||
- Credential references are also policy defaults, but credential values have
|
||||
no defaults. Operators must supply matching entries through
|
||||
`dnd_pipeline_secrets` or another environment source supported by Narratio.
|
||||
- The two credential-reference variables are nullable as a pair. Setting both
|
||||
to `null` omits the references and preserves Narratio's AWS SDK provider-chain
|
||||
path; configuring only one remains invalid under Narratio's native contract.
|
||||
- The role does not assert that the secrets map contains particular keys.
|
||||
Narratio's native loader owns typed-reference resolution, and external
|
||||
environment values remain a supported higher-precedence source.
|
||||
- `dnd_pipeline_narratio_publish_outputs` is one complete native list. A caller
|
||||
replacing it assumes responsibility for the entire selected publication
|
||||
set; the role does not merge rules.
|
||||
- Publish destinations remain omitted where Narratio can derive them. The
|
||||
configured source identity and artifact output path remain the single source
|
||||
of truth.
|
||||
- Narratio v2.1.1 is the minimum application version for this feature because
|
||||
it preserves destination derivation when `required` is explicit. Do not work
|
||||
around older behavior by duplicating destinations or omitting deliberate
|
||||
requiredness.
|
||||
- Workspace cleanup is gated by Narratio's own successful-publication
|
||||
evidence. Ansible does not independently inspect remote state or remove
|
||||
runtime content.
|
||||
- S3 audio caching is enabled, and ordinary cleanup preserves the cache.
|
||||
Explicit operator cache clearing remains a Narratio CLI operation.
|
||||
- The testing profile does not silently disable Notarius, storage,
|
||||
publication, cleanup, or credentials. It continues to differ only in the
|
||||
already selected inexpensive Scriptorium model profile.
|
||||
- S3 storage also enables Narratio's native restore and publication-lock
|
||||
commands without another configuration surface. Ansible documentation may
|
||||
describe availability but must not duplicate Narratio's detailed operating
|
||||
manual.
|
||||
- Notarius extraction-source availability and remote publication are separate
|
||||
concerns. Analyze consumes `narratio.extraction.*` sources from the accepted
|
||||
extraction catalog; `publish.outputs` selects only which sources also receive
|
||||
stable published destinations.
|
||||
|
||||
## Simplest Viable Implementation
|
||||
|
||||
Advance `dnd_pipeline_narratio_version` to v2.1.1. Add the selected variables to
|
||||
the pipeline defaults and argument specification, then map them into the
|
||||
existing `rakestraw.narratio` dependency. Update the pipeline README to document
|
||||
the policy, credential names, complete-list replacement semantics, and native
|
||||
capabilities now available. Do not add a template, task, handler, fact, probe,
|
||||
or storage-specific validation layer.
|
||||
|
||||
In the existing pipeline integration driver, extend the synthetic secrets map
|
||||
with object-storage values and explicitly supply representative S3 and publish
|
||||
variables. Use `narratio config validate` and its secret-safe effective
|
||||
configuration output to prove that the public inputs compose into a valid
|
||||
configuration. Retain the immediate clean repeat and secret non-disclosure
|
||||
checks. Do not call a real S3 service or assert that the representative values
|
||||
equal current role defaults.
|
||||
|
||||
Push the reviewed pipeline commit, import that exact public revision into the
|
||||
collection through `git subtree pull --squash`, and extend only the
|
||||
collection-level fixture inputs needed for native validation. The collection
|
||||
suite should continue to prove artifact installation and one representative
|
||||
assembled convergence rather than duplicate the detailed pipeline matrix.
|
||||
|
||||
## Rationale and Alternatives
|
||||
|
||||
Implementing storage behavior in Ansible would duplicate a mature native
|
||||
application boundary. Narratio already owns credential resolution, AWS SDK
|
||||
provider construction, immutable commits, verification, locks, restore,
|
||||
caching, and cleanup. Passing declarative configuration through the generic
|
||||
role is both smaller and more reliable.
|
||||
|
||||
Using the AWS SDK's ambient provider chain would avoid named secret files, but
|
||||
the initial deployment has explicit Backblaze application credentials and the
|
||||
suite already has a suitable managed typed-secret path. Keeping the two
|
||||
credential-reference variables nullable preserves the ambient-provider
|
||||
alternative: consumers can set both suite variables to `null` when their
|
||||
deployment uses another provider.
|
||||
|
||||
Copying destinations into every publish rule would make the output layout more
|
||||
visible locally, but it would duplicate paths already owned by Narratio's
|
||||
built-in sources and the configured artifact definitions. Native destination
|
||||
derivation keeps path coordination automatic.
|
||||
|
||||
Omitting explicit `required: true` values would avoid the v2.1.0 failure only
|
||||
by relying on an unrelated defaulting path. Changing the generic role's YAML
|
||||
format would not repair the underlying sequence-resolution mismatch. Narratio
|
||||
v2.1.1 fixes the owning application boundary, allowing the suite to state its
|
||||
requiredness policy directly without duplicating destinations.
|
||||
|
||||
Deleting spool audio after publication would minimize transient disk usage,
|
||||
but no such retention policy has been selected, and local-audio sessions may
|
||||
depend on that content. Leaving the field omitted preserves Narratio's default
|
||||
while enabling the explicitly requested S3 audio cache.
|
||||
|
||||
Publishing all ten Notarius extraction lanes would create stable remote output
|
||||
contracts that have not been requested. It is also unnecessary for the stated
|
||||
future use: analyze-stage artifacts can consume `narratio.extraction.*`
|
||||
directly. The immutable published run retains the processing result, while the
|
||||
selected transcript and human-facing artifact outputs match the known
|
||||
operational workflow.
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- The pipeline role declares and documents every suite variable shown in the
|
||||
target state, with matching argument-spec entries and direct Narratio
|
||||
dependency mappings.
|
||||
- The pipeline installs Narratio v2.1.1, whose released configuration loader
|
||||
supports explicit requiredness together with derived publish destinations.
|
||||
- The two credential-reference variables accept paired `null` overrides so a
|
||||
caller can deliberately select Narratio's ambient AWS SDK provider chain.
|
||||
- The default effective Narratio configuration selects the specified
|
||||
Backblaze-compatible S3 destination, typed credential names, S3 audio cache,
|
||||
successful-publication workspace cleanup, run upload, and seven output
|
||||
rules.
|
||||
- The publish list relies on Narratio's native destination derivation and
|
||||
retains `deranged_rumors` as optional while the other six outputs are
|
||||
required.
|
||||
- Spool-audio deletion remains unset by the pipeline role, and the testing
|
||||
profile introduces no storage or publication override.
|
||||
- `dnd_pipeline_secrets` remains a generic empty-by-default mapping; the role
|
||||
documents all four expected operational entries without embedding any
|
||||
credential value.
|
||||
- No new production task, template, handler, fact, state file, wrapper, or
|
||||
storage-validation command is introduced.
|
||||
- The standalone Narratio role remains unchanged because its `v0.1.0` role
|
||||
interface can install Narratio v2.1.1, renders the required native fields,
|
||||
and preserves null omission.
|
||||
- The existing pipeline integration scenario explicitly supplies
|
||||
representative S3 and publish inputs, resolves synthetic values for all four
|
||||
typed credentials, passes Narratio's native configuration validation, and
|
||||
exposes no synthetic value in captured output.
|
||||
- The immediate identical second pipeline application reports `changed=0`
|
||||
and `failed=0`; syntax checking, production-profile lint, the complete
|
||||
standalone integration suite, and `git diff --check` pass.
|
||||
- The exact reviewed pipeline commit is pushed to public Gitea before the
|
||||
collection imports it through the documented remote-only squashed subtree
|
||||
workflow.
|
||||
- The collection fixture supplies the added synthetic credential names, and
|
||||
its existing build, isolated installation, FQCN resolution, native
|
||||
validation, and clean-repeat suite passes without a duplicate S3 behavior
|
||||
test.
|
||||
- No real object-storage request, paid model request, application repository
|
||||
change during the remaining implementation, asset change, new release tag,
|
||||
or published artifact is required. The existing Narratio v2.1.1 release is a
|
||||
prerequisite rather than an implementation deliverable.
|
||||
- Pipeline documentation links operators to Narratio's canonical procedures
|
||||
for S3 session audio, publish, restore, locks, status, and cleanup and no
|
||||
longer describes storage and publishing as deferred.
|
||||
- Documentation distinguishes accepted Notarius extraction sources from
|
||||
published outputs and confirms that future analyze-stage artifacts may use
|
||||
`narratio.extraction.<output_key>` inputs without publishing those lanes.
|
||||
|
||||
## Dependencies and Follow-up Work
|
||||
|
||||
Features 0009, 0011, 0012, 0013, and 0014 and the public Narratio v2.1.1 bugfix
|
||||
release are prerequisites. In particular, Feature 0011 supplies the credential
|
||||
directory and Feature 0014 supplies the human-operator access needed for the
|
||||
eventual live deployment.
|
||||
|
||||
[Feature 0016](0016-deploy-dnd-pipeline-to-pipeline-dnd.md) follows this work.
|
||||
It will put the four actual values in encrypted host inventory, pin the exact
|
||||
reviewed suite revision in `rakestraw-ansible`, converge `pipeline-dnd`, and
|
||||
exercise one real session through remote publication. Bucket lifecycle policy,
|
||||
backup policy, credential rotation, additional publish outputs, automated
|
||||
scheduling, and cleanup of the proof-of-concept deployment remain separate
|
||||
future work.
|
||||
|
||||
A later artifact-development feature may add D&D-specific Narratio artifact
|
||||
definitions whose Scriptorium input mappings consume selected
|
||||
`narratio.extraction.*` sources. That feature should define each artifact's
|
||||
purpose, inputs, prompt, output contract, and publication choice; it does not
|
||||
require a storage-interface change.
|
||||
@@ -0,0 +1,145 @@
|
||||
# Feature 0016: Prepare the D&D Deployment Playbook for `pipeline-dnd`
|
||||
|
||||
Status: in_progress
|
||||
|
||||
## Summary
|
||||
|
||||
Write and commit the consumer playbook, necessary host variables, dependency
|
||||
manifest, and concise usage documentation in `rakestraw-ansible`. This feature
|
||||
ends with repository preparation. Deployment to a controller or target host
|
||||
and execution of the pipeline are separate operator work.
|
||||
|
||||
## Scope
|
||||
|
||||
- `playbooks/pipeline/dnd.yml` targets the existing `pipeline-dnd` alias and
|
||||
applies `rakestraw.dnd.pipeline`, using only hosts and roles.
|
||||
- Host choices belong in `inventory/host_vars/pipeline-dnd/`: the WhisperX
|
||||
endpoint, initial operator `eric`, default campaign `dilfs`, and credential
|
||||
references. Unknown required values are explained in adjacent comments.
|
||||
- Inherit the reviewed suite's storage, publication, caching, cleanup, and
|
||||
application defaults. Retire obsolete legacy host variables.
|
||||
- Root `requirements.yml` selects the reviewed public collection snapshot and
|
||||
its role/native collection dependencies, preserving unrelated entries.
|
||||
- Document local validation and eventual operator use. Commit the completed
|
||||
repository changes without requiring remote access or real credentials.
|
||||
|
||||
## Non-goals
|
||||
|
||||
- Installing or updating anything on the operational controller or target.
|
||||
- Connecting to `pipeline-dnd`, validating its platform or sudo access,
|
||||
converging the host, checking runtime identities, or executing a real session.
|
||||
- Obtaining API keys, choosing a Vault password, populating real credentials,
|
||||
or selecting acceptance audio/session data.
|
||||
- Changing application/role behavior, publishing a new suite release, adding
|
||||
orchestration layers, or retiring the proof-of-concept role/playbook.
|
||||
|
||||
## Target End State
|
||||
|
||||
The committed playbook is:
|
||||
|
||||
```yaml
|
||||
---
|
||||
- hosts:
|
||||
- pipeline-dnd
|
||||
roles:
|
||||
- rakestraw.dnd.pipeline
|
||||
...
|
||||
```
|
||||
|
||||
Existing connection variables remain in the host's `main.yml`. Its
|
||||
`narratio.yml` contains the following public configuration and comments:
|
||||
|
||||
```yaml
|
||||
dnd_pipeline_operator_users:
|
||||
- eric
|
||||
|
||||
dnd_pipeline_whisperx_transcribe_url: >-
|
||||
https://transcription.ai.rakestrawhome.com/transcribe
|
||||
|
||||
narratio_campaigns_default_campaign_id: dilfs
|
||||
|
||||
# Before deployment, define the four vault_dnd_* variables below in an
|
||||
# Ansible Vault-encrypted host_vars/pipeline-dnd/vault.yml file.
|
||||
# Values are not known here; leave these references unresolved until supplied.
|
||||
# Do not put API keys or object-storage credentials in this plaintext file.
|
||||
dnd_pipeline_secrets:
|
||||
AUDITA_LLM_API_KEY: "{{ vault_dnd_audita_llm_api_key }}"
|
||||
OPENROUTER_API_KEY: "{{ vault_dnd_openrouter_api_key }}"
|
||||
OBJECT_STORAGE_KEY_ID: "{{ vault_dnd_object_storage_key_id }}"
|
||||
OBJECT_STORAGE_KEY: "{{ vault_dnd_object_storage_key }}"
|
||||
```
|
||||
|
||||
The references and comments are a complete deliverable for unknown secrets.
|
||||
No encrypted file, placeholder key, or real credential is required for feature
|
||||
completion. Future operator-supplied values should be Vault-encrypted in Git;
|
||||
the suite's existing restricted runtime secrets files remain the delivery path.
|
||||
|
||||
The manifest pins the collection to
|
||||
`46913446765e86ecfb29ebad9955d579ef6a4c7e`, containing reviewed pipeline commit
|
||||
`51566afcf69083cdbb5b5718f2544c86b079001c`. Its five standalone role dependencies
|
||||
use their reviewed `v0.1.0` tags, with `ansible.posix` `2.2.0` and
|
||||
`community.general` `13.3.0`. Suite Git dependencies use public Gitea URLs;
|
||||
native collections come from Galaxy. Local sibling checkouts are not dependencies.
|
||||
|
||||
The runbook explains that the selected collection snapshot still reports
|
||||
`0.1.0`, so eventual operator installation must explicitly refresh it from
|
||||
the pinned commit if an older `0.1.0` is installed. This is an instruction
|
||||
for future use, not an operation required by this feature.
|
||||
|
||||
## Repository Ownership
|
||||
|
||||
- `rakestraw-ansible`: consumer files, comments, dependency manifest, and runbook.
|
||||
- Workspace: roadmap, staged plan, progress, and independent review record.
|
||||
- Roles, collection, applications, and assets: read-only contract references.
|
||||
|
||||
## Constraints and Settled Decisions
|
||||
|
||||
The workspace [Ansible](../../ansible.md), [testing](../../testing.md), and
|
||||
[planning](../../planning.md) policies apply. Feature 0015 has passed review
|
||||
and supplies the required suite interface.
|
||||
|
||||
- Keep playbooks minimal and settings in inventory. Preserve `pipeline-dnd`
|
||||
and its connection identity; do not rename it to `pipeline.dnd`.
|
||||
- Preserve `dilfs` using the existing standalone Narratio variable. Remove
|
||||
legacy `narratio_s3_*` and `narratio_default_campaign_id` host settings.
|
||||
- Missing operational inputs are documented, not treated as implementation
|
||||
blockers. No remote controller, SSH identity, Vault password, or session is
|
||||
required for repository preparation.
|
||||
- Local dependency installation and syntax/host-selection checks are permitted.
|
||||
Existing passing evidence may be reused when relevant inputs are unchanged.
|
||||
- `ansible-doc` rendering is not an acceptance gate. Missing role argument
|
||||
descriptions are a separate documentation improvement.
|
||||
|
||||
## Simplest Viable Implementation
|
||||
|
||||
Retain the already-committed minimal playbook, manifest, and public inventory.
|
||||
Add adjacent comments for unresolved credentials, align the runbook with the
|
||||
repository-only scope, perform appropriate local checks, and commit. No new
|
||||
Ansible task, runtime test harness, or deployment automation is needed.
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- The playbook targets only `pipeline-dnd` and applies only the collection role.
|
||||
- Host inventory expresses the known choices above and documents unknown
|
||||
credentials next to their references without real or fake secret values.
|
||||
- The dependency manifest contains the reviewed public pins and preserves
|
||||
unrelated dependencies. Public installation and local syntax/host-selection
|
||||
evidence are available; no documentation-rendering check is required.
|
||||
- The runbook distinguishes repository readiness from future deployment and
|
||||
explains how the operator supplies missing values and eventually invokes it.
|
||||
- Changes are committed in the owning repositories, with clean working trees
|
||||
for the feature's changes. No controller deployment, target convergence,
|
||||
clean-repeat run, or real pipeline execution is required or claimed.
|
||||
- Historical role/playbook files remain available.
|
||||
|
||||
## Dependencies and Follow-up Work
|
||||
|
||||
Features 0013–0015 supply public dependencies, operator enrollment, and storage
|
||||
policy. Feature 0016's consumer preparation already exists in `12b2462`, with
|
||||
initial documentation in `716a80e`; remaining work aligns comments and docs
|
||||
with the user's clarified scope.
|
||||
|
||||
The operator may later provision Vault values, install dependencies on their
|
||||
controller, deploy to the host, and validate real processing/publication.
|
||||
Those actions require a separate request and are not unfinished Feature 0016
|
||||
acceptance requirements.
|
||||
@@ -1,372 +1,193 @@
|
||||
# Implementation Plan 0014: Enroll Existing Pipeline Operators
|
||||
# Implementation Plan 0016: Prepare the D&D Deployment Playbook
|
||||
|
||||
Status: ready
|
||||
Status: in_progress
|
||||
|
||||
Target executor: GPT-5.6-Terra
|
||||
|
||||
Feature roadmap:
|
||||
`docs/roadmap/features/0014-existing-pipeline-operator-enrollment.md`
|
||||
Feature roadmap: `docs/roadmap/features/0016-deploy-dnd-pipeline-to-pipeline-dnd.md`
|
||||
|
||||
Active plan location: `docs/roadmap/implementation.md`
|
||||
|
||||
## Objective
|
||||
|
||||
Add one empty-by-default pipeline interface that enrolls explicitly selected
|
||||
existing human accounts in the suite's runtime group. Implement the behavior
|
||||
with one account-existence lookup and one additive membership task, adapt the
|
||||
existing operator integration scenario, and import the reviewed result into
|
||||
the collection from public Gitea.
|
||||
|
||||
The Feature 0014 roadmap remains authoritative for scope, rationale, target
|
||||
state, and acceptance criteria. This plan does not authorize human-account
|
||||
creation or removal, exact-list reconciliation, filesystem-policy changes, or
|
||||
a new release.
|
||||
Finish and commit repository preparation in `rakestraw-ansible`: minimal
|
||||
playbook, host variables with comments for unknown values, reviewed dependency
|
||||
pins, and concise operator instructions. This revision supersedes all previous
|
||||
deployment and live-acceptance stages. Do not execute them.
|
||||
|
||||
## Execution Rules
|
||||
|
||||
- Execute stages in numeric order.
|
||||
- Before each stage, confirm its preconditions and the working-tree state of
|
||||
its owning repository.
|
||||
- Before each stage, reread every document listed in that stage's Required
|
||||
Policy Review. Do not rely on policy recalled from an earlier stage.
|
||||
- Preserve unrelated and pre-existing changes. Stop and report unexpected
|
||||
worktree changes or branch divergence rather than overwriting them.
|
||||
- Do not make unresolved product or architecture decisions during execution;
|
||||
stop and report the decision required.
|
||||
- Keep the production behavior to the roadmap's exact direct task flow: one
|
||||
`ansible.builtin.getent` loop followed by one `ansible.builtin.user` loop.
|
||||
- Do not add an assertion, registered fact, handler, command, shell operation,
|
||||
explicit empty-list condition, membership comparison, post-change query,
|
||||
sidecar state, or account abstraction.
|
||||
- Trust `getent` to fail for a missing key and `user` to converge additive
|
||||
membership. Do not reproduce either module's checks.
|
||||
- Test the public behavior by adapting the existing temporary-operator
|
||||
scenario. Do not add a missing-user test, inspect `/etc/passwd` or
|
||||
`/etc/group`, assert task results, or duplicate the detailed test in the
|
||||
collection.
|
||||
- Run each stage's validation before committing it. Do not proceed after a
|
||||
validation failure.
|
||||
- Commit each completed stage with the stated plain-English intent. Push the
|
||||
authoritative pipeline commit before importing it into the collection.
|
||||
- Use only the public Gitea pipeline URL for the subtree update; do not import
|
||||
from the local sibling repository.
|
||||
- Execute the stages below in order, rereading each stage's listed policies.
|
||||
Check working trees first, preserve unrelated work, validate, and commit each
|
||||
completed unit with its stated plain-English intent.
|
||||
- Do not deploy to an operational controller or target, connect to the target,
|
||||
populate actual credentials, or run a real pipeline session.
|
||||
- Missing controller, SSH, Vault, and session inputs are not blockers. Express
|
||||
unknown required values through commented instructions and variable references.
|
||||
- Keep playbooks limited to hosts and roles. No new tasks, wrapper, probes,
|
||||
runtime assertions, or automated test harness is required.
|
||||
- Use existing workspace `.venv/bin` executables for local Ansible checks.
|
||||
Core 2.21.3 is installed; a missing `PATH` entry or `/etc/ansible` directory
|
||||
does not mean tooling is missing. Explicit local paths suffice.
|
||||
- Reuse passing validation whose inputs are unchanged. Optional diagnostics
|
||||
and `ansible-doc` documentation rendering are not acceptance gates.
|
||||
- Keep credential values out of plaintext files, Git, commands, and logs.
|
||||
Do not create fake keys or an empty Vault file to satisfy inventory references.
|
||||
|
||||
## Starting State and Preconditions
|
||||
## Starting State and Repository Scope
|
||||
|
||||
- Features 0009, 0011, 0012, and 0013 are complete.
|
||||
- This roadmap and implementation plan have been reviewed and committed in the
|
||||
workspace repository before execution begins.
|
||||
- The completed Feature 0013 closeout commits `eda361f` in the workspace and
|
||||
`5e695f3` in the collection are pushed to their respective `origin/main`
|
||||
branches before implementation begins. Preserve the Git-generated subtree
|
||||
squash commit that accompanies `5e695f3`.
|
||||
- `ansible-role-dnd-pipeline` starts clean on `main` at `78afffe`, equal to
|
||||
`origin/main`.
|
||||
- `ansible-collection-dnd` starts clean on `main` at `5e695f3`, equal to
|
||||
`origin/main` after the prerequisite push.
|
||||
- The workspace `.venv` supplies the tested `ansible-playbook`,
|
||||
`ansible-galaxy`, and `ansible-lint` commands used by the repository test
|
||||
drivers.
|
||||
- The local host remains the approved Ubuntu 26.04 proxy, and passwordless
|
||||
test-time sudo is available for the existing integration fixtures.
|
||||
- Git credentials can push the pipeline and collection commits to their public
|
||||
Gitea origins.
|
||||
- Feature 0015 is complete and independently reviewed.
|
||||
- Consumer `716a80e` added the initial runbook; `12b2462` committed the new
|
||||
playbook, host configuration, and dependencies. Inspect intervening commits;
|
||||
do not repeat or discard completed work.
|
||||
- Public dependency installation, target-only host listing, and syntax checking
|
||||
passed for the prepared consumer configuration. The missing option descriptions
|
||||
in three roles affect `ansible-doc` only and are not part of this feature.
|
||||
- `rakestraw-ansible` owns remaining comments/runbook changes. The workspace
|
||||
owns status and review handoff. All other repositories remain read-only.
|
||||
- No credentials, controller access, target access, or session selection is
|
||||
needed to complete this plan.
|
||||
|
||||
## Repository Scope
|
||||
## Stage 1: Finish Consumer Comments and Usage Documentation
|
||||
|
||||
- `ansible-role-dnd-pipeline` owns the variable, argument specification,
|
||||
account lookup, additive enrollment task, README, and detailed integration
|
||||
behavior.
|
||||
- `ansible-collection-dnd` receives the reviewed pipeline commit only through
|
||||
its established squashed subtree and validates the assembled artifact.
|
||||
- `ansible-workspace-dnd` records implementation progress and the review
|
||||
handoff after the two implementation repositories are complete.
|
||||
- All standalone application-role repositories, application source
|
||||
repositories, asset repositories, and `rakestraw-ansible` are out of scope
|
||||
and must remain unchanged.
|
||||
Completion: complete (consumer `7f8353d74efabb5ae408e41be9837df53010dd95`)
|
||||
|
||||
## Stage 1: Implement Existing-Operator Enrollment
|
||||
### Objective and Owning Repository
|
||||
|
||||
### Objective
|
||||
|
||||
Give the standalone pipeline role its minimal public interface for enrolling
|
||||
existing human accounts and prove the behavior through the existing operator
|
||||
integration scenario.
|
||||
Make the existing committed consumer configuration self-explanatory in
|
||||
`rakestraw-ansible` and remove live execution as a completion requirement.
|
||||
|
||||
### Required Policy Review
|
||||
|
||||
Before beginning this stage, read:
|
||||
|
||||
- `AGENTS.md`
|
||||
- `docs/orientation.md`
|
||||
- `docs/conventions.md`
|
||||
- `docs/ansible.md`
|
||||
- `docs/testing.md`
|
||||
- `docs/planning.md`
|
||||
- `docs/roadmap/features/0014-existing-pipeline-operator-enrollment.md`
|
||||
Before this stage, read `AGENTS.md`, `docs/orientation.md`,
|
||||
`docs/conventions.md`, `docs/ansible.md`, `docs/testing.md`,
|
||||
`docs/planning.md`, the Feature 0016 roadmap, and the consumer README/runbook.
|
||||
|
||||
### Preconditions
|
||||
|
||||
- The workspace planning commit is present.
|
||||
- `ansible-role-dnd-pipeline` is clean at `78afffe` on `main`, is not behind
|
||||
or diverged from `origin/main`, and contains no unrelated changes.
|
||||
- Its Narratio dependency still creates the selected runtime group before the
|
||||
pipeline role's ordinary tasks execute.
|
||||
- The existing integration driver still contains the temporary operator and
|
||||
representative group-access scenario described by the roadmap.
|
||||
|
||||
### Owning Repository
|
||||
|
||||
`ansible-role-dnd-pipeline`
|
||||
Inspect the current consumer changes and the two completed commits above.
|
||||
Local Ansible tools are available. There are no remote prerequisites.
|
||||
|
||||
### Required Changes
|
||||
|
||||
1. Add `dnd_pipeline_operator_users: []` beside the existing runtime identity
|
||||
defaults in `defaults/main.yml`.
|
||||
2. Add the matching `meta/argument_specs.yml` entry with `type: list`,
|
||||
`elements: str`, and the same required/default-backed convention used by
|
||||
the role's other public variables.
|
||||
3. Add one unprivileged `ansible.builtin.getent` task near the beginning of
|
||||
`tasks/main.yml`. Loop directly over `dnd_pipeline_operator_users`, use the
|
||||
`passwd` database and each username as `key`, and rely on the module's
|
||||
default missing-key failure. Do not register its result or add a separate
|
||||
assertion.
|
||||
4. Immediately follow it with one `become: true` `ansible.builtin.user` task
|
||||
over the same list. Set `name` to the selected account,
|
||||
`groups` to `dnd_pipeline_runtime_group`, `append: true`, and
|
||||
`state: present`. Do not set or manage any other account property.
|
||||
5. Revise the README variable reference and shared-execution section. Replace
|
||||
the consumer-owned membership example and deferred wording with the new
|
||||
interface, existing-account prerequisite, additive non-removal behavior,
|
||||
authorization consequence, and new-login-session caveat.
|
||||
6. Adapt `tests/integration` rather than adding another scenario. Create its
|
||||
temporary operator before the initial role application without membership
|
||||
in the runtime group, pass that username through
|
||||
`dnd_pipeline_operator_users` for the baseline and repeated applications,
|
||||
and retain the existing native configuration, secret-boundary, and shared
|
||||
cache exercises under that identity. Reorder cleanup as necessary so any
|
||||
final baseline convergence occurs while the selected fixture account still
|
||||
exists, then remove the fixture account.
|
||||
7. Retain the existing clean-repeat assertion. Do not add a negative
|
||||
missing-user case or assertions about module internals, task traces, the
|
||||
current default value, or raw account database entries.
|
||||
8. Run the complete existing repository validation, commit the result, push
|
||||
`main` to the public pipeline origin, and record the full pushed commit for
|
||||
Stage 2.
|
||||
1. Retain `playbooks/pipeline/dnd.yml`, which contains only the `pipeline-dnd`
|
||||
host and `rakestraw.dnd.pipeline` role. Retain reviewed dependency pins and
|
||||
known host values. Compare against the revised roadmap; correct only actual
|
||||
discrepancies, without adding suite-default copies or extra play content.
|
||||
2. In `inventory/host_vars/pipeline-dnd/narratio.yml`, add the roadmap's comments
|
||||
beside `dnd_pipeline_secrets`: define the four referenced `vault_dnd_*`
|
||||
variables in an encrypted `vault.yml` before eventual deployment. Retain
|
||||
unresolved references. Do not obtain secret values or create a Vault file.
|
||||
3. Revise `docs/pipeline-dnd.md` to describe repository preparation as the
|
||||
completed deliverable. Clearly label installation, credential provisioning,
|
||||
and deployment commands as future operator instructions, not steps the
|
||||
implementation agent must execute. Remove operational prerequisite checklists
|
||||
and session-acceptance requirements from this feature's completion path.
|
||||
4. Retain useful concise future-use instructions: the pinned collection refresh,
|
||||
Vault variable names and selection options, and minimal playbook invocation.
|
||||
Link application documentation for later pipeline operation rather than
|
||||
maintaining a live-session test script or demanding session selection now.
|
||||
5. Preserve connection inventory, unrelated dependencies, and the historical
|
||||
proof-of-concept playbook/role.
|
||||
|
||||
### Validation
|
||||
|
||||
Run from the workspace root:
|
||||
Run `git diff --check` in `rakestraw-ansible` and review the changed documentation
|
||||
and comments. Reuse the recorded successful public installation, syntax, and
|
||||
host-listing results if only comments/documentation change. No runtime tests
|
||||
or repeated dependency installation are necessary for those edits.
|
||||
|
||||
If executable configuration changes, rerun only affected local checks using
|
||||
the workspace tools and explicit local inventory/dependency paths:
|
||||
|
||||
```text
|
||||
(cd ansible-role-dnd-pipeline && tests/integration && git diff --check)
|
||||
ansible-playbook -i inventory playbooks/pipeline/dnd.yml --limit pipeline-dnd --list-hosts
|
||||
ansible-playbook -i inventory playbooks/pipeline/dnd.yml --limit pipeline-dnd --syntax-check
|
||||
```
|
||||
|
||||
Expected result: syntax and production-profile lint pass; the temporary
|
||||
existing account is enrolled through the public role variable and completes
|
||||
the existing representative operator-access scenario; native application
|
||||
checks pass; the immediate identical role application reports `changed=0`
|
||||
and `failed=0`; and no credential value appears in test output.
|
||||
Use previously installed isolated public dependencies when available. If needed,
|
||||
install from `requirements.yml` into disposable local role/collection paths.
|
||||
Do not use operational `/etc/ansible` destinations or local sibling role
|
||||
sources. Keep unresolved credentials untouched; local syntax checking does
|
||||
not claim they resolve. If unrelated encrypted inventory prevents a local
|
||||
check, use a disposable inventory containing the target alias and non-secret
|
||||
variables, and report that narrower validation scope.
|
||||
|
||||
### Completion Criteria
|
||||
|
||||
- The public interface, argument specification, two direct tasks, and README
|
||||
implement exactly the roadmap contract.
|
||||
- The existing operator scenario uses the role interface rather than fixture
|
||||
group enrollment.
|
||||
- The full standalone integration suite and `git diff --check` pass.
|
||||
- The role repository is clean after a plain-English commit.
|
||||
- The exact commit is present on public `origin/main` and its full hash is
|
||||
available for the collection import.
|
||||
The required public files exist, unknown values have explanatory comments,
|
||||
local validation evidence is sufficient, and the consumer changes are committed.
|
||||
No deployment or real credential provisioning is required.
|
||||
|
||||
### Commit
|
||||
|
||||
Commit intent: `Add optional pipeline operator enrollment`
|
||||
`Document the remaining D&D deployment inputs`.
|
||||
|
||||
## Stage 2: Import the Reviewed Pipeline Change
|
||||
## Stage 2: Record the Repository Preparation Handoff
|
||||
|
||||
### Objective
|
||||
Completion: complete (handoff recorded by this change)
|
||||
|
||||
Expose the reviewed operator interface through `rakestraw.dnd.pipeline` while
|
||||
preserving the authoritative-standalone-repository and collection-artifact
|
||||
boundaries.
|
||||
### Objective and Owning Repository
|
||||
|
||||
Record Feature 0016 implementation readiness for independent review in the
|
||||
workspace, scoped entirely to committed repository content.
|
||||
|
||||
### Required Policy Review
|
||||
|
||||
Before beginning this stage, read:
|
||||
|
||||
- `AGENTS.md`
|
||||
- `docs/orientation.md`
|
||||
- `docs/conventions.md`
|
||||
- `docs/ansible.md`
|
||||
- `docs/testing.md`
|
||||
- `docs/planning.md`
|
||||
- `docs/roadmap/features/0014-existing-pipeline-operator-enrollment.md`
|
||||
- `ansible-collection-dnd/README.md`
|
||||
Before this stage, read `AGENTS.md`, `docs/orientation.md`,
|
||||
`docs/conventions.md`, `docs/testing.md`, `docs/planning.md`, and the revised
|
||||
Feature 0016 roadmap.
|
||||
|
||||
### Preconditions
|
||||
|
||||
- Stage 1 is complete and its exact commit is publicly reachable from
|
||||
`https://gitea.maximumdirect.net/eric/ansible-role-dnd-pipeline.git`.
|
||||
- `ansible-collection-dnd` is clean at `5e695f3` on `main`, is not behind or
|
||||
diverged from `origin/main`, and contains no unrelated changes.
|
||||
- The collection's current `roles/pipeline` subtree still corresponds to the
|
||||
authoritative pipeline history through `78afffe`.
|
||||
|
||||
### Owning Repository
|
||||
|
||||
`ansible-collection-dnd`
|
||||
Stage 1 has passed local validation and is committed. No remote prerequisites.
|
||||
|
||||
### Required Changes
|
||||
|
||||
1. Set a shell variable to the full pushed Stage 1 commit hash and run the
|
||||
documented remote-only update from the collection repository:
|
||||
1. Update `docs/status.md` and this plan's progress notes with the consumer
|
||||
commit and local validation evidence. Describe real deployment as separately
|
||||
deferred operator work, not a blocker or incomplete acceptance criterion.
|
||||
2. Keep Feature 0016 `in_progress` pending independent review. Report any actual
|
||||
repository issues; do not mark it `done` solely because implementation ended.
|
||||
3. Confirm the feature's owning working trees are clean after committing.
|
||||
Preserve any unrelated changes and identify them instead of discarding them.
|
||||
|
||||
```text
|
||||
pipeline_revision=<FULL_PUSHED_STAGE_1_COMMIT>
|
||||
git subtree pull --prefix=roles/pipeline \
|
||||
https://gitea.maximumdirect.net/eric/ansible-role-dnd-pipeline.git \
|
||||
"$pipeline_revision" --squash -m "Import pipeline operator enrollment"
|
||||
```
|
||||
### Validation and Completion Criteria
|
||||
|
||||
2. Do not edit role behavior directly beneath `roles/pipeline`. The subtree
|
||||
import should carry the new default, argument specification, tasks, README,
|
||||
and standalone tests faithfully.
|
||||
3. Do not change `galaxy.yml`, dependency manifests, collection-level tests,
|
||||
or release documentation. No dependency or published version changes in
|
||||
this feature, and the detailed behavior remains covered by Stage 1.
|
||||
4. Run the existing collection integration suite against the built and
|
||||
installed artifact. Commit no additional files after the subtree command
|
||||
unless validation exposes a genuine collection-boundary issue within this
|
||||
feature's scope.
|
||||
5. Push the resulting collection history to `origin/main` and record the
|
||||
resulting merge commit and Git-generated squashed subtree commit.
|
||||
|
||||
### Validation
|
||||
|
||||
Run from the workspace root:
|
||||
|
||||
```text
|
||||
(cd ansible-collection-dnd && tests/integration && git diff --check)
|
||||
```
|
||||
|
||||
Expected result: the collection builds; the archive contains the updated
|
||||
pipeline role while retaining its development-file exclusions; isolated
|
||||
dependency and artifact installation succeeds; the FQCN resolves; native
|
||||
representative application checks pass; and the immediate repeat reports
|
||||
`changed=0` and `failed=0`.
|
||||
|
||||
### Completion Criteria
|
||||
|
||||
- Git history records a squashed subtree import from the exact public Stage 1
|
||||
commit with the stated plain-English merge message.
|
||||
- No pipeline behavior was authored directly in the collection.
|
||||
- The existing collection suite and `git diff --check` pass without an
|
||||
unnecessary duplicate operator test.
|
||||
- The collection is clean and its new history is present on `origin/main`.
|
||||
Run `git diff --check`. Evidence points to committed consumer content and
|
||||
appropriate local checks. The handoff makes no claims of live deployment or
|
||||
runtime acceptance. No paid request or host access is necessary.
|
||||
|
||||
### Commit
|
||||
|
||||
Commit intent: `Import pipeline operator enrollment`
|
||||
|
||||
The subtree command creates the stage's merge commit and its generated squash
|
||||
commit; do not add an empty or redundant manual commit.
|
||||
|
||||
## Stage 3: Record the Implementation Handoff
|
||||
|
||||
### Objective
|
||||
|
||||
Record the completed implementation evidence without marking Feature 0014
|
||||
done before independent review.
|
||||
|
||||
### Required Policy Review
|
||||
|
||||
Before beginning this stage, read:
|
||||
|
||||
- `AGENTS.md`
|
||||
- `docs/orientation.md`
|
||||
- `docs/conventions.md`
|
||||
- `docs/planning.md`
|
||||
- `docs/roadmap/features/0014-existing-pipeline-operator-enrollment.md`
|
||||
|
||||
### Preconditions
|
||||
|
||||
- Stages 1 and 2 are complete, committed, pushed, and validated.
|
||||
- Their full commit hashes and validation results are available.
|
||||
- The workspace repository is clean apart from the expected progress-record
|
||||
edits for this stage and is not behind or diverged from `origin/main`.
|
||||
|
||||
### Owning Repository
|
||||
|
||||
`ansible-workspace-dnd`
|
||||
|
||||
### Required Changes
|
||||
|
||||
1. Change Feature 0014's roadmap status and status-table state to
|
||||
`in_progress`, indicating implementation is complete but independent review
|
||||
remains outstanding.
|
||||
2. Update the workspace current focus and Feature 0014 evidence with the exact
|
||||
pipeline and collection commit hashes and the validation completed.
|
||||
3. Replace this plan's Progress Notes with a concise record of the three stage
|
||||
states and their commits. Do not copy command logs into the repository.
|
||||
4. Do not mark the feature `done`, remove the active plan, or create
|
||||
`docs/roadmap/remediation.md`; those are review and closeout decisions.
|
||||
5. Commit the workspace handoff record with the stated plain-English message.
|
||||
|
||||
### Validation
|
||||
|
||||
Run from the workspace root:
|
||||
|
||||
```text
|
||||
./scripts/status
|
||||
git diff --check
|
||||
```
|
||||
|
||||
Expected result: every repository is clean; the pipeline and collection are
|
||||
not ahead of or behind their origins; only the workspace handoff commit may be
|
||||
ahead of `origin/main`; and the dashboard points to the exact implementation
|
||||
evidence while retaining independent review as the next step.
|
||||
|
||||
### Completion Criteria
|
||||
|
||||
- Workspace status accurately identifies Feature 0014 as implemented and
|
||||
awaiting review.
|
||||
- The roadmap remains the durable target-state record and is not polluted with
|
||||
execution logs.
|
||||
- The active plan records concise implementation evidence and contains no
|
||||
false completion claim.
|
||||
- The workspace repository is clean after the handoff commit.
|
||||
|
||||
### Commit
|
||||
|
||||
Commit intent: `Record pipeline operator enrollment implementation`
|
||||
`Record D&D playbook preparation for review`.
|
||||
|
||||
## Final Validation and Handoff
|
||||
|
||||
- Confirm Stage 1's complete standalone integration result and Stage 2's
|
||||
complete collection integration result.
|
||||
- Run `./scripts/status` and confirm every repository is clean, with no
|
||||
unexpected branch divergence or untracked artifact.
|
||||
- Report the full pipeline, collection merge, collection subtree-squash, and
|
||||
workspace handoff commit hashes.
|
||||
- Summarize any deviation from this plan, unresolved finding, or added
|
||||
implementation complexity and identify the acceptance criterion that
|
||||
required it.
|
||||
- Do not mark Feature 0014 `done`; implementation results require independent
|
||||
review first.
|
||||
Report owning commits, changed consumer files, comments explaining missing
|
||||
values, and local validation performed or reused. Feature review evaluates the
|
||||
committed playbook/configuration, not operational deployment. No push is
|
||||
required unless separately requested.
|
||||
|
||||
## Progress Notes
|
||||
|
||||
- Stage 1 complete: `ansible-role-dnd-pipeline`
|
||||
`c13f726a0a846180a1aa95d110b8bbc801edaae5` implements optional enrollment
|
||||
of existing operators; the complete standalone integration suite passed and
|
||||
the commit is public on `origin/main`.
|
||||
- Stage 2 complete: `ansible-collection-dnd` merge
|
||||
`c0b0ebd13e92199f3d94e138c6cc50d1ece5dadd` imports the exact public pipeline
|
||||
change through squash `d292297b6ad179e4664c01ad07cddaefff12f769`;
|
||||
the complete collection integration suite passed and the history is public
|
||||
on `origin/main`.
|
||||
- Stage 3 complete: `ansible-workspace-dnd`
|
||||
`bf81461de60001a0d8050e560383c6ddf533209f` records the implementation
|
||||
evidence while Feature 0014 remains `in_progress` for independent review.
|
||||
- Deviations: none.
|
||||
- Consumer `716a80e` added the initial runbook, and `12b2462` committed the
|
||||
minimal playbook, public host choices, and reviewed dependency manifest.
|
||||
Public dependency installation from the mixed manifest succeeded, host
|
||||
listing selected only `pipeline-dnd`, and syntax checking passed.
|
||||
- Stage 1 is complete in consumer commit
|
||||
`7f8353d74efabb5ae408e41be9837df53010dd95`. The plaintext inventory now
|
||||
explains its unresolved Vault references, and the runbook distinguishes the
|
||||
completed repository preparation from future operator installation,
|
||||
credential provisioning, deployment, and application operation.
|
||||
- Stage 1 changed only comments and documentation, so the prior installation,
|
||||
host-listing, and syntax evidence was reused as directed. `git diff --check`
|
||||
passed, and the consumer working tree is clean.
|
||||
- Stage 2 records the implementation handoff for independent review. No live
|
||||
deployment, target validation, credential provisioning, or pipeline session
|
||||
was performed or is claimed by Feature 0016.
|
||||
|
||||
## Open Questions
|
||||
|
||||
None required for this repository-only implementation. Unknown API keys and
|
||||
future operational inputs are documented for the operator, not requested now.
|
||||
|
||||
@@ -370,6 +370,15 @@ application-role releases in one source repository.
|
||||
- The Narratio `testing` profile keeps the Notarius stage enabled and inherits
|
||||
the same D&D pipeline and extraction profile as production until a distinct
|
||||
testing extraction policy is selected.
|
||||
- The D&D suite selects its Backblaze-compatible S3 archive, typed
|
||||
`OBJECT_STORAGE_KEY_ID` and `OBJECT_STORAGE_KEY` references, maximum S3 audio
|
||||
caching, run publication, human-facing transcript and artifact outputs, and
|
||||
cleanup of successfully published workspace state. Feature 0015 owns the
|
||||
exact overrideable defaults and publication set.
|
||||
- Accepted Notarius lane outputs remain addressable inside Narratio as
|
||||
`narratio.extraction.<output_key>` sources. Future analyze-stage Scriptorium
|
||||
artifacts may consume those sources directly; publishing an extraction lane
|
||||
to a stable remote destination is an independent policy choice.
|
||||
|
||||
### Execution model
|
||||
|
||||
@@ -389,7 +398,7 @@ later design decision changes them:
|
||||
- `/usr/local/etc/notarius/pipelines` for suite-owned imported Notarius
|
||||
pipeline definitions
|
||||
- `/usr/local/etc/scriptorium/config.yml` for Scriptorium configuration
|
||||
- `/usr/local/etc/narratio/secrets` for future restricted credential material
|
||||
- `/usr/local/etc/narratio/secrets` for suite-managed restricted credentials
|
||||
- `/usr/local/share/narratio/campaigns` for version-controlled campaign data
|
||||
- `/usr/local/share/narratio/prompt-library` for version-controlled prompts
|
||||
and schemas
|
||||
|
||||
@@ -1,10 +1,13 @@
|
||||
# Workspace Status
|
||||
|
||||
Last updated: 2026-09-18
|
||||
Last updated: 2026-09-21
|
||||
|
||||
Current milestone: Existing pipeline operator enrollment
|
||||
Current milestone: Operational D&D pipeline deployment
|
||||
|
||||
Current focus: Independently review the Feature 0014 implementation.
|
||||
Current focus: Independently review the completed Feature 0016 repository
|
||||
preparation. Controller deployment, target convergence, credential provisioning,
|
||||
and session execution are separate operator work and are not claimed here.
|
||||
Feature 0015 is complete after independent review found no remediation issue.
|
||||
|
||||
| ID | Deliverable | Owner | State | Evidence or next step |
|
||||
| --- | --- | --- | --- | --- |
|
||||
@@ -30,7 +33,9 @@ Current focus: Independently review the Feature 0014 implementation.
|
||||
| F0011 | Deliver pipeline credentials and Notarius reuse | ansible-role-narratio and ansible-role-dnd-pipeline | done | Implemented and independently reviewed through remediation commit `de1b191` |
|
||||
| F0012 | Assemble the D&D collection | ansible-collection-dnd and ansible-role-dnd-pipeline | done | Pipeline `600eb21` assembled and validated through collection remediation commit `da67b34` |
|
||||
| F0013 | Publish the first coordinated suite release | all Ansible repositories and workspace | done | Public [`rakestraw.dnd` 0.1.0 release](https://gitea.maximumdirect.net/eric/ansible-collection-dnd/releases/tag/v0.1.0); documentation remediation completed in pipeline `78afffe` and collection `5e695f3` |
|
||||
| F0014 | Enroll existing pipeline operators | ansible-role-dnd-pipeline, ansible-collection-dnd, and workspace | in_progress | Review pipeline `c13f726a0a846180a1aa95d110b8bbc801edaae5` and collection `c0b0ebd13e92199f3d94e138c6cc50d1ece5dadd`; both complete integration suites passed |
|
||||
| F0014 | Enroll existing pipeline operators | ansible-role-dnd-pipeline, ansible-collection-dnd, and workspace | done | Pipeline `c13f726` and collection import `c0b0ebd` passed their complete integration suites and independent review with no remediation findings |
|
||||
| F0015 | Enable D&D S3 storage and publishing | ansible-role-dnd-pipeline, ansible-collection-dnd, and workspace | done | Pipeline `51566afcf69083cdbb5b5718f2544c86b079001c`, collection fixture `cf3de016ee09edf91afc3bc1addfd926d4a172ad`, collection squash `77f67adbc0f700e88bc15554e02052eaa22f1033`, and collection merge `46913446765e86ecfb29ebad9955d579ef6a4c7e` passed the complete repository suites and independent review with no remediation findings |
|
||||
| F0016 | Prepare the D&D deployment playbook for `pipeline-dnd` | rakestraw-ansible and workspace | in_progress | Consumer preparation is in `716a80e`, `12b2462`, and `7f8353d`; public dependency installation, target-only host listing, syntax checking, and whitespace validation passed; independent review is next |
|
||||
| A0001 | Audit completed standalone-role test suites | workspace and standalone roles | done | Findings recorded in `docs/roadmap/test-suite-audit-findings.md` |
|
||||
| R0001 | Simplify completed standalone-role test suites | workspace and standalone roles | done | Role commits `e08c63c`, `d2d0b9d`, and `cfecf37` passed implementation and independent review |
|
||||
|
||||
|
||||
Reference in New Issue
Block a user