eric/ansible-collection-dnd
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 1 Wiki Activity

Merge commit '5dbae8ab055d88ddcc61dc8ac52271fe261f67b5' as 'roles/pipeline'

Browse Source
This commit is contained in:
Eric Rakestraw
2026-09-17 03:30:23 +00:00
parent 3f7f82e40f 5dbae8ab05
commit c84aa0db8f
21 changed files with 1782 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

332
roles/pipeline/README.md Normal file
View File

@@ -0,0 +1,332 @@
# rakestraw.dnd_pipeline
Installs the D&D transcript-processing application suite by composing the
standalone Seriatim, Audita, Notarius, Scriptorium, and Narratio roles. It
supports Ubuntu 26.04 and Ansible Core 2.18 or newer. Downstream roles perform
their own task-level privilege escalation, so a play does not need `become`.
Supported host architectures are `amd64` and `arm64`.
## Installation
Install the role dependencies and their collection before applying this role:
```sh
ansible-galaxy role install -r requirements.yml
ansible-galaxy collection install -r collections/requirements.yml
```
The dependency manifest installs the roles under their canonical names:
`rakestraw.seriatim`, `rakestraw.audita`, `rakestraw.notarius`,
`rakestraw.scriptorium`, and `rakestraw.narratio`. This standalone role is
named `rakestraw.dnd_pipeline`.
```yaml
---
- name: Install the D&D pipeline foundation
hosts: pipeline_hosts
gather_facts: true
roles:
- role: rakestraw.dnd_pipeline
vars:
dnd_pipeline_whisperx_transcribe_url: https://whisperx.example/transcribe
```
The collection source exposes a reviewed snapshot of this role as
`rakestraw.dnd.pipeline`:
```yaml
---
- name: Install the D&D pipeline from the collection
hosts: pipeline_hosts
gather_facts: true
roles:
- role: rakestraw.dnd.pipeline
vars:
dnd_pipeline_whisperx_transcribe_url: https://whisperx.example/transcribe
```
Collection consumers should follow the unified bootstrap instructions in the
[`rakestraw.dnd` collection repository](https://gitea.maximumdirect.net/eric/ansible-collection-dnd).
This standalone repository remains authoritative for role changes; collection
maintainers import a reviewed commit as a snapshot. This does not imply that a
collection release has been published or tagged.
## Variables
`dnd_pipeline_whisperx_transcribe_url` is required and is passed to Narratio.
The compatible application versions are selected through these variables:
| Variable | Default |
| --- | --- |
| `dnd_pipeline_audita_version` | `v0.9.1` |
| `dnd_pipeline_narratio_version` | `v2.1.0` |
| `dnd_pipeline_notarius_version` | `v0.7.0` |
| `dnd_pipeline_seriatim_version` | `v1.5.0` |
| `dnd_pipeline_scriptorium_version` | `v0.13.0` |
| `dnd_pipeline_manage_build_dependencies` | `true` |
| `dnd_pipeline_go_snap_channel` | `1.26/stable` |
| `dnd_pipeline_go_binary` | `/snap/bin/go` |
| `dnd_pipeline_campaigns_repository_url` | `https://gitea.maximumdirect.net/eric/campaigns.git` |
| `dnd_pipeline_campaigns_revision` | `main` |
| `dnd_pipeline_campaigns_root` | `/usr/local/share/narratio/campaigns` |
| `dnd_pipeline_prompt_library_repository_url` | `https://gitea.maximumdirect.net/eric/narratio-prompts.git` |
| `dnd_pipeline_prompt_library_revision` | `main` |
| `dnd_pipeline_prompt_library_root` | `/usr/local/share/narratio/prompt-library` |
The common dependency-management switch controls whether every application
role installs Git and Go. Callers that disable it must provide both Git and Go.
The Go channel and executable are likewise passed to all five roles. Advanced
callers may supply unmapped standalone-role variables directly through normal
Ansible variable precedence.
The following values are deliberate D&D suite policy passed through the
standalone roles' generic configuration interfaces; they are not standalone
defaults. Narratio runs first and establishes the selected runtime user and
group before the remaining configuration-bearing dependencies run.
| Variable | Default policy |
| --- | --- |
| `dnd_pipeline_runtime_user` | `narratio` |
| `dnd_pipeline_runtime_group` | `narratio` |
| `dnd_pipeline_config_directory_mode` | `0755` |
| `dnd_pipeline_config_file_mode` | `0644` |
All application configuration, including pipeline-owned Narratio fragments,
is therefore `narratio:narratio`, world-readable, and Ansible-managed. It is
not an operator-editable configuration surface.
Narratio receives the standard Seriatim, Audita, Notarius, and Scriptorium
binary and configuration paths managed by the composed roles. Notarius is
enabled with the suite-owned `dnd-session` extraction pipeline.
The role deploys the campaign repository directly at the campaign root and
the prompt library directly at the prompt-library root. By default both follow
their `main` branches; callers can select an immutable commit, tag, or another
branch through the revision variables. Narratio receives the deployed campaign
root, while Scriptorium receives its `prompts/` and `schemas/` children.
### D&D processing policy
The variables below are deliberate D&D suite policy, not a copy of application
defaults. They remain ordinary inventory variables, so callers can replace a
chosen policy value when their deployment requires it.
| Variable group | Default policy |
| --- | --- |
| `dnd_pipeline_audita_pipeline_modules` | `glossary`, `homophones`, `glossary`, `spoken_word`, `grammar` |
| `dnd_pipeline_audita_output_schema` | `audita-v1` |
| `dnd_pipeline_audita_llm_proposal_*` | `https://inference.ai.rakestrawhome.com/v1`, `google/gemma-4-31b-it`, `AUDITA_LLM_API_KEY`, `900s`, `3` |
| `dnd_pipeline_audita_llm_validation_*` | `https://inference.ai.rakestrawhome.com/v1`, `google/gemma-4-31b-it`, `AUDITA_LLM_API_KEY`, `900s`, `3` |
| `dnd_pipeline_audita_concurrency_*` | total, proposal, and validation concurrency of `4` |
| `dnd_pipeline_audita_chunking_*` | `8` target sections, `6144` maximum and `2048` minimum section tokens |
| `dnd_pipeline_audita_normalization_*` | `3s` maximum and ellipsis gaps, `240s` maximum duration, `2048` maximum tokens |
| `dnd_pipeline_audita_thresholds_*` | `0.8` for glossary, homophones, spoken word, and grammar |
| `dnd_pipeline_audita_context_description` | D&D-game transcript context, including possible pre- and post-session banter |
| `dnd_pipeline_audita_diagnostics_*` | `/tmp/audita`, retained automatically |
Audita's proposal and validation settings independently remain overrideable,
but both default to the one typed `AUDITA_LLM_API_KEY` reference Narratio can
forward. Audita owns its endpoint, model, tuning, and diagnostics configuration;
Narratio receives its managed Audita configuration path and supplies the
run-specific output path.
| Variable | Default |
| --- | --- |
| `dnd_pipeline_narratio_seriatim_output_schema` | `seriatim-intermediate` |
| `dnd_pipeline_narratio_normalize_output_path` | `transcripts/final.json` |
| `dnd_pipeline_narratio_normalize_output_schema` | `seriatim-intermediate` |
| `dnd_pipeline_narratio_trim_enabled` | `true` |
| `dnd_pipeline_narratio_trim_output_path` | `transcripts/final.trimmed.json` |
| `dnd_pipeline_narratio_trim_bounds_prompt_id` | `dnd.session_bounds` |
| `dnd_pipeline_narratio_trim_bounds_transcript_input_name` | `transcript` |
| `dnd_pipeline_narratio_trim_bounds_output_path` | `artifacts/session_bounds.json` |
| `dnd_pipeline_narratio_render_enabled` | `true` |
| `dnd_pipeline_narratio_render_format` | `markdown` |
| `dnd_pipeline_narratio_testing_scriptorium_profile` | `gemini-flash-latest` |
`dnd_pipeline_audita_output_schema` is also the Narratio Audita-adapter schema;
it is the shared cross-application boundary. Other Narratio-local controls,
including reports, timeouts, debug rendering, presentation, and bounds prompt
profile selection, remain Narratio defaults.
### Artifacts and profiles
The role owns these complete native mapping variables. Replacing one mapping
changes that source's configuration as a whole; it does not change the fixed
file name, import, or source key. Set `enabled: false` to retain an available
source while excluding it from normal analysis.
| Variable | Fixed source | Default state | Output |
| --- | --- | --- | --- |
| `dnd_pipeline_narratio_session_recap_artifact` | `session_recap` | enabled | `artifacts/session_recap.md` |
| `dnd_pipeline_narratio_hostile_rumors_artifact` | `hostile_rumors` | enabled | `artifacts/hostile_rumors.md` |
| `dnd_pipeline_narratio_deranged_rumors_artifact` | `deranged_rumors` | enabled | `artifacts/deranged_rumors.md` |
| `dnd_pipeline_narratio_scene_map_artifact` | `scene_map` | disabled | `artifacts/scene_map.json` |
| `dnd_pipeline_narratio_character_analysis_family` | `character_meta_analysis` | disabled | `artifacts/character_meta_analysis/{character_id}.md` |
The character-analysis family expands over campaign `party.characters` and
passes each member's name and class summary to its prompt. Recap and both rumor
sources are enabled; the rumor input derives its recap dependency without a
redundant explicit ordering edge. Production artifacts inherit each prompt's
declared profile.
Narratio's root file remains owned by the standalone role. It imports the
suite-owned base fragments in this fixed order:
```text
/usr/local/etc/narratio/
├── pipeline.yml
├── conf.d/processing.yml
├── artifacts/session-recap.yml
├── artifacts/hostile-rumors.yml
├── artifacts/deranged-rumors.yml
├── artifacts/scene-map.yml
├── artifact-families/character-meta-analysis.yml
└── profiles/testing.yml
```
The `testing` profile is an overlay, not a second configuration tree. It only
sets the Scriptorium `profile_id` for the four artifact sources and the one
family. Its default, `gemini-flash-latest`, is an inexpensive development
choice, not a compatibility guarantee. Replace it through inventory when a
different testing profile is appropriate:
```yaml
dnd_pipeline_narratio_testing_scriptorium_profile: endpoint-profile
```
### Notarius extraction policy
The Notarius root configuration imports the fixed
`pipelines/dnd-session.yml` fragment. The suite owns that fragment at
`/usr/local/etc/notarius/pipelines/dnd-session.yml`; callers do not edit it.
| Variable | Default policy |
| --- | --- |
| `dnd_pipeline_notarius_pipelines` | Complete native map containing `dnd-session` |
| `dnd_pipeline_notarius_cache_root` | `/var/cache/notarius` |
| `dnd_pipeline_notarius_cache_chunk_plans_mode` | `auto` |
| `dnd_pipeline_notarius_cache_checkpoints_enabled` | `true` |
| `dnd_pipeline_narratio_notarius_pipeline_id` | `dnd-session` |
| `dnd_pipeline_narratio_notarius_llm_api_key` | `OPENROUTER_API_KEY` |
| `dnd_pipeline_narratio_notarius_resume` | `true` |
| `dnd_pipeline_narratio_notarius_references` | `party`, `players`, and `glossary` prepared sources |
| `dnd_pipeline_narratio_notarius_outputs` | Complete native map of ten accepted D&D lanes |
The pipeline, reference, and output values are complete native mappings.
Inventory can replace them or compose additions with Jinja's `combine` filter;
a replacement pipeline map requires a matching selected pipeline ID.
Narratio materializes the `party`, `players`, and `glossary` references for each
invocation. No campaign filesystem path appears in Notarius configuration. The
optional `spell_catalog` reference remains absent until a campaign supplies
that input. The accepted sources are
`narratio.extraction.item_registry`, `npc_registry`, `location_registry`,
`scene_descriptions`, `item_occurrences`, `spells`, `combat_turns`,
`npc_occurrences`, `location_occurrences`, and `enemy_events`.
Narratio forwards the typed `OPENROUTER_API_KEY` reference to the selected
Notarius profile and ordinary extraction requests checkpoint reuse with
`resume: true`. Chunk plans use Notarius's root-owned `auto` policy; the
pipeline deliberately leaves Narratio's optional `chunk_cache` override unset.
The `chunk-plans` and `checkpoints` paths derive from the cache root and are
created by Notarius during use. Base and `testing` configurations select the
same `dnd-extraction` profile and Notarius pipeline; the testing overlay changes
only Scriptorium profiles, so a real testing run may invoke Notarius. The
integration suite performs offline configuration inspection only.
### Inspecting configuration
The following native Narratio commands resolve configuration without running
transcription or requesting a model provider. Select a deployed campaign for
the party-driven character family:
```sh
sudo -u narratio narratio config validate \
--config /usr/local/etc/narratio/pipeline.yml --campaign centaurs
sudo -u narratio narratio config validate \
--config /usr/local/etc/narratio/pipeline.yml --campaign centaurs --profile testing
sudo -u narratio narratio config artifacts \
--config /usr/local/etc/narratio/pipeline.yml --campaign centaurs
sudo -u narratio narratio config diff default testing \
--config /usr/local/etc/narratio/pipeline.yml --campaign centaurs
```
The first two commands validate production and testing resolution. Artifact
inspection shows enabled sources, inferred dependencies, and generated family
members; the diff shows the overlay's profile selections without modifying
managed files.
Storage and publishing, operator selection and group-membership management,
scheduling, and end-to-end external execution remain deferred.
### Shared execution filesystem
`/var/lib/narratio`, `/var/spool/narratio`, `/var/cache/narratio`, and
`/var/cache/notarius` are
`narratio:narratio` mode `2775` **setgid** directories. Their access and
default ACLs retain owner and group write access plus world read/traverse access
for newly created content. The pipeline may run as `narratio` or as a human
account already in the selected runtime group; the production role neither
selects people nor manages their group membership. For example, a consumer
play can manage its selected operator separately:
```yaml
- name: Add the selected operator to the pipeline group
become: true
ansible.builtin.user:
name: "{{ pipeline_operator }}"
groups:
- "{{ dnd_pipeline_runtime_group }}"
append: true
```
Configuration, transcripts, caches, generated artifacts, campaign data,
prompts, and logs are non-secret D&D content. Credentials use the restricted
`/usr/local/etc/narratio/secrets` subtree, owned by the selected runtime user
and group with mode `0750`; managed files use mode `0640`. Runtime-group
membership authorizes use of these shared credentials, while the subtree grants
no world access. Ordinary configuration remains world-readable and contains
only reference names.
### Managed credentials
`dnd_pipeline_secrets` defaults to `{}`. Its keys must be valid environment
variable names, and each string value is managed as the exact named credential
file. Omitted names are unmanaged: the role neither mirrors nor purges the
directory. Narratio loads this conventional sibling directory before
configuration resolution; an already-set process environment value takes
precedence. The role adds no wrapper, service, or parallel credential loader.
Keep values in encrypted inventory, for example with Ansible Vault; Vault is a
consumer workflow, not role functionality:
```yaml
dnd_pipeline_secrets:
AUDITA_LLM_API_KEY: "{{ vault_dnd_audita_llm_api_key }}"
OPENROUTER_API_KEY: "{{ vault_dnd_openrouter_api_key }}"
```
Scriptorium profiles continue to own their `api_key_env` selection. To supply
that credential through the suite, add a `dnd_pipeline_secrets` entry whose
name matches the selected profile's `api_key_env`. Narratio's secrets bootstrap
loads the value, and its existing Scriptorium adapter forwards only the
profile-selected credential.
## Validation
`tests/integration` installs the pinned Git dependencies and collection into
private temporary paths, runs syntax and production-profile lint checks, then
applies the suite with a local syntactically valid WhisperX URL and explicit
immutable `file://` asset inputs. It uses native application
smoke/configuration commands, validates Notarius's imported pipeline and
effective cache policy, and validates Narratio for both deployed campaigns and
the testing overlay. It inspects enabled Notarius wiring through Narratio's
native effective configuration, validates the same imported Notarius pipeline
as a temporary existing-group-member fixture, exercises representative shared
cache access across the service and fixture identities, and confirms that
`nobody` can read ordinary configuration. It also composes an alternate pair of
version-5 pipeline maps through the public interface and requires clean repeats
for the baseline and override cases. Cleanup reapplies the asset-enabled
baseline suite and removes only the fixture account, its named test content,
and the driver's private temporary directory.