ansible-role-dnd-pipeline/README.md

rakestraw.dnd_pipeline

Installs the D&D transcript-processing application suite by composing the standalone Seriatim, Audita, Notarius, Scriptorium, and Narratio roles. It supports Ubuntu 26.04 and Ansible Core 2.18 or newer. Downstream roles perform their own task-level privilege escalation, so a play does not need become. Supported host architectures are amd64 and arm64.

Installation

Install the role dependencies and their collection before applying this role:

ansible-galaxy role install -r requirements.yml
ansible-galaxy collection install -r collections/requirements.yml

The dependency manifest installs the roles under their canonical names: rakestraw.seriatim, rakestraw.audita, rakestraw.notarius, rakestraw.scriptorium, and rakestraw.narratio. It selects the compatible v0.1.0 releases where compatible and public commit pins for Narratio and Notarius when the suite needs newer interfaces. This standalone role is named rakestraw.dnd_pipeline.

---
- name: Install the D&D pipeline foundation
  hosts: pipeline_hosts
  gather_facts: true
  roles:
    - role: rakestraw.dnd_pipeline
      vars:
        dnd_pipeline_whisperx_transcribe_url: https://whisperx.example/transcribe

The collection source exposes a reviewed snapshot of this role as rakestraw.dnd.pipeline:

---
- name: Install the D&D pipeline from the collection
  hosts: pipeline_hosts
  gather_facts: true
  roles:
    - role: rakestraw.dnd.pipeline
      vars:
        dnd_pipeline_whisperx_transcribe_url: https://whisperx.example/transcribe

Collection consumers should follow the unified bootstrap instructions in the rakestraw.dnd collection repository. This standalone repository remains authoritative for role changes; collection maintainers import a reviewed, pushed commit or immutable tag as a snapshot. Collection releases are published independently and are listed in the collection repository.

Variables

dnd_pipeline_whisperx_transcribe_url is required and is passed to Narratio. The compatible application versions are selected through these variables:

Variable	Default
dnd_pipeline_audita_version	v2.0.1
dnd_pipeline_narratio_version	v3.1.0
dnd_pipeline_notarius_version	v0.8.1
dnd_pipeline_seriatim_version	v1.5.0
dnd_pipeline_scriptorium_version	v0.15.0
dnd_pipeline_manage_build_dependencies	true
dnd_pipeline_go_snap_channel	1.26/stable
dnd_pipeline_go_binary	/snap/bin/go
dnd_pipeline_campaigns_repository_url	https://gitea.maximumdirect.net/eric/campaigns.git
dnd_pipeline_campaigns_revision	main
dnd_pipeline_campaigns_root	/usr/local/share/narratio/campaigns
dnd_pipeline_prompt_library_repository_url	https://gitea.maximumdirect.net/eric/narratio-prompts.git
dnd_pipeline_prompt_library_revision	main
dnd_pipeline_prompt_library_root	/usr/local/share/narratio/prompt-library

The common dependency-management switch controls whether every application role installs Git and Go. Callers that disable it must provide both Git and Go. The Go channel and executable are likewise passed to all five roles. Advanced callers may supply unmapped standalone-role variables directly through normal Ansible variable precedence.

The following values are deliberate D&D suite policy passed through the standalone roles' generic configuration interfaces; they are not standalone defaults. Narratio runs first and establishes the selected runtime user and group before the remaining configuration-bearing dependencies run.

Variable	Default policy
dnd_pipeline_runtime_user	narratio
dnd_pipeline_runtime_group	narratio
dnd_pipeline_operator_users	[]
dnd_pipeline_config_directory_mode	0755
dnd_pipeline_config_file_mode	0644

All application configuration, including pipeline-owned Narratio fragments, is therefore narratio:narratio, world-readable, and Ansible-managed. It is not an operator-editable configuration surface.

Narratio receives the standard Seriatim, Audita, Notarius, and Scriptorium binary and configuration paths managed by the composed roles. Notarius is enabled with the suite-owned dnd-session extraction pipeline.

The role deploys the campaign repository directly at the campaign root and the prompt library directly at the prompt-library root. By default both follow their main branches; callers can select an immutable commit, tag, or another branch through the revision variables. Narratio receives the deployed campaign root, while Scriptorium receives its prompts/ and schemas/ children.

D&D processing policy

The variables below are deliberate D&D suite policy, not a copy of application defaults. They remain ordinary inventory variables, so callers can replace a chosen policy value when their deployment requires it.

Variable group	Default policy
dnd_pipeline_audita_pipeline_modules	glossary, homophones, glossary, spoken_word, grammar
dnd_pipeline_audita_output_schema	audita-v1
dnd_pipeline_audita_backends	Native dnd backend at https://inference.ai.rakestrawhome.com/v1, credential AUDITA_LLM_API_KEY, shared concurrency 4
dnd_pipeline_audita_profiles	Complete native dnd-proposal and dnd-validation mappings using google/gemma-4-31b-it and 900-second attempt timeouts
dnd_pipeline_audita_llm_proposal_profile	dnd-proposal
dnd_pipeline_audita_llm_validation_profile	dnd-validation
dnd_pipeline_audita_llm_transient_retries, dnd_pipeline_audita_llm_repair_attempts, dnd_pipeline_audita_llm_operation_timeout	null; omit overrides and use native defaults
dnd_pipeline_audita_chunking_*	8 target sections, 6144 maximum and 2048 minimum section tokens
dnd_pipeline_audita_normalization_*	3.0 maximum and ellipsis gaps, 240.0 maximum duration, 2048 maximum tokens
dnd_pipeline_audita_thresholds_*	0.8 for glossary, homophones, spoken word, and grammar
dnd_pipeline_audita_context_description	D&D-game transcript context, including possible pre- and post-session banter
dnd_pipeline_audita_diagnostics_work_dir	/tmp/audita

Audita owns its backend and LLM policy. The suite writes each complete profile mapping to /usr/local/etc/audita/profiles/<key>.yml; keys must be filename-safe IDs matching each mapping's id. Inventory may replace the mappings and selected IDs. Narratio forwards the configured credential names, while Audita's native configuration resolves them. The role does not remove old profile files.

For inventories upgrading from the prior suite, replace the separate Audita proposal/validation endpoint, model, key, timeout, and concurrency variables with dnd_pipeline_audita_backends and dnd_pipeline_audita_profiles, then select profile IDs with the two dnd_pipeline_audita_llm_*_profile variables. Replace dnd_pipeline_narratio_notarius_llm_api_key with dnd_pipeline_narratio_notarius_credentials, and use dnd_pipeline_narratio_audita_credentials for Audita's backend credential names. The old diagnostics retention variable is no longer forwarded.

The old per-purpose Audita endpoint and API-key names now belong to the backend map; both default profiles share the dnd backend and its one concurrency limit. Old proposal and validation models and per-attempt timeouts belong in complete profile mappings. The default 900-second value is retained per attempt; dnd_pipeline_audita_llm_operation_timeout is an independent whole-operation deadline and defaults to null, leaving Audita's native policy in charge. The former *_max_retries variables do not have a direct one-to-one replacement: use the optional Audita transient retry and repair attempt settings according to the desired native behavior. Optional retry pass-throughs for Notarius PromptKit and Scriptorium likewise default to null; an explicit 0 is forwarded, not treated as omission.

Include each backend credential name in dnd_pipeline_narratio_audita_credentials and make its value available to Narratio through its process environment or supported secret files. dnd_pipeline_secrets is the suite's optional file-provisioning interface, not a requirement to store credentials on disk. Narratio forwards only names in that list and does not discover them from Audita's configuration. The Notarius list is independent. For profile overrides, use filename-safe map keys equal to each profile's native id; the suite writes each map value unchanged and does not merge individual fields. See Audita's v2 configuration migration for backend and profile migration; this role does not convert caller-owned configuration.

Existing Narratio session manifests need a separate migration before opening affected sessions with v3. Stop activity for the session and back up its local manifest. Follow Narratio's manifest overlay-provenance migration to rename effective_config.selected_profile to selected_overlay, or remove the retired informational member as upstream permits. Preserve stage records, checkpoints, and other progress; do not edit immutable remote snapshots or force all stages to rerun. The role performs no session migration.

Variable	Default
dnd_pipeline_narratio_seriatim_output_schema	seriatim-intermediate
dnd_pipeline_narratio_normalize_output_path	transcripts/final.json
dnd_pipeline_narratio_normalize_output_schema	seriatim-intermediate
dnd_pipeline_narratio_trim_enabled	true
dnd_pipeline_narratio_trim_output_path	transcripts/final.trimmed.json
dnd_pipeline_narratio_trim_bounds_prompt_id	dnd.session_bounds
dnd_pipeline_narratio_trim_bounds_transcript_input_name	transcript
dnd_pipeline_narratio_trim_bounds_output_path	artifacts/session_bounds.json
dnd_pipeline_narratio_render_enabled	true
dnd_pipeline_narratio_render_format	markdown
dnd_pipeline_narratio_testing_scriptorium_profile	gemini-flash-latest
dnd_pipeline_notarius_testing_llm_profile	gpt-luna-latest
dnd_pipeline_notarius_promptkit_transient_retries	null; native default
dnd_pipeline_scriptorium_transient_retries	null; native default
dnd_pipeline_narratio_audita_credentials	[AUDITA_LLM_API_KEY]
dnd_pipeline_narratio_notarius_credentials	[OPENROUTER_API_KEY]
dnd_pipeline_narratio_scriptorium_credentials	[OPENROUTER_API_KEY]

dnd_pipeline_audita_output_schema is also the Narratio Audita-adapter schema; it is the shared cross-application boundary. Other Narratio-local controls, including reports, timeouts, debug rendering, presentation, and bounds prompt profile selection, remain Narratio defaults.

Narratio v3.1.0 forwards the selected Scriptorium credential names for trim, analysis, character families, and render-debug calls. The Scriptorium and Notarius lists independently default to the existing OPENROUTER_API_KEY name; using the same provisioned value does not couple their selections. Inventory may replace the Scriptorium list, set [] to forward nothing, or set null to omit the field. Each selected name must resolve to a nonempty value when Narratio loads configuration, even for inspection commands. Values may come from Narratio's process environment or its existing secrets directory; dnd_pipeline_secrets is optional file provisioning, not the only source. After upgrading and provisioning the selected key, retry a failed trim or analysis normally. Do not force already successful work solely because a credential was added or rotated.

Storage and publication policy

The suite selects Narratio's S3 backend and the established Backblaze-compatible archive. These are ordinary inventory variables, so deployments may replace the selected policy through the role's public interface.

Variable	Default policy
dnd_pipeline_narratio_workspace_cleanup_after_publish	true
dnd_pipeline_narratio_storage_backend	s3
dnd_pipeline_narratio_storage_s3_bucket	rakestraw-assets
dnd_pipeline_narratio_storage_s3_root_prefix	dnd
dnd_pipeline_narratio_storage_s3_region	us-east-1
dnd_pipeline_narratio_storage_s3_endpoint	https://s3.us-east-005.backblazeb2.com
dnd_pipeline_narratio_storage_s3_force_path_style	true
dnd_pipeline_narratio_storage_s3_access_key_id	OBJECT_STORAGE_KEY_ID
dnd_pipeline_narratio_storage_s3_secret_access_key	OBJECT_STORAGE_KEY
dnd_pipeline_narratio_cache_s3_audio	true
dnd_pipeline_narratio_publish_enabled	true
dnd_pipeline_narratio_publish_upload_run	true

dnd_pipeline_narratio_publish_outputs is a complete native list, not a list merged with the suite default. It publishes final and trimmed JSON, final and trimmed Markdown, session recap, hostile rumors, and optional deranged rumors. Every rule omits dest: Narratio derives transcript destinations from its built-in sources and artifact destinations from the configured artifact output paths.

S3 session audio uses the maximum available native cache beneath /var/cache/narratio. After successful run publication, Narratio cleans the published workspace target while preserving that cache. The suite deliberately does not set narratio_spool_delete_audio_after_publish; spool-audio retention continues to use Narratio's application default.

Artifacts and profiles

The role owns these complete native mapping variables. Replacing one mapping changes that source's configuration as a whole; it does not change the fixed file name, import, or source key. Set enabled: false to retain an available source while excluding it from normal analysis.

Variable	Fixed source	Default state	Output
dnd_pipeline_narratio_session_recap_artifact	session_recap	enabled	artifacts/session_recap.md
dnd_pipeline_narratio_hostile_rumors_artifact	hostile_rumors	enabled	artifacts/hostile_rumors.md
dnd_pipeline_narratio_deranged_rumors_artifact	deranged_rumors	enabled	artifacts/deranged_rumors.md
dnd_pipeline_narratio_scene_map_artifact	scene_map	disabled	artifacts/scene_map.json
dnd_pipeline_narratio_character_analysis_family	character_meta_analysis	disabled	artifacts/character_meta_analysis/{character_id}.md

The character-analysis family expands over campaign party.characters and passes each member's name and class summary to its prompt. Recap and both rumor sources are enabled; the rumor input derives its recap dependency without a redundant explicit ordering edge. Production artifacts inherit each prompt's declared profile.

Narratio's root file remains owned by the standalone role. It imports the suite-owned base fragments in this fixed order:

/usr/local/etc/narratio/
├── pipeline.yml
├── conf.d/processing.yml
├── artifacts/session-recap.yml
├── artifacts/hostile-rumors.yml
├── artifacts/deranged-rumors.yml
├── artifacts/scene-map.yml
├── artifact-families/character-meta-analysis.yml
└── overlays/testing.yml

Notarius follows the corresponding base-and-overlay layout:

/usr/local/etc/notarius/
├── config.yml
├── pipelines/dnd-session.yml
└── overlays/testing.yml

The testing overlay is not a second configuration tree. It sets the Scriptorium profile_id for the four artifact sources and the one family, and explicitly selects Notarius's separately managed testing overlay. The Scriptorium default, gemini-flash-latest, is an inexpensive development choice, not a compatibility guarantee. Replace it through inventory when a different testing profile is appropriate:

dnd_pipeline_narratio_testing_scriptorium_profile: endpoint-profile
dnd_pipeline_notarius_testing_llm_profile: alternate-economy-profile

Narratio selects it with --overlay testing, and direct Notarius commands use --config-overlay testing; the former --profile and --config-profile selectors are retired. Previously deployed profiles/testing.yml files are ignored, not pruned. Renaming the selected overlay normally changes configuration identity, so cached work tied to the old named selection may not be reused.

Notarius extraction policy

The Notarius root configuration imports the fixed pipelines/dnd-session.yml fragment. The suite owns that fragment at /usr/local/etc/notarius/pipelines/dnd-session.yml; callers do not edit it. The separate /usr/local/etc/notarius/overlays/testing.yml overlay changes only the selected suite pipeline's LLM profile.

Variable	Default policy
dnd_pipeline_notarius_pipelines	Complete native map containing dnd-session
dnd_pipeline_notarius_cache_root	/var/cache/notarius
dnd_pipeline_notarius_cache_chunk_plans_mode	auto
dnd_pipeline_notarius_cache_checkpoints_enabled	true
dnd_pipeline_notarius_testing_llm_profile	gpt-luna-latest
dnd_pipeline_narratio_notarius_pipeline_id	dnd-session
dnd_pipeline_narratio_notarius_credentials	[OPENROUTER_API_KEY]
dnd_pipeline_narratio_notarius_resume	true
dnd_pipeline_narratio_notarius_references	party, players, and glossary prepared sources
dnd_pipeline_narratio_notarius_outputs	Complete native map of ten accepted D&D lanes

The pipeline, reference, and output values are complete native mappings. Inventory can replace them or compose additions with Jinja's combine filter; a replacement pipeline map requires a matching selected pipeline ID. The base dnd-session pipeline selects gpt-terra-latest, while the testing overlay selects gpt-luna-latest. Both are moving catalog IDs rather than immutable model promises. Pipeline and binding-level overrides retain Notarius's native precedence; the suite does not rewrite caller-supplied bindings.

Narratio materializes the party, players, and glossary references for each invocation. No campaign filesystem path appears in Notarius configuration. The optional spell_catalog reference remains absent until a campaign supplies that input. The accepted sources are narratio.extraction.item_registry, npc_registry, location_registry, scene_descriptions, item_occurrences, spells, combat_turns, npc_occurrences, location_occurrences, and enemy_events. They remain available as narratio.extraction.<output_key> inputs for future analyze artifacts whether or not a source has a stable publication rule; accepted extraction and published output selection are independent interfaces.

Narratio forwards the typed OPENROUTER_API_KEY reference to the selected Notarius profile and ordinary extraction requests checkpoint reuse with resume: true. Chunk plans use Notarius's root-owned auto policy; the pipeline deliberately leaves Narratio's optional chunk_cache override unset. The chunk-plans and checkpoints paths derive from the cache root and are created by Notarius during use. See the Notarius v0.8.1 release notes for its PromptKit dependency and profile-source identity changes. Existing checkpoints with the previous profile-source identity are not reused; source-keyed chunk plans and output artifacts remain compatible. No cache deletion or forced extraction is performed by this role. A real testing run may invoke Notarius; the integration suite performs offline configuration inspection only and makes no provider request.

Inspecting configuration

The following native Narratio commands resolve configuration without running transcription or requesting a model provider. Select a deployed campaign for the party-driven character family:

sudo -u narratio narratio config validate \
  --config /usr/local/etc/narratio/pipeline.yml --campaign centaurs
sudo -u narratio narratio config validate \
  --config /usr/local/etc/narratio/pipeline.yml --campaign centaurs --overlay testing
sudo -u narratio narratio config artifacts \
  --config /usr/local/etc/narratio/pipeline.yml --campaign centaurs
sudo -u narratio narratio config diff default testing \
  --config /usr/local/etc/narratio/pipeline.yml --campaign centaurs

The first two commands validate production and testing resolution. Artifact inspection shows enabled sources, inferred dependencies, and generated family members; the diff shows the overlay's profile selections without modifying managed files.

With valid object-storage credentials, the selected policy enables Narratio's native S3 session-audio, publication, restore, publication locks, session status, and cleanup workflows. These remain application commands and lifecycle rules; scheduling and real end-to-end execution remain deployment concerns.

Shared execution filesystem

/var/lib/narratio, /var/spool/narratio, /var/cache/narratio, and /var/cache/notarius are narratio:narratio mode 2775 setgid directories. Their access and default ACLs retain owner and group write access plus world read/traverse access for newly created content. The pipeline may run as narratio or as a human account in the selected runtime group. Consumers can enroll selected existing human accounts through the role interface:

dnd_pipeline_operator_users:
  - eric
  - alice

Every selected account must already exist; the role does not create human accounts or manage their identity and login policy. Enrollment adds the selected runtime group without replacing other supplementary groups. Removing or omitting a name later does not revoke membership. A newly enrolled operator must begin a new login session before the operating system exposes the added supplementary group to that session.

Configuration, transcripts, caches, generated artifacts, campaign data, prompts, and logs are non-secret D&D content. Credentials use the restricted /usr/local/etc/narratio/secrets subtree, owned by the selected runtime user and group with mode 0750; managed files use mode 0640. Runtime-group membership, including enrollment through dnd_pipeline_operator_users, authorizes use of these shared credentials, while the subtree grants no world access. Ordinary configuration remains world-readable and contains only reference names.

Managed credentials

dnd_pipeline_secrets defaults to {}. Its keys must be valid environment variable names, and each string value is managed as the exact named credential file. Omitted names are unmanaged: the role neither mirrors nor purges the directory. Narratio loads this conventional sibling directory before configuration resolution; an already-set process environment value takes precedence. The role adds no wrapper, service, or parallel credential loader.

Keep values in encrypted inventory, for example with Ansible Vault; Vault is a consumer workflow, not role functionality:

dnd_pipeline_secrets:
  AUDITA_LLM_API_KEY: "{{ vault_dnd_audita_llm_api_key }}"
  OPENROUTER_API_KEY: "{{ vault_dnd_openrouter_api_key }}"
  OBJECT_STORAGE_KEY_ID: "{{ vault_dnd_object_storage_key_id }}"
  OBJECT_STORAGE_KEY: "{{ vault_dnd_object_storage_key }}"

Only the reference names enter ordinary Narratio configuration; credential values remain in encrypted inventory and the restricted managed secrets directory. A deployment using Narratio's ambient AWS SDK provider chain can set both dnd_pipeline_narratio_storage_s3_access_key_id and dnd_pipeline_narratio_storage_s3_secret_access_key to null. The pair must be omitted together.

Scriptorium profiles continue to own their api_key_env selection. To supply that credential through the suite, add a dnd_pipeline_secrets entry whose name matches the selected profile's api_key_env. Narratio's secrets bootstrap loads the value, and its existing Scriptorium adapter forwards only the profile-selected credential.

Validation

tests/integration installs the pinned Git dependencies and collection into private temporary paths, runs syntax and production-profile lint checks, then applies the suite with a local syntactically valid WhisperX URL and explicit immutable file:// asset inputs. It uses native application smoke/configuration commands, validates Notarius's imported pipeline and effective cache policy, and validates Narratio for both deployed campaigns and the testing overlay. It inspects enabled Notarius wiring through Narratio's native effective configuration, validates the same imported Notarius pipeline as a temporary existing-group-member fixture, exercises representative shared cache access across the service and fixture identities, and confirms that nobody can read ordinary configuration. It also composes an alternate pair of version-5 pipeline maps through the public interface and requires clean repeats for the baseline and override cases. Cleanup reapplies the asset-enabled baseline suite and removes only the fixture account, its named test content, and the driver's private temporary directory.