commit 5dbae8ab055d88ddcc61dc8ac52271fe261f67b5 Author: Eric Rakestraw Date: Thu Sep 17 03:30:23 2026 +0000 Squashed 'roles/pipeline/' content from commit 600eb21 git-subtree-dir: roles/pipeline git-subtree-split: 600eb21cd967831607c087f1773063636ff2c31b diff --git a/.gitignore b/.gitignore new file mode 100644 index 0000000..5ed72a7 --- /dev/null +++ b/.gitignore @@ -0,0 +1,86 @@ +# ---> Ansible +*.retry + +# ---> Linux +*~ + +# temporary files which can be created if a process still has a handle open of a deleted file +.fuse_hidden* + +# KDE directory preferences +.directory + +# Linux trash folder which might appear on any partition or disk +.Trash-* + +# .nfs files are created when an open file is removed but is still being accessed +.nfs* + +# ---> macOS +# General +.DS_Store +.AppleDouble +.LSOverride + +# Icon must end with two \r +Icon + +# Thumbnails +._* + +# Files that might appear in the root of a volume +.DocumentRevisions-V100 +.fseventsd +.Spotlight-V100 +.TemporaryItems +.Trashes +.VolumeIcon.icns +.com.apple.timemachine.donotpresent + +# Directories potentially created on remote AFP share +.AppleDB +.AppleDesktop +Network Trash Folder +Temporary Items +.apdisk + +# ---> Windows +# Windows thumbnail cache files +Thumbs.db +Thumbs.db:encryptable +ehthumbs.db +ehthumbs_vista.db + +# Dump file +*.stackdump + +# Folder config file +[Dd]esktop.ini + +# Recycle Bin used on file shares +$RECYCLE.BIN/ + +# Windows Installer files +*.cab +*.msi +*.msix +*.msm +*.msp + +# Windows shortcuts +*.lnk + +# ---> VisualStudioCode +.vscode/* +!.vscode/settings.json +!.vscode/tasks.json +!.vscode/launch.json +!.vscode/extensions.json +!.vscode/*.code-snippets + +# Local History for Visual Studio Code +.history/ + +# Built Visual Studio Code Extensions +*.vsix + diff --git a/LICENSE b/LICENSE new file mode 100644 index 0000000..0e259d4 --- /dev/null +++ b/LICENSE @@ -0,0 +1,121 @@ +Creative Commons Legal Code + +CC0 1.0 Universal + + CREATIVE COMMONS CORPORATION IS NOT A LAW FIRM AND DOES NOT PROVIDE + LEGAL SERVICES. DISTRIBUTION OF THIS DOCUMENT DOES NOT CREATE AN + ATTORNEY-CLIENT RELATIONSHIP. CREATIVE COMMONS PROVIDES THIS + INFORMATION ON AN "AS-IS" BASIS. CREATIVE COMMONS MAKES NO WARRANTIES + REGARDING THE USE OF THIS DOCUMENT OR THE INFORMATION OR WORKS + PROVIDED HEREUNDER, AND DISCLAIMS LIABILITY FOR DAMAGES RESULTING FROM + THE USE OF THIS DOCUMENT OR THE INFORMATION OR WORKS PROVIDED + HEREUNDER. + +Statement of Purpose + +The laws of most jurisdictions throughout the world automatically confer +exclusive Copyright and Related Rights (defined below) upon the creator +and subsequent owner(s) (each and all, an "owner") of an original work of +authorship and/or a database (each, a "Work"). + +Certain owners wish to permanently relinquish those rights to a Work for +the purpose of contributing to a commons of creative, cultural and +scientific works ("Commons") that the public can reliably and without fear +of later claims of infringement build upon, modify, incorporate in other +works, reuse and redistribute as freely as possible in any form whatsoever +and for any purposes, including without limitation commercial purposes. +These owners may contribute to the Commons to promote the ideal of a free +culture and the further production of creative, cultural and scientific +works, or to gain reputation or greater distribution for their Work in +part through the use and efforts of others. + +For these and/or other purposes and motivations, and without any +expectation of additional consideration or compensation, the person +associating CC0 with a Work (the "Affirmer"), to the extent that he or she +is an owner of Copyright and Related Rights in the Work, voluntarily +elects to apply CC0 to the Work and publicly distribute the Work under its +terms, with knowledge of his or her Copyright and Related Rights in the +Work and the meaning and intended legal effect of CC0 on those rights. + +1. Copyright and Related Rights. A Work made available under CC0 may be +protected by copyright and related or neighboring rights ("Copyright and +Related Rights"). Copyright and Related Rights include, but are not +limited to, the following: + + i. the right to reproduce, adapt, distribute, perform, display, + communicate, and translate a Work; + ii. moral rights retained by the original author(s) and/or performer(s); +iii. publicity and privacy rights pertaining to a person's image or + likeness depicted in a Work; + iv. rights protecting against unfair competition in regards to a Work, + subject to the limitations in paragraph 4(a), below; + v. rights protecting the extraction, dissemination, use and reuse of data + in a Work; + vi. database rights (such as those arising under Directive 96/9/EC of the + European Parliament and of the Council of 11 March 1996 on the legal + protection of databases, and under any national implementation + thereof, including any amended or successor version of such + directive); and +vii. other similar, equivalent or corresponding rights throughout the + world based on applicable law or treaty, and any national + implementations thereof. + +2. Waiver. To the greatest extent permitted by, but not in contravention +of, applicable law, Affirmer hereby overtly, fully, permanently, +irrevocably and unconditionally waives, abandons, and surrenders all of +Affirmer's Copyright and Related Rights and associated claims and causes +of action, whether now known or unknown (including existing as well as +future claims and causes of action), in the Work (i) in all territories +worldwide, (ii) for the maximum duration provided by applicable law or +treaty (including future time extensions), (iii) in any current or future +medium and for any number of copies, and (iv) for any purpose whatsoever, +including without limitation commercial, advertising or promotional +purposes (the "Waiver"). Affirmer makes the Waiver for the benefit of each +member of the public at large and to the detriment of Affirmer's heirs and +successors, fully intending that such Waiver shall not be subject to +revocation, rescission, cancellation, termination, or any other legal or +equitable action to disrupt the quiet enjoyment of the Work by the public +as contemplated by Affirmer's express Statement of Purpose. + +3. Public License Fallback. Should any part of the Waiver for any reason +be judged legally invalid or ineffective under applicable law, then the +Waiver shall be preserved to the maximum extent permitted taking into +account Affirmer's express Statement of Purpose. In addition, to the +extent the Waiver is so judged Affirmer hereby grants to each affected +person a royalty-free, non transferable, non sublicensable, non exclusive, +irrevocable and unconditional license to exercise Affirmer's Copyright and +Related Rights in the Work (i) in all territories worldwide, (ii) for the +maximum duration provided by applicable law or treaty (including future +time extensions), (iii) in any current or future medium and for any number +of copies, and (iv) for any purpose whatsoever, including without +limitation commercial, advertising or promotional purposes (the +"License"). The License shall be deemed effective as of the date CC0 was +applied by Affirmer to the Work. Should any part of the License for any +reason be judged legally invalid or ineffective under applicable law, such +partial invalidity or ineffectiveness shall not invalidate the remainder +of the License, and in such case Affirmer hereby affirms that he or she +will not (i) exercise any of his or her remaining Copyright and Related +Rights in the Work or (ii) assert any associated claims and causes of +action with respect to the Work, in either case contrary to Affirmer's +express Statement of Purpose. + +4. Limitations and Disclaimers. + + a. No trademark or patent rights held by Affirmer are waived, abandoned, + surrendered, licensed or otherwise affected by this document. + b. Affirmer offers the Work as-is and makes no representations or + warranties of any kind concerning the Work, express, implied, + statutory or otherwise, including without limitation warranties of + title, merchantability, fitness for a particular purpose, non + infringement, or the absence of latent or other defects, accuracy, or + the present or absence of errors, whether or not discoverable, all to + the greatest extent permissible under applicable law. + c. Affirmer disclaims responsibility for clearing rights of other persons + that may apply to the Work or any use thereof, including without + limitation any person's Copyright and Related Rights in the Work. + Further, Affirmer disclaims responsibility for obtaining any necessary + consents, permissions or other rights required for any use of the + Work. + d. Affirmer understands and acknowledges that Creative Commons is not a + party to this document and has no duty or obligation with respect to + this CC0 or use of the Work. diff --git a/README.md b/README.md new file mode 100644 index 0000000..c23b98c --- /dev/null +++ b/README.md @@ -0,0 +1,332 @@ +# rakestraw.dnd_pipeline + +Installs the D&D transcript-processing application suite by composing the +standalone Seriatim, Audita, Notarius, Scriptorium, and Narratio roles. It +supports Ubuntu 26.04 and Ansible Core 2.18 or newer. Downstream roles perform +their own task-level privilege escalation, so a play does not need `become`. +Supported host architectures are `amd64` and `arm64`. + +## Installation + +Install the role dependencies and their collection before applying this role: + +```sh +ansible-galaxy role install -r requirements.yml +ansible-galaxy collection install -r collections/requirements.yml +``` + +The dependency manifest installs the roles under their canonical names: +`rakestraw.seriatim`, `rakestraw.audita`, `rakestraw.notarius`, +`rakestraw.scriptorium`, and `rakestraw.narratio`. This standalone role is +named `rakestraw.dnd_pipeline`. + +```yaml +--- +- name: Install the D&D pipeline foundation + hosts: pipeline_hosts + gather_facts: true + roles: + - role: rakestraw.dnd_pipeline + vars: + dnd_pipeline_whisperx_transcribe_url: https://whisperx.example/transcribe +``` + +The collection source exposes a reviewed snapshot of this role as +`rakestraw.dnd.pipeline`: + +```yaml +--- +- name: Install the D&D pipeline from the collection + hosts: pipeline_hosts + gather_facts: true + roles: + - role: rakestraw.dnd.pipeline + vars: + dnd_pipeline_whisperx_transcribe_url: https://whisperx.example/transcribe +``` + +Collection consumers should follow the unified bootstrap instructions in the +[`rakestraw.dnd` collection repository](https://gitea.maximumdirect.net/eric/ansible-collection-dnd). +This standalone repository remains authoritative for role changes; collection +maintainers import a reviewed commit as a snapshot. This does not imply that a +collection release has been published or tagged. + +## Variables + +`dnd_pipeline_whisperx_transcribe_url` is required and is passed to Narratio. +The compatible application versions are selected through these variables: + +| Variable | Default | +| --- | --- | +| `dnd_pipeline_audita_version` | `v0.9.1` | +| `dnd_pipeline_narratio_version` | `v2.1.0` | +| `dnd_pipeline_notarius_version` | `v0.7.0` | +| `dnd_pipeline_seriatim_version` | `v1.5.0` | +| `dnd_pipeline_scriptorium_version` | `v0.13.0` | +| `dnd_pipeline_manage_build_dependencies` | `true` | +| `dnd_pipeline_go_snap_channel` | `1.26/stable` | +| `dnd_pipeline_go_binary` | `/snap/bin/go` | +| `dnd_pipeline_campaigns_repository_url` | `https://gitea.maximumdirect.net/eric/campaigns.git` | +| `dnd_pipeline_campaigns_revision` | `main` | +| `dnd_pipeline_campaigns_root` | `/usr/local/share/narratio/campaigns` | +| `dnd_pipeline_prompt_library_repository_url` | `https://gitea.maximumdirect.net/eric/narratio-prompts.git` | +| `dnd_pipeline_prompt_library_revision` | `main` | +| `dnd_pipeline_prompt_library_root` | `/usr/local/share/narratio/prompt-library` | + +The common dependency-management switch controls whether every application +role installs Git and Go. Callers that disable it must provide both Git and Go. +The Go channel and executable are likewise passed to all five roles. Advanced +callers may supply unmapped standalone-role variables directly through normal +Ansible variable precedence. + +The following values are deliberate D&D suite policy passed through the +standalone roles' generic configuration interfaces; they are not standalone +defaults. Narratio runs first and establishes the selected runtime user and +group before the remaining configuration-bearing dependencies run. + +| Variable | Default policy | +| --- | --- | +| `dnd_pipeline_runtime_user` | `narratio` | +| `dnd_pipeline_runtime_group` | `narratio` | +| `dnd_pipeline_config_directory_mode` | `0755` | +| `dnd_pipeline_config_file_mode` | `0644` | + +All application configuration, including pipeline-owned Narratio fragments, +is therefore `narratio:narratio`, world-readable, and Ansible-managed. It is +not an operator-editable configuration surface. + +Narratio receives the standard Seriatim, Audita, Notarius, and Scriptorium +binary and configuration paths managed by the composed roles. Notarius is +enabled with the suite-owned `dnd-session` extraction pipeline. + +The role deploys the campaign repository directly at the campaign root and +the prompt library directly at the prompt-library root. By default both follow +their `main` branches; callers can select an immutable commit, tag, or another +branch through the revision variables. Narratio receives the deployed campaign +root, while Scriptorium receives its `prompts/` and `schemas/` children. + +### D&D processing policy + +The variables below are deliberate D&D suite policy, not a copy of application +defaults. They remain ordinary inventory variables, so callers can replace a +chosen policy value when their deployment requires it. + +| Variable group | Default policy | +| --- | --- | +| `dnd_pipeline_audita_pipeline_modules` | `glossary`, `homophones`, `glossary`, `spoken_word`, `grammar` | +| `dnd_pipeline_audita_output_schema` | `audita-v1` | +| `dnd_pipeline_audita_llm_proposal_*` | `https://inference.ai.rakestrawhome.com/v1`, `google/gemma-4-31b-it`, `AUDITA_LLM_API_KEY`, `900s`, `3` | +| `dnd_pipeline_audita_llm_validation_*` | `https://inference.ai.rakestrawhome.com/v1`, `google/gemma-4-31b-it`, `AUDITA_LLM_API_KEY`, `900s`, `3` | +| `dnd_pipeline_audita_concurrency_*` | total, proposal, and validation concurrency of `4` | +| `dnd_pipeline_audita_chunking_*` | `8` target sections, `6144` maximum and `2048` minimum section tokens | +| `dnd_pipeline_audita_normalization_*` | `3s` maximum and ellipsis gaps, `240s` maximum duration, `2048` maximum tokens | +| `dnd_pipeline_audita_thresholds_*` | `0.8` for glossary, homophones, spoken word, and grammar | +| `dnd_pipeline_audita_context_description` | D&D-game transcript context, including possible pre- and post-session banter | +| `dnd_pipeline_audita_diagnostics_*` | `/tmp/audita`, retained automatically | + +Audita's proposal and validation settings independently remain overrideable, +but both default to the one typed `AUDITA_LLM_API_KEY` reference Narratio can +forward. Audita owns its endpoint, model, tuning, and diagnostics configuration; +Narratio receives its managed Audita configuration path and supplies the +run-specific output path. + +| Variable | Default | +| --- | --- | +| `dnd_pipeline_narratio_seriatim_output_schema` | `seriatim-intermediate` | +| `dnd_pipeline_narratio_normalize_output_path` | `transcripts/final.json` | +| `dnd_pipeline_narratio_normalize_output_schema` | `seriatim-intermediate` | +| `dnd_pipeline_narratio_trim_enabled` | `true` | +| `dnd_pipeline_narratio_trim_output_path` | `transcripts/final.trimmed.json` | +| `dnd_pipeline_narratio_trim_bounds_prompt_id` | `dnd.session_bounds` | +| `dnd_pipeline_narratio_trim_bounds_transcript_input_name` | `transcript` | +| `dnd_pipeline_narratio_trim_bounds_output_path` | `artifacts/session_bounds.json` | +| `dnd_pipeline_narratio_render_enabled` | `true` | +| `dnd_pipeline_narratio_render_format` | `markdown` | +| `dnd_pipeline_narratio_testing_scriptorium_profile` | `gemini-flash-latest` | + +`dnd_pipeline_audita_output_schema` is also the Narratio Audita-adapter schema; +it is the shared cross-application boundary. Other Narratio-local controls, +including reports, timeouts, debug rendering, presentation, and bounds prompt +profile selection, remain Narratio defaults. + +### Artifacts and profiles + +The role owns these complete native mapping variables. Replacing one mapping +changes that source's configuration as a whole; it does not change the fixed +file name, import, or source key. Set `enabled: false` to retain an available +source while excluding it from normal analysis. + +| Variable | Fixed source | Default state | Output | +| --- | --- | --- | --- | +| `dnd_pipeline_narratio_session_recap_artifact` | `session_recap` | enabled | `artifacts/session_recap.md` | +| `dnd_pipeline_narratio_hostile_rumors_artifact` | `hostile_rumors` | enabled | `artifacts/hostile_rumors.md` | +| `dnd_pipeline_narratio_deranged_rumors_artifact` | `deranged_rumors` | enabled | `artifacts/deranged_rumors.md` | +| `dnd_pipeline_narratio_scene_map_artifact` | `scene_map` | disabled | `artifacts/scene_map.json` | +| `dnd_pipeline_narratio_character_analysis_family` | `character_meta_analysis` | disabled | `artifacts/character_meta_analysis/{character_id}.md` | + +The character-analysis family expands over campaign `party.characters` and +passes each member's name and class summary to its prompt. Recap and both rumor +sources are enabled; the rumor input derives its recap dependency without a +redundant explicit ordering edge. Production artifacts inherit each prompt's +declared profile. + +Narratio's root file remains owned by the standalone role. It imports the +suite-owned base fragments in this fixed order: + +```text +/usr/local/etc/narratio/ +├── pipeline.yml +├── conf.d/processing.yml +├── artifacts/session-recap.yml +├── artifacts/hostile-rumors.yml +├── artifacts/deranged-rumors.yml +├── artifacts/scene-map.yml +├── artifact-families/character-meta-analysis.yml +└── profiles/testing.yml +``` + +The `testing` profile is an overlay, not a second configuration tree. It only +sets the Scriptorium `profile_id` for the four artifact sources and the one +family. Its default, `gemini-flash-latest`, is an inexpensive development +choice, not a compatibility guarantee. Replace it through inventory when a +different testing profile is appropriate: + +```yaml +dnd_pipeline_narratio_testing_scriptorium_profile: endpoint-profile +``` + +### Notarius extraction policy + +The Notarius root configuration imports the fixed +`pipelines/dnd-session.yml` fragment. The suite owns that fragment at +`/usr/local/etc/notarius/pipelines/dnd-session.yml`; callers do not edit it. + +| Variable | Default policy | +| --- | --- | +| `dnd_pipeline_notarius_pipelines` | Complete native map containing `dnd-session` | +| `dnd_pipeline_notarius_cache_root` | `/var/cache/notarius` | +| `dnd_pipeline_notarius_cache_chunk_plans_mode` | `auto` | +| `dnd_pipeline_notarius_cache_checkpoints_enabled` | `true` | +| `dnd_pipeline_narratio_notarius_pipeline_id` | `dnd-session` | +| `dnd_pipeline_narratio_notarius_llm_api_key` | `OPENROUTER_API_KEY` | +| `dnd_pipeline_narratio_notarius_resume` | `true` | +| `dnd_pipeline_narratio_notarius_references` | `party`, `players`, and `glossary` prepared sources | +| `dnd_pipeline_narratio_notarius_outputs` | Complete native map of ten accepted D&D lanes | + +The pipeline, reference, and output values are complete native mappings. +Inventory can replace them or compose additions with Jinja's `combine` filter; +a replacement pipeline map requires a matching selected pipeline ID. + +Narratio materializes the `party`, `players`, and `glossary` references for each +invocation. No campaign filesystem path appears in Notarius configuration. The +optional `spell_catalog` reference remains absent until a campaign supplies +that input. The accepted sources are +`narratio.extraction.item_registry`, `npc_registry`, `location_registry`, +`scene_descriptions`, `item_occurrences`, `spells`, `combat_turns`, +`npc_occurrences`, `location_occurrences`, and `enemy_events`. + +Narratio forwards the typed `OPENROUTER_API_KEY` reference to the selected +Notarius profile and ordinary extraction requests checkpoint reuse with +`resume: true`. Chunk plans use Notarius's root-owned `auto` policy; the +pipeline deliberately leaves Narratio's optional `chunk_cache` override unset. +The `chunk-plans` and `checkpoints` paths derive from the cache root and are +created by Notarius during use. Base and `testing` configurations select the +same `dnd-extraction` profile and Notarius pipeline; the testing overlay changes +only Scriptorium profiles, so a real testing run may invoke Notarius. The +integration suite performs offline configuration inspection only. + +### Inspecting configuration + +The following native Narratio commands resolve configuration without running +transcription or requesting a model provider. Select a deployed campaign for +the party-driven character family: + +```sh +sudo -u narratio narratio config validate \ + --config /usr/local/etc/narratio/pipeline.yml --campaign centaurs +sudo -u narratio narratio config validate \ + --config /usr/local/etc/narratio/pipeline.yml --campaign centaurs --profile testing +sudo -u narratio narratio config artifacts \ + --config /usr/local/etc/narratio/pipeline.yml --campaign centaurs +sudo -u narratio narratio config diff default testing \ + --config /usr/local/etc/narratio/pipeline.yml --campaign centaurs +``` + +The first two commands validate production and testing resolution. Artifact +inspection shows enabled sources, inferred dependencies, and generated family +members; the diff shows the overlay's profile selections without modifying +managed files. + +Storage and publishing, operator selection and group-membership management, +scheduling, and end-to-end external execution remain deferred. + +### Shared execution filesystem + +`/var/lib/narratio`, `/var/spool/narratio`, `/var/cache/narratio`, and +`/var/cache/notarius` are +`narratio:narratio` mode `2775` **setgid** directories. Their access and +default ACLs retain owner and group write access plus world read/traverse access +for newly created content. The pipeline may run as `narratio` or as a human +account already in the selected runtime group; the production role neither +selects people nor manages their group membership. For example, a consumer +play can manage its selected operator separately: + +```yaml +- name: Add the selected operator to the pipeline group + become: true + ansible.builtin.user: + name: "{{ pipeline_operator }}" + groups: + - "{{ dnd_pipeline_runtime_group }}" + append: true +``` + +Configuration, transcripts, caches, generated artifacts, campaign data, +prompts, and logs are non-secret D&D content. Credentials use the restricted +`/usr/local/etc/narratio/secrets` subtree, owned by the selected runtime user +and group with mode `0750`; managed files use mode `0640`. Runtime-group +membership authorizes use of these shared credentials, while the subtree grants +no world access. Ordinary configuration remains world-readable and contains +only reference names. + +### Managed credentials + +`dnd_pipeline_secrets` defaults to `{}`. Its keys must be valid environment +variable names, and each string value is managed as the exact named credential +file. Omitted names are unmanaged: the role neither mirrors nor purges the +directory. Narratio loads this conventional sibling directory before +configuration resolution; an already-set process environment value takes +precedence. The role adds no wrapper, service, or parallel credential loader. + +Keep values in encrypted inventory, for example with Ansible Vault; Vault is a +consumer workflow, not role functionality: + +```yaml +dnd_pipeline_secrets: + AUDITA_LLM_API_KEY: "{{ vault_dnd_audita_llm_api_key }}" + OPENROUTER_API_KEY: "{{ vault_dnd_openrouter_api_key }}" +``` + +Scriptorium profiles continue to own their `api_key_env` selection. To supply +that credential through the suite, add a `dnd_pipeline_secrets` entry whose +name matches the selected profile's `api_key_env`. Narratio's secrets bootstrap +loads the value, and its existing Scriptorium adapter forwards only the +profile-selected credential. + +## Validation + +`tests/integration` installs the pinned Git dependencies and collection into +private temporary paths, runs syntax and production-profile lint checks, then +applies the suite with a local syntactically valid WhisperX URL and explicit +immutable `file://` asset inputs. It uses native application +smoke/configuration commands, validates Notarius's imported pipeline and +effective cache policy, and validates Narratio for both deployed campaigns and +the testing overlay. It inspects enabled Notarius wiring through Narratio's +native effective configuration, validates the same imported Notarius pipeline +as a temporary existing-group-member fixture, exercises representative shared +cache access across the service and fixture identities, and confirms that +`nobody` can read ordinary configuration. It also composes an alternate pair of +version-5 pipeline maps through the public interface and requires clean repeats +for the baseline and override cases. Cleanup reapplies the asset-enabled +baseline suite and removes only the fixture account, its named test content, +and the driver's private temporary directory. diff --git a/ansible.cfg b/ansible.cfg new file mode 100644 index 0000000..2d4c5f9 --- /dev/null +++ b/ansible.cfg @@ -0,0 +1,2 @@ +[defaults] +roles_path = .. diff --git a/collections/requirements.yml b/collections/requirements.yml new file mode 100644 index 0000000..9d6edd5 --- /dev/null +++ b/collections/requirements.yml @@ -0,0 +1,6 @@ +--- +collections: + - name: ansible.posix + version: 2.2.0 + - name: community.general + version: 13.3.0 diff --git a/defaults/main.yml b/defaults/main.yml new file mode 100644 index 0000000..4e9d9f4 --- /dev/null +++ b/defaults/main.yml @@ -0,0 +1,337 @@ +--- +dnd_pipeline_audita_version: v0.9.1 +dnd_pipeline_narratio_version: v2.1.0 +dnd_pipeline_notarius_version: v0.7.0 +dnd_pipeline_seriatim_version: v1.5.0 +dnd_pipeline_scriptorium_version: v0.13.0 +dnd_pipeline_manage_build_dependencies: true +dnd_pipeline_go_snap_channel: 1.26/stable +dnd_pipeline_go_binary: /snap/bin/go +dnd_pipeline_runtime_user: narratio +dnd_pipeline_runtime_group: narratio +dnd_pipeline_secrets: {} +dnd_pipeline_config_directory_mode: "0755" +dnd_pipeline_config_file_mode: "0644" +dnd_pipeline_campaigns_repository_url: https://gitea.maximumdirect.net/eric/campaigns.git +dnd_pipeline_campaigns_revision: main +dnd_pipeline_campaigns_root: /usr/local/share/narratio/campaigns +dnd_pipeline_prompt_library_repository_url: https://gitea.maximumdirect.net/eric/narratio-prompts.git +dnd_pipeline_prompt_library_revision: main +dnd_pipeline_prompt_library_root: /usr/local/share/narratio/prompt-library +dnd_pipeline_audita_pipeline_modules: + - glossary + - homophones + - glossary + - spoken_word + - grammar +dnd_pipeline_audita_output_schema: audita-v1 +dnd_pipeline_audita_llm_proposal_base_url: https://inference.ai.rakestrawhome.com/v1 +dnd_pipeline_audita_llm_proposal_model: google/gemma-4-31b-it +dnd_pipeline_audita_llm_proposal_api_key_env: AUDITA_LLM_API_KEY +dnd_pipeline_audita_llm_proposal_timeout: 900s +dnd_pipeline_audita_llm_proposal_max_retries: 3 +dnd_pipeline_audita_llm_validation_base_url: https://inference.ai.rakestrawhome.com/v1 +dnd_pipeline_audita_llm_validation_model: google/gemma-4-31b-it +dnd_pipeline_audita_llm_validation_api_key_env: AUDITA_LLM_API_KEY +dnd_pipeline_audita_llm_validation_timeout: 900s +dnd_pipeline_audita_llm_validation_max_retries: 3 +dnd_pipeline_audita_concurrency_total_llm: 4 +dnd_pipeline_audita_concurrency_proposal_llm: 4 +dnd_pipeline_audita_concurrency_validation_llm: 4 +dnd_pipeline_audita_chunking_target_sections: 8 +dnd_pipeline_audita_chunking_max_section_tokens: 6144 +dnd_pipeline_audita_chunking_min_section_tokens: 2048 +dnd_pipeline_audita_normalization_max_segment_gap: 3s +dnd_pipeline_audita_normalization_ellipsis_gap: 3s +dnd_pipeline_audita_normalization_max_segment_duration: 240s +dnd_pipeline_audita_normalization_max_segment_tokens: 2048 +dnd_pipeline_audita_thresholds_glossary: 0.8 +dnd_pipeline_audita_thresholds_homophones: 0.8 +dnd_pipeline_audita_thresholds_spoken_word: 0.8 +dnd_pipeline_audita_thresholds_grammar: 0.8 +dnd_pipeline_audita_context_description: >- + The following transcript was recorded during a Dungeons & Dragons game session. + It may contain some unrelated pre-session or post-session banter, in addition + to actual D&D gameplay. +dnd_pipeline_audita_diagnostics_work_dir: /tmp/audita +dnd_pipeline_audita_diagnostics_retention: auto +dnd_pipeline_notarius_pipelines: + dnd-session: + llm_profile: dnd-extraction + input: + module: seriatim + chunk: + module: dnd/scenes + retries: 2 + output: + module: json + options: + include_chunk_map: true + evidence_context: + enabled: true + window_units: 3 + lanes: + - item-occurrences + - item-registry + - location-registry + - location-occurrences + - npc-registry + - spells + - combat-turns + - npc-occurrences + - enemy-events + steps: + - id: describe-session + artifacts: + item-registry: + extract: + module: dnd/item-registry + retries: 2 + merge: + module: appendorder + normalize: + module: dnd/item-registry + npc-registry: + extract: + module: dnd/npc-registry + retries: 2 + merge: + module: appendorder + normalize: + module: dnd/npc-registry + retries: 2 + location-registry: + extract: + module: dnd/location-registry + retries: 2 + merge: + module: appendorder + normalize: + module: dnd/location-registry + retries: 2 + scene-descriptions: + extract: + module: dnd/scene-descriptions + retries: 2 + merge: + module: appendorder + normalize: + module: dnd/scene-descriptions + - id: extract-events + references: + location_registry: + artifact: + step: describe-session + lane: location-registry + npc_registry: + artifact: + step: describe-session + lane: npc-registry + scene_descriptions: + artifact: + step: describe-session + lane: scene-descriptions + item_registry: + artifact: + step: describe-session + lane: item-registry + artifacts: + item-occurrences: + extract: + module: dnd/item-occurrences + retries: 2 + merge: + module: appendorder + normalize: + module: dnd/item-occurrences + spells: + extract: + module: dnd/spells + retries: 2 + merge: + module: appendorder + normalize: + module: dnd/spells + combat-turns: + extract: + module: dnd/combat-turns + retries: 2 + merge: + module: appendorder + normalize: + module: dnd/combat-turns + npc-occurrences: + extract: + module: dnd/npc-occurrences + retries: 2 + merge: + module: appendorder + normalize: + module: dnd/npc-occurrences + location-occurrences: + extract: + module: dnd/location-occurrences + retries: 2 + merge: + module: appendorder + normalize: + module: dnd/location-occurrences + - id: track-enemies + references: + npc_registry: + artifact: + step: describe-session + lane: npc-registry + scene_descriptions: + artifact: + step: describe-session + lane: scene-descriptions + combat_turns: + artifact: + step: extract-events + lane: combat-turns + npc_occurrences: + artifact: + step: extract-events + lane: npc-occurrences + artifacts: + enemy-events: + extract: + module: dnd/enemy-events + retries: 2 + merge: + module: appendorder + normalize: + module: dnd/enemy-events +dnd_pipeline_notarius_cache_root: /var/cache/notarius +dnd_pipeline_notarius_cache_chunk_plans_mode: auto +dnd_pipeline_notarius_cache_checkpoints_enabled: true +dnd_pipeline_narratio_seriatim_output_schema: seriatim-intermediate +dnd_pipeline_narratio_normalize_output_path: transcripts/final.json +dnd_pipeline_narratio_normalize_output_schema: seriatim-intermediate +dnd_pipeline_narratio_trim_enabled: true +dnd_pipeline_narratio_trim_output_path: transcripts/final.trimmed.json +dnd_pipeline_narratio_trim_bounds_prompt_id: dnd.session_bounds +dnd_pipeline_narratio_trim_bounds_transcript_input_name: transcript +dnd_pipeline_narratio_trim_bounds_output_path: artifacts/session_bounds.json +dnd_pipeline_narratio_notarius_pipeline_id: dnd-session +dnd_pipeline_narratio_notarius_llm_api_key: OPENROUTER_API_KEY +dnd_pipeline_narratio_notarius_resume: true +dnd_pipeline_narratio_notarius_references: + party: narratio.input.party + players: narratio.input.players + glossary: narratio.input.glossary +dnd_pipeline_narratio_notarius_outputs: + item_registry: + lane_id: item-registry + media_type: application/json + schema_id: notarius.dnd.item_registry + schema_version: v1 + module_key: dnd/item-registry + npc_registry: + lane_id: npc-registry + media_type: application/json + schema_id: notarius.dnd.npc_registry + schema_version: v1 + module_key: dnd/npc-registry + location_registry: + lane_id: location-registry + media_type: application/json + schema_id: notarius.dnd.location_registry + schema_version: v1 + module_key: dnd/location-registry + scene_descriptions: + lane_id: scene-descriptions + media_type: application/json + schema_id: notarius.dnd.scene_descriptions + schema_version: v1 + module_key: dnd/scene-descriptions + item_occurrences: + lane_id: item-occurrences + media_type: application/json + schema_id: notarius.dnd.item_occurrences + schema_version: v1 + module_key: dnd/item-occurrences + spells: + lane_id: spells + media_type: application/json + schema_id: notarius.dnd.spells + schema_version: v1 + module_key: dnd/spells + combat_turns: + lane_id: combat-turns + media_type: application/json + schema_id: notarius.dnd.combat_turns + schema_version: v1 + module_key: dnd/combat-turns + npc_occurrences: + lane_id: npc-occurrences + media_type: application/json + schema_id: notarius.dnd.npc_occurrences + schema_version: v1 + module_key: dnd/npc-occurrences + location_occurrences: + lane_id: location-occurrences + media_type: application/json + schema_id: notarius.dnd.location_occurrences + schema_version: v1 + module_key: dnd/location-occurrences + enemy_events: + lane_id: enemy-events + media_type: application/json + schema_id: notarius.dnd.enemy_events + schema_version: v1 + module_key: dnd/enemy-events +dnd_pipeline_narratio_render_enabled: true +dnd_pipeline_narratio_render_format: markdown +dnd_pipeline_narratio_session_recap_artifact: + enabled: true + prompt_id: dnd.session_recap + output_path: artifacts/session_recap.md + inputs: + transcript: + source: narratio.transcript.final_trimmed + required: true + recap: + source: narratio.previous_session.artifact.session_recap + required: false +dnd_pipeline_narratio_hostile_rumors_artifact: + enabled: true + prompt_id: dnd.hostile_rumors + output_path: artifacts/hostile_rumors.md + inputs: + recap: + source: narratio.artifact.session_recap + required: true +dnd_pipeline_narratio_deranged_rumors_artifact: + enabled: true + prompt_id: dnd.deranged_rumors + output_path: artifacts/deranged_rumors.md + inputs: + recap: + source: narratio.artifact.session_recap + required: true +dnd_pipeline_narratio_scene_map_artifact: + enabled: false + prompt_id: dnd.scene_map + output_path: artifacts/scene_map.json + inputs: + transcript: + source: narratio.transcript.final_trimmed + required: true +dnd_pipeline_narratio_character_analysis_family: + enabled: false + for_each: party.characters + prompt_id: dnd.character_meta_analysis + output_path_pattern: artifacts/character_meta_analysis/{character_id}.md + inputs: + transcript: + source: narratio.transcript.final_trimmed + required: true + recap: + source: narratio.previous_session.artifact.session_recap + required: false + member_vars: + character_name: character.name + character_class: character.class_summary +dnd_pipeline_narratio_testing_scriptorium_profile: gemini-flash-latest diff --git a/meta/argument_specs.yml b/meta/argument_specs.yml new file mode 100644 index 0000000..52f7506 --- /dev/null +++ b/meta/argument_specs.yml @@ -0,0 +1,302 @@ +--- +argument_specs: + main: + short_description: Compose the D&D transcript-processing application suite + options: + dnd_pipeline_audita_version: + type: str + required: true + description: Audita application revision for the compatible suite. + dnd_pipeline_narratio_version: + type: str + required: true + description: Narratio application revision for the compatible suite. + dnd_pipeline_notarius_version: + type: str + required: true + description: Notarius application revision for the compatible suite. + dnd_pipeline_seriatim_version: + type: str + required: true + description: Seriatim application revision for the compatible suite. + dnd_pipeline_scriptorium_version: + type: str + required: true + description: Scriptorium application revision for the compatible suite. + dnd_pipeline_manage_build_dependencies: + type: bool + required: true + description: Whether every application role manages Git and Go. + dnd_pipeline_go_snap_channel: + type: str + required: true + description: Go Snap channel passed to every application role. + dnd_pipeline_go_binary: + type: path + required: true + description: Go executable passed to every application role. + dnd_pipeline_runtime_user: + type: str + required: true + description: Service account that owns suite-managed configuration. + dnd_pipeline_runtime_group: + type: str + required: true + description: Shared group for suite-managed configuration. + dnd_pipeline_secrets: + type: dict + required: true + description: Explicit credential-file names and values managed beneath Narratio's secrets directory. + dnd_pipeline_config_directory_mode: + type: str + required: true + description: Mode for suite-managed configuration directories. + dnd_pipeline_config_file_mode: + type: str + required: true + description: Mode for suite-managed configuration files. + dnd_pipeline_whisperx_transcribe_url: + type: str + required: true + description: WhisperX transcription endpoint passed to Narratio. + dnd_pipeline_campaigns_repository_url: + type: str + required: true + description: Git origin containing the campaign registry. + dnd_pipeline_campaigns_revision: + type: str + required: true + description: Campaign branch, tag, or commit to deploy. + dnd_pipeline_campaigns_root: + type: path + required: true + description: Campaign checkout and Narratio registry root. + dnd_pipeline_prompt_library_repository_url: + type: str + required: true + description: Git origin containing the prompt library. + dnd_pipeline_prompt_library_revision: + type: str + required: true + description: Prompt-library branch, tag, or commit to deploy. + dnd_pipeline_prompt_library_root: + type: path + required: true + description: Prompt-library checkout root. + dnd_pipeline_audita_pipeline_modules: + type: list + elements: str + required: true + description: Ordered Audita processing modules. + dnd_pipeline_audita_output_schema: + type: str + required: true + description: Audita output schema shared with Narratio. + dnd_pipeline_audita_llm_proposal_base_url: + type: str + required: true + description: Audita proposal LLM endpoint. + dnd_pipeline_audita_llm_proposal_model: + type: str + required: true + description: Audita proposal LLM model. + dnd_pipeline_audita_llm_proposal_api_key_env: + type: str + required: true + description: Proposal LLM API-key environment variable name. + dnd_pipeline_audita_llm_proposal_timeout: + type: str + required: true + description: Proposal LLM timeout. + dnd_pipeline_audita_llm_proposal_max_retries: + type: int + required: true + description: Proposal LLM retry count. + dnd_pipeline_audita_llm_validation_base_url: + type: str + required: true + description: Audita validation LLM endpoint. + dnd_pipeline_audita_llm_validation_model: + type: str + required: true + description: Audita validation LLM model. + dnd_pipeline_audita_llm_validation_api_key_env: + type: str + required: true + description: Validation LLM API-key environment variable name. + dnd_pipeline_audita_llm_validation_timeout: + type: str + required: true + description: Validation LLM timeout. + dnd_pipeline_audita_llm_validation_max_retries: + type: int + required: true + description: Validation LLM retry count. + dnd_pipeline_audita_concurrency_total_llm: + type: int + required: true + description: Total Audita LLM concurrency. + dnd_pipeline_audita_concurrency_proposal_llm: + type: int + required: true + description: Proposal Audita LLM concurrency. + dnd_pipeline_audita_concurrency_validation_llm: + type: int + required: true + description: Validation Audita LLM concurrency. + dnd_pipeline_audita_chunking_target_sections: + type: int + required: true + description: Audita target section count. + dnd_pipeline_audita_chunking_max_section_tokens: + type: int + required: true + description: Audita maximum section tokens. + dnd_pipeline_audita_chunking_min_section_tokens: + type: int + required: true + description: Audita minimum section tokens. + dnd_pipeline_audita_normalization_max_segment_gap: + type: str + required: true + description: Audita maximum segment gap. + dnd_pipeline_audita_normalization_ellipsis_gap: + type: str + required: true + description: Audita ellipsis gap. + dnd_pipeline_audita_normalization_max_segment_duration: + type: str + required: true + description: Audita maximum segment duration. + dnd_pipeline_audita_normalization_max_segment_tokens: + type: int + required: true + description: Audita maximum segment tokens. + dnd_pipeline_audita_thresholds_glossary: + type: float + required: true + description: Audita glossary confidence threshold. + dnd_pipeline_audita_thresholds_homophones: + type: float + required: true + description: Audita homophone confidence threshold. + dnd_pipeline_audita_thresholds_spoken_word: + type: float + required: true + description: Audita spoken-word confidence threshold. + dnd_pipeline_audita_thresholds_grammar: + type: float + required: true + description: Audita grammar confidence threshold. + dnd_pipeline_audita_context_description: + type: str + required: true + description: Audita transcript context description. + dnd_pipeline_audita_diagnostics_work_dir: + type: str + required: true + description: Audita diagnostics work directory. + dnd_pipeline_audita_diagnostics_retention: + type: str + required: true + description: Audita diagnostics retention policy. + dnd_pipeline_notarius_pipelines: + type: dict + required: true + description: Complete native map of Notarius pipeline definitions. + dnd_pipeline_notarius_cache_root: + type: path + required: true + description: Shared root for Notarius cache state. + dnd_pipeline_notarius_cache_chunk_plans_mode: + type: str + required: true + description: Notarius chunk-plan cache policy. + dnd_pipeline_notarius_cache_checkpoints_enabled: + type: bool + required: true + description: Whether Notarius records checkpoints. + dnd_pipeline_narratio_seriatim_output_schema: + type: str + required: true + description: Seriatim schema consumed by the D&D processing workflow. + dnd_pipeline_narratio_normalize_output_path: + type: str + required: true + description: Canonical normalized transcript output path. + dnd_pipeline_narratio_normalize_output_schema: + type: str + required: true + description: Schema for the normalized transcript output. + dnd_pipeline_narratio_trim_enabled: + type: bool + required: true + description: Whether D&D transcript trimming is enabled. + dnd_pipeline_narratio_trim_output_path: + type: str + required: true + description: Canonical trimmed transcript output path. + dnd_pipeline_narratio_trim_bounds_prompt_id: + type: str + required: true + description: Prompt that identifies D&D session bounds. + dnd_pipeline_narratio_trim_bounds_transcript_input_name: + type: str + required: true + description: Bounds prompt input receiving the transcript. + dnd_pipeline_narratio_trim_bounds_output_path: + type: str + required: true + description: Canonical session-bounds artifact output path. + dnd_pipeline_narratio_notarius_pipeline_id: + type: str + required: true + description: Notarius pipeline selected by Narratio. + dnd_pipeline_narratio_notarius_llm_api_key: + type: str + required: true + description: Environment-variable name forwarded by Narratio to Notarius. + dnd_pipeline_narratio_notarius_resume: + type: bool + required: true + description: Whether ordinary Narratio extraction requests Notarius checkpoint reuse. + dnd_pipeline_narratio_notarius_references: + type: dict + required: true + description: Complete native map of Narratio prepared-reference bindings. + dnd_pipeline_narratio_notarius_outputs: + type: dict + required: true + description: Complete native map of accepted Notarius lane outputs. + dnd_pipeline_narratio_render_enabled: + type: bool + required: true + description: Whether transcript rendering is enabled. + dnd_pipeline_narratio_render_format: + type: str + required: true + description: Format for rendered D&D transcripts. + dnd_pipeline_narratio_session_recap_artifact: + type: dict + required: true + description: Complete native mapping for the session recap artifact. + dnd_pipeline_narratio_hostile_rumors_artifact: + type: dict + required: true + description: Complete native mapping for the hostile rumors artifact. + dnd_pipeline_narratio_deranged_rumors_artifact: + type: dict + required: true + description: Complete native mapping for the deranged rumors artifact. + dnd_pipeline_narratio_scene_map_artifact: + type: dict + required: true + description: Complete native mapping for the scene map artifact. + dnd_pipeline_narratio_character_analysis_family: + type: dict + required: true + description: Complete native mapping for the character analysis family. + dnd_pipeline_narratio_testing_scriptorium_profile: + type: str + required: true + description: Scriptorium profile selected by the testing overlay. diff --git a/meta/main.yml b/meta/main.yml new file mode 100644 index 0000000..d0b175d --- /dev/null +++ b/meta/main.yml @@ -0,0 +1,129 @@ +--- +galaxy_info: + role_name: dnd_pipeline + namespace: rakestraw + author: Eric Rakestraw + description: Compose the D&D transcript-processing application suite + license: CC0-1.0 + min_ansible_version: "2.18" + issue_tracker_url: https://gitea.maximumdirect.net/eric/ansible-role-dnd-pipeline/issues + platforms: + - name: Ubuntu + versions: + - resolute + galaxy_tags: + - dnd + - pipeline + - transcript + - processing +dependencies: + - role: rakestraw.narratio + vars: + narratio_version: "{{ dnd_pipeline_narratio_version }}" + narratio_manage_build_dependencies: "{{ dnd_pipeline_manage_build_dependencies }}" + narratio_go_snap_channel: "{{ dnd_pipeline_go_snap_channel }}" + narratio_go_binary: "{{ dnd_pipeline_go_binary }}" + narratio_user: "{{ dnd_pipeline_runtime_user }}" + narratio_group: "{{ dnd_pipeline_runtime_group }}" + narratio_config_path: /usr/local/etc/narratio/pipeline.yml + narratio_config_owner: "{{ dnd_pipeline_runtime_user }}" + narratio_config_group: "{{ dnd_pipeline_runtime_group }}" + narratio_config_directory_mode: "{{ dnd_pipeline_config_directory_mode }}" + narratio_config_file_mode: "{{ dnd_pipeline_config_file_mode }}" + narratio_whisperx_transcribe_url: "{{ dnd_pipeline_whisperx_transcribe_url }}" + narratio_campaigns_root: "{{ dnd_pipeline_campaigns_root }}" + narratio_configkit_imports: + - path: conf.d/processing.yml + - path: artifacts/session-recap.yml + - path: artifacts/hostile-rumors.yml + - path: artifacts/deranged-rumors.yml + - path: artifacts/scene-map.yml + - path: artifact-families/character-meta-analysis.yml + narratio_seriatim_binary: /usr/local/bin/seriatim + narratio_audita_binary: /usr/local/bin/audita + narratio_audita_config_path: /usr/local/etc/audita/config.yml + narratio_audita_llm_api_key: "{{ dnd_pipeline_audita_llm_proposal_api_key_env }}" + narratio_notarius_binary: /usr/local/bin/notarius + narratio_notarius_config_path: /usr/local/etc/notarius/config.yml + narratio_notarius_enabled: true + narratio_notarius_pipeline_id: "{{ dnd_pipeline_narratio_notarius_pipeline_id }}" + narratio_notarius_working_directory: /usr/local/etc/notarius + narratio_notarius_llm_api_key: "{{ dnd_pipeline_narratio_notarius_llm_api_key }}" + narratio_notarius_resume: "{{ dnd_pipeline_narratio_notarius_resume }}" + narratio_notarius_references: "{{ dnd_pipeline_narratio_notarius_references }}" + narratio_notarius_outputs: "{{ dnd_pipeline_narratio_notarius_outputs }}" + narratio_scriptorium_binary: /usr/local/bin/scriptorium + narratio_scriptorium_config_path: /usr/local/etc/scriptorium/config.yml + - role: rakestraw.seriatim + vars: + seriatim_version: "{{ dnd_pipeline_seriatim_version }}" + seriatim_manage_build_dependencies: "{{ dnd_pipeline_manage_build_dependencies }}" + seriatim_go_snap_channel: "{{ dnd_pipeline_go_snap_channel }}" + seriatim_go_binary: "{{ dnd_pipeline_go_binary }}" + - role: rakestraw.audita + vars: + audita_version: "{{ dnd_pipeline_audita_version }}" + audita_manage_build_dependencies: "{{ dnd_pipeline_manage_build_dependencies }}" + audita_go_snap_channel: "{{ dnd_pipeline_go_snap_channel }}" + audita_go_binary: "{{ dnd_pipeline_go_binary }}" + audita_config_owner: "{{ dnd_pipeline_runtime_user }}" + audita_config_group: "{{ dnd_pipeline_runtime_group }}" + audita_config_directory_mode: "{{ dnd_pipeline_config_directory_mode }}" + audita_config_file_mode: "{{ dnd_pipeline_config_file_mode }}" + audita_pipeline_modules: "{{ dnd_pipeline_audita_pipeline_modules }}" + audita_output_schema: "{{ dnd_pipeline_audita_output_schema }}" + audita_llm_proposal_base_url: "{{ dnd_pipeline_audita_llm_proposal_base_url }}" + audita_llm_proposal_model: "{{ dnd_pipeline_audita_llm_proposal_model }}" + audita_llm_proposal_api_key_env: "{{ dnd_pipeline_audita_llm_proposal_api_key_env }}" + audita_llm_proposal_timeout: "{{ dnd_pipeline_audita_llm_proposal_timeout }}" + audita_llm_proposal_max_retries: "{{ dnd_pipeline_audita_llm_proposal_max_retries }}" + audita_llm_validation_base_url: "{{ dnd_pipeline_audita_llm_validation_base_url }}" + audita_llm_validation_model: "{{ dnd_pipeline_audita_llm_validation_model }}" + audita_llm_validation_api_key_env: "{{ dnd_pipeline_audita_llm_validation_api_key_env }}" + audita_llm_validation_timeout: "{{ dnd_pipeline_audita_llm_validation_timeout }}" + audita_llm_validation_max_retries: "{{ dnd_pipeline_audita_llm_validation_max_retries }}" + audita_concurrency_total_llm: "{{ dnd_pipeline_audita_concurrency_total_llm }}" + audita_concurrency_proposal_llm: "{{ dnd_pipeline_audita_concurrency_proposal_llm }}" + audita_concurrency_validation_llm: "{{ dnd_pipeline_audita_concurrency_validation_llm }}" + audita_chunking_target_sections: "{{ dnd_pipeline_audita_chunking_target_sections }}" + audita_chunking_max_section_tokens: "{{ dnd_pipeline_audita_chunking_max_section_tokens }}" + audita_chunking_min_section_tokens: "{{ dnd_pipeline_audita_chunking_min_section_tokens }}" + audita_normalization_max_segment_gap: "{{ dnd_pipeline_audita_normalization_max_segment_gap }}" + audita_normalization_ellipsis_gap: "{{ dnd_pipeline_audita_normalization_ellipsis_gap }}" + audita_normalization_max_segment_duration: "{{ dnd_pipeline_audita_normalization_max_segment_duration }}" + audita_normalization_max_segment_tokens: "{{ dnd_pipeline_audita_normalization_max_segment_tokens }}" + audita_thresholds_glossary: "{{ dnd_pipeline_audita_thresholds_glossary }}" + audita_thresholds_homophones: "{{ dnd_pipeline_audita_thresholds_homophones }}" + audita_thresholds_spoken_word: "{{ dnd_pipeline_audita_thresholds_spoken_word }}" + audita_thresholds_grammar: "{{ dnd_pipeline_audita_thresholds_grammar }}" + audita_context_description: "{{ dnd_pipeline_audita_context_description }}" + audita_diagnostics_work_dir: "{{ dnd_pipeline_audita_diagnostics_work_dir }}" + audita_diagnostics_retention: "{{ dnd_pipeline_audita_diagnostics_retention }}" + - role: rakestraw.notarius + vars: + notarius_version: "{{ dnd_pipeline_notarius_version }}" + notarius_manage_build_dependencies: "{{ dnd_pipeline_manage_build_dependencies }}" + notarius_go_snap_channel: "{{ dnd_pipeline_go_snap_channel }}" + notarius_go_binary: "{{ dnd_pipeline_go_binary }}" + notarius_config_owner: "{{ dnd_pipeline_runtime_user }}" + notarius_config_group: "{{ dnd_pipeline_runtime_group }}" + notarius_config_directory_mode: "{{ dnd_pipeline_config_directory_mode }}" + notarius_config_file_mode: "{{ dnd_pipeline_config_file_mode }}" + notarius_configkit_imports: + - path: pipelines/dnd-session.yml + notarius_cache_chunk_plans_mode: "{{ dnd_pipeline_notarius_cache_chunk_plans_mode }}" + notarius_cache_chunk_plans_directory: "{{ dnd_pipeline_notarius_cache_root ~ '/chunk-plans' }}" + notarius_cache_checkpoints_enabled: "{{ dnd_pipeline_notarius_cache_checkpoints_enabled }}" + notarius_cache_checkpoints_directory: "{{ dnd_pipeline_notarius_cache_root ~ '/checkpoints' }}" + - role: rakestraw.scriptorium + vars: + scriptorium_version: "{{ dnd_pipeline_scriptorium_version }}" + scriptorium_manage_build_dependencies: "{{ dnd_pipeline_manage_build_dependencies }}" + scriptorium_go_snap_channel: "{{ dnd_pipeline_go_snap_channel }}" + scriptorium_go_binary: "{{ dnd_pipeline_go_binary }}" + scriptorium_config_owner: "{{ dnd_pipeline_runtime_user }}" + scriptorium_config_group: "{{ dnd_pipeline_runtime_group }}" + scriptorium_config_directory_mode: "{{ dnd_pipeline_config_directory_mode }}" + scriptorium_config_file_mode: "{{ dnd_pipeline_config_file_mode }}" + scriptorium_prompt_dir: "{{ dnd_pipeline_prompt_library_root ~ '/prompts' }}" + scriptorium_schema_dir: "{{ dnd_pipeline_prompt_library_root ~ '/schemas' }}" diff --git a/requirements.yml b/requirements.yml new file mode 100644 index 0000000..bb8af90 --- /dev/null +++ b/requirements.yml @@ -0,0 +1,22 @@ +--- +roles: + - name: rakestraw.seriatim + src: https://gitea.maximumdirect.net/eric/ansible-role-seriatim.git + scm: git + version: e08c63c0b078658bdfa2d10a2e6e1e4f9372a075 + - name: rakestraw.audita + src: https://gitea.maximumdirect.net/eric/ansible-role-audita.git + scm: git + version: 66ebea9a47ce82bfdbdc5e2855f3dfaaff8e2a16 + - name: rakestraw.notarius + src: https://gitea.maximumdirect.net/eric/ansible-role-notarius.git + scm: git + version: 14a38e04e35ef524d6a97fa7fb5a04981a0a2161 + - name: rakestraw.scriptorium + src: https://gitea.maximumdirect.net/eric/ansible-role-scriptorium.git + scm: git + version: 8b4c86e48e7d8655e479b26dc510bf01683a0c8f + - name: rakestraw.narratio + src: https://gitea.maximumdirect.net/eric/ansible-role-narratio.git + scm: git + version: 292d8669e7aad5cfe80b0c469d22b6177e8cf0f1 diff --git a/tasks/main.yml b/tasks/main.yml new file mode 100644 index 0000000..5cffe1c --- /dev/null +++ b/tasks/main.yml @@ -0,0 +1,174 @@ +--- +- name: Install ACL support + become: true + ansible.builtin.apt: + name: acl + state: present + update_cache: true + cache_valid_time: 3600 + +- name: Validate managed Narratio credential entries + ansible.builtin.assert: + that: + - item.key is string + - item.key is match('^[A-Za-z_][A-Za-z0-9_]*$') + - item.value is string + loop: "{{ dnd_pipeline_secrets | dict2items }}" + loop_control: + label: "{{ item.key }}" + no_log: true + +- name: Create the Narratio secrets directory + become: true + ansible.builtin.file: + path: /usr/local/etc/narratio/secrets + state: directory + owner: "{{ dnd_pipeline_runtime_user }}" + group: "{{ dnd_pipeline_runtime_group }}" + mode: "0750" + +- name: Converge managed Narratio credentials + become: true + ansible.builtin.copy: + content: "{{ item.value }}" + dest: "/usr/local/etc/narratio/secrets/{{ item.key }}" + owner: "{{ dnd_pipeline_runtime_user }}" + group: "{{ dnd_pipeline_runtime_group }}" + mode: "0640" + loop: "{{ dnd_pipeline_secrets | dict2items }}" + loop_control: + label: "{{ item.key }}" + no_log: true + +- name: Create the Notarius cache root + become: true + ansible.builtin.file: + path: "{{ dnd_pipeline_notarius_cache_root }}" + state: directory + owner: "{{ dnd_pipeline_runtime_user }}" + group: "{{ dnd_pipeline_runtime_group }}" + mode: "2775" + +- name: Converge access ACLs on shared pipeline writable roots + become: true + ansible.posix.acl: + path: "{{ item.0 }}" + etype: "{{ item.1.etype }}" + permissions: "{{ item.1.permissions }}" + state: present + loop: >- + {{ + ['/var/lib/narratio', '/var/spool/narratio', '/var/cache/narratio', dnd_pipeline_notarius_cache_root] | + product([ + {'etype': 'user', 'permissions': 'rwx'}, + {'etype': 'group', 'permissions': 'rwx'}, + {'etype': 'other', 'permissions': 'r-x'}, + {'etype': 'mask', 'permissions': 'rwx'} + ]) | list + }} + loop_control: + label: "{{ item.0 }} {{ item.1.etype }}::{{ item.1.permissions }}" + +- name: Converge default ACLs on shared pipeline writable roots + become: true + ansible.posix.acl: + path: "{{ item.0 }}" + etype: "{{ item.1.etype }}" + permissions: "{{ item.1.permissions }}" + default: true + state: present + loop: >- + {{ + ['/var/lib/narratio', '/var/spool/narratio', '/var/cache/narratio', dnd_pipeline_notarius_cache_root] | + product([ + {'etype': 'user', 'permissions': 'rwx'}, + {'etype': 'group', 'permissions': 'rwx'}, + {'etype': 'other', 'permissions': 'r-x'}, + {'etype': 'mask', 'permissions': 'rwx'} + ]) | list + }} + loop_control: + label: "{{ item.0 }} default:{{ item.1.etype }}::{{ item.1.permissions }}" + +- name: Create the prompt-library root + become: true + ansible.builtin.file: + path: "{{ dnd_pipeline_prompt_library_root }}" + state: directory + owner: root + group: "{{ dnd_pipeline_runtime_group }}" + mode: "0755" + +- name: Deploy the campaign registry + become: true + ansible.builtin.git: + repo: "{{ dnd_pipeline_campaigns_repository_url }}" + dest: "{{ dnd_pipeline_campaigns_root }}" + version: "{{ dnd_pipeline_campaigns_revision }}" + update: true + force: true + +- name: Deploy the prompt library + become: true + ansible.builtin.git: + repo: "{{ dnd_pipeline_prompt_library_repository_url }}" + dest: "{{ dnd_pipeline_prompt_library_root }}" + version: "{{ dnd_pipeline_prompt_library_revision }}" + update: true + force: true + +- name: Create Narratio configuration directories + become: true + ansible.builtin.file: + path: "/usr/local/etc/narratio/{{ item }}" + state: directory + owner: "{{ dnd_pipeline_runtime_user }}" + group: "{{ dnd_pipeline_runtime_group }}" + mode: "{{ dnd_pipeline_config_directory_mode }}" + loop: + - conf.d + - artifacts + - artifact-families + - profiles + +- name: Create the Notarius pipeline configuration directory + become: true + ansible.builtin.file: + path: /usr/local/etc/notarius/pipelines + state: directory + owner: "{{ dnd_pipeline_runtime_user }}" + group: "{{ dnd_pipeline_runtime_group }}" + mode: "{{ dnd_pipeline_config_directory_mode }}" + +- name: Deploy the Notarius D&D pipeline configuration + become: true + ansible.builtin.template: + src: dnd-session.yml.j2 + dest: /usr/local/etc/notarius/pipelines/dnd-session.yml + owner: "{{ dnd_pipeline_runtime_user }}" + group: "{{ dnd_pipeline_runtime_group }}" + mode: "{{ dnd_pipeline_config_file_mode }}" + +- name: Deploy Narratio D&D configuration fragments + become: true + ansible.builtin.template: + src: "{{ item.src }}" + dest: "/usr/local/etc/narratio/{{ item.dest }}" + owner: "{{ dnd_pipeline_runtime_user }}" + group: "{{ dnd_pipeline_runtime_group }}" + mode: "{{ dnd_pipeline_config_file_mode }}" + loop: + - src: processing.yml.j2 + dest: conf.d/processing.yml + - src: session-recap.yml.j2 + dest: artifacts/session-recap.yml + - src: hostile-rumors.yml.j2 + dest: artifacts/hostile-rumors.yml + - src: deranged-rumors.yml.j2 + dest: artifacts/deranged-rumors.yml + - src: scene-map.yml.j2 + dest: artifacts/scene-map.yml + - src: character-meta-analysis.yml.j2 + dest: artifact-families/character-meta-analysis.yml + - src: testing.yml.j2 + dest: profiles/testing.yml diff --git a/templates/character-meta-analysis.yml.j2 b/templates/character-meta-analysis.yml.j2 new file mode 100644 index 0000000..2c9d666 --- /dev/null +++ b/templates/character-meta-analysis.yml.j2 @@ -0,0 +1,4 @@ +scriptorium: + artifact_families: + character_meta_analysis: +{{ dnd_pipeline_narratio_character_analysis_family | to_nice_yaml(indent=2) | indent(6, true) }} diff --git a/templates/deranged-rumors.yml.j2 b/templates/deranged-rumors.yml.j2 new file mode 100644 index 0000000..f99dafe --- /dev/null +++ b/templates/deranged-rumors.yml.j2 @@ -0,0 +1,4 @@ +scriptorium: + artifacts: + deranged_rumors: +{{ dnd_pipeline_narratio_deranged_rumors_artifact | to_nice_yaml(indent=2) | indent(6, true) }} diff --git a/templates/dnd-session.yml.j2 b/templates/dnd-session.yml.j2 new file mode 100644 index 0000000..5ea5b89 --- /dev/null +++ b/templates/dnd-session.yml.j2 @@ -0,0 +1,2 @@ +pipelines: +{{ dnd_pipeline_notarius_pipelines | to_nice_yaml(indent=2) | indent(2, true) }} diff --git a/templates/hostile-rumors.yml.j2 b/templates/hostile-rumors.yml.j2 new file mode 100644 index 0000000..95d2456 --- /dev/null +++ b/templates/hostile-rumors.yml.j2 @@ -0,0 +1,4 @@ +scriptorium: + artifacts: + hostile_rumors: +{{ dnd_pipeline_narratio_hostile_rumors_artifact | to_nice_yaml(indent=2) | indent(6, true) }} diff --git a/templates/processing.yml.j2 b/templates/processing.yml.j2 new file mode 100644 index 0000000..514ecdc --- /dev/null +++ b/templates/processing.yml.j2 @@ -0,0 +1,17 @@ +audita: + output_schema: {{ dnd_pipeline_audita_output_schema | to_json }} +seriatim: + output_schema: {{ dnd_pipeline_narratio_seriatim_output_schema | to_json }} +normalize: + output_path: {{ dnd_pipeline_narratio_normalize_output_path | to_json }} + output_schema: {{ dnd_pipeline_narratio_normalize_output_schema | to_json }} +trim: + enabled: {{ dnd_pipeline_narratio_trim_enabled | to_json }} + output_path: {{ dnd_pipeline_narratio_trim_output_path | to_json }} + bounds: + prompt_id: {{ dnd_pipeline_narratio_trim_bounds_prompt_id | to_json }} + transcript_input_name: {{ dnd_pipeline_narratio_trim_bounds_transcript_input_name | to_json }} + output_path: {{ dnd_pipeline_narratio_trim_bounds_output_path | to_json }} +render: + enabled: {{ dnd_pipeline_narratio_render_enabled | to_json }} + format: {{ dnd_pipeline_narratio_render_format | to_json }} diff --git a/templates/scene-map.yml.j2 b/templates/scene-map.yml.j2 new file mode 100644 index 0000000..c5f5cd0 --- /dev/null +++ b/templates/scene-map.yml.j2 @@ -0,0 +1,4 @@ +scriptorium: + artifacts: + scene_map: +{{ dnd_pipeline_narratio_scene_map_artifact | to_nice_yaml(indent=2) | indent(6, true) }} diff --git a/templates/session-recap.yml.j2 b/templates/session-recap.yml.j2 new file mode 100644 index 0000000..1ee818f --- /dev/null +++ b/templates/session-recap.yml.j2 @@ -0,0 +1,4 @@ +scriptorium: + artifacts: + session_recap: +{{ dnd_pipeline_narratio_session_recap_artifact | to_nice_yaml(indent=2) | indent(6, true) }} diff --git a/templates/testing.yml.j2 b/templates/testing.yml.j2 new file mode 100644 index 0000000..51dacc7 --- /dev/null +++ b/templates/testing.yml.j2 @@ -0,0 +1,13 @@ +scriptorium: + artifacts: + session_recap: + profile_id: {{ dnd_pipeline_narratio_testing_scriptorium_profile | to_json }} + hostile_rumors: + profile_id: {{ dnd_pipeline_narratio_testing_scriptorium_profile | to_json }} + deranged_rumors: + profile_id: {{ dnd_pipeline_narratio_testing_scriptorium_profile | to_json }} + scene_map: + profile_id: {{ dnd_pipeline_narratio_testing_scriptorium_profile | to_json }} + artifact_families: + character_meta_analysis: + profile_id: {{ dnd_pipeline_narratio_testing_scriptorium_profile | to_json }} diff --git a/tests/converge.yml b/tests/converge.yml new file mode 100644 index 0000000..a3dffa8 --- /dev/null +++ b/tests/converge.yml @@ -0,0 +1,6 @@ +--- +- name: Converge the D&D pipeline role + hosts: localhost + gather_facts: true + roles: + - role: ansible-role-dnd-pipeline diff --git a/tests/integration b/tests/integration new file mode 100755 index 0000000..6d6f7c4 --- /dev/null +++ b/tests/integration @@ -0,0 +1,212 @@ +#!/usr/bin/env bash +set -euo pipefail + +repository_root=$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd) +workspace_venv="$repository_root/../.venv" +test_tmp=$(mktemp -d) +chmod 0700 "$test_tmp" + +export ANSIBLE_COLLECTIONS_PATH="$test_tmp/collections" +export ANSIBLE_ROLES_PATH="$test_tmp/roles:$(dirname "$repository_root")" + +inventory="$repository_root/tests/inventory" +converge="$repository_root/tests/converge.yml" +ansible_playbook="$workspace_venv/bin/ansible-playbook" +profile_dir="$test_tmp/profiles" +campaigns_repository=$(cd "$repository_root/../assets/campaigns" && pwd) +prompt_library_repository=$(cd "$repository_root/../assets/narratio-prompts" && pwd) +prompt_library_root=/usr/local/share/narratio/prompt-library +campaigns_revision=$(git -C "$campaigns_repository" rev-parse HEAD) +prompt_library_revision=$(git -C "$prompt_library_repository" rev-parse HEAD) +operator_user="dnd_f0009_${$}" +shared_fixture_root="/var/cache/notarius/${operator_user}" +service_fixture_file="$shared_fixture_root/service.txt" +operator_fixture_file="$shared_fixture_root/operator.txt" +synthetic_audita_api_key='audita-pipeline-synthetic-value' +synthetic_notarius_api_key='notarius-pipeline-synthetic-value' +baseline_args=( + -e 'dnd_pipeline_whisperx_transcribe_url=http://127.0.0.1:8000/transcribe' + -e "dnd_pipeline_campaigns_repository_url=file://$campaigns_repository" + -e "dnd_pipeline_campaigns_revision=$campaigns_revision" + -e "dnd_pipeline_prompt_library_repository_url=file://$prompt_library_repository" + -e "dnd_pipeline_prompt_library_revision=$prompt_library_revision" + -e "dnd_pipeline_prompt_library_root=$prompt_library_root" + -e "{\"dnd_pipeline_secrets\":{\"AUDITA_LLM_API_KEY\":\"$synthetic_audita_api_key\",\"OPENROUTER_API_KEY\":\"$synthetic_notarius_api_key\"}}" +) + +cleanup() { + exit_status=$? + trap - EXIT + /usr/bin/sudo /bin/rm -rf -- "$shared_fixture_root" || true + /usr/bin/sudo /usr/sbin/userdel --remove "$operator_user" 2>/dev/null || true + "$ansible_playbook" -i "$inventory" "${baseline_args[@]}" "$converge" >/dev/null 2>&1 || true + rm -rf -- "$test_tmp" + exit "$exit_status" +} +trap cleanup EXIT + +require_clean_recap() { + grep -Eq 'changed=0 .*failed=0' "$1" +} + +run_converge() { + output_file=$1 + shift + env -u AUDITA_LLM_API_KEY -u OPENROUTER_API_KEY \ + "$ansible_playbook" -i "$inventory" "$@" "$converge" | tee "$output_file" +} + +cd "$repository_root" +mkdir -p "$profile_dir" +printf '%s\n' \ + 'id: endpoint-profile' \ + 'endpoint: http://127.0.0.1:8000/v1' \ + 'model: offline-fixture' > "$profile_dir/endpoint-profile.yml" +"$workspace_venv/bin/ansible-galaxy" collection install \ + -r collections/requirements.yml \ + -p "$ANSIBLE_COLLECTIONS_PATH" +"$workspace_venv/bin/ansible-galaxy" role install \ + -r requirements.yml \ + -p "$test_tmp/roles" +"$ansible_playbook" -i "$inventory" --syntax-check "$converge" +"$workspace_venv/bin/ansible-lint" --offline --profile production "$converge" + +baseline_log="$test_tmp/baseline.log" +run_converge "$baseline_log" "${baseline_args[@]}" +/usr/local/bin/seriatim --version +/usr/local/bin/audita config validate --config /usr/local/etc/audita/config.yml +/usr/local/bin/notarius config validate --config /usr/local/etc/notarius/config.yml \ + --pipeline dnd-session +notarius_effective_config="$test_tmp/notarius-effective.yml" +/usr/local/bin/notarius config effective --config /usr/local/etc/notarius/config.yml \ + > "$notarius_effective_config" +grep -Fq 'mode: auto' "$notarius_effective_config" +grep -Fq 'step: describe-session' "$notarius_effective_config" +/usr/local/bin/scriptorium inspect profile --config /usr/local/etc/scriptorium/config.yml \ + --profile-dir "$profile_dir" --profile endpoint-profile +/usr/bin/sudo -u narratio /usr/local/bin/narratio config validate \ + --config /usr/local/etc/narratio/pipeline.yml --campaign centaurs +/usr/bin/sudo -u narratio /usr/local/bin/narratio config validate \ + --config /usr/local/etc/narratio/pipeline.yml --campaign dilfs +/usr/bin/sudo -u narratio /usr/local/bin/narratio config validate \ + --config /usr/local/etc/narratio/pipeline.yml --campaign centaurs --profile testing +narratio_effective_config="$test_tmp/narratio-effective.yml" +/usr/bin/sudo -u narratio /usr/local/bin/narratio config show \ + --config /usr/local/etc/narratio/pipeline.yml --campaign centaurs \ + > "$narratio_effective_config" +grep -Fq 'enabled: true' "$narratio_effective_config" +grep -Fq 'llm_api_key: AUDITA_LLM_API_KEY' "$narratio_effective_config" +grep -Fq 'pipeline_id: dnd-session' "$narratio_effective_config" +grep -Fq 'llm_api_key: OPENROUTER_API_KEY' "$narratio_effective_config" +grep -Fq 'resume: true' "$narratio_effective_config" +! grep -Fq 'chunk_cache:' "$narratio_effective_config" +grep -Fq 'party: narratio.input.party' "$narratio_effective_config" +grep -Fq 'lane_id: item-registry' "$narratio_effective_config" +! grep -Fq "$synthetic_audita_api_key" "$baseline_log" "$narratio_effective_config" "$notarius_effective_config" +! grep -Fq "$synthetic_notarius_api_key" "$baseline_log" "$narratio_effective_config" "$notarius_effective_config" +/usr/bin/sudo -u narratio /usr/local/bin/narratio config artifacts \ + --config /usr/local/etc/narratio/pipeline.yml --campaign centaurs | \ + awk -F '\t' '$1 == "session_recap" && $4 == "true" { found = 1 } END { exit !found }' +/usr/bin/sudo -u narratio /usr/local/bin/narratio config artifacts \ + --config /usr/local/etc/narratio/pipeline.yml --campaign centaurs | \ + awk -F '\t' '$1 == "hostile_rumors" && $6 == "session_recap" { found = 1 } END { exit !found }' +/usr/bin/sudo -u narratio /usr/local/bin/narratio config artifacts \ + --config /usr/local/etc/narratio/pipeline.yml --campaign centaurs | \ + awk -F '\t' 'index($1, "character_meta_analysis_") == 1 && $4 == "false" && $7 == "character_meta_analysis" { found = 1 } END { exit !found }' +/usr/bin/sudo -u narratio /usr/local/bin/narratio config diff default testing \ + --config /usr/local/etc/narratio/pipeline.yml --campaign centaurs | \ + awk -F '\t' '$1 != "changed" || $2 !~ /\.profile_id$/ { invalid = 1 } END { exit invalid || NR == 0 }' +/usr/bin/sudo /usr/sbin/useradd --create-home --groups narratio \ + --shell /usr/sbin/nologin "$operator_user" +/usr/bin/sudo -u "$operator_user" /usr/local/bin/narratio config validate \ + --config /usr/local/etc/narratio/pipeline.yml --campaign centaurs +/usr/bin/sudo -u "$operator_user" /usr/local/bin/notarius config validate \ + --config /usr/local/etc/notarius/config.yml --pipeline dnd-session +/usr/bin/sudo -u nobody /bin/cat /usr/local/etc/narratio/pipeline.yml >/dev/null +if /usr/bin/sudo -u nobody /usr/local/bin/narratio config validate \ + --config /usr/local/etc/narratio/pipeline.yml --campaign centaurs; then + exit 1 +fi +/usr/bin/sudo -u narratio /bin/sh -c \ + "umask 077; mkdir '$shared_fixture_root'; printf '%s\\n' service-created > '$service_fixture_file'" +/usr/bin/sudo -u "$operator_user" /bin/sh -c \ + "grep -Fx service-created '$service_fixture_file'; printf '%s\\n' operator-appended >> '$service_fixture_file'" +/usr/bin/sudo -u narratio /bin/grep -Fx operator-appended "$service_fixture_file" +/usr/bin/sudo -u "$operator_user" /bin/sh -c \ + "umask 077; printf '%s\\n' operator-created > '$operator_fixture_file'" +/usr/bin/sudo -u narratio /bin/grep -Fx operator-created "$operator_fixture_file" +session_recap_prompt_json=$(/usr/local/bin/scriptorium inspect prompt \ + --config /usr/local/etc/scriptorium/config.yml \ + --prompt dnd.session_recap --prompt-version 2.0.0 --format json) +session_recap_profile=$(printf '%s\n' "$session_recap_prompt_json" | \ + "$workspace_venv/bin/python" -c 'import json, sys; print(json.load(sys.stdin)["default_profile_id"])') +/usr/local/bin/scriptorium inspect profile --config /usr/local/etc/scriptorium/config.yml \ + --profile "$session_recap_profile" +/usr/local/bin/scriptorium inspect prompt --config /usr/local/etc/scriptorium/config.yml \ + --prompt dnd.scene_map --prompt-version 1.0.0 --format json | \ + "$workspace_venv/bin/python" -c ' +import json +import sys +from pathlib import Path + +schema_root = Path(sys.argv[1]).resolve() +schema_path = json.load(sys.stdin)["output_contract"]["schema_path"] +schema = (schema_root / schema_path).resolve() +if not schema.is_relative_to(schema_root) or not schema.is_file(): + raise SystemExit(f"invalid deployed schema path: {schema}") +' "$prompt_library_root/schemas" +baseline_repeat_log="$test_tmp/baseline-repeat.log" +run_converge "$baseline_repeat_log" "${baseline_args[@]}" +require_clean_recap "$baseline_repeat_log" + +invalid_key_log="$test_tmp/invalid-key.log" +if run_converge "$invalid_key_log" "${baseline_args[@]}" \ + -e '{"dnd_pipeline_secrets":{"INVALID-KEY":"invalid-key-synthetic-value"}}'; then + exit 1 +fi + +testing_profile_log="$test_tmp/testing-profile.log" +run_converge "$testing_profile_log" "${baseline_args[@]}" \ + -e 'dnd_pipeline_narratio_testing_scriptorium_profile=endpoint-profile' +/usr/bin/sudo -u narratio /usr/local/bin/narratio config validate \ + --config /usr/local/etc/narratio/pipeline.yml --campaign centaurs --profile testing +/usr/bin/sudo -u narratio /usr/local/bin/narratio config diff default testing \ + --config /usr/local/etc/narratio/pipeline.yml --campaign centaurs | \ + grep -F '"endpoint-profile"' +/usr/local/bin/scriptorium inspect profile --config /usr/local/etc/scriptorium/config.yml \ + --profile-dir "$profile_dir" --profile endpoint-profile +testing_profile_repeat_log="$test_tmp/testing-profile-repeat.log" +run_converge "$testing_profile_repeat_log" "${baseline_args[@]}" \ + -e 'dnd_pipeline_narratio_testing_scriptorium_profile=endpoint-profile' +require_clean_recap "$testing_profile_repeat_log" + +selected_version_log="$test_tmp/selected-version.log" +run_converge "$selected_version_log" "${baseline_args[@]}" \ + -e 'dnd_pipeline_seriatim_version=v1.4.0' +/usr/local/bin/seriatim --version | grep -Fx 'seriatim version v1.4.0' +selected_version_repeat_log="$test_tmp/selected-version-repeat.log" +run_converge "$selected_version_repeat_log" "${baseline_args[@]}" \ + -e 'dnd_pipeline_seriatim_version=v1.4.0' +require_clean_recap "$selected_version_repeat_log" + +pipeline_group_one='{"archive-session":{"llm_profile":"dnd-extraction","input":{"module":"seriatim"},"artifacts":{"spells":{"extract":{"module":"dnd/spells"},"normalize":{"module":"dnd/spells"}}}}}' +pipeline_group_two='{"research-session":{"llm_profile":"dnd-extraction","input":{"module":"seriatim"},"artifacts":{"spells":{"extract":{"module":"dnd/spells"},"normalize":{"module":"dnd/spells"}}}}}' +alternate_config="{\"notarius_pipeline_group_one\":$pipeline_group_one,\"notarius_pipeline_group_two\":$pipeline_group_two,\"dnd_pipeline_notarius_pipelines\":\"{{ notarius_pipeline_group_one | combine(notarius_pipeline_group_two) }}\",\"dnd_pipeline_narratio_notarius_pipeline_id\":\"archive-session\",\"dnd_pipeline_notarius_cache_chunk_plans_mode\":\"bypass\"}" +alternate_log="$test_tmp/alternate.log" +run_converge "$alternate_log" "${baseline_args[@]}" -e "$alternate_config" +/usr/local/bin/notarius config validate --config /usr/local/etc/notarius/config.yml \ + --pipeline archive-session +alternate_narratio_config="$test_tmp/alternate-narratio.yml" +/usr/bin/sudo -u narratio /usr/local/bin/narratio config show \ + --config /usr/local/etc/narratio/pipeline.yml --campaign centaurs \ + > "$alternate_narratio_config" +grep -Fq 'pipeline_id: archive-session' "$alternate_narratio_config" +alternate_effective_config="$test_tmp/alternate-effective.yml" +/usr/local/bin/notarius config effective --config /usr/local/etc/notarius/config.yml \ + > "$alternate_effective_config" +grep -Fq 'archive-session:' "$alternate_effective_config" +grep -Fq 'research-session:' "$alternate_effective_config" +grep -Fq 'mode: bypass' "$alternate_effective_config" +alternate_repeat_log="$test_tmp/alternate-repeat.log" +run_converge "$alternate_repeat_log" "${baseline_args[@]}" -e "$alternate_config" +require_clean_recap "$alternate_repeat_log" diff --git a/tests/inventory b/tests/inventory new file mode 100644 index 0000000..2302eda --- /dev/null +++ b/tests/inventory @@ -0,0 +1 @@ +localhost ansible_connection=local