Compare commits
17 Commits
v0.1.0
...
3d77eb382b
| Author | SHA1 | Date | |
|---|---|---|---|
| 3d77eb382b | |||
| a78c3092e5 | |||
| 3599f39f49 | |||
| ce5b838d95 | |||
| e1238eb4fa | |||
| b6d7130046 | |||
| 68a8bbc462 | |||
| 54889e6b29 | |||
| 19b1dcabcf | |||
| 2a53979069 | |||
| 77f67adbc0 | |||
| 4691344676 | |||
| cf3de016ee | |||
| d292297b6a | |||
| c0b0ebd13e | |||
| f62445ad1e | |||
| 5e695f38d0 |
@@ -8,6 +8,14 @@ Release `0.1.0` contains the reviewed `rakestraw.dnd_pipeline` `v0.1.0`
|
||||
snapshot and pins the Seriatim, Audita, Notarius, Scriptorium, and Narratio
|
||||
roles to their respective `v0.1.0` releases.
|
||||
|
||||
The current development snapshot integrates Narratio v3.0.0, Notarius v0.8.1,
|
||||
Audita v2.0.1, and Scriptorium v0.15.0 through public standalone role pins.
|
||||
It includes the suite's Audita backend/profile configuration and Narratio and
|
||||
Notarius testing overlays. This does not replace or modify the published
|
||||
`0.1.0` release artifact. Review the [pipeline role migration notes](roles/pipeline/README.md)
|
||||
before upgrading inventory or application-owned manifests; this collection
|
||||
does not convert existing manifests. Seriatim remains v1.5.0.
|
||||
|
||||
## Install release 0.1.0
|
||||
|
||||
Download the tagged compatibility manifest and install its exact standalone
|
||||
|
||||
@@ -7,19 +7,19 @@ roles:
|
||||
- name: rakestraw.audita
|
||||
src: https://gitea.maximumdirect.net/eric/ansible-role-audita.git
|
||||
scm: git
|
||||
version: v0.1.0
|
||||
version: c3dd59f5ac9b942306ffe540ae9e71d493998137
|
||||
- name: rakestraw.notarius
|
||||
src: https://gitea.maximumdirect.net/eric/ansible-role-notarius.git
|
||||
scm: git
|
||||
version: v0.1.0
|
||||
version: 6aef48b7e12e3651971444b8c224d96ff1879e6c
|
||||
- name: rakestraw.scriptorium
|
||||
src: https://gitea.maximumdirect.net/eric/ansible-role-scriptorium.git
|
||||
scm: git
|
||||
version: v0.1.0
|
||||
version: b174c8c2990bb79f8149d8f288c5e06ccb42bd73
|
||||
- name: rakestraw.narratio
|
||||
src: https://gitea.maximumdirect.net/eric/ansible-role-narratio.git
|
||||
scm: git
|
||||
version: v0.1.0
|
||||
version: 3c0700b5323293d44e3b1e67a50074d82d7a3bff
|
||||
collections:
|
||||
- name: ansible.posix
|
||||
version: 2.2.0
|
||||
|
||||
@@ -18,7 +18,8 @@ ansible-galaxy collection install -r collections/requirements.yml
|
||||
The dependency manifest installs the roles under their canonical names:
|
||||
`rakestraw.seriatim`, `rakestraw.audita`, `rakestraw.notarius`,
|
||||
`rakestraw.scriptorium`, and `rakestraw.narratio`. It selects the compatible
|
||||
`v0.1.0` release of each role. This standalone role is named
|
||||
`v0.1.0` releases where compatible and public commit pins for Narratio and
|
||||
Notarius when the suite needs newer interfaces. This standalone role is named
|
||||
`rakestraw.dnd_pipeline`.
|
||||
|
||||
```yaml
|
||||
@@ -49,8 +50,9 @@ The collection source exposes a reviewed snapshot of this role as
|
||||
Collection consumers should follow the unified bootstrap instructions in the
|
||||
[`rakestraw.dnd` collection repository](https://gitea.maximumdirect.net/eric/ansible-collection-dnd).
|
||||
This standalone repository remains authoritative for role changes; collection
|
||||
maintainers import a reviewed release as a snapshot. Collection releases are
|
||||
published independently and are listed in the collection repository.
|
||||
maintainers import a reviewed, pushed commit or immutable tag as a snapshot.
|
||||
Collection releases are published independently and are listed in the
|
||||
collection repository.
|
||||
|
||||
## Variables
|
||||
|
||||
@@ -59,11 +61,11 @@ The compatible application versions are selected through these variables:
|
||||
|
||||
| Variable | Default |
|
||||
| --- | --- |
|
||||
| `dnd_pipeline_audita_version` | `v0.9.1` |
|
||||
| `dnd_pipeline_narratio_version` | `v2.1.0` |
|
||||
| `dnd_pipeline_notarius_version` | `v0.7.0` |
|
||||
| `dnd_pipeline_audita_version` | `v2.0.1` |
|
||||
| `dnd_pipeline_narratio_version` | `v3.0.0` |
|
||||
| `dnd_pipeline_notarius_version` | `v0.8.1` |
|
||||
| `dnd_pipeline_seriatim_version` | `v1.5.0` |
|
||||
| `dnd_pipeline_scriptorium_version` | `v0.13.0` |
|
||||
| `dnd_pipeline_scriptorium_version` | `v0.15.0` |
|
||||
| `dnd_pipeline_manage_build_dependencies` | `true` |
|
||||
| `dnd_pipeline_go_snap_channel` | `1.26/stable` |
|
||||
| `dnd_pipeline_go_binary` | `/snap/bin/go` |
|
||||
@@ -89,6 +91,7 @@ group before the remaining configuration-bearing dependencies run.
|
||||
| --- | --- |
|
||||
| `dnd_pipeline_runtime_user` | `narratio` |
|
||||
| `dnd_pipeline_runtime_group` | `narratio` |
|
||||
| `dnd_pipeline_operator_users` | `[]` |
|
||||
| `dnd_pipeline_config_directory_mode` | `0755` |
|
||||
| `dnd_pipeline_config_file_mode` | `0644` |
|
||||
|
||||
@@ -116,20 +119,53 @@ chosen policy value when their deployment requires it.
|
||||
| --- | --- |
|
||||
| `dnd_pipeline_audita_pipeline_modules` | `glossary`, `homophones`, `glossary`, `spoken_word`, `grammar` |
|
||||
| `dnd_pipeline_audita_output_schema` | `audita-v1` |
|
||||
| `dnd_pipeline_audita_llm_proposal_*` | `https://inference.ai.rakestrawhome.com/v1`, `google/gemma-4-31b-it`, `AUDITA_LLM_API_KEY`, `900s`, `3` |
|
||||
| `dnd_pipeline_audita_llm_validation_*` | `https://inference.ai.rakestrawhome.com/v1`, `google/gemma-4-31b-it`, `AUDITA_LLM_API_KEY`, `900s`, `3` |
|
||||
| `dnd_pipeline_audita_concurrency_*` | total, proposal, and validation concurrency of `4` |
|
||||
| `dnd_pipeline_audita_backends` | Native `dnd` backend at `https://inference.ai.rakestrawhome.com/v1`, credential `AUDITA_LLM_API_KEY`, shared concurrency `4` |
|
||||
| `dnd_pipeline_audita_profiles` | Complete native `dnd-proposal` and `dnd-validation` mappings using `google/gemma-4-31b-it` and 900-second attempt timeouts |
|
||||
| `dnd_pipeline_audita_llm_proposal_profile` | `dnd-proposal` |
|
||||
| `dnd_pipeline_audita_llm_validation_profile` | `dnd-validation` |
|
||||
| `dnd_pipeline_audita_llm_transient_retries`, `dnd_pipeline_audita_llm_repair_attempts`, `dnd_pipeline_audita_llm_operation_timeout` | `null`; omit overrides and use native defaults |
|
||||
| `dnd_pipeline_audita_chunking_*` | `8` target sections, `6144` maximum and `2048` minimum section tokens |
|
||||
| `dnd_pipeline_audita_normalization_*` | `3s` maximum and ellipsis gaps, `240s` maximum duration, `2048` maximum tokens |
|
||||
| `dnd_pipeline_audita_normalization_*` | `3.0` maximum and ellipsis gaps, `240.0` maximum duration, `2048` maximum tokens |
|
||||
| `dnd_pipeline_audita_thresholds_*` | `0.8` for glossary, homophones, spoken word, and grammar |
|
||||
| `dnd_pipeline_audita_context_description` | D&D-game transcript context, including possible pre- and post-session banter |
|
||||
| `dnd_pipeline_audita_diagnostics_*` | `/tmp/audita`, retained automatically |
|
||||
| `dnd_pipeline_audita_diagnostics_work_dir` | `/tmp/audita` |
|
||||
|
||||
Audita's proposal and validation settings independently remain overrideable,
|
||||
but both default to the one typed `AUDITA_LLM_API_KEY` reference Narratio can
|
||||
forward. Audita owns its endpoint, model, tuning, and diagnostics configuration;
|
||||
Narratio receives its managed Audita configuration path and supplies the
|
||||
run-specific output path.
|
||||
Audita owns its backend and LLM policy. The suite writes each complete profile
|
||||
mapping to `/usr/local/etc/audita/profiles/<key>.yml`; keys must be filename-safe
|
||||
IDs matching each mapping's `id`. Inventory may replace the mappings and selected
|
||||
IDs. Narratio forwards the configured credential names, while Audita's native
|
||||
configuration resolves them. The role does not remove old profile files.
|
||||
|
||||
For inventories upgrading from the prior suite, replace the separate Audita
|
||||
proposal/validation endpoint, model, key, timeout, and concurrency variables
|
||||
with `dnd_pipeline_audita_backends` and `dnd_pipeline_audita_profiles`, then
|
||||
select profile IDs with the two `dnd_pipeline_audita_llm_*_profile` variables.
|
||||
Replace `dnd_pipeline_narratio_notarius_llm_api_key` with
|
||||
`dnd_pipeline_narratio_notarius_credentials`, and use
|
||||
`dnd_pipeline_narratio_audita_credentials` for Audita's backend credential
|
||||
names. The old diagnostics retention variable is no longer forwarded.
|
||||
|
||||
The old per-purpose Audita endpoint and API-key names now belong to the
|
||||
backend map; both default profiles share the `dnd` backend and its one
|
||||
concurrency limit. Old proposal and validation models and per-attempt timeouts
|
||||
belong in complete profile mappings. The default 900-second value is retained
|
||||
per attempt; `dnd_pipeline_audita_llm_operation_timeout` is an independent
|
||||
whole-operation deadline and defaults to `null`, leaving Audita's native
|
||||
policy in charge. The former `*_max_retries` variables do not have a direct
|
||||
one-to-one replacement: use the optional Audita transient retry and repair
|
||||
attempt settings according to the desired native behavior. Optional retry
|
||||
pass-throughs for Notarius PromptKit and Scriptorium likewise default to
|
||||
`null`; an explicit `0` is forwarded, not treated as omission.
|
||||
|
||||
Inventory must supply each named backend credential through
|
||||
`dnd_pipeline_secrets` and include its name in
|
||||
`dnd_pipeline_narratio_audita_credentials`. Narratio forwards only names in
|
||||
that list and does not discover them from Audita's configuration. The
|
||||
Notarius list is independent. For profile overrides, use filename-safe map
|
||||
keys equal to each profile's native `id`; the suite writes each map value
|
||||
unchanged and does not merge individual fields. See [Audita's v2 manifest
|
||||
migration](https://gitea.maximumdirect.net/eric/audita/src/tag/v2.0.0/docs/migrations/v2.md)
|
||||
for application-owned manifests; this role does not convert them.
|
||||
|
||||
| Variable | Default |
|
||||
| --- | --- |
|
||||
@@ -144,12 +180,51 @@ run-specific output path.
|
||||
| `dnd_pipeline_narratio_render_enabled` | `true` |
|
||||
| `dnd_pipeline_narratio_render_format` | `markdown` |
|
||||
| `dnd_pipeline_narratio_testing_scriptorium_profile` | `gemini-flash-latest` |
|
||||
| `dnd_pipeline_notarius_testing_llm_profile` | `gpt-luna-latest` |
|
||||
| `dnd_pipeline_notarius_promptkit_transient_retries` | `null`; native default |
|
||||
| `dnd_pipeline_scriptorium_transient_retries` | `null`; native default |
|
||||
| `dnd_pipeline_narratio_audita_credentials` | `[AUDITA_LLM_API_KEY]` |
|
||||
| `dnd_pipeline_narratio_notarius_credentials` | `[OPENROUTER_API_KEY]` |
|
||||
|
||||
`dnd_pipeline_audita_output_schema` is also the Narratio Audita-adapter schema;
|
||||
it is the shared cross-application boundary. Other Narratio-local controls,
|
||||
including reports, timeouts, debug rendering, presentation, and bounds prompt
|
||||
profile selection, remain Narratio defaults.
|
||||
|
||||
### Storage and publication policy
|
||||
|
||||
The suite selects Narratio's S3 backend and the established Backblaze-compatible
|
||||
archive. These are ordinary inventory variables, so deployments may replace
|
||||
the selected policy through the role's public interface.
|
||||
|
||||
| Variable | Default policy |
|
||||
| --- | --- |
|
||||
| `dnd_pipeline_narratio_workspace_cleanup_after_publish` | `true` |
|
||||
| `dnd_pipeline_narratio_storage_backend` | `s3` |
|
||||
| `dnd_pipeline_narratio_storage_s3_bucket` | `rakestraw-assets` |
|
||||
| `dnd_pipeline_narratio_storage_s3_root_prefix` | `dnd` |
|
||||
| `dnd_pipeline_narratio_storage_s3_region` | `us-east-1` |
|
||||
| `dnd_pipeline_narratio_storage_s3_endpoint` | `https://s3.us-east-005.backblazeb2.com` |
|
||||
| `dnd_pipeline_narratio_storage_s3_force_path_style` | `true` |
|
||||
| `dnd_pipeline_narratio_storage_s3_access_key_id` | `OBJECT_STORAGE_KEY_ID` |
|
||||
| `dnd_pipeline_narratio_storage_s3_secret_access_key` | `OBJECT_STORAGE_KEY` |
|
||||
| `dnd_pipeline_narratio_cache_s3_audio` | `true` |
|
||||
| `dnd_pipeline_narratio_publish_enabled` | `true` |
|
||||
| `dnd_pipeline_narratio_publish_upload_run` | `true` |
|
||||
|
||||
`dnd_pipeline_narratio_publish_outputs` is a complete native list, not a list
|
||||
merged with the suite default. It publishes final and trimmed JSON, final and
|
||||
trimmed Markdown, session recap, hostile rumors, and optional deranged rumors.
|
||||
Every rule omits `dest`: Narratio derives transcript destinations from its
|
||||
built-in sources and artifact destinations from the configured artifact output
|
||||
paths.
|
||||
|
||||
S3 session audio uses the maximum available native cache beneath
|
||||
`/var/cache/narratio`. After successful run publication, Narratio cleans the
|
||||
published workspace target while preserving that cache. The suite deliberately
|
||||
does not set `narratio_spool_delete_audio_after_publish`; spool-audio retention
|
||||
continues to use Narratio's application default.
|
||||
|
||||
### Artifacts and profiles
|
||||
|
||||
The role owns these complete native mapping variables. Replacing one mapping
|
||||
@@ -183,24 +258,44 @@ suite-owned base fragments in this fixed order:
|
||||
├── artifacts/deranged-rumors.yml
|
||||
├── artifacts/scene-map.yml
|
||||
├── artifact-families/character-meta-analysis.yml
|
||||
└── profiles/testing.yml
|
||||
└── overlays/testing.yml
|
||||
```
|
||||
|
||||
The `testing` profile is an overlay, not a second configuration tree. It only
|
||||
sets the Scriptorium `profile_id` for the four artifact sources and the one
|
||||
family. Its default, `gemini-flash-latest`, is an inexpensive development
|
||||
Notarius follows the corresponding base-and-overlay layout:
|
||||
|
||||
```text
|
||||
/usr/local/etc/notarius/
|
||||
├── config.yml
|
||||
├── pipelines/dnd-session.yml
|
||||
└── overlays/testing.yml
|
||||
```
|
||||
|
||||
The `testing` overlay is not a second configuration tree. It sets
|
||||
the Scriptorium `profile_id` for the four artifact sources and the one family,
|
||||
and explicitly selects Notarius's separately managed `testing` overlay. The
|
||||
Scriptorium default, `gemini-flash-latest`, is an inexpensive development
|
||||
choice, not a compatibility guarantee. Replace it through inventory when a
|
||||
different testing profile is appropriate:
|
||||
|
||||
```yaml
|
||||
dnd_pipeline_narratio_testing_scriptorium_profile: endpoint-profile
|
||||
dnd_pipeline_notarius_testing_llm_profile: alternate-economy-profile
|
||||
```
|
||||
|
||||
Narratio selects it with `--overlay testing`, and direct Notarius commands
|
||||
use `--config-overlay testing`; the former `--profile` and
|
||||
`--config-profile` selectors are retired. Previously deployed
|
||||
`profiles/testing.yml` files are ignored, not pruned. Renaming the selected
|
||||
overlay normally changes configuration identity, so cached work tied to the
|
||||
old named selection may not be reused.
|
||||
|
||||
### Notarius extraction policy
|
||||
|
||||
The Notarius root configuration imports the fixed
|
||||
`pipelines/dnd-session.yml` fragment. The suite owns that fragment at
|
||||
`/usr/local/etc/notarius/pipelines/dnd-session.yml`; callers do not edit it.
|
||||
The separate `/usr/local/etc/notarius/overlays/testing.yml` overlay changes only
|
||||
the selected suite pipeline's LLM profile.
|
||||
|
||||
| Variable | Default policy |
|
||||
| --- | --- |
|
||||
@@ -208,15 +303,20 @@ The Notarius root configuration imports the fixed
|
||||
| `dnd_pipeline_notarius_cache_root` | `/var/cache/notarius` |
|
||||
| `dnd_pipeline_notarius_cache_chunk_plans_mode` | `auto` |
|
||||
| `dnd_pipeline_notarius_cache_checkpoints_enabled` | `true` |
|
||||
| `dnd_pipeline_notarius_testing_llm_profile` | `gpt-luna-latest` |
|
||||
| `dnd_pipeline_narratio_notarius_pipeline_id` | `dnd-session` |
|
||||
| `dnd_pipeline_narratio_notarius_llm_api_key` | `OPENROUTER_API_KEY` |
|
||||
| `dnd_pipeline_narratio_notarius_credentials` | `[OPENROUTER_API_KEY]` |
|
||||
| `dnd_pipeline_narratio_notarius_resume` | `true` |
|
||||
| `dnd_pipeline_narratio_notarius_references` | `party`, `players`, and `glossary` prepared sources |
|
||||
| `dnd_pipeline_narratio_notarius_outputs` | Complete native map of ten accepted D&D lanes |
|
||||
|
||||
The pipeline, reference, and output values are complete native mappings.
|
||||
Inventory can replace them or compose additions with Jinja's `combine` filter;
|
||||
a replacement pipeline map requires a matching selected pipeline ID.
|
||||
a replacement pipeline map requires a matching selected pipeline ID. The base
|
||||
`dnd-session` pipeline selects `gpt-terra-latest`, while the testing overlay
|
||||
selects `gpt-luna-latest`. Both are moving catalog IDs rather than immutable
|
||||
model promises. Pipeline and binding-level overrides retain Notarius's native
|
||||
precedence; the suite does not rewrite caller-supplied bindings.
|
||||
|
||||
Narratio materializes the `party`, `players`, and `glossary` references for each
|
||||
invocation. No campaign filesystem path appears in Notarius configuration. The
|
||||
@@ -225,16 +325,22 @@ that input. The accepted sources are
|
||||
`narratio.extraction.item_registry`, `npc_registry`, `location_registry`,
|
||||
`scene_descriptions`, `item_occurrences`, `spells`, `combat_turns`,
|
||||
`npc_occurrences`, `location_occurrences`, and `enemy_events`.
|
||||
They remain available as `narratio.extraction.<output_key>` inputs for future
|
||||
analyze artifacts whether or not a source has a stable publication rule;
|
||||
accepted extraction and published output selection are independent interfaces.
|
||||
|
||||
Narratio forwards the typed `OPENROUTER_API_KEY` reference to the selected
|
||||
Notarius profile and ordinary extraction requests checkpoint reuse with
|
||||
`resume: true`. Chunk plans use Notarius's root-owned `auto` policy; the
|
||||
pipeline deliberately leaves Narratio's optional `chunk_cache` override unset.
|
||||
The `chunk-plans` and `checkpoints` paths derive from the cache root and are
|
||||
created by Notarius during use. Base and `testing` configurations select the
|
||||
same `dnd-extraction` profile and Notarius pipeline; the testing overlay changes
|
||||
only Scriptorium profiles, so a real testing run may invoke Notarius. The
|
||||
integration suite performs offline configuration inspection only.
|
||||
created by Notarius during use. Notarius v0.8.1 selects PromptKit v0.10.0 and
|
||||
OpenRouter catalog v2.20260922.1, which supplies both suite-selected IDs.
|
||||
Existing checkpoints with the previous profile-source identity are not reused;
|
||||
source-keyed chunk plans and output artifacts remain compatible. No cache
|
||||
deletion or forced extraction is performed by this role. A real testing run may
|
||||
invoke Notarius; the integration suite performs offline configuration inspection
|
||||
only and makes no provider request.
|
||||
|
||||
### Inspecting configuration
|
||||
|
||||
@@ -246,7 +352,7 @@ the party-driven character family:
|
||||
sudo -u narratio narratio config validate \
|
||||
--config /usr/local/etc/narratio/pipeline.yml --campaign centaurs
|
||||
sudo -u narratio narratio config validate \
|
||||
--config /usr/local/etc/narratio/pipeline.yml --campaign centaurs --profile testing
|
||||
--config /usr/local/etc/narratio/pipeline.yml --campaign centaurs --overlay testing
|
||||
sudo -u narratio narratio config artifacts \
|
||||
--config /usr/local/etc/narratio/pipeline.yml --campaign centaurs
|
||||
sudo -u narratio narratio config diff default testing \
|
||||
@@ -258,8 +364,15 @@ inspection shows enabled sources, inferred dependencies, and generated family
|
||||
members; the diff shows the overlay's profile selections without modifying
|
||||
managed files.
|
||||
|
||||
Storage and publishing, operator selection and group-membership management,
|
||||
scheduling, and end-to-end external execution remain deferred.
|
||||
With valid object-storage credentials, the selected policy enables Narratio's
|
||||
native [S3 session-audio](https://gitea.maximumdirect.net/eric/narratio/src/tag/v2.1.1/docs/config.md#session),
|
||||
[publication](https://gitea.maximumdirect.net/eric/narratio/src/tag/v2.1.1/docs/operations.md#publish-workflow),
|
||||
[restore](https://gitea.maximumdirect.net/eric/narratio/src/tag/v2.1.1/docs/operations.md#restore-workflow),
|
||||
[publication locks](https://gitea.maximumdirect.net/eric/narratio/src/tag/v2.1.1/docs/operations.md#publish-locks),
|
||||
[session status](https://gitea.maximumdirect.net/eric/narratio/src/tag/v2.1.1/docs/operations.md#standard-session-workflow),
|
||||
and [cleanup](https://gitea.maximumdirect.net/eric/narratio/src/tag/v2.1.1/docs/operations.md#cleanup)
|
||||
workflows. These remain application commands and lifecycle rules; scheduling
|
||||
and real end-to-end execution remain deployment concerns.
|
||||
|
||||
### Shared execution filesystem
|
||||
|
||||
@@ -268,27 +381,30 @@ scheduling, and end-to-end external execution remain deferred.
|
||||
`narratio:narratio` mode `2775` **setgid** directories. Their access and
|
||||
default ACLs retain owner and group write access plus world read/traverse access
|
||||
for newly created content. The pipeline may run as `narratio` or as a human
|
||||
account already in the selected runtime group; the production role neither
|
||||
selects people nor manages their group membership. For example, a consumer
|
||||
play can manage its selected operator separately:
|
||||
account in the selected runtime group. Consumers can enroll selected existing
|
||||
human accounts through the role interface:
|
||||
|
||||
```yaml
|
||||
- name: Add the selected operator to the pipeline group
|
||||
become: true
|
||||
ansible.builtin.user:
|
||||
name: "{{ pipeline_operator }}"
|
||||
groups:
|
||||
- "{{ dnd_pipeline_runtime_group }}"
|
||||
append: true
|
||||
dnd_pipeline_operator_users:
|
||||
- eric
|
||||
- alice
|
||||
```
|
||||
|
||||
Every selected account must already exist; the role does not create human
|
||||
accounts or manage their identity and login policy. Enrollment adds the
|
||||
selected runtime group without replacing other supplementary groups. Removing
|
||||
or omitting a name later does not revoke membership. A newly enrolled operator
|
||||
must begin a new login session before the operating system exposes the added
|
||||
supplementary group to that session.
|
||||
|
||||
Configuration, transcripts, caches, generated artifacts, campaign data,
|
||||
prompts, and logs are non-secret D&D content. Credentials use the restricted
|
||||
`/usr/local/etc/narratio/secrets` subtree, owned by the selected runtime user
|
||||
and group with mode `0750`; managed files use mode `0640`. Runtime-group
|
||||
membership authorizes use of these shared credentials, while the subtree grants
|
||||
no world access. Ordinary configuration remains world-readable and contains
|
||||
only reference names.
|
||||
membership, including enrollment through `dnd_pipeline_operator_users`,
|
||||
authorizes use of these shared credentials, while the subtree grants no world
|
||||
access. Ordinary configuration remains world-readable and contains only
|
||||
reference names.
|
||||
|
||||
### Managed credentials
|
||||
|
||||
@@ -306,8 +422,17 @@ consumer workflow, not role functionality:
|
||||
dnd_pipeline_secrets:
|
||||
AUDITA_LLM_API_KEY: "{{ vault_dnd_audita_llm_api_key }}"
|
||||
OPENROUTER_API_KEY: "{{ vault_dnd_openrouter_api_key }}"
|
||||
OBJECT_STORAGE_KEY_ID: "{{ vault_dnd_object_storage_key_id }}"
|
||||
OBJECT_STORAGE_KEY: "{{ vault_dnd_object_storage_key }}"
|
||||
```
|
||||
|
||||
Only the reference names enter ordinary Narratio configuration; credential
|
||||
values remain in encrypted inventory and the restricted managed secrets
|
||||
directory. A deployment using Narratio's ambient AWS SDK provider chain can
|
||||
set both `dnd_pipeline_narratio_storage_s3_access_key_id` and
|
||||
`dnd_pipeline_narratio_storage_s3_secret_access_key` to `null`. The pair must
|
||||
be omitted together.
|
||||
|
||||
Scriptorium profiles continue to own their `api_key_env` selection. To supply
|
||||
that credential through the suite, add a `dnd_pipeline_secrets` entry whose
|
||||
name matches the selected profile's `api_key_env`. Narratio's secrets bootstrap
|
||||
|
||||
@@ -1,14 +1,15 @@
|
||||
---
|
||||
dnd_pipeline_audita_version: v0.9.1
|
||||
dnd_pipeline_narratio_version: v2.1.0
|
||||
dnd_pipeline_notarius_version: v0.7.0
|
||||
dnd_pipeline_audita_version: v2.0.1
|
||||
dnd_pipeline_narratio_version: v3.0.0
|
||||
dnd_pipeline_notarius_version: v0.8.1
|
||||
dnd_pipeline_seriatim_version: v1.5.0
|
||||
dnd_pipeline_scriptorium_version: v0.13.0
|
||||
dnd_pipeline_scriptorium_version: v0.15.0
|
||||
dnd_pipeline_manage_build_dependencies: true
|
||||
dnd_pipeline_go_snap_channel: 1.26/stable
|
||||
dnd_pipeline_go_binary: /snap/bin/go
|
||||
dnd_pipeline_runtime_user: narratio
|
||||
dnd_pipeline_runtime_group: narratio
|
||||
dnd_pipeline_operator_users: []
|
||||
dnd_pipeline_secrets: {}
|
||||
dnd_pipeline_config_directory_mode: "0755"
|
||||
dnd_pipeline_config_file_mode: "0644"
|
||||
@@ -25,25 +26,33 @@ dnd_pipeline_audita_pipeline_modules:
|
||||
- spoken_word
|
||||
- grammar
|
||||
dnd_pipeline_audita_output_schema: audita-v1
|
||||
dnd_pipeline_audita_llm_proposal_base_url: https://inference.ai.rakestrawhome.com/v1
|
||||
dnd_pipeline_audita_llm_proposal_model: google/gemma-4-31b-it
|
||||
dnd_pipeline_audita_llm_proposal_api_key_env: AUDITA_LLM_API_KEY
|
||||
dnd_pipeline_audita_llm_proposal_timeout: 900s
|
||||
dnd_pipeline_audita_llm_proposal_max_retries: 3
|
||||
dnd_pipeline_audita_llm_validation_base_url: https://inference.ai.rakestrawhome.com/v1
|
||||
dnd_pipeline_audita_llm_validation_model: google/gemma-4-31b-it
|
||||
dnd_pipeline_audita_llm_validation_api_key_env: AUDITA_LLM_API_KEY
|
||||
dnd_pipeline_audita_llm_validation_timeout: 900s
|
||||
dnd_pipeline_audita_llm_validation_max_retries: 3
|
||||
dnd_pipeline_audita_concurrency_total_llm: 4
|
||||
dnd_pipeline_audita_concurrency_proposal_llm: 4
|
||||
dnd_pipeline_audita_concurrency_validation_llm: 4
|
||||
dnd_pipeline_audita_backends:
|
||||
dnd:
|
||||
endpoint: https://inference.ai.rakestrawhome.com/v1
|
||||
credential: AUDITA_LLM_API_KEY
|
||||
concurrency: 4
|
||||
dnd_pipeline_audita_profiles:
|
||||
dnd-proposal:
|
||||
id: dnd-proposal
|
||||
backend: dnd
|
||||
model: google/gemma-4-31b-it
|
||||
timeout_seconds: 900
|
||||
dnd-validation:
|
||||
id: dnd-validation
|
||||
backend: dnd
|
||||
model: google/gemma-4-31b-it
|
||||
timeout_seconds: 900
|
||||
dnd_pipeline_audita_llm_proposal_profile: dnd-proposal
|
||||
dnd_pipeline_audita_llm_validation_profile: dnd-validation
|
||||
dnd_pipeline_audita_llm_transient_retries: null
|
||||
dnd_pipeline_audita_llm_repair_attempts: null
|
||||
dnd_pipeline_audita_llm_operation_timeout: null
|
||||
dnd_pipeline_audita_chunking_target_sections: 8
|
||||
dnd_pipeline_audita_chunking_max_section_tokens: 6144
|
||||
dnd_pipeline_audita_chunking_min_section_tokens: 2048
|
||||
dnd_pipeline_audita_normalization_max_segment_gap: 3s
|
||||
dnd_pipeline_audita_normalization_ellipsis_gap: 3s
|
||||
dnd_pipeline_audita_normalization_max_segment_duration: 240s
|
||||
dnd_pipeline_audita_normalization_max_segment_gap: 3.0
|
||||
dnd_pipeline_audita_normalization_ellipsis_gap: 3.0
|
||||
dnd_pipeline_audita_normalization_max_segment_duration: 240.0
|
||||
dnd_pipeline_audita_normalization_max_segment_tokens: 2048
|
||||
dnd_pipeline_audita_thresholds_glossary: 0.8
|
||||
dnd_pipeline_audita_thresholds_homophones: 0.8
|
||||
@@ -54,10 +63,9 @@ dnd_pipeline_audita_context_description: >-
|
||||
It may contain some unrelated pre-session or post-session banter, in addition
|
||||
to actual D&D gameplay.
|
||||
dnd_pipeline_audita_diagnostics_work_dir: /tmp/audita
|
||||
dnd_pipeline_audita_diagnostics_retention: auto
|
||||
dnd_pipeline_notarius_pipelines:
|
||||
dnd-session:
|
||||
llm_profile: dnd-extraction
|
||||
llm_profile: gpt-terra-latest
|
||||
input:
|
||||
module: seriatim
|
||||
chunk:
|
||||
@@ -206,6 +214,36 @@ dnd_pipeline_notarius_pipelines:
|
||||
dnd_pipeline_notarius_cache_root: /var/cache/notarius
|
||||
dnd_pipeline_notarius_cache_chunk_plans_mode: auto
|
||||
dnd_pipeline_notarius_cache_checkpoints_enabled: true
|
||||
dnd_pipeline_notarius_testing_llm_profile: gpt-luna-latest
|
||||
dnd_pipeline_notarius_promptkit_transient_retries: null
|
||||
dnd_pipeline_scriptorium_transient_retries: null
|
||||
dnd_pipeline_narratio_workspace_cleanup_after_publish: true
|
||||
dnd_pipeline_narratio_storage_backend: s3
|
||||
dnd_pipeline_narratio_storage_s3_bucket: rakestraw-assets
|
||||
dnd_pipeline_narratio_storage_s3_root_prefix: dnd
|
||||
dnd_pipeline_narratio_storage_s3_region: us-east-1
|
||||
dnd_pipeline_narratio_storage_s3_endpoint: https://s3.us-east-005.backblazeb2.com
|
||||
dnd_pipeline_narratio_storage_s3_force_path_style: true
|
||||
dnd_pipeline_narratio_storage_s3_access_key_id: OBJECT_STORAGE_KEY_ID
|
||||
dnd_pipeline_narratio_storage_s3_secret_access_key: OBJECT_STORAGE_KEY
|
||||
dnd_pipeline_narratio_cache_s3_audio: true
|
||||
dnd_pipeline_narratio_publish_enabled: true
|
||||
dnd_pipeline_narratio_publish_upload_run: true
|
||||
dnd_pipeline_narratio_publish_outputs:
|
||||
- source: narratio.transcript.final
|
||||
required: true
|
||||
- source: narratio.transcript.final_trimmed
|
||||
required: true
|
||||
- source: narratio.transcript.final_markdown
|
||||
required: true
|
||||
- source: narratio.transcript.final_trimmed_markdown
|
||||
required: true
|
||||
- source: narratio.artifact.session_recap
|
||||
required: true
|
||||
- source: narratio.artifact.hostile_rumors
|
||||
required: true
|
||||
- source: narratio.artifact.deranged_rumors
|
||||
required: false
|
||||
dnd_pipeline_narratio_seriatim_output_schema: seriatim-intermediate
|
||||
dnd_pipeline_narratio_normalize_output_path: transcripts/final.json
|
||||
dnd_pipeline_narratio_normalize_output_schema: seriatim-intermediate
|
||||
@@ -215,7 +253,10 @@ dnd_pipeline_narratio_trim_bounds_prompt_id: dnd.session_bounds
|
||||
dnd_pipeline_narratio_trim_bounds_transcript_input_name: transcript
|
||||
dnd_pipeline_narratio_trim_bounds_output_path: artifacts/session_bounds.json
|
||||
dnd_pipeline_narratio_notarius_pipeline_id: dnd-session
|
||||
dnd_pipeline_narratio_notarius_llm_api_key: OPENROUTER_API_KEY
|
||||
dnd_pipeline_narratio_audita_credentials:
|
||||
- AUDITA_LLM_API_KEY
|
||||
dnd_pipeline_narratio_notarius_credentials:
|
||||
- OPENROUTER_API_KEY
|
||||
dnd_pipeline_narratio_notarius_resume: true
|
||||
dnd_pipeline_narratio_notarius_references:
|
||||
party: narratio.input.party
|
||||
|
||||
@@ -43,6 +43,11 @@ argument_specs:
|
||||
type: str
|
||||
required: true
|
||||
description: Shared group for suite-managed configuration.
|
||||
dnd_pipeline_operator_users:
|
||||
type: list
|
||||
elements: str
|
||||
required: true
|
||||
description: Existing human accounts enrolled in the shared runtime group.
|
||||
dnd_pipeline_secrets:
|
||||
type: dict
|
||||
required: true
|
||||
@@ -92,58 +97,34 @@ argument_specs:
|
||||
type: str
|
||||
required: true
|
||||
description: Audita output schema shared with Narratio.
|
||||
dnd_pipeline_audita_llm_proposal_base_url:
|
||||
dnd_pipeline_audita_backends:
|
||||
type: dict
|
||||
required: true
|
||||
description: Native Audita backend definitions.
|
||||
dnd_pipeline_audita_profiles:
|
||||
type: dict
|
||||
required: true
|
||||
description: Native Audita profiles keyed by filename-safe IDs matching each mapping's id.
|
||||
dnd_pipeline_audita_llm_proposal_profile:
|
||||
type: str
|
||||
required: true
|
||||
description: Audita proposal LLM endpoint.
|
||||
dnd_pipeline_audita_llm_proposal_model:
|
||||
description: Audita proposal profile ID.
|
||||
dnd_pipeline_audita_llm_validation_profile:
|
||||
type: str
|
||||
required: true
|
||||
description: Audita proposal LLM model.
|
||||
dnd_pipeline_audita_llm_proposal_api_key_env:
|
||||
type: str
|
||||
required: true
|
||||
description: Proposal LLM API-key environment variable name.
|
||||
dnd_pipeline_audita_llm_proposal_timeout:
|
||||
type: str
|
||||
required: true
|
||||
description: Proposal LLM timeout.
|
||||
dnd_pipeline_audita_llm_proposal_max_retries:
|
||||
description: Audita validation profile ID.
|
||||
dnd_pipeline_audita_llm_transient_retries:
|
||||
type: int
|
||||
required: true
|
||||
description: Proposal LLM retry count.
|
||||
dnd_pipeline_audita_llm_validation_base_url:
|
||||
type: str
|
||||
required: true
|
||||
description: Audita validation LLM endpoint.
|
||||
dnd_pipeline_audita_llm_validation_model:
|
||||
type: str
|
||||
required: true
|
||||
description: Audita validation LLM model.
|
||||
dnd_pipeline_audita_llm_validation_api_key_env:
|
||||
type: str
|
||||
required: true
|
||||
description: Validation LLM API-key environment variable name.
|
||||
dnd_pipeline_audita_llm_validation_timeout:
|
||||
type: str
|
||||
required: true
|
||||
description: Validation LLM timeout.
|
||||
dnd_pipeline_audita_llm_validation_max_retries:
|
||||
required: false
|
||||
description: Optional Audita transient retry override.
|
||||
dnd_pipeline_audita_llm_repair_attempts:
|
||||
type: int
|
||||
required: true
|
||||
description: Validation LLM retry count.
|
||||
dnd_pipeline_audita_concurrency_total_llm:
|
||||
type: int
|
||||
required: true
|
||||
description: Total Audita LLM concurrency.
|
||||
dnd_pipeline_audita_concurrency_proposal_llm:
|
||||
type: int
|
||||
required: true
|
||||
description: Proposal Audita LLM concurrency.
|
||||
dnd_pipeline_audita_concurrency_validation_llm:
|
||||
type: int
|
||||
required: true
|
||||
description: Validation Audita LLM concurrency.
|
||||
required: false
|
||||
description: Optional Audita repair attempt override.
|
||||
dnd_pipeline_audita_llm_operation_timeout:
|
||||
type: str
|
||||
required: false
|
||||
description: Optional Audita whole-operation deadline override.
|
||||
dnd_pipeline_audita_chunking_target_sections:
|
||||
type: int
|
||||
required: true
|
||||
@@ -157,15 +138,15 @@ argument_specs:
|
||||
required: true
|
||||
description: Audita minimum section tokens.
|
||||
dnd_pipeline_audita_normalization_max_segment_gap:
|
||||
type: str
|
||||
type: float
|
||||
required: true
|
||||
description: Audita maximum segment gap.
|
||||
dnd_pipeline_audita_normalization_ellipsis_gap:
|
||||
type: str
|
||||
type: float
|
||||
required: true
|
||||
description: Audita ellipsis gap.
|
||||
dnd_pipeline_audita_normalization_max_segment_duration:
|
||||
type: str
|
||||
type: float
|
||||
required: true
|
||||
description: Audita maximum segment duration.
|
||||
dnd_pipeline_audita_normalization_max_segment_tokens:
|
||||
@@ -196,10 +177,6 @@ argument_specs:
|
||||
type: str
|
||||
required: true
|
||||
description: Audita diagnostics work directory.
|
||||
dnd_pipeline_audita_diagnostics_retention:
|
||||
type: str
|
||||
required: true
|
||||
description: Audita diagnostics retention policy.
|
||||
dnd_pipeline_notarius_pipelines:
|
||||
type: dict
|
||||
required: true
|
||||
@@ -216,6 +193,70 @@ argument_specs:
|
||||
type: bool
|
||||
required: true
|
||||
description: Whether Notarius records checkpoints.
|
||||
dnd_pipeline_notarius_testing_llm_profile:
|
||||
type: str
|
||||
required: true
|
||||
description: Notarius LLM profile selected by the testing overlay.
|
||||
dnd_pipeline_notarius_promptkit_transient_retries:
|
||||
type: int
|
||||
required: false
|
||||
description: Optional Notarius PromptKit transient retry override.
|
||||
dnd_pipeline_scriptorium_transient_retries:
|
||||
type: int
|
||||
required: false
|
||||
description: Optional Scriptorium transient retry override.
|
||||
dnd_pipeline_narratio_workspace_cleanup_after_publish:
|
||||
type: bool
|
||||
required: true
|
||||
description: Whether Narratio cleans successfully published workspace state.
|
||||
dnd_pipeline_narratio_storage_backend:
|
||||
type: str
|
||||
required: true
|
||||
description: Narratio object-storage backend.
|
||||
dnd_pipeline_narratio_storage_s3_bucket:
|
||||
type: str
|
||||
required: false
|
||||
description: S3 bucket used by Narratio.
|
||||
dnd_pipeline_narratio_storage_s3_root_prefix:
|
||||
type: str
|
||||
required: false
|
||||
description: Root prefix for Narratio objects in the S3 bucket.
|
||||
dnd_pipeline_narratio_storage_s3_region:
|
||||
type: str
|
||||
required: false
|
||||
description: S3 region used by Narratio.
|
||||
dnd_pipeline_narratio_storage_s3_endpoint:
|
||||
type: str
|
||||
required: false
|
||||
description: S3-compatible endpoint used by Narratio.
|
||||
dnd_pipeline_narratio_storage_s3_force_path_style:
|
||||
type: bool
|
||||
required: false
|
||||
description: Whether Narratio uses path-style S3 requests.
|
||||
dnd_pipeline_narratio_storage_s3_access_key_id:
|
||||
type: str
|
||||
required: false
|
||||
description: Nullable access-key environment-variable reference.
|
||||
dnd_pipeline_narratio_storage_s3_secret_access_key:
|
||||
type: str
|
||||
required: false
|
||||
description: Nullable secret-key environment-variable reference.
|
||||
dnd_pipeline_narratio_cache_s3_audio:
|
||||
type: bool
|
||||
required: true
|
||||
description: Whether Narratio caches downloaded S3 audio.
|
||||
dnd_pipeline_narratio_publish_enabled:
|
||||
type: bool
|
||||
required: true
|
||||
description: Whether Narratio publication is enabled.
|
||||
dnd_pipeline_narratio_publish_upload_run:
|
||||
type: bool
|
||||
required: true
|
||||
description: Whether Narratio uploads immutable run content.
|
||||
dnd_pipeline_narratio_publish_outputs:
|
||||
type: list
|
||||
required: true
|
||||
description: Complete native list of Narratio output publication rules.
|
||||
dnd_pipeline_narratio_seriatim_output_schema:
|
||||
type: str
|
||||
required: true
|
||||
@@ -252,10 +293,16 @@ argument_specs:
|
||||
type: str
|
||||
required: true
|
||||
description: Notarius pipeline selected by Narratio.
|
||||
dnd_pipeline_narratio_notarius_llm_api_key:
|
||||
type: str
|
||||
dnd_pipeline_narratio_audita_credentials:
|
||||
type: list
|
||||
elements: str
|
||||
required: true
|
||||
description: Environment-variable name forwarded by Narratio to Notarius.
|
||||
description: Credential names forwarded by Narratio to Audita.
|
||||
dnd_pipeline_narratio_notarius_credentials:
|
||||
type: list
|
||||
elements: str
|
||||
required: true
|
||||
description: Credential names forwarded by Narratio to Notarius.
|
||||
dnd_pipeline_narratio_notarius_resume:
|
||||
type: bool
|
||||
required: true
|
||||
|
||||
@@ -39,16 +39,29 @@ dependencies:
|
||||
- path: artifacts/deranged-rumors.yml
|
||||
- path: artifacts/scene-map.yml
|
||||
- path: artifact-families/character-meta-analysis.yml
|
||||
narratio_workspace_cleanup_after_publish: "{{ dnd_pipeline_narratio_workspace_cleanup_after_publish }}"
|
||||
narratio_storage_backend: "{{ dnd_pipeline_narratio_storage_backend }}"
|
||||
narratio_storage_s3_bucket: "{{ dnd_pipeline_narratio_storage_s3_bucket }}"
|
||||
narratio_storage_s3_root_prefix: "{{ dnd_pipeline_narratio_storage_s3_root_prefix }}"
|
||||
narratio_storage_s3_region: "{{ dnd_pipeline_narratio_storage_s3_region }}"
|
||||
narratio_storage_s3_endpoint: "{{ dnd_pipeline_narratio_storage_s3_endpoint }}"
|
||||
narratio_storage_s3_force_path_style: "{{ dnd_pipeline_narratio_storage_s3_force_path_style }}"
|
||||
narratio_storage_s3_access_key_id: "{{ dnd_pipeline_narratio_storage_s3_access_key_id }}"
|
||||
narratio_storage_s3_secret_access_key: "{{ dnd_pipeline_narratio_storage_s3_secret_access_key }}"
|
||||
narratio_cache_s3_audio: "{{ dnd_pipeline_narratio_cache_s3_audio }}"
|
||||
narratio_publish_enabled: "{{ dnd_pipeline_narratio_publish_enabled }}"
|
||||
narratio_publish_upload_run: "{{ dnd_pipeline_narratio_publish_upload_run }}"
|
||||
narratio_publish_outputs: "{{ dnd_pipeline_narratio_publish_outputs }}"
|
||||
narratio_seriatim_binary: /usr/local/bin/seriatim
|
||||
narratio_audita_binary: /usr/local/bin/audita
|
||||
narratio_audita_config_path: /usr/local/etc/audita/config.yml
|
||||
narratio_audita_llm_api_key: "{{ dnd_pipeline_audita_llm_proposal_api_key_env }}"
|
||||
narratio_audita_credentials: "{{ dnd_pipeline_narratio_audita_credentials }}"
|
||||
narratio_notarius_binary: /usr/local/bin/notarius
|
||||
narratio_notarius_config_path: /usr/local/etc/notarius/config.yml
|
||||
narratio_notarius_enabled: true
|
||||
narratio_notarius_pipeline_id: "{{ dnd_pipeline_narratio_notarius_pipeline_id }}"
|
||||
narratio_notarius_working_directory: /usr/local/etc/notarius
|
||||
narratio_notarius_llm_api_key: "{{ dnd_pipeline_narratio_notarius_llm_api_key }}"
|
||||
narratio_notarius_credentials: "{{ dnd_pipeline_narratio_notarius_credentials }}"
|
||||
narratio_notarius_resume: "{{ dnd_pipeline_narratio_notarius_resume }}"
|
||||
narratio_notarius_references: "{{ dnd_pipeline_narratio_notarius_references }}"
|
||||
narratio_notarius_outputs: "{{ dnd_pipeline_narratio_notarius_outputs }}"
|
||||
@@ -72,19 +85,13 @@ dependencies:
|
||||
audita_config_file_mode: "{{ dnd_pipeline_config_file_mode }}"
|
||||
audita_pipeline_modules: "{{ dnd_pipeline_audita_pipeline_modules }}"
|
||||
audita_output_schema: "{{ dnd_pipeline_audita_output_schema }}"
|
||||
audita_llm_proposal_base_url: "{{ dnd_pipeline_audita_llm_proposal_base_url }}"
|
||||
audita_llm_proposal_model: "{{ dnd_pipeline_audita_llm_proposal_model }}"
|
||||
audita_llm_proposal_api_key_env: "{{ dnd_pipeline_audita_llm_proposal_api_key_env }}"
|
||||
audita_llm_proposal_timeout: "{{ dnd_pipeline_audita_llm_proposal_timeout }}"
|
||||
audita_llm_proposal_max_retries: "{{ dnd_pipeline_audita_llm_proposal_max_retries }}"
|
||||
audita_llm_validation_base_url: "{{ dnd_pipeline_audita_llm_validation_base_url }}"
|
||||
audita_llm_validation_model: "{{ dnd_pipeline_audita_llm_validation_model }}"
|
||||
audita_llm_validation_api_key_env: "{{ dnd_pipeline_audita_llm_validation_api_key_env }}"
|
||||
audita_llm_validation_timeout: "{{ dnd_pipeline_audita_llm_validation_timeout }}"
|
||||
audita_llm_validation_max_retries: "{{ dnd_pipeline_audita_llm_validation_max_retries }}"
|
||||
audita_concurrency_total_llm: "{{ dnd_pipeline_audita_concurrency_total_llm }}"
|
||||
audita_concurrency_proposal_llm: "{{ dnd_pipeline_audita_concurrency_proposal_llm }}"
|
||||
audita_concurrency_validation_llm: "{{ dnd_pipeline_audita_concurrency_validation_llm }}"
|
||||
audita_backends: "{{ dnd_pipeline_audita_backends }}"
|
||||
audita_profiles_dir: /usr/local/etc/audita/profiles
|
||||
audita_llm_proposal_profile: "{{ dnd_pipeline_audita_llm_proposal_profile }}"
|
||||
audita_llm_validation_profile: "{{ dnd_pipeline_audita_llm_validation_profile }}"
|
||||
audita_llm_transient_retries: "{{ dnd_pipeline_audita_llm_transient_retries }}"
|
||||
audita_llm_repair_attempts: "{{ dnd_pipeline_audita_llm_repair_attempts }}"
|
||||
audita_llm_operation_timeout: "{{ dnd_pipeline_audita_llm_operation_timeout }}"
|
||||
audita_chunking_target_sections: "{{ dnd_pipeline_audita_chunking_target_sections }}"
|
||||
audita_chunking_max_section_tokens: "{{ dnd_pipeline_audita_chunking_max_section_tokens }}"
|
||||
audita_chunking_min_section_tokens: "{{ dnd_pipeline_audita_chunking_min_section_tokens }}"
|
||||
@@ -98,7 +105,6 @@ dependencies:
|
||||
audita_thresholds_grammar: "{{ dnd_pipeline_audita_thresholds_grammar }}"
|
||||
audita_context_description: "{{ dnd_pipeline_audita_context_description }}"
|
||||
audita_diagnostics_work_dir: "{{ dnd_pipeline_audita_diagnostics_work_dir }}"
|
||||
audita_diagnostics_retention: "{{ dnd_pipeline_audita_diagnostics_retention }}"
|
||||
- role: rakestraw.notarius
|
||||
vars:
|
||||
notarius_version: "{{ dnd_pipeline_notarius_version }}"
|
||||
@@ -115,6 +121,7 @@ dependencies:
|
||||
notarius_cache_chunk_plans_directory: "{{ dnd_pipeline_notarius_cache_root ~ '/chunk-plans' }}"
|
||||
notarius_cache_checkpoints_enabled: "{{ dnd_pipeline_notarius_cache_checkpoints_enabled }}"
|
||||
notarius_cache_checkpoints_directory: "{{ dnd_pipeline_notarius_cache_root ~ '/checkpoints' }}"
|
||||
notarius_promptkit_transient_retries: "{{ dnd_pipeline_notarius_promptkit_transient_retries }}"
|
||||
- role: rakestraw.scriptorium
|
||||
vars:
|
||||
scriptorium_version: "{{ dnd_pipeline_scriptorium_version }}"
|
||||
@@ -127,3 +134,4 @@ dependencies:
|
||||
scriptorium_config_file_mode: "{{ dnd_pipeline_config_file_mode }}"
|
||||
scriptorium_prompt_dir: "{{ dnd_pipeline_prompt_library_root ~ '/prompts' }}"
|
||||
scriptorium_schema_dir: "{{ dnd_pipeline_prompt_library_root ~ '/schemas' }}"
|
||||
scriptorium_transient_retries: "{{ dnd_pipeline_scriptorium_transient_retries }}"
|
||||
|
||||
@@ -7,16 +7,16 @@ roles:
|
||||
- name: rakestraw.audita
|
||||
src: https://gitea.maximumdirect.net/eric/ansible-role-audita.git
|
||||
scm: git
|
||||
version: v0.1.0
|
||||
version: c3dd59f5ac9b942306ffe540ae9e71d493998137
|
||||
- name: rakestraw.notarius
|
||||
src: https://gitea.maximumdirect.net/eric/ansible-role-notarius.git
|
||||
scm: git
|
||||
version: v0.1.0
|
||||
version: 6aef48b7e12e3651971444b8c224d96ff1879e6c
|
||||
- name: rakestraw.scriptorium
|
||||
src: https://gitea.maximumdirect.net/eric/ansible-role-scriptorium.git
|
||||
scm: git
|
||||
version: v0.1.0
|
||||
version: b174c8c2990bb79f8149d8f288c5e06ccb42bd73
|
||||
- name: rakestraw.narratio
|
||||
src: https://gitea.maximumdirect.net/eric/ansible-role-narratio.git
|
||||
scm: git
|
||||
version: v0.1.0
|
||||
version: 3c0700b5323293d44e3b1e67a50074d82d7a3bff
|
||||
|
||||
@@ -1,4 +1,19 @@
|
||||
---
|
||||
- name: Validate selected pipeline operator accounts
|
||||
ansible.builtin.getent:
|
||||
database: passwd
|
||||
key: "{{ item }}"
|
||||
loop: "{{ dnd_pipeline_operator_users }}"
|
||||
|
||||
- name: Enroll selected pipeline operators
|
||||
become: true
|
||||
ansible.builtin.user:
|
||||
name: "{{ item }}"
|
||||
groups: "{{ dnd_pipeline_runtime_group }}"
|
||||
append: true
|
||||
state: present
|
||||
loop: "{{ dnd_pipeline_operator_users }}"
|
||||
|
||||
- name: Install ACL support
|
||||
become: true
|
||||
ansible.builtin.apt:
|
||||
@@ -129,16 +144,38 @@
|
||||
- conf.d
|
||||
- artifacts
|
||||
- artifact-families
|
||||
- profiles
|
||||
- overlays
|
||||
|
||||
- name: Create the Notarius pipeline configuration directory
|
||||
- name: Create Notarius configuration directories
|
||||
become: true
|
||||
ansible.builtin.file:
|
||||
path: /usr/local/etc/notarius/pipelines
|
||||
path: "/usr/local/etc/notarius/{{ item }}"
|
||||
state: directory
|
||||
owner: "{{ dnd_pipeline_runtime_user }}"
|
||||
group: "{{ dnd_pipeline_runtime_group }}"
|
||||
mode: "{{ dnd_pipeline_config_directory_mode }}"
|
||||
loop:
|
||||
- pipelines
|
||||
- overlays
|
||||
|
||||
- name: Create the Audita profile directory
|
||||
become: true
|
||||
ansible.builtin.file:
|
||||
path: /usr/local/etc/audita/profiles
|
||||
state: directory
|
||||
owner: "{{ dnd_pipeline_runtime_user }}"
|
||||
group: "{{ dnd_pipeline_runtime_group }}"
|
||||
mode: "{{ dnd_pipeline_config_directory_mode }}"
|
||||
|
||||
- name: Deploy Audita profiles
|
||||
become: true
|
||||
ansible.builtin.template:
|
||||
src: audita-profile.yml.j2
|
||||
dest: "/usr/local/etc/audita/profiles/{{ item.key }}.yml"
|
||||
owner: "{{ dnd_pipeline_runtime_user }}"
|
||||
group: "{{ dnd_pipeline_runtime_group }}"
|
||||
mode: "{{ dnd_pipeline_config_file_mode }}"
|
||||
loop: "{{ dnd_pipeline_audita_profiles | dict2items }}"
|
||||
|
||||
- name: Deploy the Notarius D&D pipeline configuration
|
||||
become: true
|
||||
@@ -149,6 +186,15 @@
|
||||
group: "{{ dnd_pipeline_runtime_group }}"
|
||||
mode: "{{ dnd_pipeline_config_file_mode }}"
|
||||
|
||||
- name: Deploy the Notarius testing overlay
|
||||
become: true
|
||||
ansible.builtin.template:
|
||||
src: notarius-testing.yml.j2
|
||||
dest: /usr/local/etc/notarius/overlays/testing.yml
|
||||
owner: "{{ dnd_pipeline_runtime_user }}"
|
||||
group: "{{ dnd_pipeline_runtime_group }}"
|
||||
mode: "{{ dnd_pipeline_config_file_mode }}"
|
||||
|
||||
- name: Deploy Narratio D&D configuration fragments
|
||||
become: true
|
||||
ansible.builtin.template:
|
||||
@@ -171,4 +217,4 @@
|
||||
- src: character-meta-analysis.yml.j2
|
||||
dest: artifact-families/character-meta-analysis.yml
|
||||
- src: testing.yml.j2
|
||||
dest: profiles/testing.yml
|
||||
dest: overlays/testing.yml
|
||||
|
||||
1
roles/pipeline/templates/audita-profile.yml.j2
Normal file
1
roles/pipeline/templates/audita-profile.yml.j2
Normal file
@@ -0,0 +1 @@
|
||||
{{ item.value | to_nice_yaml }}
|
||||
3
roles/pipeline/templates/notarius-testing.yml.j2
Normal file
3
roles/pipeline/templates/notarius-testing.yml.j2
Normal file
@@ -0,0 +1,3 @@
|
||||
pipelines:
|
||||
{{ dnd_pipeline_narratio_notarius_pipeline_id | to_json }}:
|
||||
llm_profile: {{ dnd_pipeline_notarius_testing_llm_profile | to_json }}
|
||||
@@ -1,3 +1,5 @@
|
||||
notarius:
|
||||
config_overlay: testing
|
||||
scriptorium:
|
||||
artifacts:
|
||||
session_recap:
|
||||
|
||||
@@ -24,6 +24,8 @@ service_fixture_file="$shared_fixture_root/service.txt"
|
||||
operator_fixture_file="$shared_fixture_root/operator.txt"
|
||||
synthetic_audita_api_key='audita-pipeline-synthetic-value'
|
||||
synthetic_notarius_api_key='notarius-pipeline-synthetic-value'
|
||||
synthetic_object_storage_key_id='object-storage-key-id-pipeline-synthetic-value'
|
||||
synthetic_object_storage_key='object-storage-key-pipeline-synthetic-value'
|
||||
baseline_args=(
|
||||
-e 'dnd_pipeline_whisperx_transcribe_url=http://127.0.0.1:8000/transcribe'
|
||||
-e "dnd_pipeline_campaigns_repository_url=file://$campaigns_repository"
|
||||
@@ -31,15 +33,28 @@ baseline_args=(
|
||||
-e "dnd_pipeline_prompt_library_repository_url=file://$prompt_library_repository"
|
||||
-e "dnd_pipeline_prompt_library_revision=$prompt_library_revision"
|
||||
-e "dnd_pipeline_prompt_library_root=$prompt_library_root"
|
||||
-e "{\"dnd_pipeline_secrets\":{\"AUDITA_LLM_API_KEY\":\"$synthetic_audita_api_key\",\"OPENROUTER_API_KEY\":\"$synthetic_notarius_api_key\"}}"
|
||||
-e "{\"dnd_pipeline_operator_users\":[\"$operator_user\"]}"
|
||||
-e "{\"dnd_pipeline_secrets\":{\"AUDITA_LLM_API_KEY\":\"$synthetic_audita_api_key\",\"OPENROUTER_API_KEY\":\"$synthetic_notarius_api_key\",\"OBJECT_STORAGE_KEY_ID\":\"$synthetic_object_storage_key_id\",\"OBJECT_STORAGE_KEY\":\"$synthetic_object_storage_key\"}}"
|
||||
-e 'dnd_pipeline_narratio_storage_backend=s3'
|
||||
-e 'dnd_pipeline_narratio_storage_s3_bucket=dnd-pipeline-fixture'
|
||||
-e 'dnd_pipeline_narratio_storage_s3_root_prefix=fixture/dnd'
|
||||
-e 'dnd_pipeline_narratio_storage_s3_region=fixture-1'
|
||||
-e 'dnd_pipeline_narratio_storage_s3_endpoint=https://storage.fixture.invalid'
|
||||
-e 'dnd_pipeline_narratio_storage_s3_access_key_id=OBJECT_STORAGE_KEY_ID'
|
||||
-e 'dnd_pipeline_narratio_storage_s3_secret_access_key=OBJECT_STORAGE_KEY'
|
||||
-e '{"dnd_pipeline_narratio_workspace_cleanup_after_publish":false,"dnd_pipeline_narratio_storage_s3_force_path_style":false,"dnd_pipeline_narratio_cache_s3_audio":false,"dnd_pipeline_narratio_publish_enabled":true,"dnd_pipeline_narratio_publish_upload_run":true}'
|
||||
-e '{"dnd_pipeline_narratio_publish_outputs":[{"source":"narratio.transcript.final","required":true},{"source":"narratio.artifact.session_recap","required":true}]}'
|
||||
-e 'dnd_pipeline_narratio_testing_scriptorium_profile=endpoint-profile'
|
||||
-e 'dnd_pipeline_notarius_testing_llm_profile=gpt-terra-latest'
|
||||
-e '{"dnd_pipeline_audita_llm_transient_retries":null,"dnd_pipeline_audita_llm_repair_attempts":null,"dnd_pipeline_audita_llm_operation_timeout":null,"dnd_pipeline_notarius_promptkit_transient_retries":null,"dnd_pipeline_scriptorium_transient_retries":null}'
|
||||
)
|
||||
|
||||
cleanup() {
|
||||
exit_status=$?
|
||||
trap - EXIT
|
||||
/usr/bin/sudo /bin/rm -rf -- "$shared_fixture_root" || true
|
||||
/usr/bin/sudo /usr/sbin/userdel --remove "$operator_user" 2>/dev/null || true
|
||||
"$ansible_playbook" -i "$inventory" "${baseline_args[@]}" "$converge" >/dev/null 2>&1 || true
|
||||
/usr/bin/sudo /usr/sbin/userdel --remove "$operator_user" 2>/dev/null || true
|
||||
rm -rf -- "$test_tmp"
|
||||
exit "$exit_status"
|
||||
}
|
||||
@@ -53,6 +68,7 @@ run_converge() {
|
||||
output_file=$1
|
||||
shift
|
||||
env -u AUDITA_LLM_API_KEY -u OPENROUTER_API_KEY \
|
||||
-u OBJECT_STORAGE_KEY_ID -u OBJECT_STORAGE_KEY \
|
||||
"$ansible_playbook" -i "$inventory" "$@" "$converge" | tee "$output_file"
|
||||
}
|
||||
|
||||
@@ -70,18 +86,42 @@ printf '%s\n' \
|
||||
-p "$test_tmp/roles"
|
||||
"$ansible_playbook" -i "$inventory" --syntax-check "$converge"
|
||||
"$workspace_venv/bin/ansible-lint" --offline --profile production "$converge"
|
||||
/usr/bin/sudo /usr/sbin/useradd --create-home --shell /usr/sbin/nologin "$operator_user"
|
||||
|
||||
baseline_log="$test_tmp/baseline.log"
|
||||
run_converge "$baseline_log" "${baseline_args[@]}"
|
||||
/usr/local/bin/seriatim --version
|
||||
/usr/local/bin/audita config validate --config /usr/local/etc/audita/config.yml
|
||||
AUDITA_LLM_API_KEY="$synthetic_audita_api_key" \
|
||||
/usr/local/bin/audita config validate --config /usr/local/etc/audita/config.yml
|
||||
"$workspace_venv/bin/python" - <<'PY'
|
||||
import yaml
|
||||
|
||||
with open("/usr/local/etc/audita/config.yml", encoding="utf-8") as config_file:
|
||||
audita = yaml.safe_load(config_file)
|
||||
with open("/usr/local/etc/notarius/config.yml", encoding="utf-8") as config_file:
|
||||
notarius = yaml.safe_load(config_file)
|
||||
with open("/usr/local/etc/scriptorium/config.yml", encoding="utf-8") as config_file:
|
||||
scriptorium = yaml.safe_load(config_file)
|
||||
|
||||
assert "transient_retries" not in audita["llm"]
|
||||
assert "repair_attempts" not in audita["llm"]
|
||||
assert "operation_timeout" not in audita["llm"]
|
||||
assert "transient_retries" not in notarius.get("promptkit", {})
|
||||
assert "transient_retries" not in scriptorium
|
||||
PY
|
||||
/usr/local/bin/notarius config validate --config /usr/local/etc/notarius/config.yml \
|
||||
--pipeline dnd-session
|
||||
/usr/local/bin/notarius config validate --config /usr/local/etc/notarius/config.yml \
|
||||
--config-overlay testing --pipeline dnd-session
|
||||
notarius_effective_config="$test_tmp/notarius-effective.yml"
|
||||
/usr/local/bin/notarius config effective --config /usr/local/etc/notarius/config.yml \
|
||||
> "$notarius_effective_config"
|
||||
notarius_testing_effective_config="$test_tmp/notarius-testing-effective.yml"
|
||||
/usr/local/bin/notarius config effective --config /usr/local/etc/notarius/config.yml \
|
||||
--config-overlay testing > "$notarius_testing_effective_config"
|
||||
grep -Fq 'mode: auto' "$notarius_effective_config"
|
||||
grep -Fq 'step: describe-session' "$notarius_effective_config"
|
||||
/usr/local/bin/scriptorium config validate --config /usr/local/etc/scriptorium/config.yml
|
||||
/usr/local/bin/scriptorium inspect profile --config /usr/local/etc/scriptorium/config.yml \
|
||||
--profile-dir "$profile_dir" --profile endpoint-profile
|
||||
/usr/bin/sudo -u narratio /usr/local/bin/narratio config validate \
|
||||
@@ -89,21 +129,79 @@ grep -Fq 'step: describe-session' "$notarius_effective_config"
|
||||
/usr/bin/sudo -u narratio /usr/local/bin/narratio config validate \
|
||||
--config /usr/local/etc/narratio/pipeline.yml --campaign dilfs
|
||||
/usr/bin/sudo -u narratio /usr/local/bin/narratio config validate \
|
||||
--config /usr/local/etc/narratio/pipeline.yml --campaign centaurs --profile testing
|
||||
--config /usr/local/etc/narratio/pipeline.yml --campaign centaurs --overlay testing
|
||||
narratio_effective_config="$test_tmp/narratio-effective.yml"
|
||||
/usr/bin/sudo -u narratio /usr/local/bin/narratio config show \
|
||||
--config /usr/local/etc/narratio/pipeline.yml --campaign centaurs \
|
||||
> "$narratio_effective_config"
|
||||
narratio_testing_effective_config="$test_tmp/narratio-testing-effective.yml"
|
||||
/usr/bin/sudo -u narratio /usr/local/bin/narratio config show \
|
||||
--config /usr/local/etc/narratio/pipeline.yml --campaign centaurs --overlay testing \
|
||||
> "$narratio_testing_effective_config"
|
||||
grep -Fq 'enabled: true' "$narratio_effective_config"
|
||||
grep -Fq 'llm_api_key: AUDITA_LLM_API_KEY' "$narratio_effective_config"
|
||||
grep -Fq 'pipeline_id: dnd-session' "$narratio_effective_config"
|
||||
grep -Fq 'llm_api_key: OPENROUTER_API_KEY' "$narratio_effective_config"
|
||||
grep -Fq 'resume: true' "$narratio_effective_config"
|
||||
! grep -Fq 'chunk_cache:' "$narratio_effective_config"
|
||||
grep -Fq 'party: narratio.input.party' "$narratio_effective_config"
|
||||
grep -Fq 'lane_id: item-registry' "$narratio_effective_config"
|
||||
! grep -Fq "$synthetic_audita_api_key" "$baseline_log" "$narratio_effective_config" "$notarius_effective_config"
|
||||
! grep -Fq "$synthetic_notarius_api_key" "$baseline_log" "$narratio_effective_config" "$notarius_effective_config"
|
||||
"$workspace_venv/bin/python" - \
|
||||
"$narratio_effective_config" "$narratio_testing_effective_config" <<'PY'
|
||||
import sys
|
||||
|
||||
import yaml
|
||||
|
||||
with open(sys.argv[1], encoding="utf-8") as config_file:
|
||||
config = yaml.safe_load(config_file)
|
||||
with open(sys.argv[2], encoding="utf-8") as config_file:
|
||||
testing_config = yaml.safe_load(config_file)
|
||||
|
||||
assert config["workspace"]["cleanup_after_publish"] is False
|
||||
assert config["storage"] == {
|
||||
"backend": "s3",
|
||||
"s3": {
|
||||
"bucket": "dnd-pipeline-fixture",
|
||||
"root_prefix": "fixture/dnd",
|
||||
"region": "fixture-1",
|
||||
"endpoint": "https://storage.fixture.invalid",
|
||||
"force_path_style": False,
|
||||
"access_key_id": "OBJECT_STORAGE_KEY_ID",
|
||||
"secret_access_key": "OBJECT_STORAGE_KEY",
|
||||
},
|
||||
}
|
||||
assert config["cache"]["s3_audio"] is False
|
||||
assert config["audita"]["credentials"] == ["AUDITA_LLM_API_KEY"]
|
||||
assert config["notarius"]["credentials"] == ["OPENROUTER_API_KEY"]
|
||||
assert config["publish"] == {
|
||||
"enabled": True,
|
||||
"upload_run": True,
|
||||
"outputs": [
|
||||
{
|
||||
"source": "narratio.transcript.final",
|
||||
"dest": "transcripts/final.json",
|
||||
"required": True,
|
||||
},
|
||||
{
|
||||
"source": "narratio.artifact.session_recap",
|
||||
"dest": "artifacts/session_recap.md",
|
||||
"required": True,
|
||||
},
|
||||
],
|
||||
}
|
||||
assert "config_overlay" not in config["notarius"]
|
||||
assert testing_config["notarius"]["config_overlay"] == "testing"
|
||||
assert all(
|
||||
artifact["profile_id"] == "endpoint-profile"
|
||||
for artifact in testing_config["scriptorium"]["artifacts"].values()
|
||||
)
|
||||
assert any(
|
||||
artifact_id.startswith("character_meta_analysis_")
|
||||
for artifact_id in testing_config["scriptorium"]["artifacts"]
|
||||
)
|
||||
PY
|
||||
! grep -Fq "$synthetic_audita_api_key" "$baseline_log" "$narratio_effective_config" "$narratio_testing_effective_config" "$notarius_effective_config" "$notarius_testing_effective_config"
|
||||
! grep -Fq "$synthetic_notarius_api_key" "$baseline_log" "$narratio_effective_config" "$narratio_testing_effective_config" "$notarius_effective_config" "$notarius_testing_effective_config"
|
||||
! grep -Fq "$synthetic_object_storage_key_id" "$baseline_log" "$narratio_effective_config" "$narratio_testing_effective_config" "$notarius_effective_config" "$notarius_testing_effective_config"
|
||||
! grep -Fq "$synthetic_object_storage_key" "$baseline_log" "$narratio_effective_config" "$narratio_testing_effective_config" "$notarius_effective_config" "$notarius_testing_effective_config"
|
||||
/usr/bin/sudo -u narratio /usr/local/bin/narratio config artifacts \
|
||||
--config /usr/local/etc/narratio/pipeline.yml --campaign centaurs | \
|
||||
awk -F '\t' '$1 == "session_recap" && $4 == "true" { found = 1 } END { exit !found }'
|
||||
@@ -113,11 +211,6 @@ grep -Fq 'lane_id: item-registry' "$narratio_effective_config"
|
||||
/usr/bin/sudo -u narratio /usr/local/bin/narratio config artifacts \
|
||||
--config /usr/local/etc/narratio/pipeline.yml --campaign centaurs | \
|
||||
awk -F '\t' 'index($1, "character_meta_analysis_") == 1 && $4 == "false" && $7 == "character_meta_analysis" { found = 1 } END { exit !found }'
|
||||
/usr/bin/sudo -u narratio /usr/local/bin/narratio config diff default testing \
|
||||
--config /usr/local/etc/narratio/pipeline.yml --campaign centaurs | \
|
||||
awk -F '\t' '$1 != "changed" || $2 !~ /\.profile_id$/ { invalid = 1 } END { exit invalid || NR == 0 }'
|
||||
/usr/bin/sudo /usr/sbin/useradd --create-home --groups narratio \
|
||||
--shell /usr/sbin/nologin "$operator_user"
|
||||
/usr/bin/sudo -u "$operator_user" /usr/local/bin/narratio config validate \
|
||||
--config /usr/local/etc/narratio/pipeline.yml --campaign centaurs
|
||||
/usr/bin/sudo -u "$operator_user" /usr/local/bin/notarius config validate \
|
||||
@@ -165,21 +258,6 @@ if run_converge "$invalid_key_log" "${baseline_args[@]}" \
|
||||
exit 1
|
||||
fi
|
||||
|
||||
testing_profile_log="$test_tmp/testing-profile.log"
|
||||
run_converge "$testing_profile_log" "${baseline_args[@]}" \
|
||||
-e 'dnd_pipeline_narratio_testing_scriptorium_profile=endpoint-profile'
|
||||
/usr/bin/sudo -u narratio /usr/local/bin/narratio config validate \
|
||||
--config /usr/local/etc/narratio/pipeline.yml --campaign centaurs --profile testing
|
||||
/usr/bin/sudo -u narratio /usr/local/bin/narratio config diff default testing \
|
||||
--config /usr/local/etc/narratio/pipeline.yml --campaign centaurs | \
|
||||
grep -F '"endpoint-profile"'
|
||||
/usr/local/bin/scriptorium inspect profile --config /usr/local/etc/scriptorium/config.yml \
|
||||
--profile-dir "$profile_dir" --profile endpoint-profile
|
||||
testing_profile_repeat_log="$test_tmp/testing-profile-repeat.log"
|
||||
run_converge "$testing_profile_repeat_log" "${baseline_args[@]}" \
|
||||
-e 'dnd_pipeline_narratio_testing_scriptorium_profile=endpoint-profile'
|
||||
require_clean_recap "$testing_profile_repeat_log"
|
||||
|
||||
selected_version_log="$test_tmp/selected-version.log"
|
||||
run_converge "$selected_version_log" "${baseline_args[@]}" \
|
||||
-e 'dnd_pipeline_seriatim_version=v1.4.0'
|
||||
@@ -189,24 +267,120 @@ run_converge "$selected_version_repeat_log" "${baseline_args[@]}" \
|
||||
-e 'dnd_pipeline_seriatim_version=v1.4.0'
|
||||
require_clean_recap "$selected_version_repeat_log"
|
||||
|
||||
pipeline_group_one='{"archive-session":{"llm_profile":"dnd-extraction","input":{"module":"seriatim"},"artifacts":{"spells":{"extract":{"module":"dnd/spells"},"normalize":{"module":"dnd/spells"}}}}}'
|
||||
pipeline_group_two='{"research-session":{"llm_profile":"dnd-extraction","input":{"module":"seriatim"},"artifacts":{"spells":{"extract":{"module":"dnd/spells"},"normalize":{"module":"dnd/spells"}}}}}'
|
||||
alternate_config="{\"notarius_pipeline_group_one\":$pipeline_group_one,\"notarius_pipeline_group_two\":$pipeline_group_two,\"dnd_pipeline_notarius_pipelines\":\"{{ notarius_pipeline_group_one | combine(notarius_pipeline_group_two) }}\",\"dnd_pipeline_narratio_notarius_pipeline_id\":\"archive-session\",\"dnd_pipeline_notarius_cache_chunk_plans_mode\":\"bypass\"}"
|
||||
pipeline_group_one='{"archive-session":{"llm_profile":"gpt-luna-latest","input":{"module":"seriatim"},"artifacts":{"spells":{"extract":{"module":"dnd/spells"},"normalize":{"module":"dnd/spells"}}}}}'
|
||||
pipeline_group_two='{"research-session":{"llm_profile":"gpt-luna-latest","input":{"module":"seriatim"},"artifacts":{"spells":{"extract":{"module":"dnd/spells"},"normalize":{"module":"dnd/spells"}}}}}'
|
||||
alternate_config="{\"notarius_pipeline_group_one\":$pipeline_group_one,\"notarius_pipeline_group_two\":$pipeline_group_two,\"dnd_pipeline_notarius_pipelines\":\"{{ notarius_pipeline_group_one | combine(notarius_pipeline_group_two) }}\",\"dnd_pipeline_narratio_notarius_pipeline_id\":\"archive-session\",\"dnd_pipeline_notarius_testing_llm_profile\":\"gpt-terra-latest\",\"dnd_pipeline_notarius_cache_chunk_plans_mode\":\"bypass\"}"
|
||||
alternate_log="$test_tmp/alternate.log"
|
||||
run_converge "$alternate_log" "${baseline_args[@]}" -e "$alternate_config"
|
||||
/usr/local/bin/notarius config validate --config /usr/local/etc/notarius/config.yml \
|
||||
--pipeline archive-session
|
||||
/usr/local/bin/notarius config validate --config /usr/local/etc/notarius/config.yml \
|
||||
--config-overlay testing --pipeline archive-session
|
||||
alternate_narratio_config="$test_tmp/alternate-narratio.yml"
|
||||
/usr/bin/sudo -u narratio /usr/local/bin/narratio config show \
|
||||
--config /usr/local/etc/narratio/pipeline.yml --campaign centaurs \
|
||||
> "$alternate_narratio_config"
|
||||
grep -Fq 'pipeline_id: archive-session' "$alternate_narratio_config"
|
||||
alternate_testing_narratio_config="$test_tmp/alternate-testing-narratio.yml"
|
||||
/usr/bin/sudo -u narratio /usr/local/bin/narratio config show \
|
||||
--config /usr/local/etc/narratio/pipeline.yml --campaign centaurs --overlay testing \
|
||||
> "$alternate_testing_narratio_config"
|
||||
alternate_effective_config="$test_tmp/alternate-effective.yml"
|
||||
/usr/local/bin/notarius config effective --config /usr/local/etc/notarius/config.yml \
|
||||
> "$alternate_effective_config"
|
||||
grep -Fq 'archive-session:' "$alternate_effective_config"
|
||||
grep -Fq 'research-session:' "$alternate_effective_config"
|
||||
grep -Fq 'mode: bypass' "$alternate_effective_config"
|
||||
alternate_testing_effective_config="$test_tmp/alternate-testing-effective.yml"
|
||||
/usr/local/bin/notarius config effective --config /usr/local/etc/notarius/config.yml \
|
||||
--config-overlay testing > "$alternate_testing_effective_config"
|
||||
"$workspace_venv/bin/python" - \
|
||||
"$alternate_narratio_config" "$alternate_testing_narratio_config" \
|
||||
"$alternate_effective_config" "$alternate_testing_effective_config" <<'PY'
|
||||
import sys
|
||||
|
||||
import yaml
|
||||
|
||||
with open(sys.argv[1], encoding="utf-8") as config_file:
|
||||
narratio_base = yaml.safe_load(config_file)
|
||||
with open(sys.argv[2], encoding="utf-8") as config_file:
|
||||
narratio_testing = yaml.safe_load(config_file)
|
||||
with open(sys.argv[3], encoding="utf-8") as config_file:
|
||||
notarius_base = yaml.safe_load(config_file)
|
||||
with open(sys.argv[4], encoding="utf-8") as config_file:
|
||||
notarius_testing = yaml.safe_load(config_file)
|
||||
|
||||
assert narratio_base["notarius"]["pipeline_id"] == "archive-session"
|
||||
assert "config_overlay" not in narratio_base["notarius"]
|
||||
assert narratio_testing["notarius"]["pipeline_id"] == "archive-session"
|
||||
assert narratio_testing["notarius"]["config_overlay"] == "testing"
|
||||
assert notarius_base["pipelines"]["archive-session"]["llm_profile"] == "gpt-luna-latest"
|
||||
assert notarius_testing["pipelines"]["archive-session"]["llm_profile"] == "gpt-terra-latest"
|
||||
assert notarius_testing["pipelines"]["research-session"]["llm_profile"] == "gpt-luna-latest"
|
||||
assert notarius_base["cache"]["chunk_plans"]["mode"] == "bypass"
|
||||
PY
|
||||
alternate_repeat_log="$test_tmp/alternate-repeat.log"
|
||||
run_converge "$alternate_repeat_log" "${baseline_args[@]}" -e "$alternate_config"
|
||||
require_clean_recap "$alternate_repeat_log"
|
||||
|
||||
synthetic_fixture_audita_key='audita-override-synthetic-value'
|
||||
synthetic_fixture_notarius_key='notarius-override-synthetic-value'
|
||||
override_config="{\"dnd_pipeline_audita_backends\":{\"fixture\":{\"endpoint\":\"https://inference.fixture.invalid/v1\",\"credential\":\"FIXTURE_AUDITA_KEY\",\"concurrency\":2}},"\
|
||||
"\"dnd_pipeline_audita_profiles\":{\"fixture-proposal\":{\"id\":\"fixture-proposal\",\"backend\":\"fixture\",\"model\":\"fixture/proposal\",\"timeout_seconds\":900},"\
|
||||
"\"fixture-validation\":{\"id\":\"fixture-validation\",\"backend\":\"fixture\",\"model\":\"fixture/validation\",\"timeout_seconds\":900}},"\
|
||||
"\"dnd_pipeline_audita_llm_proposal_profile\":\"fixture-proposal\",\"dnd_pipeline_audita_llm_validation_profile\":\"fixture-validation\","\
|
||||
"\"dnd_pipeline_narratio_audita_credentials\":[\"FIXTURE_AUDITA_KEY\"],\"dnd_pipeline_narratio_notarius_credentials\":[\"FIXTURE_NOTARIUS_KEY\"],"\
|
||||
"\"dnd_pipeline_audita_llm_transient_retries\":0,\"dnd_pipeline_audita_llm_repair_attempts\":0,\"dnd_pipeline_audita_llm_operation_timeout\":\"20m\","\
|
||||
"\"dnd_pipeline_notarius_promptkit_transient_retries\":0,\"dnd_pipeline_scriptorium_transient_retries\":0,"\
|
||||
"\"dnd_pipeline_secrets\":{\"AUDITA_LLM_API_KEY\":\"$synthetic_audita_api_key\",\"OPENROUTER_API_KEY\":\"$synthetic_notarius_api_key\","\
|
||||
"\"FIXTURE_AUDITA_KEY\":\"$synthetic_fixture_audita_key\",\"FIXTURE_NOTARIUS_KEY\":\"$synthetic_fixture_notarius_key\","\
|
||||
"\"OBJECT_STORAGE_KEY_ID\":\"$synthetic_object_storage_key_id\",\"OBJECT_STORAGE_KEY\":\"$synthetic_object_storage_key\"}}"
|
||||
override_log="$test_tmp/override.log"
|
||||
run_converge "$override_log" "${baseline_args[@]}" -e "$override_config"
|
||||
FIXTURE_AUDITA_KEY="$synthetic_fixture_audita_key" \
|
||||
/usr/local/bin/audita config validate --config /usr/local/etc/audita/config.yml
|
||||
audita_override_settings="$test_tmp/audita-override-settings.yml"
|
||||
audita_override_profiles="$test_tmp/audita-override-profiles.yml"
|
||||
FIXTURE_AUDITA_KEY="$synthetic_fixture_audita_key" \
|
||||
/usr/local/bin/audita config print-effective --config /usr/local/etc/audita/config.yml \
|
||||
--view settings > "$audita_override_settings"
|
||||
FIXTURE_AUDITA_KEY="$synthetic_fixture_audita_key" \
|
||||
/usr/local/bin/audita config print-effective --config /usr/local/etc/audita/config.yml \
|
||||
--view profiles > "$audita_override_profiles"
|
||||
notarius_override_config="$test_tmp/notarius-override.yml"
|
||||
scriptorium_override_config="$test_tmp/scriptorium-override.yml"
|
||||
narratio_override_config="$test_tmp/narratio-override.yml"
|
||||
/usr/local/bin/notarius config effective --config /usr/local/etc/notarius/config.yml \
|
||||
> "$notarius_override_config"
|
||||
/usr/local/bin/scriptorium config show --config /usr/local/etc/scriptorium/config.yml \
|
||||
> "$scriptorium_override_config"
|
||||
/usr/bin/sudo -u narratio /usr/local/bin/narratio config show \
|
||||
--config /usr/local/etc/narratio/pipeline.yml --campaign centaurs \
|
||||
> "$narratio_override_config"
|
||||
"$workspace_venv/bin/python" - \
|
||||
"$audita_override_settings" "$audita_override_profiles" \
|
||||
"$notarius_override_config" "$scriptorium_override_config" \
|
||||
"$narratio_override_config" <<'PY'
|
||||
import sys
|
||||
|
||||
import yaml
|
||||
|
||||
settings, profiles, notarius, scriptorium, narratio = (
|
||||
yaml.safe_load(open(path, encoding="utf-8")) for path in sys.argv[1:]
|
||||
)
|
||||
assert settings["backends"]["fixture"]["endpoint"] == "https://inference.fixture.invalid/v1"
|
||||
assert settings["backends"]["fixture"]["concurrency"] == 2
|
||||
assert settings["llm"]["proposal_profile"] == "fixture-proposal"
|
||||
assert settings["llm"]["validation_profile"] == "fixture-validation"
|
||||
assert settings["llm"]["transient_retries"] == 0
|
||||
assert settings["llm"]["repair_attempts"] == 0
|
||||
assert settings["llm"]["operation_timeout"] == "20m0s"
|
||||
assert profiles["proposal"]["model"] == "fixture/proposal"
|
||||
assert profiles["validation"]["model"] == "fixture/validation"
|
||||
assert notarius["promptkit"]["transient_retries"] == 0
|
||||
assert scriptorium["transient_retries"] == 0
|
||||
assert narratio["audita"]["credentials"] == ["FIXTURE_AUDITA_KEY"]
|
||||
assert narratio["notarius"]["credentials"] == ["FIXTURE_NOTARIUS_KEY"]
|
||||
PY
|
||||
! grep -Fq "$synthetic_fixture_audita_key" "$override_log" "$narratio_override_config"
|
||||
! grep -Fq "$synthetic_fixture_notarius_key" "$override_log" "$narratio_override_config"
|
||||
override_repeat_log="$test_tmp/override-repeat.log"
|
||||
run_converge "$override_repeat_log" "${baseline_args[@]}" -e "$override_config"
|
||||
require_clean_recap "$override_repeat_log"
|
||||
|
||||
@@ -22,7 +22,8 @@ baseline_args=(
|
||||
-e "dnd_pipeline_campaigns_revision=$campaigns_revision"
|
||||
-e "dnd_pipeline_prompt_library_repository_url=file://$prompt_library_repository"
|
||||
-e "dnd_pipeline_prompt_library_revision=$prompt_library_revision"
|
||||
-e '{"dnd_pipeline_secrets":{"AUDITA_LLM_API_KEY":"audita-pipeline-synthetic-value","OPENROUTER_API_KEY":"notarius-pipeline-synthetic-value"}}'
|
||||
-e 'dnd_pipeline_notarius_testing_llm_profile=gpt-terra-latest'
|
||||
-e '{"dnd_pipeline_secrets":{"AUDITA_LLM_API_KEY":"audita-pipeline-synthetic-value","OPENROUTER_API_KEY":"notarius-pipeline-synthetic-value","OBJECT_STORAGE_KEY_ID":"object-storage-key-id-pipeline-synthetic-value","OBJECT_STORAGE_KEY":"object-storage-key-pipeline-synthetic-value"}}'
|
||||
)
|
||||
|
||||
cleanup() {
|
||||
@@ -36,6 +37,7 @@ trap cleanup EXIT
|
||||
run_converge() {
|
||||
output_file=$1
|
||||
env -u AUDITA_LLM_API_KEY -u OPENROUTER_API_KEY \
|
||||
-u OBJECT_STORAGE_KEY_ID -u OBJECT_STORAGE_KEY \
|
||||
"$ansible_playbook" -i "$inventory" "${baseline_args[@]}" "$converge" | \
|
||||
tee "$output_file"
|
||||
}
|
||||
@@ -61,14 +63,51 @@ fi
|
||||
baseline_log="$test_tmp/baseline.log"
|
||||
run_converge "$baseline_log"
|
||||
/usr/local/bin/seriatim --version
|
||||
/usr/local/bin/audita config validate --config /usr/local/etc/audita/config.yml
|
||||
AUDITA_LLM_API_KEY='audita-pipeline-synthetic-value' \
|
||||
/usr/local/bin/audita config validate --config /usr/local/etc/audita/config.yml
|
||||
audita_profiles="$test_tmp/audita-profiles.yml"
|
||||
AUDITA_LLM_API_KEY='audita-pipeline-synthetic-value' \
|
||||
/usr/local/bin/audita config print-effective --config /usr/local/etc/audita/config.yml \
|
||||
--view profiles > "$audita_profiles"
|
||||
/usr/local/bin/notarius config validate --config /usr/local/etc/notarius/config.yml \
|
||||
--pipeline dnd-session
|
||||
/usr/local/bin/notarius config validate --config /usr/local/etc/notarius/config.yml \
|
||||
--config-overlay testing --pipeline dnd-session
|
||||
notarius_testing_config="$test_tmp/notarius-testing.yml"
|
||||
/usr/local/bin/notarius config effective --config /usr/local/etc/notarius/config.yml \
|
||||
--config-overlay testing > "$notarius_testing_config"
|
||||
/usr/local/bin/scriptorium config validate --config /usr/local/etc/scriptorium/config.yml
|
||||
/usr/local/bin/scriptorium inspect prompt \
|
||||
--config /usr/local/etc/scriptorium/config.yml \
|
||||
--prompt dnd.session_recap --prompt-version 2.0.0 --format json >/dev/null
|
||||
/usr/bin/sudo -u narratio /usr/local/bin/narratio config validate \
|
||||
--config /usr/local/etc/narratio/pipeline.yml --campaign centaurs
|
||||
/usr/bin/sudo -u narratio /usr/local/bin/narratio config validate \
|
||||
--config /usr/local/etc/narratio/pipeline.yml --campaign centaurs --overlay testing
|
||||
narratio_testing_config="$test_tmp/narratio-testing.yml"
|
||||
/usr/bin/sudo -u narratio /usr/local/bin/narratio config show \
|
||||
--config /usr/local/etc/narratio/pipeline.yml --campaign centaurs --overlay testing \
|
||||
> "$narratio_testing_config"
|
||||
"$workspace_venv/bin/python" - \
|
||||
"$audita_profiles" "$notarius_testing_config" "$narratio_testing_config" <<'PY'
|
||||
import sys
|
||||
|
||||
import yaml
|
||||
|
||||
with open(sys.argv[1], encoding="utf-8") as config_file:
|
||||
audita_profiles = yaml.safe_load(config_file)
|
||||
with open(sys.argv[2], encoding="utf-8") as config_file:
|
||||
notarius = yaml.safe_load(config_file)
|
||||
with open(sys.argv[3], encoding="utf-8") as config_file:
|
||||
narratio = yaml.safe_load(config_file)
|
||||
|
||||
assert audita_profiles["proposal"]["id"] == "dnd-proposal"
|
||||
assert audita_profiles["validation"]["id"] == "dnd-validation"
|
||||
assert notarius["pipelines"]["dnd-session"]["llm_profile"] == "gpt-terra-latest"
|
||||
assert narratio["notarius"]["config_overlay"] == "testing"
|
||||
assert narratio["audita"]["credentials"] == ["AUDITA_LLM_API_KEY"]
|
||||
assert narratio["notarius"]["credentials"] == ["OPENROUTER_API_KEY"]
|
||||
PY
|
||||
|
||||
repeat_log="$test_tmp/repeat.log"
|
||||
run_converge "$repeat_log"
|
||||
|
||||
Reference in New Issue
Block a user