diff --git a/README.md b/README.md index bfc0f44..31a1aa2 100644 --- a/README.md +++ b/README.md @@ -18,7 +18,8 @@ ansible-galaxy collection install -r collections/requirements.yml The dependency manifest installs the roles under their canonical names: `rakestraw.seriatim`, `rakestraw.audita`, `rakestraw.notarius`, `rakestraw.scriptorium`, and `rakestraw.narratio`. It selects the compatible -`v0.1.0` release of each role. This standalone role is named +`v0.1.0` releases where compatible and public commit pins for Narratio and +Notarius when the suite needs newer interfaces. This standalone role is named `rakestraw.dnd_pipeline`. ```yaml @@ -62,7 +63,7 @@ The compatible application versions are selected through these variables: | --- | --- | | `dnd_pipeline_audita_version` | `v0.9.1` | | `dnd_pipeline_narratio_version` | `v2.2.0` | -| `dnd_pipeline_notarius_version` | `v0.7.0` | +| `dnd_pipeline_notarius_version` | `v0.7.1` | | `dnd_pipeline_seriatim_version` | `v1.5.0` | | `dnd_pipeline_scriptorium_version` | `v0.13.0` | | `dnd_pipeline_manage_build_dependencies` | `true` | @@ -146,6 +147,7 @@ run-specific output path. | `dnd_pipeline_narratio_render_enabled` | `true` | | `dnd_pipeline_narratio_render_format` | `markdown` | | `dnd_pipeline_narratio_testing_scriptorium_profile` | `gemini-flash-latest` | +| `dnd_pipeline_notarius_testing_llm_profile` | `gpt-luna-latest` | `dnd_pipeline_audita_output_schema` is also the Narratio Audita-adapter schema; it is the shared cross-application boundary. Other Narratio-local controls, @@ -222,14 +224,25 @@ suite-owned base fragments in this fixed order: └── profiles/testing.yml ``` -The `testing` profile is an overlay, not a second configuration tree. It only -sets the Scriptorium `profile_id` for the four artifact sources and the one -family. Its default, `gemini-flash-latest`, is an inexpensive development +Notarius follows the corresponding base-and-overlay layout: + +```text +/usr/local/etc/notarius/ +├── config.yml +├── pipelines/dnd-session.yml +└── profiles/testing.yml +``` + +The `testing` profile is an overlay, not a second configuration tree. It sets +the Scriptorium `profile_id` for the four artifact sources and the one family, +and explicitly selects Notarius's separately managed `testing` overlay. The +Scriptorium default, `gemini-flash-latest`, is an inexpensive development choice, not a compatibility guarantee. Replace it through inventory when a different testing profile is appropriate: ```yaml dnd_pipeline_narratio_testing_scriptorium_profile: endpoint-profile +dnd_pipeline_notarius_testing_llm_profile: alternate-economy-profile ``` ### Notarius extraction policy @@ -237,6 +250,8 @@ dnd_pipeline_narratio_testing_scriptorium_profile: endpoint-profile The Notarius root configuration imports the fixed `pipelines/dnd-session.yml` fragment. The suite owns that fragment at `/usr/local/etc/notarius/pipelines/dnd-session.yml`; callers do not edit it. +The separate `/usr/local/etc/notarius/profiles/testing.yml` overlay changes only +the selected suite pipeline's LLM profile. | Variable | Default policy | | --- | --- | @@ -244,6 +259,7 @@ The Notarius root configuration imports the fixed | `dnd_pipeline_notarius_cache_root` | `/var/cache/notarius` | | `dnd_pipeline_notarius_cache_chunk_plans_mode` | `auto` | | `dnd_pipeline_notarius_cache_checkpoints_enabled` | `true` | +| `dnd_pipeline_notarius_testing_llm_profile` | `gpt-luna-latest` | | `dnd_pipeline_narratio_notarius_pipeline_id` | `dnd-session` | | `dnd_pipeline_narratio_notarius_llm_api_key` | `OPENROUTER_API_KEY` | | `dnd_pipeline_narratio_notarius_resume` | `true` | @@ -252,7 +268,11 @@ The Notarius root configuration imports the fixed The pipeline, reference, and output values are complete native mappings. Inventory can replace them or compose additions with Jinja's `combine` filter; -a replacement pipeline map requires a matching selected pipeline ID. +a replacement pipeline map requires a matching selected pipeline ID. The base +`dnd-session` pipeline selects `gpt-terra-latest`, while the testing overlay +selects `gpt-luna-latest`. Both are moving catalog IDs rather than immutable +model promises. Pipeline and binding-level overrides retain Notarius's native +precedence; the suite does not rewrite caller-supplied bindings. Narratio materializes the `party`, `players`, and `glossary` references for each invocation. No campaign filesystem path appears in Notarius configuration. The @@ -270,10 +290,13 @@ Notarius profile and ordinary extraction requests checkpoint reuse with `resume: true`. Chunk plans use Notarius's root-owned `auto` policy; the pipeline deliberately leaves Narratio's optional `chunk_cache` override unset. The `chunk-plans` and `checkpoints` paths derive from the cache root and are -created by Notarius during use. Base and `testing` configurations select the -same `dnd-extraction` profile and Notarius pipeline; the testing overlay changes -only Scriptorium profiles, so a real testing run may invoke Notarius. The -integration suite performs offline configuration inspection only. +created by Notarius during use. Notarius v0.7.1 selects PromptKit v0.10.0 and +OpenRouter catalog v2.20260922.1, which supplies both suite-selected IDs. +Existing checkpoints with the previous profile-source identity are not reused; +source-keyed chunk plans and output artifacts remain compatible. No cache +deletion or forced extraction is performed by this role. A real testing run may +invoke Notarius; the integration suite performs offline configuration inspection +only and makes no provider request. ### Inspecting configuration diff --git a/defaults/main.yml b/defaults/main.yml index 02f20fd..aea4243 100644 --- a/defaults/main.yml +++ b/defaults/main.yml @@ -1,7 +1,7 @@ --- dnd_pipeline_audita_version: v0.9.1 dnd_pipeline_narratio_version: v2.2.0 -dnd_pipeline_notarius_version: v0.7.0 +dnd_pipeline_notarius_version: v0.7.1 dnd_pipeline_seriatim_version: v1.5.0 dnd_pipeline_scriptorium_version: v0.13.0 dnd_pipeline_manage_build_dependencies: true @@ -58,7 +58,7 @@ dnd_pipeline_audita_diagnostics_work_dir: /tmp/audita dnd_pipeline_audita_diagnostics_retention: auto dnd_pipeline_notarius_pipelines: dnd-session: - llm_profile: dnd-extraction + llm_profile: gpt-terra-latest input: module: seriatim chunk: @@ -207,6 +207,7 @@ dnd_pipeline_notarius_pipelines: dnd_pipeline_notarius_cache_root: /var/cache/notarius dnd_pipeline_notarius_cache_chunk_plans_mode: auto dnd_pipeline_notarius_cache_checkpoints_enabled: true +dnd_pipeline_notarius_testing_llm_profile: gpt-luna-latest dnd_pipeline_narratio_workspace_cleanup_after_publish: true dnd_pipeline_narratio_storage_backend: s3 dnd_pipeline_narratio_storage_s3_bucket: rakestraw-assets diff --git a/meta/argument_specs.yml b/meta/argument_specs.yml index ea96720..c740e28 100644 --- a/meta/argument_specs.yml +++ b/meta/argument_specs.yml @@ -221,6 +221,10 @@ argument_specs: type: bool required: true description: Whether Notarius records checkpoints. + dnd_pipeline_notarius_testing_llm_profile: + type: str + required: true + description: Notarius LLM profile selected by the testing overlay. dnd_pipeline_narratio_workspace_cleanup_after_publish: type: bool required: true diff --git a/requirements.yml b/requirements.yml index 91b29a6..5082424 100644 --- a/requirements.yml +++ b/requirements.yml @@ -11,7 +11,7 @@ roles: - name: rakestraw.notarius src: https://gitea.maximumdirect.net/eric/ansible-role-notarius.git scm: git - version: v0.1.0 + version: a82a83dfc23bb80a59f45f94f78d6754c47285e7 - name: rakestraw.scriptorium src: https://gitea.maximumdirect.net/eric/ansible-role-scriptorium.git scm: git @@ -19,4 +19,4 @@ roles: - name: rakestraw.narratio src: https://gitea.maximumdirect.net/eric/ansible-role-narratio.git scm: git - version: v0.1.0 + version: 330afe7856da94b2402a75447f4b60aba767843f diff --git a/tasks/main.yml b/tasks/main.yml index 08ef3b8..4e3ebea 100644 --- a/tasks/main.yml +++ b/tasks/main.yml @@ -146,14 +146,17 @@ - artifact-families - profiles -- name: Create the Notarius pipeline configuration directory +- name: Create Notarius configuration directories become: true ansible.builtin.file: - path: /usr/local/etc/notarius/pipelines + path: "/usr/local/etc/notarius/{{ item }}" state: directory owner: "{{ dnd_pipeline_runtime_user }}" group: "{{ dnd_pipeline_runtime_group }}" mode: "{{ dnd_pipeline_config_directory_mode }}" + loop: + - pipelines + - profiles - name: Deploy the Notarius D&D pipeline configuration become: true @@ -164,6 +167,15 @@ group: "{{ dnd_pipeline_runtime_group }}" mode: "{{ dnd_pipeline_config_file_mode }}" +- name: Deploy the Notarius testing overlay + become: true + ansible.builtin.template: + src: notarius-testing.yml.j2 + dest: /usr/local/etc/notarius/profiles/testing.yml + owner: "{{ dnd_pipeline_runtime_user }}" + group: "{{ dnd_pipeline_runtime_group }}" + mode: "{{ dnd_pipeline_config_file_mode }}" + - name: Deploy Narratio D&D configuration fragments become: true ansible.builtin.template: diff --git a/templates/notarius-testing.yml.j2 b/templates/notarius-testing.yml.j2 new file mode 100644 index 0000000..905de8e --- /dev/null +++ b/templates/notarius-testing.yml.j2 @@ -0,0 +1,3 @@ +pipelines: + {{ dnd_pipeline_narratio_notarius_pipeline_id | to_json }}: + llm_profile: {{ dnd_pipeline_notarius_testing_llm_profile | to_json }} diff --git a/templates/testing.yml.j2 b/templates/testing.yml.j2 index 51dacc7..33aaea9 100644 --- a/templates/testing.yml.j2 +++ b/templates/testing.yml.j2 @@ -1,3 +1,5 @@ +notarius: + config_profile: testing scriptorium: artifacts: session_recap: diff --git a/tests/integration b/tests/integration index c90905c..d242c17 100755 --- a/tests/integration +++ b/tests/integration @@ -44,6 +44,8 @@ baseline_args=( -e 'dnd_pipeline_narratio_storage_s3_secret_access_key=OBJECT_STORAGE_KEY' -e '{"dnd_pipeline_narratio_workspace_cleanup_after_publish":false,"dnd_pipeline_narratio_storage_s3_force_path_style":false,"dnd_pipeline_narratio_cache_s3_audio":false,"dnd_pipeline_narratio_publish_enabled":true,"dnd_pipeline_narratio_publish_upload_run":true}' -e '{"dnd_pipeline_narratio_publish_outputs":[{"source":"narratio.transcript.final","required":true},{"source":"narratio.artifact.session_recap","required":true}]}' + -e 'dnd_pipeline_narratio_testing_scriptorium_profile=endpoint-profile' + -e 'dnd_pipeline_notarius_testing_llm_profile=gpt-terra-latest' ) cleanup() { @@ -91,9 +93,14 @@ run_converge "$baseline_log" "${baseline_args[@]}" /usr/local/bin/audita config validate --config /usr/local/etc/audita/config.yml /usr/local/bin/notarius config validate --config /usr/local/etc/notarius/config.yml \ --pipeline dnd-session +/usr/local/bin/notarius config validate --config /usr/local/etc/notarius/config.yml \ + --config-profile testing --pipeline dnd-session notarius_effective_config="$test_tmp/notarius-effective.yml" /usr/local/bin/notarius config effective --config /usr/local/etc/notarius/config.yml \ > "$notarius_effective_config" +notarius_testing_effective_config="$test_tmp/notarius-testing-effective.yml" +/usr/local/bin/notarius config effective --config /usr/local/etc/notarius/config.yml \ + --config-profile testing > "$notarius_testing_effective_config" grep -Fq 'mode: auto' "$notarius_effective_config" grep -Fq 'step: describe-session' "$notarius_effective_config" /usr/local/bin/scriptorium inspect profile --config /usr/local/etc/scriptorium/config.yml \ @@ -108,6 +115,10 @@ narratio_effective_config="$test_tmp/narratio-effective.yml" /usr/bin/sudo -u narratio /usr/local/bin/narratio config show \ --config /usr/local/etc/narratio/pipeline.yml --campaign centaurs \ > "$narratio_effective_config" +narratio_testing_effective_config="$test_tmp/narratio-testing-effective.yml" +/usr/bin/sudo -u narratio /usr/local/bin/narratio config show \ + --config /usr/local/etc/narratio/pipeline.yml --campaign centaurs --profile testing \ + > "$narratio_testing_effective_config" grep -Fq 'enabled: true' "$narratio_effective_config" grep -Fq 'llm_api_key: AUDITA_LLM_API_KEY' "$narratio_effective_config" grep -Fq 'pipeline_id: dnd-session' "$narratio_effective_config" @@ -116,13 +127,16 @@ grep -Fq 'resume: true' "$narratio_effective_config" ! grep -Fq 'chunk_cache:' "$narratio_effective_config" grep -Fq 'party: narratio.input.party' "$narratio_effective_config" grep -Fq 'lane_id: item-registry' "$narratio_effective_config" -"$workspace_venv/bin/python" - "$narratio_effective_config" <<'PY' +"$workspace_venv/bin/python" - \ + "$narratio_effective_config" "$narratio_testing_effective_config" <<'PY' import sys import yaml with open(sys.argv[1], encoding="utf-8") as config_file: config = yaml.safe_load(config_file) +with open(sys.argv[2], encoding="utf-8") as config_file: + testing_config = yaml.safe_load(config_file) assert config["workspace"]["cleanup_after_publish"] is False assert config["storage"] == { @@ -154,11 +168,21 @@ assert config["publish"] == { }, ], } +assert "config_profile" not in config["notarius"] +assert testing_config["notarius"]["config_profile"] == "testing" +assert all( + artifact["profile_id"] == "endpoint-profile" + for artifact in testing_config["scriptorium"]["artifacts"].values() +) +assert any( + artifact_id.startswith("character_meta_analysis_") + for artifact_id in testing_config["scriptorium"]["artifacts"] +) PY -! grep -Fq "$synthetic_audita_api_key" "$baseline_log" "$narratio_effective_config" "$notarius_effective_config" -! grep -Fq "$synthetic_notarius_api_key" "$baseline_log" "$narratio_effective_config" "$notarius_effective_config" -! grep -Fq "$synthetic_object_storage_key_id" "$baseline_log" "$narratio_effective_config" "$notarius_effective_config" -! grep -Fq "$synthetic_object_storage_key" "$baseline_log" "$narratio_effective_config" "$notarius_effective_config" +! grep -Fq "$synthetic_audita_api_key" "$baseline_log" "$narratio_effective_config" "$narratio_testing_effective_config" "$notarius_effective_config" "$notarius_testing_effective_config" +! grep -Fq "$synthetic_notarius_api_key" "$baseline_log" "$narratio_effective_config" "$narratio_testing_effective_config" "$notarius_effective_config" "$notarius_testing_effective_config" +! grep -Fq "$synthetic_object_storage_key_id" "$baseline_log" "$narratio_effective_config" "$narratio_testing_effective_config" "$notarius_effective_config" "$notarius_testing_effective_config" +! grep -Fq "$synthetic_object_storage_key" "$baseline_log" "$narratio_effective_config" "$narratio_testing_effective_config" "$notarius_effective_config" "$notarius_testing_effective_config" /usr/bin/sudo -u narratio /usr/local/bin/narratio config artifacts \ --config /usr/local/etc/narratio/pipeline.yml --campaign centaurs | \ awk -F '\t' '$1 == "session_recap" && $4 == "true" { found = 1 } END { exit !found }' @@ -168,9 +192,6 @@ PY /usr/bin/sudo -u narratio /usr/local/bin/narratio config artifacts \ --config /usr/local/etc/narratio/pipeline.yml --campaign centaurs | \ awk -F '\t' 'index($1, "character_meta_analysis_") == 1 && $4 == "false" && $7 == "character_meta_analysis" { found = 1 } END { exit !found }' -/usr/bin/sudo -u narratio /usr/local/bin/narratio config diff default testing \ - --config /usr/local/etc/narratio/pipeline.yml --campaign centaurs | \ - awk -F '\t' '$1 != "changed" || $2 !~ /\.profile_id$/ { invalid = 1 } END { exit invalid || NR == 0 }' /usr/bin/sudo -u "$operator_user" /usr/local/bin/narratio config validate \ --config /usr/local/etc/narratio/pipeline.yml --campaign centaurs /usr/bin/sudo -u "$operator_user" /usr/local/bin/notarius config validate \ @@ -218,21 +239,6 @@ if run_converge "$invalid_key_log" "${baseline_args[@]}" \ exit 1 fi -testing_profile_log="$test_tmp/testing-profile.log" -run_converge "$testing_profile_log" "${baseline_args[@]}" \ - -e 'dnd_pipeline_narratio_testing_scriptorium_profile=endpoint-profile' -/usr/bin/sudo -u narratio /usr/local/bin/narratio config validate \ - --config /usr/local/etc/narratio/pipeline.yml --campaign centaurs --profile testing -/usr/bin/sudo -u narratio /usr/local/bin/narratio config diff default testing \ - --config /usr/local/etc/narratio/pipeline.yml --campaign centaurs | \ - grep -F '"endpoint-profile"' -/usr/local/bin/scriptorium inspect profile --config /usr/local/etc/scriptorium/config.yml \ - --profile-dir "$profile_dir" --profile endpoint-profile -testing_profile_repeat_log="$test_tmp/testing-profile-repeat.log" -run_converge "$testing_profile_repeat_log" "${baseline_args[@]}" \ - -e 'dnd_pipeline_narratio_testing_scriptorium_profile=endpoint-profile' -require_clean_recap "$testing_profile_repeat_log" - selected_version_log="$test_tmp/selected-version.log" run_converge "$selected_version_log" "${baseline_args[@]}" \ -e 'dnd_pipeline_seriatim_version=v1.4.0' @@ -242,24 +248,55 @@ run_converge "$selected_version_repeat_log" "${baseline_args[@]}" \ -e 'dnd_pipeline_seriatim_version=v1.4.0' require_clean_recap "$selected_version_repeat_log" -pipeline_group_one='{"archive-session":{"llm_profile":"dnd-extraction","input":{"module":"seriatim"},"artifacts":{"spells":{"extract":{"module":"dnd/spells"},"normalize":{"module":"dnd/spells"}}}}}' -pipeline_group_two='{"research-session":{"llm_profile":"dnd-extraction","input":{"module":"seriatim"},"artifacts":{"spells":{"extract":{"module":"dnd/spells"},"normalize":{"module":"dnd/spells"}}}}}' -alternate_config="{\"notarius_pipeline_group_one\":$pipeline_group_one,\"notarius_pipeline_group_two\":$pipeline_group_two,\"dnd_pipeline_notarius_pipelines\":\"{{ notarius_pipeline_group_one | combine(notarius_pipeline_group_two) }}\",\"dnd_pipeline_narratio_notarius_pipeline_id\":\"archive-session\",\"dnd_pipeline_notarius_cache_chunk_plans_mode\":\"bypass\"}" +pipeline_group_one='{"archive-session":{"llm_profile":"gpt-luna-latest","input":{"module":"seriatim"},"artifacts":{"spells":{"extract":{"module":"dnd/spells"},"normalize":{"module":"dnd/spells"}}}}}' +pipeline_group_two='{"research-session":{"llm_profile":"gpt-luna-latest","input":{"module":"seriatim"},"artifacts":{"spells":{"extract":{"module":"dnd/spells"},"normalize":{"module":"dnd/spells"}}}}}' +alternate_config="{\"notarius_pipeline_group_one\":$pipeline_group_one,\"notarius_pipeline_group_two\":$pipeline_group_two,\"dnd_pipeline_notarius_pipelines\":\"{{ notarius_pipeline_group_one | combine(notarius_pipeline_group_two) }}\",\"dnd_pipeline_narratio_notarius_pipeline_id\":\"archive-session\",\"dnd_pipeline_notarius_testing_llm_profile\":\"gpt-terra-latest\",\"dnd_pipeline_notarius_cache_chunk_plans_mode\":\"bypass\"}" alternate_log="$test_tmp/alternate.log" run_converge "$alternate_log" "${baseline_args[@]}" -e "$alternate_config" /usr/local/bin/notarius config validate --config /usr/local/etc/notarius/config.yml \ --pipeline archive-session +/usr/local/bin/notarius config validate --config /usr/local/etc/notarius/config.yml \ + --config-profile testing --pipeline archive-session alternate_narratio_config="$test_tmp/alternate-narratio.yml" /usr/bin/sudo -u narratio /usr/local/bin/narratio config show \ --config /usr/local/etc/narratio/pipeline.yml --campaign centaurs \ > "$alternate_narratio_config" grep -Fq 'pipeline_id: archive-session' "$alternate_narratio_config" +alternate_testing_narratio_config="$test_tmp/alternate-testing-narratio.yml" +/usr/bin/sudo -u narratio /usr/local/bin/narratio config show \ + --config /usr/local/etc/narratio/pipeline.yml --campaign centaurs --profile testing \ + > "$alternate_testing_narratio_config" alternate_effective_config="$test_tmp/alternate-effective.yml" /usr/local/bin/notarius config effective --config /usr/local/etc/notarius/config.yml \ > "$alternate_effective_config" -grep -Fq 'archive-session:' "$alternate_effective_config" -grep -Fq 'research-session:' "$alternate_effective_config" -grep -Fq 'mode: bypass' "$alternate_effective_config" +alternate_testing_effective_config="$test_tmp/alternate-testing-effective.yml" +/usr/local/bin/notarius config effective --config /usr/local/etc/notarius/config.yml \ + --config-profile testing > "$alternate_testing_effective_config" +"$workspace_venv/bin/python" - \ + "$alternate_narratio_config" "$alternate_testing_narratio_config" \ + "$alternate_effective_config" "$alternate_testing_effective_config" <<'PY' +import sys + +import yaml + +with open(sys.argv[1], encoding="utf-8") as config_file: + narratio_base = yaml.safe_load(config_file) +with open(sys.argv[2], encoding="utf-8") as config_file: + narratio_testing = yaml.safe_load(config_file) +with open(sys.argv[3], encoding="utf-8") as config_file: + notarius_base = yaml.safe_load(config_file) +with open(sys.argv[4], encoding="utf-8") as config_file: + notarius_testing = yaml.safe_load(config_file) + +assert narratio_base["notarius"]["pipeline_id"] == "archive-session" +assert "config_profile" not in narratio_base["notarius"] +assert narratio_testing["notarius"]["pipeline_id"] == "archive-session" +assert narratio_testing["notarius"]["config_profile"] == "testing" +assert notarius_base["pipelines"]["archive-session"]["llm_profile"] == "gpt-luna-latest" +assert notarius_testing["pipelines"]["archive-session"]["llm_profile"] == "gpt-terra-latest" +assert notarius_testing["pipelines"]["research-session"]["llm_profile"] == "gpt-luna-latest" +assert notarius_base["cache"]["chunk_plans"]["mode"] == "bypass" +PY alternate_repeat_log="$test_tmp/alternate-repeat.log" run_converge "$alternate_repeat_log" "${baseline_args[@]}" -e "$alternate_config" require_clean_recap "$alternate_repeat_log"