ansible-workspace-dnd/docs/roadmap/features/0008-opinionated-dnd-processing-configuration.md

# Feature 0008: Opinionated D&D Processing Configuration

Status: done

## Summary

Turn the installed application suite and deployed D&D assets into a coherent
processing configuration. The pipeline role will supply deliberately
opinionated Audita and Narratio policy, split Narratio's suite-owned processing
and artifact definitions into explicit ConfigKit fragments, and provide a
named `testing` profile for inexpensive prompt and model iteration without
changing production policy.

## Problem or Opportunity

Features 0006 and 0007 install the five applications, connect their standard
paths, and deploy valid campaigns, prompts, and schemas. The assembled role
still leaves most optional application policy unset. Narratio can validate its
minimal root document and discover content, but it does not yet describe the
D&D trimming, rendering, recap, rumor, scene-map, or per-character analysis
workflow.

A current test-machine configuration provides useful evidence of the intended
workflow, but it cannot be copied. It includes pre-Narratio-2.0 syntax,
installation-specific storage and inference settings, obsolete paths,
campaign-specific character entries, and redundant artifact dependencies.
The current application repositories remain authoritative for supported
configuration.

Narratio 2.0 also provides two useful composition mechanisms that the suite
has not yet adopted: additive root imports for independently owned base
fragments and a fixed `profiles/testing.yml` overlay. Using both will make the
production policy reviewable and make inexpensive development runs easy to
select without editing the production configuration.

## Governing Policies

- [The Ansible house style](../../ansible.md) governs the suite-owned public
  interface, templates, direct desired-state tasks, and omission of
  unselected application defaults.
- [The testing house style](../../testing.md) governs native behavioral
  validation, explicit testing inputs, and clean repeated convergence.
- [The MVP roadmap](../mvp.md) assigns opinionated cross-application and D&D
  policy to the pipeline role while keeping standalone application roles
  generic.
- [Decision 0001](../../decisions/0001-minimal-module-driven-ansible.md)
  requires direct module-owned convergence without supplemental state or
  redundant verification.

## Scope

- Supply the complete reviewed Audita configuration as deliberate D&D suite
  policy, including its context, schema, module sequence, endpoint, models,
  credential environment-variable names, timeouts, retries, concurrency,
  chunking, normalization, thresholds, and diagnostics settings.
- Configure Narratio's normalization, trimming, and Markdown rendering stages
  through its current public contract, explicitly pinning only the schemas,
  identifiers, enablement choices, and paths that form the D&D suite's
  cross-stage contract.
- Use the deployed `dnd.session_bounds` prompt for transcript-bound detection.
- Define each ordinary Narratio Scriptorium artifact in its own managed YAML
  fragment:
  - session recap;
  - hostile rumors;
  - deranged rumors; and
  - scene map.
- Define per-character gameplay analysis as one Narratio artifact-family
  fragment over `party.characters`, not as campaign-specific artifact entries.
- Supply only the inputs declared by each current prompt definition.
- Let each production prompt select its own declared Promptkit default profile
  unless the roadmap records a concrete reason for a suite override.
- Manage a valid `profiles/testing.yml` Narratio overlay that selects an
  inexpensive Scriptorium profile while leaving the base production policy
  and inherited Audita configuration unchanged.
- Expose the suite-owned testing-profile choice through normal Ansible
  inventory precedence.
- Compose the base fragments through an explicit ordered
  `narratio_configkit_imports` list in the standalone Narratio role's root
  configuration.
- Create and manage the required `conf.d`, `artifacts`,
  `artifact-families`, and `profiles` directories and their suite-owned files
  under `/usr/local/etc/narratio`.
- Document production and testing invocations, the imported layout, public
  variables, and the boundary with later pipeline features.
- Extend focused integration coverage for both base and `testing` resolution,
  representative artifact relationships, party-driven family expansion, and
  immediate clean repeated convergence.

## Non-goals

- Enabling Narratio's Notarius stage or installing the uploaded Notarius
  pipeline. Its multi-stage D&D extraction, campaign reference bindings,
  output descriptors, and runtime directories remain a separate feature.
- Configuring S3 or other remote storage, publishing outputs, upload behavior,
  or cleanup-after-publish.
- Selecting a default campaign. Campaign choice remains an inventory or
  invocation decision.
- Delivering API keys, object-storage credentials, environment files, or a
  general secret-management mechanism.
- Selecting human operators or managing their Narratio group membership.
- Adding a scheduler, service, or automated invocation.
- Running WhisperX, Audita model calls, Scriptorium model calls, Notarius, or
  a complete transcript during integration tests.
- Copying the test-machine configuration verbatim or preserving its obsolete
  Narratio, campaign, Scriptorium, or credential paths.
- Duplicating prompt content, prompt schemas, built-in Promptkit profiles, or
  application validation rules in the pipeline role.
- Modifying the standalone application roles or upstream application source.
- Collection assembly, role releases, asset release policy, upgrade/removal
  behavior, or end-to-end production validation.

## Target End State

The standalone Narratio role continues to own
`/usr/local/etc/narratio/pipeline.yml`. That root document contains the
installation paths already supplied by the pipeline role and an explicit list
of suite-owned relative imports. The pipeline role owns the imported files:

```text
/usr/local/etc/narratio/
├── pipeline.yml
├── conf.d/
│   └── processing.yml
├── artifacts/
│   ├── session-recap.yml
│   ├── hostile-rumors.yml
│   ├── deranged-rumors.yml
│   └── scene-map.yml
├── artifact-families/
│   └── character-meta-analysis.yml
└── profiles/
    └── testing.yml
```

Each base artifact or family key is declared by exactly one fragment. The
root uses explicit `path` imports rather than globs; Narratio therefore loads
only the files selected by current suite policy, even if an obsolete file is
left on a previously managed host. Imported maps remain additive and no two
base sources claim the same leaf.

The production configuration uses the deployed prompt definitions and their
frontier-oriented default profiles. Audita uses the selected
`https://inference.ai.rakestrawhome.com/v1` endpoint and
`google/gemma-4-31b-it` proposal and validation model. Narratio's canonical
processing paths and schemas are suite-owned contracts rather than incidental
copies of application defaults. Session recap consumes the final trimmed
transcript and may consume the previous session's recap. Hostile and deranged
rumors consume the current recap, allowing Narratio to infer their dependency
without a duplicate `depends_on` declaration. Scene map is configured as an
available, normally disabled artifact. Character analysis is an available
party-driven family whose generated members receive `character.name` and
`character.class_summary` through family member variables.

The installed `testing` profile is a valid overlay selected with
`--profile testing`. It selects `gemini-flash-latest` for each Scriptorium
artifact and artifact family while inheriting the base topology, paths,
prompts, inputs, outputs, and enabled state. Audita inherits the same endpoint
and model in both modes for now, so the overlay does not restate its base
configuration. The testing profile neither imports files nor contains
credentials. Operators can validate, inspect, or run the same session under
base or testing policy without modifying an Ansible-managed file.

## Repository Ownership

- `ansible-role-dnd-pipeline` owns the suite variables, Narratio fragment and
  testing-profile templates, direct directory/file convergence, dependency
  variable wiring, consumer documentation, and integration coverage. It is
  the only implementation repository expected to change.
- `ansible-role-narratio` remains unchanged. It owns the root pipeline
  template and the public `narratio_configkit_imports` and stage variables
  through which the suite composes it.
- `ansible-role-audita` remains unchanged. It owns Audita's field-specific
  configuration interface; the pipeline role selects only intentional D&D
  policy through that interface.
- `ansible-role-scriptorium`, `assets/narratio-prompts`, and `../promptkit`
  remain unchanged authorities for source-directory configuration, prompt
  contracts, and built-in production/testing profile choices.
- `../audita` and `../narratio` remain unchanged authorities for current
  application configuration and native inspection behavior.
- `ansible-role-notarius` and `../notarius` are read-only inputs for the later
  D&D extraction feature.
- `rakestraw-ansible` and the uploaded test-machine configurations are
  historical and operational references, not application contracts.
- `ansible-workspace-dnd` owns this roadmap, the later implementation plan,
  settled MVP decisions, and aggregate status.

## Constraints and Settled Decisions

- Standalone application roles remain generic. Every concrete value added by
  this feature must represent policy deliberately owned by the D&D suite.
- Application defaults observed in documentation or the uploaded files are
  not copied merely to restate them. An explicit suite value must preserve a
  chosen outcome across upstream default changes.
- The pipeline explicitly owns a value when another stage, artifact, prompt,
  or future publishing rule relies on its schema, identifier, enabled state,
  or path. Behavior confined to Narratio inherits Narratio's application
  default unless the suite has independently selected that behavior as policy.
- The pipeline role does not duplicate Narratio's root template. It acts as a
  caller: the standalone role renders the root and the pipeline role deploys
  the suite-owned fragments named by that root.
- Every ordinary artifact and every artifact family has its own source and
  installed file. General stage policy may share `conf.d/processing.yml`.
- Imports are explicit relative `path` entries. Globs are not used because
  the role does not need directory discovery and must not activate stale
  files accidentally.
- The imported files cannot contain `configkit`, nested imports, or profile
  directives. The `testing` overlay cannot import or stack other profiles.
- Suite-managed configuration directories are `root:narratio` mode `0750`;
  files are `root:narratio` mode `0640`, matching the established Narratio
  configuration boundary.
- The role declares directory and template state directly. It does not
  inspect files first, validate production configuration during convergence,
  keep manifests, or track fragment digests.
- The base artifact definitions omit `profile_id` when the deployed prompt's
  declared default is the selected production policy.
- Artifact inputs must match the current prompt contracts:
  - session recap: required `transcript`, optional `recap`;
  - hostile and deranged rumors: required `recap`;
  - scene map: required `transcript`; and
  - character analysis family: required `transcript`, optional `recap`, plus
    member variables for `character_name` and `character_class`.
- A `narratio.artifact.session_recap` input already creates the rumor
  dependency. The base fragments do not repeat that edge with `depends_on`.
- Scene map and character analysis are configured but disabled by default;
  Narratio's explicit artifact selection may still run their concrete entries.
- The character family uses `for_each: party.characters`, an output path with
  exactly one `{character_id}` token, and member selectors
  `character.name` and `character.class_summary`. It contains no hard-coded
  campaign character names.
- Production and testing policy never embeds secret values. Credential
  delivery remains a later feature.
- The testing profile is a single valid overlay at the native fixed location.
  It overrides each Scriptorium artifact and family `profile_id` with the
  suite's `gemini-flash-latest` testing-profile variable rather than
  duplicating complete definitions. Audita deliberately uses the same base
  endpoint and models under both configurations.
- Each fixed artifact or family fragment has one complete native-mapping
  variable: `dnd_pipeline_narratio_session_recap_artifact`,
  `dnd_pipeline_narratio_hostile_rumors_artifact`,
  `dnd_pipeline_narratio_deranged_rumors_artifact`,
  `dnd_pipeline_narratio_scene_map_artifact`, and
  `dnd_pipeline_narratio_character_analysis_family`. The file names, imports,
  and top-level source keys remain fixed suite topology; callers customize the
  mapping and disable an available source with `enabled: false` rather than
  removing its managed fragment or import.

## Selected Narratio Coordination Policy

The pipeline role pins the following effective Narratio fields because they
connect processing stages, prompt contracts, or stable suite outputs. A value
that currently equals an application default is still intentional suite policy
when an upstream change would otherwise break that connection.

| Native Narratio field | D&D suite default | Coordination purpose |
| --- | --- | --- |
| `pipeline.seriatim.output_schema` | `seriatim-intermediate` | Establishes the merge-stage transcript contract. |
| `pipeline.audita.output_schema` | `audita-v1` | Matches the Audita configuration and adapter output contract. |
| `pipeline.normalize.output_path` | `transcripts/final.json` | Provides the canonical normalized transcript source. |
| `pipeline.normalize.output_schema` | `seriatim-intermediate` | Provides the canonical schema consumed by later stages. |
| `pipeline.trim.enabled` | `true` | Makes the trimmed transcript part of the selected D&D workflow. |
| `pipeline.trim.output_path` | `transcripts/final.trimmed.json` | Provides the canonical transcript consumed by D&D artifacts. |
| `pipeline.trim.bounds.prompt_id` | `dnd.session_bounds` | Selects the deployed D&D bounds prompt. |
| `pipeline.trim.bounds.transcript_input_name` | `transcript` | Matches that prompt's declared input contract. |
| `pipeline.trim.bounds.output_path` | `artifacts/session_bounds.json` | Provides a stable bounds artifact location. |
| `pipeline.render.enabled` | `true` | Selects rendered transcripts as a suite outcome. |
| `pipeline.render.format` | `markdown` | Selects the suite's rendered transcript format. |

The fixed artifact mappings own these output paths:

| Source | D&D suite output |
| --- | --- |
| `session_recap` | `artifacts/session_recap.md` |
| `hostile_rumors` | `artifacts/hostile_rumors.md` |
| `deranged_rumors` | `artifacts/deranged_rumors.md` |
| `scene_map` | `artifacts/scene_map.json` |
| `character_meta_analysis` family | `artifacts/character_meta_analysis/{character_id}.md` |

The processing fragment omits Narratio-local choices such as reports,
timeouts, debug rendering, work-directory retention, render title, timestamp,
segment-ID, and metadata presentation, and the bounds prompt profile. Narratio
continues to supply those defaults. Likewise, the Narratio Audita adapter does
not repeat Audita's endpoint, models, module sequence, tuning, or diagnostic
settings: it uses the pipeline-managed Audita configuration file. The Audita
output schema is the deliberate exception because it spans that application
boundary, and one suite value must feed both configurations.

Audita does not choose a fixed final transcript path in this integration.
Narratio supplies the run-specific `--output` path when invoking Audita and
then normalizes the result to the canonical path above. The role therefore
does not invent a separate Audita output-path setting.

## Selected Audita and Testing Policy

The pipeline role deliberately owns all of the following Audita values. They
are suite defaults, not assertions about Audita's upstream defaults, and each
remains replaceable through the corresponding suite-prefixed role variable.

| Native Audita field | D&D suite default |
| --- | --- |
| `pipeline.modules` | `[glossary, homophones, glossary, spoken_word, grammar]` |
| `output.schema` | `audita-v1` |
| `llm.proposal.base_url` | `https://inference.ai.rakestrawhome.com/v1` |
| `llm.proposal.model` | `google/gemma-4-31b-it` |
| `llm.proposal.api_key_env` | `AUDITA_LLM_API_KEY` |
| `llm.proposal.timeout` | `900s` |
| `llm.proposal.max_retries` | `3` |
| `llm.validation.base_url` | `https://inference.ai.rakestrawhome.com/v1` |
| `llm.validation.model` | `google/gemma-4-31b-it` |
| `llm.validation.api_key_env` | `AUDITA_VALIDATION_LLM_API_KEY` |
| `llm.validation.timeout` | `900s` |
| `llm.validation.max_retries` | `3` |
| `concurrency.total_llm` | `4` |
| `concurrency.proposal_llm` | `4` |
| `concurrency.validation_llm` | `4` |
| `chunking.target_sections` | `8` |
| `chunking.max_section_tokens` | `6144` |
| `chunking.min_section_tokens` | `2048` |
| `normalization.max_segment_gap` | `3s` |
| `normalization.ellipsis_gap` | `3s` |
| `normalization.max_segment_duration` | `240s` |
| `normalization.max_segment_tokens` | `2048` |
| `thresholds.glossary` | `0.8` |
| `thresholds.homophones` | `0.8` |
| `thresholds.spoken_word` | `0.8` |
| `thresholds.grammar` | `0.8` |
| `context.description` | `The following transcript was recorded during a Dungeons & Dragons game session. It may contain some unrelated pre-session or post-session banter, in addition to actual D&D gameplay.` |
| `diagnostics.work_dir` | `/tmp/audita` |
| `diagnostics.retention` | `auto` |

The repeated `glossary` entry is intentional and preserves the selected
multi-pass module sequence. Environment-variable names are configuration, not
secret values; the later secret-delivery feature will provide their contents.

The testing overlay uses one suite variable whose default is
`gemini-flash-latest` for every Scriptorium artifact and artifact family.
Callers may replace that profile normally. Audita uses the production endpoint
and models in testing until the suite deliberately selects alternatives; the
overlay omits identical Audita fields rather than writing them twice. These
model choices are expected to evolve rapidly and can be updated as ordinary
suite policy without changing the fragment architecture.

## Simplest Viable Implementation

The production change should remain a small configuration extension:

1. Add only the suite variables required by the selected Audita policy,
   selected Narratio coordination policy, fragment mappings, and
   testing-profile override; align the argument specification.
2. Pass the Audita choices and explicit Narratio import list through the
   existing dependency interfaces.
3. Create the four Narratio configuration subdirectories with one
   `ansible.builtin.file` loop.
4. Render the processing fragment, four artifact fragments, one family
   fragment, and testing overlay with one `ansible.builtin.template` loop.
5. Update the README and extend the existing integration driver with native
   base/testing validation and a clean repeat.

No preliminary `stat`, configuration assembly fact, handler, validation
command, checksum, manifest, or fragment cleanup task is required. Explicit
imports make an obsolete unreferenced file inert without destructive directory
reconciliation.

## Rationale and Alternatives

ConfigKit fragments preserve the application's native composition model and
give each artifact a focused review boundary. Rendering one monolithic
Narratio mapping through the standalone root variable would produce a valid
file but would discard that operational and editorial advantage. Reimplementing
the entire root template in the pipeline role would duplicate the standalone
role and create competing configuration ownership.

Explicit imports are preferred to globs because the suite already knows every
managed fragment. A glob would add discovery behavior and could keep a removed
artifact active through a stale file. Deleting and recreating the directory on
every run would harm idempotence and caller safety merely to compensate for
that avoidable ambiguity.

One testing overlay is preferable to a second complete configuration tree.
Profiles replace only selected leaves after base composition, so topology and
paths cannot drift between production and development. The selected
`gemini-flash-latest` default provides immediate value while remaining one
ordinary role-default change when model economics evolve.

The uploaded Notarius pipeline is concrete and valid, but adding it here would
combine a ten-lane extraction contract, campaign-reference mapping, output
descriptor mapping, runtime directories, and model policy with an already
coherent Narratio/Scriptorium feature. Keeping it separate makes both reviews
smaller and preserves a clear acceptance boundary.

## Acceptance Criteria

### Required Configuration Behavior

- The pipeline role produces the documented imported configuration layout
  with one file per artifact and artifact family.
- The root pipeline document contains the exact explicit relative imports and
  no glob imports.
- Base Narratio configuration validates natively for either deployed campaign.
- The effective base configuration uses the selected Seriatim, Audita,
  normalization, trimming, bounds, and rendering contracts and the canonical
  artifact output paths documented above.
- `narratio config artifacts` reports a valid recap-to-rumor graph and expands
  the character-analysis family for the selected campaign.
- The recap, rumor, scene-map, and character-family definitions use only inputs
  and variables supported by their deployed prompt definitions and Narratio's
  current source vocabulary.
- The two rumor dependencies are inferred from their recap input sources and
  are not duplicated as explicit ordering edges.
- Scene map and the character-analysis family are available but disabled by
  default; recap and both rumor artifacts are enabled.
- Production artifacts inherit their selected profiles from the prompt
  definitions unless an expressly agreed suite override is documented.
- Audita's managed configuration contains the selected D&D policy and remains
  valid through Audita's native validator.
- The same suite-owned `audita-v1` schema is supplied to Audita's managed
  configuration and Narratio's Audita adapter; Narratio supplies Audita's
  run-specific output path rather than relying on a second fixed path.
- `profiles/testing.yml` is a valid, independently selectable Narratio profile
  that selects the configured inexpensive Scriptorium profile for every
  artifact and family and inherits base workflow structure.
- The testing profile does not repeat Audita configuration while production
  and testing deliberately share the same Audita endpoint and models.
- Base and testing configuration can be compared with
  `narratio config diff default testing` without requiring a model call.
- All suite-managed configuration remains readable by the `narratio` group and
  is not made writable by runtime or human operator accounts.
- An immediate repeated role application with unchanged inputs reports no
  changes.
- The role contains no configuration probes, production self-validation,
  fragment manifests, duplicate root template, secrets, Notarius integration,
  storage, or publishing behavior.

### Required Validation

- YAML syntax checking and production-profile ansible-lint pass for the role
  and test playbooks.
- The complete existing pipeline integration entry point continues to install
  the pinned role combination and validate asset discovery.
- Audita natively validates the role-managed D&D configuration.
- Narratio natively validates both base and `testing` configuration with an
  explicitly selected deployed campaign.
- Narratio's native artifact inspection demonstrates one enabled ordinary
  artifact, one inferred dependency, and representative family expansion.
- A test explicitly supplies a representative testing Scriptorium profile and
  observes its effective selection; it does not assert that the same value
  remains the role's current default.
- Scriptorium can resolve the representative production and testing profiles
  through its native inspection interface without making a provider request.
- The immediate repeat reports `changed=0` and `failed=0`.
- Tests do not snapshot full rendered files, enumerate every character or
  artifact, inspect template/file task results, test filesystem metadata, or
  reproduce ConfigKit, Narratio, Audita, Scriptorium, or Promptkit validation.

## Dependencies and Follow-up Work

Features 0001 through 0007 and the standalone-role test-suite simplification
are complete. Feature 0008 depends on the current Narratio 2.0 ConfigKit
contract, the canonical party files deployed by Feature 0007, and prompt
definitions whose production default profiles are available from Promptkit.

Feature 0010 should adapt the validated D&D Notarius pipeline, remove its
hard-coded campaign references in favor of Narratio invocation bindings,
declare its runtime-directory policy, map its accepted outputs into Narratio,
and enable extraction after Feature 0009 establishes shared filesystem and
execution policy. Later features should address storage/publishing and secret
delivery together, optional operator selection and membership management,
collection assembly, and an end-to-end workflow that can use actual external
services.

## Potential Future Hardening

- Use ConfigKit source inspection in operational diagnostics to show which
  managed fragment supplied an effective value.
- Add a controlled cleanup policy for obsolete suite fragments if accumulated
  inert files become an operational problem.