Plan the Audita v2 migration

2026-09-25 14:50:25 +00:00
parent 70f1dbfb59
commit 55c102c9ed
2 changed files with 554 additions and 271 deletions
--- a/docs/roadmap/audita.md
+++ b/docs/roadmap/audita.md
@@ -0,0 +1,250 @@
+# Adopt Audita v2
+
+Status: agreed scope; implementation has not started.
+
+## Goal
+
+Migrate Narratio's `polish` integration from Audita's former command/configuration
+contract to Audita v2.0.0. Keep Narratio responsible for orchestration and Audita
+responsible for correction behavior and LLM execution configuration.
+
+Adopt a consistent provider-named credential list for both Audita and Notarius,
+so operators define each secret once and explicitly share it with the downstream
+applications that need it. This is a clean cutover without backward-compatibility
+shims.
+
+Production runs use Audita's default reporting configuration: Narratio passes
+neither `--debug` nor `--reporting=false`. Normal reports remain available without
+enabling debug diagnostics. Adding debug reporting through Narratio's testing
+profile is deferred.
+
+This document defines the intended end state, migration scope, and policy.
+The [implementation plan](implementation.md) owns implementation sequencing and
+verification steps.
+
+## Baseline and upstream contract
+
+The baseline is Narratio v2.2.1 and the Audita checkout tagged v2.0.0. Upstream
+references are Audita's `docs/migrations/v2.md`, `docs/cli.md`,
+`docs/integrations/subprocess.md`, and `docs/integrations/output-files.md`,
+available in the sibling `../audita/` checkout. Narratio's current contract is
+documented in [Integration: Audita](../integrations/audita.md).
+
+The current integration is not compatible with Audita v2:
+
+- It requests `--report-json` by default; that flag has been removed.
+- It can pass retired endpoint, model, concurrency, and work-directory-retention
+  flags.
+- It expects one JSON report instead of the normal v2 report bundle.
+- Its single forwarded `AUDITA_LLM_API_KEY` has no implicit meaning in v2;
+  an Audita backend must explicitly reference that environment variable.
+- Its semantic fingerprint sorts modules, although Audita treats module order
+  and repetitions as meaningful.
+
+Audita v2 requires a version-2 configuration file. An explicit `--config` selects
+one; otherwise Audita performs its own documented discovery. Backend definitions
+own endpoints, credentials, and capacity. PromptKit profiles own models and
+per-attempt controls. Audita owns retry/repair policy and operation deadlines.
+Narratio's generated invocation YAML is a diagnostic record, not a substitute
+for that Audita configuration file.
+
+## Target architecture
+
+Retain the existing subprocess boundary and `polish` stage. Do not introduce a
+second correction engine, an in-process Audita dependency, or a generic external
+tool configuration framework.
+
+Narratio owns:
+
+- Selection of the executable and the overall invocation timeout/cancellation.
+- Transcript and glossary inputs, run-local output placement, validation of the
+  corrected transcript, and promotion to the existing polished transcript
+  identity.
+- Explicit configuration/profile selections and supported domain overrides.
+- Deliberate credential forwarding, secret-safe invocation diagnostics, and
+  manifest references to results and reports.
+- Existing stage completion, resume, force, publish, restore, and cleanup
+  behavior.
+
+Audita owns its configuration schema and discovery, backend/model configuration,
+provider execution, correction sequence, recovery behavior, and report contents.
+Narratio does not translate old settings into a generated v2 configuration or
+duplicate Audita's complete validation logic.
+
+## Configuration and invocation migration
+
+Keep binary, timeout, explicit config-path selection, and surviving explicit
+domain overrides such as transcript description, output schema, and modules.
+Allow Audita's native configuration discovery when no config path is selected;
+do not impose an additional requirement to supply an explicit path.
+
+Provide optional Audita overlay, proposal-profile, and validation-profile
+selectors, passed through to their corresponding CLI flags. Omission leaves
+Audita's own defaults and validation-profile inheritance in control. These are
+Audita selectors, not automatic translations of Narratio profile names.
+
+Retire the Narratio settings that mirror removed Audita endpoint/model/capacity
+flags and work-directory retention. Move their operational meaning into the
+operator-maintained Audita configuration and profiles. Remove stale adapter
+fields, validation, invocation metadata, examples, and tests along with them.
+Remove the old report toggle and both adapters' singular `llm_api_key` fields
+from accepted Narratio configuration. Do not retain deprecated aliases,
+automatic conversion, ignored fields, or a v1 execution path. Existing strict
+configuration decoding rejects removed fields. Provide migration guidance and a
+release compatibility notice rather than runtime compatibility machinery.
+
+Preserve the distinction between omitted modules and an explicitly empty module
+list: omission requests Audita's configured/default sequence; an empty list
+requests no correction modules. Preserve order and repetitions rather than
+sorting or deduplicating. Do not broaden this into a change to unrelated
+adapters' module semantics.
+
+Normal invocations no longer pass `--report-json`, `--work-dir-retention`, or
+debug-work-directory flags as a way to obtain reports. They need no reporting
+override: Audita writes its ordinary report bundle by default. The old
+Narratio `report` toggle must not silently become `--reporting=false`.
+
+The production Audita configuration must also retain reporting enabled and debug
+disabled. Omitting CLI flags does not override an operator's explicit external
+configuration. Document this distinction rather than silently adding forced
+boolean overrides or rejecting otherwise valid Audita configuration.
+
+## Credentials
+
+Define secrets once under provider-oriented environment-variable names, such as
+`OPENROUTER_API_KEY` and `RAKESTRAWHOME_API_KEY`, through the process environment
+or Narratio's existing secrets-directory bootstrap. These names are examples,
+not a fixed provider registry; application-specific names are not required.
+
+Add `pipeline.audita.credentials` and `pipeline.notarius.credentials` as lists
+of Narratio typed secret references. Each entry names the environment variable
+whose captured value is forwarded under that exact name. Omission or an empty
+list forwards no credentials. The same reference may appear in both adapters'
+lists without duplicating its stored secret value. An illustrative configuration
+fragment is:
+
+```yaml
+pipeline:
+  audita:
+    credentials:
+      - OPENROUTER_API_KEY
+      - RAKESTRAWHOME_API_KEY
+  notarius:
+    credentials:
+      - OPENROUTER_API_KEY
+```
+
+Resolve references through the existing configuration/secret-loading mechanism
+before adapter execution, preserving its nonempty-value requirement. Forward
+the captured values rather than rereading the parent environment during a run.
+Retain the minimal child execution environment plus each adapter's explicit
+credential list; do not forward every secret found during bootstrap.
+
+Audita backends select these names through `credential`; the current Notarius
+PromptKit profiles select them through `api_key_env`. Narratio neither inspects
+downstream configuration to infer required credentials nor translates names,
+registers providers, or duplicates backend configuration. Audita resolves
+credentials for every surviving configured backend, even one not selected by
+a profile, so its list must cover all those references.
+
+Generalize Notarius's current single-name forwarding and replace Audita's fixed
+`AUDITA_LLM_API_KEY` destination with this shared convention. Use existing shared
+secret and subprocess facilities, extending narrow helpers where needed rather
+than introducing a new secret-management framework. Register every forwarded
+credential with captured-output redaction, including custom names and multiple
+keys. Values must not enter arguments, generated configuration, manifests,
+logs, or examples. As today, downstream tools remain responsible for files they
+write directly; this does not add arbitrary downstream-file scanning.
+
+## Ordinary reports and result handling
+
+Replace the old `audita.report.json` handling with Audita's normal `errors.yml`,
+`recoveries.yml`, and `report.md` bundle. With a file output, Audita writes these
+beside the corrected transcript; an arbitrary report destination is not supplied
+through the removed report flag.
+
+Discover the reports at their defined run-local locations and retain them as
+diagnostics using Narratio's existing manifest and diagnostic lifecycle. Keep
+them distinguishable from the canonical polished transcript. Adapt report
+references, publication/restore coverage, and operator inspection guidance so
+the bundle is not lost or confused with an older run's JSON report. Do not
+fabricate a legacy JSON report or delete historical diagnostics as part of the
+migration.
+
+Retain existing bounded, confined file handling. Replace JSON-specific report
+handling with only the validation needed for consuming the v2 bundle; do not
+introduce a second comprehensive report-schema validator or make incidental
+Markdown wording a contract. Report locations must be discoverable without
+parsing human-readable stderr. Detailed report-counter ingestion is not required
+for this migration.
+
+Audita's exit status remains authoritative. Exit zero with a valid corrected
+transcript may include recoverable LLM failures, including a normalized result
+with no model corrections. Preserve the reports so operators can assess that
+outcome; do not add an unrequested fail-on-recovery or minimum-correction policy.
+
+Conversely, a nonzero exit must fail polish even if a transcript file exists:
+Audita can fail while writing required reports after writing the transcript.
+Preserve available failure diagnostics without requiring early failures to have
+produced a complete report bundle. Failed output must not become the canonical
+successful result.
+
+## Resume and semantic evidence
+
+Update polish's semantic evidence for the new result-affecting selections and
+remove retired endpoint/model fields. Include explicit config/overlay/profile
+selection and preserve ordered module semantics, including omitted versus empty
+selection. Version the changed evidence so incompatible older evidence is not
+mistaken for a valid match when polish is considered for reuse.
+
+Keep the existing boundary for opaque downstream configuration: Narratio does
+not recursively hash Audita configuration, overlays, and profile contents.
+Operators force polish when changing contents behind unchanged selectors.
+Timeouts, credential values, reporting details, and other execution-only settings
+must not become blanket semantic invalidation keys. Do not rerun unrelated or
+unselected stages merely because the integration was upgraded.
+
+## Migration guidance and verification
+
+Update the canonical [configuration reference](../config.md),
+[Audita integration contract](../integrations/audita.md),
+[Notarius integration contract](../integrations/notarius.md),
+[polish internals](../internal/stage-polish.md), maintained examples, and relevant
+operations/troubleshooting guidance when implementation lands. Cover:
+
+- Preparing and validating an Audita v2 configuration and its profiles before
+  switching the executable; copying an old v1 configuration is insufficient.
+- Moving endpoint, model, capacity, and credential selections to their v2 owners.
+- Replacing both singular `llm_api_key` settings with explicit `credentials`
+  lists, storing each provider key once, and matching downstream credential
+  references to those names. State clearly that old fields are removed.
+- Using normal reports to inspect recoveries and failures without debug mode.
+- Force/resume expectations and the treatment of existing completed sessions.
+- The explicitly deferred nature of Narratio testing-profile debug support.
+
+Verification should exercise the real adapter boundary with offline subprocess
+fixtures and focused stage/configuration tests: accepted v2 arguments, omitted
+defaults, ordered/empty modules, secret-safe forwarding, successful report
+capture, recoverable-error success, nonzero exit with an existing transcript,
+failure diagnostics, and semantic reuse/invalidation. Update existing behavioral
+tests rather than retaining assertions for removed flags or exact stderr prose.
+Where practical, verify invocation against the tagged Audita CLI without live
+provider calls. Follow the repository's testing and documentation policies.
+
+Credential tests must cover both adapters, multiple custom names, reuse of one
+secret by both tools, no forwarding of unselected secrets, captured-value
+semantics, and redaction of every forwarded value. Verify removal of obsolete
+configuration fields without preserving their compatibility behavior.
+
+## Scope boundaries
+
+No changes to pipeline order, transcript identity, downstream artifact
+definitions, or provider algorithms are intended. No automatic v1 configuration
+converter, Audita-internal resume mechanism, runtime binary-version negotiation,
+or live-service release gate is required. Debug-profile integration and ingestion
+of Audita's debug-only execution artifacts are future work.
+
+The Notarius change is limited to credential configuration, forwarding, and
+associated diagnostics/documentation/tests; extraction and cache behavior are
+unchanged. This migration breaks existing pipeline configurations using removed
+fields; account for that explicitly in Narratio release versioning.