notarius/docs/roadmap/4-portable-audita-infrastructure.md

Checkpoint 4: Portable Audita Infrastructure

Status

This document describes planned work, not implemented behavior.

Goal

Port or adapt reusable Audita infrastructure that directly supports Notarius contracts while avoiding Audita's transcript-correction model.

This checkpoint should add reusable runtime plumbing, not real extraction behavior.

Scope

In scope:

• structured LLM client interface implementation;
• LLM scheduler;
• prompt registry pattern;
• response-schema registry pattern;
• diagnostics run directory pattern;
• config loading and validation for named pipeline profiles and implemented runtime pieces.

Out of scope:

• correction proposals;
• replacement policies;
• transcript mutation;
• correction ledger terminology;
• Audita module or validator behavior;
• real D&D prompts or schemas unless needed as inert registry tests.
• embedded built-in pipeline profiles.

Proposed Stages

Stage 1: LLM Runtime

Port or adapt the OpenAI-compatible structured-output client and scheduler.

Keep the contract transport-neutral:

• framework code should depend on a StructuredLLMClient interface;
• provider-specific HTTP details should remain in the LLM runtime package;
• errors must redact configured secrets.

Stage 2: Response Schema Registry

Port or adapt the embedded JSON response-schema registry pattern.

The registry should track:

• schema key;
• schema ID;
• schema version;
• schema name;
• JSON schema content;
• schema hash.

Use placeholder or test schemas if real extractor schemas are not ready.

Stage 3: Prompt Registry

Port or adapt the embedded prompt registry pattern.

The registry should track:

• prompt ID;
• prompt version;
• prompt source;
• embedded path;
• prompt hash.

Do not add D&D prompt assets here unless the implementation naturally overlaps with checkpoint 6. Test prompts are acceptable for registry tests.

Stage 4: Diagnostics Run Directory

Port or adapt the diagnostics run directory pattern.

Initial diagnostics should cover:

• invocation metadata;
• redacted effective config;
• selected pipeline ID;
• resolved pipeline definition and digest;
• source document artifact;
• run report placeholder;
• error log on failure.

Avoid Audita-specific artifact names such as correction ledger.

Stage 5: Pipeline Profile Config

Add config structs, loading, defaults, and validation for named pipeline profiles.

Initial config areas:

• llm_profiles;
• pipelines.<pipeline_id>.input;
• pipelines.<pipeline_id>.chunk;
• pipelines.<pipeline_id>.artifacts.<lane>.extract;
• lane merge, normalize, and validator settings;
• output module selection;
• inline module-binding object form and string shorthand;
• default chunk, merge, normalize, output, and llm_profile;
• selected pipeline ID and lane filtering for runtime use;
• concurrency;
• work directory;
• diagnostics retention.

Config loading should support the standard precedence model:

1. built-in defaults
2. configuration file
3. environment variables
4. CLI flags

Structural module selection should come from pipeline config. CLI flags may override operational settings and artifact lane selection, but should not offer ad hoc --extractor or --chunker wiring.

Add validation for unknown pipeline IDs, unknown module keys, missing required slots, missing capabilities, unknown LLM profiles, empty artifact-lane sets, and invalid lane selections.

If the CLI shell is ready, add:

notarius config validate
notarius pipelines list

Done Criteria

• go test ./... passes.
• Audita runtime pieces are adapted to Notarius package names and contracts.
• No correction proposal, replacement policy, transcript mutation, or correction ledger code has been copied.
• Runtime tests cover secret redaction, schema registry lookup, prompt metadata, and scheduler behavior where applicable.
• Config tests cover named pipeline profiles, defaults, lane selection, capability validation, and resolved pipeline digesting.

Review Questions

• Did we copy only reusable infrastructure?
• Do provider-specific types stay behind adapter/runtime boundaries?
• Are diagnostics names and report concepts extraction-oriented?
• Is config limited to named pipeline profiles and implemented behavior?
• Are structural pipeline changes kept out of ad hoc CLI flags?