115 lines
3.8 KiB
Markdown
115 lines
3.8 KiB
Markdown
# Checkpoint 4: Portable Audita Infrastructure
|
|
|
|
## Status
|
|
|
|
This document describes planned work, not implemented behavior.
|
|
|
|
## Goal
|
|
|
|
Port or adapt reusable Audita infrastructure that directly supports Notarius
|
|
contracts while avoiding Audita's transcript-correction model.
|
|
|
|
This checkpoint should add reusable runtime plumbing, not real extraction
|
|
behavior.
|
|
|
|
## Scope
|
|
|
|
In scope:
|
|
|
|
- structured LLM client interface implementation;
|
|
- LLM scheduler;
|
|
- prompt registry pattern;
|
|
- response-schema registry pattern;
|
|
- diagnostics run directory pattern;
|
|
- config loading and validation for named pipeline profiles and implemented
|
|
runtime pieces.
|
|
|
|
Out of scope:
|
|
|
|
- correction proposals;
|
|
- replacement policies;
|
|
- transcript mutation;
|
|
- correction ledger terminology;
|
|
- Audita module or validator behavior;
|
|
- real D&D prompts or schemas;
|
|
- domain-specific prompt or schema assets;
|
|
- embedded built-in pipeline profiles.
|
|
|
|
## Target End State
|
|
|
|
The repository should contain reusable runtime infrastructure adapted from
|
|
Audita where it directly supports Notarius contracts:
|
|
|
|
- an OpenAI-compatible structured-output LLM client behind the existing
|
|
`StructuredLLMClient` interface;
|
|
- an LLM scheduler for bounded concurrency;
|
|
- an embedded response-schema registry pattern;
|
|
- an embedded prompt registry pattern;
|
|
- a diagnostics run directory pattern using extraction-oriented artifact names;
|
|
- config structs, loading, defaults, redaction, and validation for named
|
|
pipeline profiles.
|
|
|
|
Framework code should remain source-agnostic and domain-agnostic. Provider HTTP
|
|
details should stay inside the LLM runtime package. Prompt and schema registries
|
|
should use placeholder/test assets until real extractor prompts and schemas are
|
|
introduced by later checkpoints.
|
|
|
|
Config should support:
|
|
|
|
- `llm_profiles`;
|
|
- `pipelines.<pipeline_id>.input`;
|
|
- `pipelines.<pipeline_id>.chunk`;
|
|
- `pipelines.<pipeline_id>.artifacts.<lane>.extract`;
|
|
- lane `merge`, `normalize`, and validator settings;
|
|
- output module selection;
|
|
- inline module-binding object form and string shorthand;
|
|
- default `chunk`, `merge`, `normalize`, `output`, and `llm_profile`;
|
|
- selected pipeline ID and lane filtering for runtime use;
|
|
- concurrency;
|
|
- work directory;
|
|
- diagnostics retention.
|
|
|
|
Config loading should support the standard precedence model:
|
|
|
|
1. built-in defaults
|
|
2. configuration file
|
|
3. environment variables
|
|
4. CLI flags
|
|
|
|
Structural module selection should come from pipeline config. CLI flags may
|
|
override operational settings and artifact lane selection, but should not offer
|
|
ad hoc `--extractor` or `--chunker` wiring.
|
|
|
|
Config validation should fail fast for unknown pipeline IDs, unknown module
|
|
keys, missing required slots, missing capabilities, unknown LLM profiles, empty
|
|
artifact-lane sets, and invalid lane selections.
|
|
|
|
If the CLI shell is ready, the checkpoint should expose discovery/validation
|
|
commands for config and pipeline profiles:
|
|
|
|
```sh
|
|
notarius config validate
|
|
notarius pipelines list
|
|
```
|
|
|
|
Implementation staging belongs in [`implementation.md`](implementation.md).
|
|
|
|
## Done Criteria
|
|
|
|
- `go test ./...` passes.
|
|
- Audita runtime pieces are adapted to Notarius package names and contracts.
|
|
- No correction proposal, replacement policy, transcript mutation, or correction
|
|
ledger code has been copied.
|
|
- Runtime tests cover secret redaction, schema registry lookup, prompt metadata,
|
|
and scheduler behavior where applicable.
|
|
- Config tests cover named pipeline profiles, defaults, lane selection,
|
|
capability validation, and resolved pipeline digesting.
|
|
|
|
## Review Questions
|
|
|
|
- Did we copy only reusable infrastructure?
|
|
- Do provider-specific types stay behind adapter/runtime boundaries?
|
|
- Are diagnostics names and report concepts extraction-oriented?
|
|
- Is config limited to named pipeline profiles and implemented behavior?
|
|
- Are structural pipeline changes kept out of ad hoc CLI flags?
|