155 lines
4.3 KiB
Markdown
155 lines
4.3 KiB
Markdown
# Checkpoint 4: Portable Audita Infrastructure
|
|
|
|
## Status
|
|
|
|
This document describes planned work, not implemented behavior.
|
|
|
|
## Goal
|
|
|
|
Port or adapt reusable Audita infrastructure that directly supports Notarius
|
|
contracts while avoiding Audita's transcript-correction model.
|
|
|
|
This checkpoint should add reusable runtime plumbing, not real extraction
|
|
behavior.
|
|
|
|
## Scope
|
|
|
|
In scope:
|
|
|
|
- structured LLM client interface implementation;
|
|
- LLM scheduler;
|
|
- prompt registry pattern;
|
|
- response-schema registry pattern;
|
|
- diagnostics run directory pattern;
|
|
- config loading and validation for named pipeline profiles and implemented
|
|
runtime pieces.
|
|
|
|
Out of scope:
|
|
|
|
- correction proposals;
|
|
- replacement policies;
|
|
- transcript mutation;
|
|
- correction ledger terminology;
|
|
- Audita module or validator behavior;
|
|
- real D&D prompts or schemas unless needed as inert registry tests.
|
|
- embedded built-in pipeline profiles.
|
|
|
|
## Proposed Stages
|
|
|
|
### Stage 1: LLM Runtime
|
|
|
|
Port or adapt the OpenAI-compatible structured-output client and scheduler.
|
|
|
|
Keep the contract transport-neutral:
|
|
|
|
- framework code should depend on a `StructuredLLMClient` interface;
|
|
- provider-specific HTTP details should remain in the LLM runtime package;
|
|
- errors must redact configured secrets.
|
|
|
|
### Stage 2: Response Schema Registry
|
|
|
|
Port or adapt the embedded JSON response-schema registry pattern.
|
|
|
|
The registry should track:
|
|
|
|
- schema key;
|
|
- schema ID;
|
|
- schema version;
|
|
- schema name;
|
|
- JSON schema content;
|
|
- schema hash.
|
|
|
|
Use placeholder or test schemas if real extractor schemas are not ready.
|
|
|
|
### Stage 3: Prompt Registry
|
|
|
|
Port or adapt the embedded prompt registry pattern.
|
|
|
|
The registry should track:
|
|
|
|
- prompt ID;
|
|
- prompt version;
|
|
- prompt source;
|
|
- embedded path;
|
|
- prompt hash.
|
|
|
|
Do not add D&D prompt assets here unless the implementation naturally overlaps
|
|
with checkpoint 6. Test prompts are acceptable for registry tests.
|
|
|
|
### Stage 4: Diagnostics Run Directory
|
|
|
|
Port or adapt the diagnostics run directory pattern.
|
|
|
|
Initial diagnostics should cover:
|
|
|
|
- invocation metadata;
|
|
- redacted effective config;
|
|
- selected pipeline ID;
|
|
- resolved pipeline definition and digest;
|
|
- source document artifact;
|
|
- run report placeholder;
|
|
- error log on failure.
|
|
|
|
Avoid Audita-specific artifact names such as correction ledger.
|
|
|
|
### Stage 5: Pipeline Profile Config
|
|
|
|
Add config structs, loading, defaults, and validation for named pipeline
|
|
profiles.
|
|
|
|
Initial config areas:
|
|
|
|
- `llm_profiles`;
|
|
- `pipelines.<pipeline_id>.input`;
|
|
- `pipelines.<pipeline_id>.chunk`;
|
|
- `pipelines.<pipeline_id>.artifacts.<lane>.extract`;
|
|
- lane `merge`, `normalize`, and validator settings;
|
|
- output module selection;
|
|
- inline module-binding object form and string shorthand;
|
|
- default `chunk`, `merge`, `normalize`, `output`, and `llm_profile`;
|
|
- selected pipeline ID and lane filtering for runtime use;
|
|
- concurrency;
|
|
- work directory;
|
|
- diagnostics retention.
|
|
|
|
Config loading should support the standard precedence model:
|
|
|
|
1. built-in defaults
|
|
2. configuration file
|
|
3. environment variables
|
|
4. CLI flags
|
|
|
|
Structural module selection should come from pipeline config. CLI flags may
|
|
override operational settings and artifact lane selection, but should not offer
|
|
ad hoc `--extractor` or `--chunker` wiring.
|
|
|
|
Add validation for unknown pipeline IDs, unknown module keys, missing required
|
|
slots, missing capabilities, unknown LLM profiles, empty artifact-lane sets, and
|
|
invalid lane selections.
|
|
|
|
If the CLI shell is ready, add:
|
|
|
|
```sh
|
|
notarius config validate
|
|
notarius pipelines list
|
|
```
|
|
|
|
## Done Criteria
|
|
|
|
- `go test ./...` passes.
|
|
- Audita runtime pieces are adapted to Notarius package names and contracts.
|
|
- No correction proposal, replacement policy, transcript mutation, or correction
|
|
ledger code has been copied.
|
|
- Runtime tests cover secret redaction, schema registry lookup, prompt metadata,
|
|
and scheduler behavior where applicable.
|
|
- Config tests cover named pipeline profiles, defaults, lane selection,
|
|
capability validation, and resolved pipeline digesting.
|
|
|
|
## Review Questions
|
|
|
|
- Did we copy only reusable infrastructure?
|
|
- Do provider-specific types stay behind adapter/runtime boundaries?
|
|
- Are diagnostics names and report concepts extraction-oriented?
|
|
- Is config limited to named pipeline profiles and implemented behavior?
|
|
- Are structural pipeline changes kept out of ad hoc CLI flags?
|