Migrate internal architecture docs to docs/internal
This commit is contained in:
@@ -1,14 +1,16 @@
|
||||
# Audita Architecture Index
|
||||
|
||||
This file is the entrypoint for architecture documentation.
|
||||
This file is the architecture documentation entrypoint.
|
||||
|
||||
Core architecture overview:
|
||||
- [Architecture Overview](./architecture/architecture.md)
|
||||
Implemented internals:
|
||||
- [Internal Overview](./internal/overview.md)
|
||||
- [Pipeline](./internal/pipeline.md)
|
||||
- [Modules](./internal/modules.md)
|
||||
- [Validators](./internal/validators.md)
|
||||
- [LLM Runtime](./internal/llm-runtime.md)
|
||||
- [Diagnostics and Reporting](./internal/diagnostics-reporting.md)
|
||||
- [Prompts](./internal/prompts.md)
|
||||
- [Output Schemas](./internal/output-schemas.md)
|
||||
|
||||
Focused architecture contracts:
|
||||
External/runtime contract summary:
|
||||
- [Public Contract](./architecture/public-contract.md)
|
||||
- [Diagnostics](./architecture/diagnostics.md)
|
||||
- [Structured LLM](./architecture/structured-llm.md)
|
||||
- [Validators](./architecture/validators.md)
|
||||
- [Prompts](./architecture/prompts.md)
|
||||
- [Output Schemas](./architecture/output-schemas.md)
|
||||
|
||||
@@ -1,153 +1,5 @@
|
||||
# Audita Architecture
|
||||
# Moved: Internal Overview
|
||||
|
||||
## Scope
|
||||
This document describes the production architecture implemented in this repository today.
|
||||
Implemented internal architecture docs now live in [`docs/internal/`](../internal/).
|
||||
|
||||
Audita is a single-process Go CLI that:
|
||||
- loads effective runtime configuration;
|
||||
- reads transcript and glossary inputs;
|
||||
- normalizes and sections transcripts;
|
||||
- runs a built-in module pipeline with validator chains;
|
||||
- writes transcript output and run diagnostics.
|
||||
|
||||
## Runtime entrypoints
|
||||
Primary CLI commands:
|
||||
- `audita process <transcript.json> --glossary <glossary.yaml> [flags]`
|
||||
- `audita config validate --config <config.yml>`
|
||||
- `audita config print-effective [--config <config.yml>]`
|
||||
|
||||
Command ownership lives in `internal/cli/run.go`.
|
||||
|
||||
## Configuration model
|
||||
`internal/core/config` owns defaults, file parsing, environment overrides, CLI overrides, and validation.
|
||||
|
||||
Effective-config loading for `process` and `config print-effective` is centralized in:
|
||||
- `ResolveConfigPath`
|
||||
- `LoadEffectiveConfig`
|
||||
|
||||
Effective precedence for `audita process`:
|
||||
1. defaults
|
||||
2. config file
|
||||
3. environment overrides
|
||||
4. CLI overrides
|
||||
|
||||
`audita config validate` is intentionally file-only validation:
|
||||
- load versioned file;
|
||||
- apply onto defaults;
|
||||
- validate;
|
||||
- do not apply environment overrides.
|
||||
|
||||
Supported module and output-schema keys are validated through shared catalogs:
|
||||
- module keys: `internal/core/modulecatalog`
|
||||
- output schemas: `internal/core/outputschema`
|
||||
|
||||
## Pipeline and module orchestration
|
||||
The built-in module sequence is configured in runtime config and executed by `internal/framework/runner` through resolved module specs.
|
||||
|
||||
Current default sequence:
|
||||
- `glossary`
|
||||
- `homophones`
|
||||
- `glossary`
|
||||
- `spoken_word`
|
||||
- `grammar`
|
||||
|
||||
Execution behavior:
|
||||
- modules execute serially over the working transcript;
|
||||
- section proposal work can run concurrently within a module;
|
||||
- validator execution happens on generated proposals before application;
|
||||
- approved proposals are applied once per module in deterministic proposal-index order.
|
||||
|
||||
Production modules remain separate packages:
|
||||
- `internal/modules/glossary`
|
||||
- `internal/modules/homophones`
|
||||
- `internal/modules/spoken_word`
|
||||
- `internal/modules/grammar`
|
||||
|
||||
## Proposal generation and prompt context
|
||||
Shared proposal plumbing is centralized in `internal/framework/proposal_generation`.
|
||||
|
||||
Module packages provide:
|
||||
- module identity and replacement policy;
|
||||
- module-specific prompt message building;
|
||||
- built-in validator chain selection.
|
||||
|
||||
Shared prompt payload helpers are in `internal/framework/promptcontext`.
|
||||
|
||||
## Validator architecture
|
||||
Built-in validator construction and chain composition live in `internal/validators`.
|
||||
|
||||
Shared validator runtime mechanics live in `internal/framework/validators`.
|
||||
|
||||
Execution class metadata (deterministic vs LLM-backed) is centralized in `internal/validators/metadata` and used for ordering and reporting classification.
|
||||
|
||||
## Structured LLM boundary
|
||||
All production LLM calls go through the internal contract:
|
||||
- `contracts.StructuredLLMClient`
|
||||
- `CompleteStructured(ctx, req, out)`
|
||||
|
||||
The OpenAI-compatible HTTP adapter is implemented in `internal/framework/llm`.
|
||||
|
||||
Structured response schemas are registered in `internal/framework/responseschema` and attached to requests via `response_format` metadata.
|
||||
|
||||
Malformed structured-output detection is centralized in `internal/framework/structuredoutput` and reused by proposal generation and validator execution so downgrade behavior stays consistent.
|
||||
|
||||
## Stage naming and diagnostics metadata
|
||||
Diagnostics stage naming is centralized in `internal/framework/stagename`:
|
||||
- module proposal stage names;
|
||||
- proposal-generation stage names;
|
||||
- validator batch stage names.
|
||||
|
||||
Prompt metadata and response-schema metadata each expose canonical diagnostics maps via:
|
||||
- `prompts.Metadata.DiagnosticsMap()`
|
||||
- `responseschema.Schema.DiagnosticsMap()`
|
||||
|
||||
## Diagnostics and reporting
|
||||
Run-directory artifacts are owned by `internal/core/diagnostics`.
|
||||
|
||||
Stable artifact names are centralized constants (for example transcript artifacts, `invocation.json`, `effective-config.json`, `utilization-diagnostics.json`, `correction-ledger.json`, `report.json`, `error.log`).
|
||||
|
||||
Report diagnostics path metadata is constructed through `BuildDiagnosticsMetadata`, which keeps run-directory artifact references consistent between success and failure reports.
|
||||
|
||||
## Secret redaction
|
||||
Redaction responsibilities are split by concern:
|
||||
- structural config redaction: `config.Config.Redacted()`
|
||||
- byte/string payload redaction for diagnostics and surfaced errors: framework redaction utilities.
|
||||
|
||||
Configured LLM secret extraction is centralized in `llm.ConfiguredSecrets(cfg)` and reused across proposal and validator diagnostics paths.
|
||||
|
||||
## Output contracts
|
||||
Transcript output schema selection is owned by `internal/core/outputschema`.
|
||||
|
||||
Supported schemas:
|
||||
- `bare-segments`
|
||||
- `audita-v1`
|
||||
|
||||
Unknown schema keys fail validation and runtime resolution.
|
||||
|
||||
## Key package map
|
||||
Core packages:
|
||||
- `internal/core/config`
|
||||
- `internal/core/schema`
|
||||
- `internal/core/normalization`
|
||||
- `internal/core/chunking`
|
||||
- `internal/core/diagnostics`
|
||||
- `internal/core/reporting`
|
||||
- `internal/core/modulecatalog`
|
||||
- `internal/core/outputschema`
|
||||
|
||||
Framework packages:
|
||||
- `internal/framework/contracts`
|
||||
- `internal/framework/proposals`
|
||||
- `internal/framework/proposal_generation`
|
||||
- `internal/framework/promptcontext`
|
||||
- `internal/framework/runner`
|
||||
- `internal/framework/validators`
|
||||
- `internal/framework/llm`
|
||||
- `internal/framework/responseschema`
|
||||
- `internal/framework/stagename`
|
||||
- `internal/framework/structuredoutput`
|
||||
|
||||
Domain packages:
|
||||
- `internal/modules/*`
|
||||
- `internal/validators/*`
|
||||
- `internal/prompts`
|
||||
Start with [`docs/internal/overview.md`](../internal/overview.md).
|
||||
|
||||
@@ -1,104 +1,5 @@
|
||||
# Audita Diagnostics
|
||||
# Moved: Diagnostics and Reporting
|
||||
|
||||
This document describes the run-directory diagnostics artifacts produced by `audita process`.
|
||||
|
||||
## Purpose
|
||||
|
||||
Diagnostics provide machine-readable run context and execution artifacts for:
|
||||
- failure debugging;
|
||||
- validator/correction review;
|
||||
- post-run performance analysis.
|
||||
|
||||
Diagnostics are written under the configured work directory (`--work-dir`) when run-directory initialization succeeds.
|
||||
|
||||
## Core artifacts
|
||||
|
||||
Typical artifacts in each run directory:
|
||||
- `source-transcript.json`
|
||||
- `source-transcript-parsed.json`
|
||||
- `normalized-transcript.json`
|
||||
- `normalization-summary.json`
|
||||
- `chunking-summary.json`
|
||||
- `invocation.json`
|
||||
- `effective-config.json` (redacted)
|
||||
- module/validator LLM interaction artifacts
|
||||
- `report.json`
|
||||
- `error.log` on failure
|
||||
|
||||
## Utilization diagnostics artifact
|
||||
|
||||
Artifact:
|
||||
- `utilization-diagnostics.json`
|
||||
|
||||
High-level fields:
|
||||
- `effective_concurrency`:
|
||||
- total/proposal/validation LLM concurrency limits in effect.
|
||||
- `run_timing`:
|
||||
- run wall time;
|
||||
- scheduler queue wait time;
|
||||
- LLM execution time;
|
||||
- deterministic validator time;
|
||||
- max/average in-flight LLM calls.
|
||||
- `llm_calls`:
|
||||
- total proposal and validation LLM call counts.
|
||||
- `modules`:
|
||||
- module-level timing summaries.
|
||||
- `validators`:
|
||||
- per-validator timing summaries keyed by stable validator key.
|
||||
|
||||
## Correction ledger artifact
|
||||
|
||||
Artifact:
|
||||
- `correction-ledger.json`
|
||||
|
||||
Ledger records are flattened review entries derived from module results and include:
|
||||
- module/proposal identity (`module_key`, `module_instance`, `proposal_index`, `segment_id`);
|
||||
- correction text fields and replacement policy when available;
|
||||
- disposition:
|
||||
- `applied`
|
||||
- `rejected`
|
||||
- `skipped`
|
||||
- `failed`
|
||||
- stable reason codes/messages;
|
||||
- deterministic and LLM validator decision snapshots using stable validator keys.
|
||||
|
||||
Validator rejection and proposal-application skip are distinct dispositions.
|
||||
Module warnings are reported in module results and diagnostics metadata, but do not create standalone correction-ledger rows.
|
||||
|
||||
## Report references
|
||||
|
||||
`report.json` and optional `--report-json` output include diagnostics metadata paths for:
|
||||
- utilization diagnostics artifact;
|
||||
- correction ledger artifact;
|
||||
- existing transcript/normalization/chunking/invocation/effective-config artifacts.
|
||||
|
||||
Module report entries also include warning records for malformed proposal-generation payloads and malformed validator batches.
|
||||
|
||||
## Retention behavior
|
||||
|
||||
Run-directory retention follows configured policy:
|
||||
- `always`: keep all run directories;
|
||||
- `never`: keep successful run directories;
|
||||
- `auto`: keep failed runs and successful runs with skipped/rejected corrections.
|
||||
|
||||
## Redaction guarantees
|
||||
|
||||
API keys and other configured secrets are redacted from:
|
||||
- `effective-config.json`;
|
||||
- LLM interaction diagnostics artifacts;
|
||||
- reports and surfaced errors.
|
||||
|
||||
## Debugging guide
|
||||
|
||||
When debugging:
|
||||
- slow runs:
|
||||
- inspect `utilization-diagnostics.json` (`run_timing`, `modules`, `validators`, in-flight metrics).
|
||||
- validator rejections:
|
||||
- inspect `correction-ledger.json` rejected entries and matching validator decisions;
|
||||
- inspect validator response diagnostics payloads.
|
||||
- module warnings:
|
||||
- inspect module `warnings` entries in `report.json` or `--report-json`;
|
||||
- follow any diagnostic artifact path on the warning to the recorded error/response payload.
|
||||
- application skips:
|
||||
- inspect `correction-ledger.json` skipped entries and skip reason codes;
|
||||
- compare with validator decisions to distinguish validation rejection vs apply-time skip.
|
||||
Implemented diagnostics and reporting internals now live at:
|
||||
- [`docs/internal/diagnostics-reporting.md`](../internal/diagnostics-reporting.md)
|
||||
- [`docs/operations.md`](../operations.md)
|
||||
|
||||
@@ -1,88 +1,3 @@
|
||||
# Audita Output Schemas
|
||||
# Moved: Output Schemas
|
||||
|
||||
This document describes the built-in transcript output schema registry used by `audita process`.
|
||||
|
||||
## Supported schema names
|
||||
|
||||
### `bare-segments`
|
||||
|
||||
Status:
|
||||
- implemented
|
||||
- default output schema
|
||||
|
||||
Shape:
|
||||
- top-level JSON array of transcript segments
|
||||
|
||||
Segment fields:
|
||||
- `id`
|
||||
- `speaker`
|
||||
- `start`
|
||||
- `end`
|
||||
- `text`
|
||||
- optional `categories`
|
||||
|
||||
Compatibility:
|
||||
- this preserves the long-standing output shape used by existing consumers.
|
||||
|
||||
### `audita-v1`
|
||||
|
||||
Status:
|
||||
- implemented
|
||||
|
||||
Shape:
|
||||
- top-level JSON object:
|
||||
- `schema`: `"audita-v1"`
|
||||
- `version`: `"v1"`
|
||||
- `segments`: transcript segment array
|
||||
|
||||
Segment fields inside `segments` match `bare-segments` segment fields.
|
||||
|
||||
Compatibility:
|
||||
- this is the Audita-native object format with explicit schema/version metadata.
|
||||
|
||||
### `seriatim-intermediate`
|
||||
|
||||
Status:
|
||||
- deferred / not implemented
|
||||
|
||||
Current behavior:
|
||||
- selecting `seriatim-intermediate` fails clearly as an unsupported output schema.
|
||||
|
||||
Reason:
|
||||
- a concrete, repository-backed contract for this schema has not been finalized yet.
|
||||
|
||||
## Selection
|
||||
|
||||
Choose output schema with CLI:
|
||||
|
||||
```sh
|
||||
audita process <transcript.json> --glossary <glossary.yaml> --output-schema audita-v1
|
||||
```
|
||||
|
||||
Or in file config:
|
||||
|
||||
```yaml
|
||||
version: 1
|
||||
output:
|
||||
schema: audita-v1
|
||||
```
|
||||
|
||||
Precedence remains:
|
||||
1. defaults
|
||||
2. file config
|
||||
3. environment overrides
|
||||
4. CLI overrides
|
||||
|
||||
`--output-schema` overrides `output.schema` when both are supplied.
|
||||
|
||||
## Output routing behavior
|
||||
|
||||
- With `--output`, transcript JSON is written to file using the selected schema and stdout stays empty on success.
|
||||
- Without `--output`, stdout contains transcript JSON only, using the selected schema.
|
||||
- `--report-json` writes report JSON to file and does not write report payloads to stdout.
|
||||
|
||||
## Backward-compatibility expectations
|
||||
|
||||
- default schema stays `bare-segments` for compatibility unless explicitly changed in a future breaking release;
|
||||
- supported schema names are treated as stable public contract values;
|
||||
- unsupported schema names fail before output write.
|
||||
Implemented output schema registry docs now live at [`docs/internal/output-schemas.md`](../internal/output-schemas.md).
|
||||
|
||||
@@ -1,118 +1,3 @@
|
||||
# Audita Prompts
|
||||
# Moved: Prompt Registry
|
||||
|
||||
This document describes Audita's built-in embedded prompt assets and prompt registry behavior.
|
||||
|
||||
## Why embedded prompt assets
|
||||
|
||||
Audita embeds production prompt text into the binary so runtime behavior is:
|
||||
- deterministic;
|
||||
- auditable;
|
||||
- dependency-light;
|
||||
- not dependent on external prompt files at execution time.
|
||||
|
||||
Prompt text is authored as Markdown assets and rendered by Go code using typed template data.
|
||||
|
||||
## Built-in prompt registry
|
||||
|
||||
The prompt registry lives in `internal/prompts` and is responsible for:
|
||||
- loading embedded prompt assets;
|
||||
- registering stable prompt IDs and versions;
|
||||
- recording prompt source metadata;
|
||||
- computing deterministic SHA-256 source hashes;
|
||||
- rendering system/user prompts with strict missing-key failures.
|
||||
|
||||
Current prompt source behavior:
|
||||
- built-in embedded prompts only (`prompt_source = builtin`).
|
||||
- filesystem prompt overrides are not supported.
|
||||
|
||||
## Built-in prompt IDs
|
||||
|
||||
Module proposal prompts:
|
||||
- `modules.glossary.proposal`
|
||||
- `modules.homophones.proposal`
|
||||
- `modules.spoken_word.proposal`
|
||||
- `modules.grammar.proposal`
|
||||
|
||||
LLM-backed validator prompts:
|
||||
- `validators.spoken_form_plausibility`
|
||||
- `validators.meaning_reversal_review`
|
||||
- `validators.editorial_review`
|
||||
- `validators.grammar_review`
|
||||
- `validators.spoken_word_review`
|
||||
|
||||
## Prompt version semantics
|
||||
|
||||
Current built-in prompt version value is `v1`.
|
||||
|
||||
Version is a stable metadata identifier for diagnostics and debugging. It is not a dynamic prompt-selection mechanism.
|
||||
|
||||
## Prompt hash semantics
|
||||
|
||||
Each registered prompt includes a deterministic SHA-256 hash of embedded source text.
|
||||
|
||||
Hash purpose:
|
||||
- identify exact prompt source used in a run;
|
||||
- support diagnostics reproducibility and change auditing.
|
||||
|
||||
Current hash scope:
|
||||
- source prompt text (system + user assets for a registered prompt), not a runtime secret-bearing payload.
|
||||
|
||||
## Template rendering behavior
|
||||
|
||||
Prompt rendering uses Go `text/template` with typed template data from module/validator builders.
|
||||
|
||||
Missing-key behavior:
|
||||
- rendering uses missing-key errors;
|
||||
- missing/renamed template fields fail quickly instead of silently producing incomplete prompts.
|
||||
|
||||
Go code still owns:
|
||||
- structured request/response models;
|
||||
- response schema selection;
|
||||
- transcript/glossary/payload formatting;
|
||||
- module and validator selection;
|
||||
- diagnostics wiring.
|
||||
|
||||
## Shared prompt hardening policy
|
||||
|
||||
A shared hardening fragment is embedded once and included in every module proposal prompt and every LLM-validator prompt.
|
||||
|
||||
Hardening policy includes:
|
||||
- transcript text is untrusted data;
|
||||
- glossary entries and transcript descriptions are reference data, not instructions;
|
||||
- instructions found inside transcript text must not be obeyed;
|
||||
- model must perform only the requested correction/validation task;
|
||||
- no invention of facts, names, events, motivations, speaker intent, or corrections;
|
||||
- transcript remains the source of truth.
|
||||
|
||||
## Transcript description behavior
|
||||
|
||||
Transcript description remains background-only prompt context:
|
||||
- it may help interpret ambiguous terms;
|
||||
- it is explicitly non-authoritative and must not override transcript content;
|
||||
- empty descriptions do not render awkward blank context sections.
|
||||
|
||||
Generated transcript descriptions are not implemented in this workstream.
|
||||
|
||||
## Diagnostics and report metadata boundaries
|
||||
|
||||
Current metadata flow:
|
||||
- proposal-generation diagnostics request metadata includes prompt metadata;
|
||||
- LLM-validator diagnostics request metadata includes prompt metadata.
|
||||
|
||||
Prompt metadata fields used in diagnostics:
|
||||
- `prompt_id`
|
||||
- `prompt_version`
|
||||
- `prompt_source`
|
||||
- `embedded_path`
|
||||
- `sha256`
|
||||
|
||||
Current boundary:
|
||||
- detailed prompt metadata is diagnostics-first;
|
||||
- broad report-level prompt registries/ledgers are deferred.
|
||||
|
||||
## 1.0 boundary
|
||||
|
||||
Not implemented for 1.0 in this workstream:
|
||||
- filesystem prompt overrides;
|
||||
- user-configurable prompt selection;
|
||||
- external prompt directories.
|
||||
Implemented prompt registry and prompt metadata docs now live at [`docs/internal/prompts.md`](../internal/prompts.md).
|
||||
|
||||
@@ -1,72 +1,3 @@
|
||||
# Structured LLM Architecture
|
||||
# Moved: LLM Runtime
|
||||
|
||||
## Scope
|
||||
This document describes Audita's structured LLM runtime boundary and adapter behavior.
|
||||
|
||||
## Runtime boundary
|
||||
Production LLM integration depends on the internal contract only:
|
||||
- `contracts.StructuredLLMClient`
|
||||
- `CompleteStructured(ctx, req, out)`
|
||||
|
||||
Provider SDK types do not leak past this boundary.
|
||||
|
||||
## Adapter ownership
|
||||
`internal/framework/llm` owns the OpenAI-compatible HTTP adapter and shared LLM runtime utilities.
|
||||
|
||||
Key responsibilities:
|
||||
- request assembly;
|
||||
- timeout/cancellation propagation;
|
||||
- bounded retry behavior;
|
||||
- scheduler integration;
|
||||
- provider response decoding;
|
||||
- error redaction.
|
||||
|
||||
## Structured schema registry
|
||||
Structured response schemas are registered in `internal/framework/responseschema` and include stable metadata:
|
||||
- `id`
|
||||
- `version`
|
||||
- `name`
|
||||
- `json_schema`
|
||||
- `sha256`
|
||||
|
||||
Current schema keys:
|
||||
- `correction_set`
|
||||
- `validator_decision_set`
|
||||
|
||||
Schema metadata is attached to diagnostics through `Schema.DiagnosticsMap()`.
|
||||
|
||||
## Request shape assumptions
|
||||
Audita targets OpenAI-compatible chat-completions endpoints and sends structured requests with:
|
||||
- model;
|
||||
- chat messages;
|
||||
- `response_format.type = json_schema`;
|
||||
- schema name and JSON schema payload.
|
||||
|
||||
## Local validation remains mandatory
|
||||
Provider schema enforcement is treated as transport-level guardrails.
|
||||
|
||||
Audita still validates output locally before applying behavior changes:
|
||||
- proposal decoding and proposal invariants;
|
||||
- validator decision decoding and cardinality checks;
|
||||
- deterministic validation and apply-time rules.
|
||||
|
||||
## Shared malformed-output policy
|
||||
Malformed structured-output classification is centralized in `internal/framework/structuredoutput`.
|
||||
|
||||
Proposal generation and validator execution both use this shared classifier so downgrade behavior cannot drift between the two paths.
|
||||
|
||||
## Secrets and redaction
|
||||
Secret extraction for LLM redaction is centralized in `llm.ConfiguredSecrets(cfg)` and reused by proposal and validator diagnostics writers.
|
||||
|
||||
Secrets are redacted from:
|
||||
- diagnostics artifacts;
|
||||
- report artifacts;
|
||||
- surfaced adapter/runtime errors.
|
||||
|
||||
## Concurrency and scheduling
|
||||
LLM execution is constrained by composed scheduler limits:
|
||||
- total LLM concurrency;
|
||||
- proposal LLM concurrency;
|
||||
- validation LLM concurrency.
|
||||
|
||||
The scheduler is FIFO and context-aware so permits are released on success, failure, and cancellation.
|
||||
Implemented structured LLM runtime docs now live at [`docs/internal/llm-runtime.md`](../internal/llm-runtime.md).
|
||||
|
||||
@@ -1,96 +1,3 @@
|
||||
# Audita Validators
|
||||
# Moved: Validators
|
||||
|
||||
## Scope
|
||||
This document defines the built-in validator system used by production module runs.
|
||||
|
||||
## Ownership boundaries
|
||||
Built-in validator keys, constructors, and module chains are owned by `internal/validators`.
|
||||
|
||||
Shared runtime execution mechanics are owned by `internal/framework/validators`, including:
|
||||
- validator request/result models;
|
||||
- deterministic proposal checks;
|
||||
- LLM validator batching and execution;
|
||||
- decision-cardinality enforcement;
|
||||
- diagnostics integration.
|
||||
|
||||
Execution class metadata is owned by `internal/validators/metadata`.
|
||||
|
||||
## Stable validator keys
|
||||
Deterministic:
|
||||
- `proposal_shape`
|
||||
- `confidence_threshold`
|
||||
- `original_text_presence`
|
||||
- `non_empty_corrected_text`
|
||||
- `no_effect`
|
||||
- `protected_terms`
|
||||
|
||||
LLM-backed:
|
||||
- `spoken_form_plausibility`
|
||||
- `meaning_reversal_review`
|
||||
- `editorial_review`
|
||||
|
||||
## Built-in module chains
|
||||
`glossary`:
|
||||
- `proposal_shape`
|
||||
- `no_effect`
|
||||
- `original_text_presence`
|
||||
- `confidence_threshold`
|
||||
- `protected_terms`
|
||||
- `non_empty_corrected_text`
|
||||
- `spoken_form_plausibility`
|
||||
- `meaning_reversal_review`
|
||||
|
||||
`homophones`:
|
||||
- `proposal_shape`
|
||||
- `no_effect`
|
||||
- `original_text_presence`
|
||||
- `confidence_threshold`
|
||||
- `protected_terms`
|
||||
- `non_empty_corrected_text`
|
||||
- `spoken_form_plausibility`
|
||||
- `meaning_reversal_review`
|
||||
|
||||
`spoken_word`:
|
||||
- `proposal_shape`
|
||||
- `no_effect`
|
||||
- `original_text_presence`
|
||||
- `confidence_threshold`
|
||||
- `protected_terms`
|
||||
- `non_empty_corrected_text`
|
||||
- `editorial_review`
|
||||
- `meaning_reversal_review`
|
||||
|
||||
`grammar`:
|
||||
- `proposal_shape`
|
||||
- `no_effect`
|
||||
- `original_text_presence`
|
||||
- `confidence_threshold`
|
||||
- `protected_terms`
|
||||
- `non_empty_corrected_text`
|
||||
- `editorial_review`
|
||||
- `meaning_reversal_review`
|
||||
|
||||
## Ordering and execution semantics
|
||||
Validator ordering is based on canonical metadata:
|
||||
- deterministic validators run before LLM-backed validators.
|
||||
|
||||
Within each module stage:
|
||||
- proposals are generated per section;
|
||||
- validator chains execute on those proposals;
|
||||
- approved proposals are applied once after section work settles.
|
||||
|
||||
## Malformed payload behavior
|
||||
Malformed structured-output from proposal generation and LLM validator calls is downgraded, not treated as a process-fatal transport error.
|
||||
|
||||
Current outcomes:
|
||||
- malformed proposal-generation payloads produce section/module warnings and zero proposals for the affected section;
|
||||
- malformed validator decision payloads reject the affected validator batch with warnings;
|
||||
- deterministic validator behavior and runner order remain unchanged.
|
||||
|
||||
## Reporting identity
|
||||
Reports and diagnostics use stable validator keys as identifiers.
|
||||
|
||||
Correction-ledger deterministic-vs-LLM classification is derived from canonical validator metadata, not package-local hardcoded maps.
|
||||
|
||||
## Prompt assets
|
||||
LLM validator prompt assets and prompt metadata are documented in [Prompts](./prompts.md).
|
||||
Implemented validator architecture docs now live at [`docs/internal/validators.md`](../internal/validators.md).
|
||||
|
||||
79
docs/internal/diagnostics-reporting.md
Normal file
79
docs/internal/diagnostics-reporting.md
Normal file
@@ -0,0 +1,79 @@
|
||||
# Audita Diagnostics and Reporting
|
||||
|
||||
## Scope
|
||||
|
||||
This document describes diagnostics artifacts, process report mapping, and correction ledger generation.
|
||||
|
||||
## Run Directory Ownership
|
||||
|
||||
`internal/core/diagnostics` owns run-directory creation, artifact writes, and retention decisions.
|
||||
|
||||
Stable artifact names include:
|
||||
- `source-transcript.json`
|
||||
- `source-transcript-parsed.json`
|
||||
- `normalized-transcript.json`
|
||||
- `normalization-summary.json`
|
||||
- `chunking-summary.json`
|
||||
- `utilization-diagnostics.json`
|
||||
- `correction-ledger.json`
|
||||
- `invocation.json`
|
||||
- `effective-config.json`
|
||||
- `report.json`
|
||||
- `error.log` (failure)
|
||||
|
||||
## Process Report Mapping
|
||||
|
||||
`internal/framework/processreport` maps runner/CLI execution facts into `reporting.ProcessReport`.
|
||||
|
||||
Report metadata fields include:
|
||||
- `report_schema_name` (`audita-process-report`)
|
||||
- `report_schema_version` (`v1`)
|
||||
- `output_schema`
|
||||
- `config_version` (when file config exists)
|
||||
|
||||
The report includes:
|
||||
- top-level status/error phase/error message;
|
||||
- normalization/chunking summaries;
|
||||
- diagnostics metadata paths;
|
||||
- per-module results and module summary.
|
||||
|
||||
## Correction Ledger
|
||||
|
||||
`internal/framework/processreport/BuildCorrectionLedger` flattens run results into `correction-ledger.json` entries.
|
||||
|
||||
Dispositions:
|
||||
- `applied`
|
||||
- `skipped`
|
||||
- `rejected`
|
||||
- `failed`
|
||||
|
||||
Validator decisions are split into deterministic and LLM-backed groups using validator metadata classification.
|
||||
|
||||
## Report Write Paths
|
||||
|
||||
- run directory always attempts to write `report.json` when possible;
|
||||
- optional `--report-json` writes an external report file;
|
||||
- on failure paths, report writing is best-effort and does not mask primary run errors.
|
||||
|
||||
## Retention Interaction
|
||||
|
||||
Current retention behavior:
|
||||
- failed runs are retained;
|
||||
- `always` keeps successful runs;
|
||||
- `auto` removes only clean successful runs;
|
||||
- `never` currently retains successful runs in current implementation.
|
||||
|
||||
## Redaction
|
||||
|
||||
Redacted data expectations:
|
||||
- effective config artifact uses config redaction;
|
||||
- diagnostics payloads and surfaced errors use LLM secret redaction;
|
||||
- reports should not include raw API key values.
|
||||
|
||||
## Key Tests
|
||||
|
||||
- `internal/core/diagnostics/*_test.go`
|
||||
- `internal/framework/processreport/*_test.go`
|
||||
- `internal/core/reporting/report_test.go`
|
||||
- `internal/cli/run_test.go`
|
||||
- `cmd/audita/main_integration_test.go`
|
||||
67
docs/internal/llm-runtime.md
Normal file
67
docs/internal/llm-runtime.md
Normal file
@@ -0,0 +1,67 @@
|
||||
# Audita LLM Runtime
|
||||
|
||||
## Scope
|
||||
|
||||
This document describes the structured LLM runtime and scheduler behavior.
|
||||
|
||||
## Client Boundary
|
||||
|
||||
All runtime LLM calls go through `contracts.StructuredLLMClient`.
|
||||
|
||||
Primary adapter:
|
||||
- `internal/framework/llm/OpenAICompatibleClient`
|
||||
|
||||
## Request/Response Behavior
|
||||
|
||||
The OpenAI-compatible adapter sends chat completions requests with:
|
||||
- model;
|
||||
- messages;
|
||||
- `response_format.type = json_schema`;
|
||||
- strict schema envelope (`name`, `schema`, `strict=true`).
|
||||
|
||||
The response is decoded into the requested structured output target.
|
||||
|
||||
## Response Schema Registry
|
||||
|
||||
Structured response schemas are registered in `internal/framework/responseschema`:
|
||||
- `correction_set`
|
||||
- `validator_decision_set`
|
||||
|
||||
Each schema includes stable diagnostics metadata (`id`, `version`, `name`, `sha256`).
|
||||
|
||||
## Retries and Error Handling
|
||||
|
||||
Adapter retries apply to retryable conditions (for example transport/decoding/retryable status classes) up to configured `max_retries`.
|
||||
|
||||
Errors are sanitized to redact configured API-key values before surfacing.
|
||||
|
||||
Malformed structured output detection is shared through `internal/framework/structuredoutput` and is used by:
|
||||
- proposal generation;
|
||||
- LLM-backed validators.
|
||||
|
||||
## Scheduling and Concurrency
|
||||
|
||||
`internal/framework/llm/Scheduler` provides FIFO, context-aware permit gating.
|
||||
|
||||
Runner composes scheduler limits across:
|
||||
- total LLM concurrency;
|
||||
- proposal LLM concurrency;
|
||||
- validation LLM concurrency.
|
||||
|
||||
Scheduler release is guarded to avoid permit leaks on cancellation/error.
|
||||
|
||||
## Diagnostics and Redaction
|
||||
|
||||
`internal/framework/llm/DiagnosticsWriter` writes request/response/error artifacts.
|
||||
|
||||
Configured secrets are derived from `llm.ConfiguredSecrets(cfg)` and redacted from:
|
||||
- diagnostics payloads;
|
||||
- surfaced runtime/adapter errors.
|
||||
|
||||
## Key Tests
|
||||
|
||||
- `internal/framework/llm/openai_compatible_client_test.go`
|
||||
- `internal/framework/llm/scheduler_test.go`
|
||||
- `internal/framework/llm/diagnostics_test.go`
|
||||
- `internal/framework/responseschema/registry_test.go`
|
||||
- `internal/framework/structuredoutput/malformed_test.go`
|
||||
58
docs/internal/modules.md
Normal file
58
docs/internal/modules.md
Normal file
@@ -0,0 +1,58 @@
|
||||
# Audita Modules
|
||||
|
||||
## Scope
|
||||
|
||||
This document covers module contracts and built-in module packages.
|
||||
|
||||
## Module Contract
|
||||
|
||||
Modules implement `contracts.TranscriptModule`:
|
||||
- `Key()`
|
||||
- `ReplacementPolicy()`
|
||||
- `Validators()`
|
||||
- `Propose(ctx, req)`
|
||||
|
||||
Runner resolves configured module specs to module instances through `internal/framework/modules`.
|
||||
|
||||
## Built-In Modules
|
||||
|
||||
Current module packages:
|
||||
- `internal/modules/glossary`
|
||||
- `internal/modules/homophones`
|
||||
- `internal/modules/spoken_word`
|
||||
- `internal/modules/grammar`
|
||||
|
||||
Current replacement policies:
|
||||
- `glossary`: `replace_all`
|
||||
- `homophones`: `require_unique`
|
||||
- `spoken_word`: `require_unique`
|
||||
- `grammar`: `require_unique`
|
||||
|
||||
## Proposal Generation Ownership
|
||||
|
||||
Shared proposal-generation plumbing is centralized in:
|
||||
- `internal/framework/proposal_generation`
|
||||
|
||||
Module packages own:
|
||||
- prompt selection (`internal/prompts` prompt IDs);
|
||||
- module-specific prompt payload construction.
|
||||
|
||||
Shared prompt helpers live in `internal/framework/promptcontext`.
|
||||
|
||||
## Validator Chain Ownership
|
||||
|
||||
Built-in chains are resolved in `internal/validators` per module key.
|
||||
Module packages call the built-in chain resolver at construction.
|
||||
|
||||
## Failure and Warning Behavior
|
||||
|
||||
- module setup failures surface as `runner_setup` or module setup errors;
|
||||
- module runtime failures surface as `runner_execution` with partial module results preserved;
|
||||
- malformed structured proposal payloads are downgraded to warnings and section-level proposal rejection.
|
||||
|
||||
## Key Tests
|
||||
|
||||
- `internal/modules/*/module_test.go`
|
||||
- `internal/framework/modules/registry_test.go`
|
||||
- `internal/framework/proposal_generation/*_test.go`
|
||||
- `internal/cli/run_test.go` (pipeline/report integration)
|
||||
46
docs/internal/output-schemas.md
Normal file
46
docs/internal/output-schemas.md
Normal file
@@ -0,0 +1,46 @@
|
||||
# Audita Output Schemas
|
||||
|
||||
## Scope
|
||||
|
||||
This document describes the implemented transcript output schema registry.
|
||||
|
||||
## Registry Ownership
|
||||
|
||||
Output schema registry is owned by `internal/core/outputschema`.
|
||||
|
||||
Supported schema keys:
|
||||
- `bare-segments`
|
||||
- `audita-v1`
|
||||
|
||||
## Schemas
|
||||
|
||||
`bare-segments`:
|
||||
- top-level JSON array of transcript segments.
|
||||
|
||||
`audita-v1`:
|
||||
- top-level JSON object with:
|
||||
- `schema: "audita-v1"`
|
||||
- `version: "v1"`
|
||||
- `segments: [...]`
|
||||
|
||||
Segment fields include `id`, `speaker`, `start`, `end`, `text`, and optional `categories`.
|
||||
|
||||
## Validation and Resolution
|
||||
|
||||
Config validation and runtime resolution both reject unsupported schema keys.
|
||||
|
||||
Unknown schema keys fail with `unsupported output schema` before output emission.
|
||||
|
||||
## Output Emission
|
||||
|
||||
The selected schema is used by `audita process` when writing:
|
||||
- output file (`--output`) or
|
||||
- stdout (when no `--output`).
|
||||
|
||||
Report metadata records selected `output_schema`.
|
||||
|
||||
## Key Tests
|
||||
|
||||
- `internal/core/outputschema/registry_test.go`
|
||||
- `internal/core/config/config_test.go`
|
||||
- `internal/cli/run_test.go`
|
||||
92
docs/internal/overview.md
Normal file
92
docs/internal/overview.md
Normal file
@@ -0,0 +1,92 @@
|
||||
# Audita Internal Overview
|
||||
|
||||
## Scope
|
||||
|
||||
This document is the internal architecture entry point for developers and coding agents.
|
||||
|
||||
It summarizes:
|
||||
- package boundaries;
|
||||
- the main `process` execution path;
|
||||
- where to add new code safely.
|
||||
|
||||
## Package Map
|
||||
|
||||
CLI and command orchestration:
|
||||
- `internal/cli`
|
||||
|
||||
Core deterministic components:
|
||||
- `internal/core/config`
|
||||
- `internal/core/schema`
|
||||
- `internal/core/normalization`
|
||||
- `internal/core/chunking`
|
||||
- `internal/core/outputschema`
|
||||
- `internal/core/diagnostics`
|
||||
- `internal/core/reporting`
|
||||
- `internal/core/modulecatalog`
|
||||
|
||||
Framework orchestration and contracts:
|
||||
- `internal/framework/contracts`
|
||||
- `internal/framework/modules`
|
||||
- `internal/framework/proposal_generation`
|
||||
- `internal/framework/proposals`
|
||||
- `internal/framework/runner`
|
||||
- `internal/framework/validators`
|
||||
- `internal/framework/llm`
|
||||
- `internal/framework/responseschema`
|
||||
- `internal/framework/structuredoutput`
|
||||
- `internal/framework/processreport`
|
||||
- `internal/framework/promptcontext`
|
||||
- `internal/framework/stagename`
|
||||
|
||||
Domain implementations:
|
||||
- `internal/modules/*`
|
||||
- `internal/validators/*`
|
||||
- `internal/prompts`
|
||||
|
||||
## Main Execution Path (`audita process`)
|
||||
|
||||
High-level flow:
|
||||
1. CLI loads effective config and validates CLI requirements.
|
||||
2. Run directory is created and invocation/effective config artifacts are written.
|
||||
3. Transcript/glossary files are loaded and parsed.
|
||||
4. Transcript is normalized and chunked.
|
||||
5. `runner.Run` executes configured module instances.
|
||||
6. Proposals are validated, applied deterministically, and serialized in selected output schema.
|
||||
7. Process report, utilization diagnostics, correction ledger, and retention decisions are finalized.
|
||||
|
||||
## Boundary Summary
|
||||
|
||||
- `internal/core/*` owns deterministic, reusable logic and persistence-independent rules.
|
||||
- `internal/framework/*` owns orchestration contracts and reusable runtime plumbing.
|
||||
- `internal/modules/*` owns module-specific proposal behavior and prompt usage.
|
||||
- `internal/validators/*` owns validator composition and built-in chain assembly.
|
||||
- `internal/prompts` owns embedded prompt assets and metadata registry.
|
||||
|
||||
## Where To Add New Code
|
||||
|
||||
Add config fields:
|
||||
- `internal/core/config`
|
||||
|
||||
Add module behavior:
|
||||
- one package under `internal/modules/<module_key>`
|
||||
- registration/wiring through `internal/framework/modules` and config module list
|
||||
|
||||
Add validators:
|
||||
- implementation under `internal/validators/<validator_key>`
|
||||
- registry/chain wiring in `internal/validators`
|
||||
|
||||
Add runtime orchestration behavior:
|
||||
- `internal/framework/*` (runner/proposal/validator/LLM plumbing)
|
||||
|
||||
Add CLI surface:
|
||||
- `internal/cli`
|
||||
|
||||
## Related Internal Docs
|
||||
|
||||
- [`docs/internal/pipeline.md`](pipeline.md)
|
||||
- [`docs/internal/modules.md`](modules.md)
|
||||
- [`docs/internal/validators.md`](validators.md)
|
||||
- [`docs/internal/llm-runtime.md`](llm-runtime.md)
|
||||
- [`docs/internal/diagnostics-reporting.md`](diagnostics-reporting.md)
|
||||
- [`docs/internal/prompts.md`](prompts.md)
|
||||
- [`docs/internal/output-schemas.md`](output-schemas.md)
|
||||
79
docs/internal/pipeline.md
Normal file
79
docs/internal/pipeline.md
Normal file
@@ -0,0 +1,79 @@
|
||||
# Audita Internal Pipeline
|
||||
|
||||
## Scope
|
||||
|
||||
This document describes the implemented `audita process` pipeline.
|
||||
|
||||
## Inputs
|
||||
|
||||
Pipeline inputs are:
|
||||
- effective config (`internal/core/config`);
|
||||
- transcript JSON (`internal/core/schema`);
|
||||
- glossary YAML (`internal/core/schema`).
|
||||
|
||||
## Pipeline Phases
|
||||
|
||||
1. Input loading and schema validation
|
||||
- transcript and glossary files are read and parsed.
|
||||
- schema failures stop the run with `transcript_schema` or `glossary_schema`.
|
||||
|
||||
2. Normalization
|
||||
- canonical transcript segments are normalized by configured gap/duration/token settings.
|
||||
- normalization summary artifacts are written.
|
||||
|
||||
3. Chunking
|
||||
- normalized transcript is chunked with configured max/min tokens and target sections.
|
||||
|
||||
4. Module proposal generation
|
||||
- runner executes configured module instances in sequence.
|
||||
- each module proposes corrections per section.
|
||||
- per-section proposal generation can run concurrently.
|
||||
|
||||
5. Validator filtering
|
||||
- validators run on candidate proposals before apply.
|
||||
- deterministic validators run before LLM-backed validators.
|
||||
- LLM validator inputs are batched by max prompt token limit.
|
||||
|
||||
6. Deterministic apply
|
||||
- approved proposals are applied via replacement policy.
|
||||
- applied/skipped/rejected outcomes are recorded.
|
||||
|
||||
7. Output and reporting
|
||||
- final transcript is serialized with selected output schema.
|
||||
- report, utilization diagnostics, and correction ledger are written.
|
||||
- retention policy is applied to run directory.
|
||||
|
||||
## Runner Outputs
|
||||
|
||||
`runner.Run` returns:
|
||||
- final transcript;
|
||||
- per-module results;
|
||||
- utilization diagnostics.
|
||||
|
||||
CLI/reporting then map this into process report and diagnostics artifacts.
|
||||
|
||||
## Failure Behavior
|
||||
|
||||
Representative failure phases include:
|
||||
- `run_dir_creation`
|
||||
- `transcript_read`, `glossary_read`
|
||||
- `transcript_schema`, `glossary_schema`
|
||||
- `chunking`
|
||||
- `runner_setup`, `runner_execution`
|
||||
- `output_schema`, `serialization`, `output_write`, `stdout_write`
|
||||
|
||||
When diagnostics are available, failure stderr includes diagnostics path.
|
||||
|
||||
## Invariants
|
||||
|
||||
- module execution order follows configured module sequence;
|
||||
- proposal/validator nondeterminism is isolated before deterministic apply;
|
||||
- proposal indices are assigned deterministically by section order;
|
||||
- output/report artifacts are generated from run results, not speculative state.
|
||||
|
||||
## Key Tests
|
||||
|
||||
- `internal/framework/runner/runner_test.go`
|
||||
- `internal/framework/proposal_generation/*_test.go`
|
||||
- `internal/cli/run_test.go`
|
||||
- `cmd/audita/main_integration_test.go`
|
||||
62
docs/internal/prompts.md
Normal file
62
docs/internal/prompts.md
Normal file
@@ -0,0 +1,62 @@
|
||||
# Audita Prompt Registry
|
||||
|
||||
## Scope
|
||||
|
||||
This document describes embedded prompt assets, prompt metadata, and rendering behavior.
|
||||
|
||||
## Registry Ownership
|
||||
|
||||
Prompt registry lives in `internal/prompts` and embeds assets under `internal/prompts/assets/**`.
|
||||
|
||||
Registered prompt IDs:
|
||||
- `modules.glossary.proposal`
|
||||
- `modules.homophones.proposal`
|
||||
- `modules.spoken_word.proposal`
|
||||
- `modules.grammar.proposal`
|
||||
- `validators.spoken_form_plausibility`
|
||||
- `validators.meaning_reversal_review`
|
||||
- `validators.editorial_review`
|
||||
- `validators.grammar_review`
|
||||
- `validators.spoken_word_review`
|
||||
|
||||
## Metadata Model
|
||||
|
||||
Each prompt has metadata:
|
||||
- `prompt_id`
|
||||
- `prompt_version`
|
||||
- `prompt_source`
|
||||
- `embedded_path`
|
||||
- `sha256`
|
||||
|
||||
Current source/version values:
|
||||
- `prompt_source = builtin`
|
||||
- `prompt_version = v1`
|
||||
|
||||
## Rendering
|
||||
|
||||
`prompts.RenderUserSystem(promptID, data)` renders system/user templates.
|
||||
|
||||
Template behavior:
|
||||
- uses Go `text/template`;
|
||||
- `missingkey=error` is enabled;
|
||||
- output is trimmed.
|
||||
|
||||
A shared hardening fragment is embedded once and referenced by prompt templates.
|
||||
|
||||
## Prompt Context Inputs
|
||||
|
||||
Shared prompt payload helpers:
|
||||
- transcript section JSON (`internal/framework/promptcontext/MarshalTranscriptSectionJSON`)
|
||||
- transcript description block (`TranscriptDescriptionBlock`)
|
||||
|
||||
Modules and LLM validators provide typed data maps to render prompt assets.
|
||||
|
||||
## Diagnostics Integration
|
||||
|
||||
Prompt metadata is attached to proposal/validator diagnostics request metadata using `Metadata.DiagnosticsMap()`.
|
||||
|
||||
## Key Tests
|
||||
|
||||
- `internal/prompts/registry_test.go`
|
||||
- `internal/framework/promptcontext/*_test.go`
|
||||
- module and validator prompt builder tests
|
||||
72
docs/internal/validators.md
Normal file
72
docs/internal/validators.md
Normal file
@@ -0,0 +1,72 @@
|
||||
# Audita Validators
|
||||
|
||||
## Scope
|
||||
|
||||
This document describes validator composition, execution order, and decision handling.
|
||||
|
||||
## Ownership
|
||||
|
||||
Built-in validator keys and chains:
|
||||
- `internal/validators`
|
||||
|
||||
Shared validator runtime mechanics:
|
||||
- `internal/framework/validators`
|
||||
|
||||
Execution-class metadata:
|
||||
- `internal/validators/metadata`
|
||||
|
||||
## Built-In Validator Keys
|
||||
|
||||
Deterministic:
|
||||
- `proposal_shape`
|
||||
- `confidence_threshold`
|
||||
- `original_text_presence`
|
||||
- `non_empty_corrected_text`
|
||||
- `no_effect`
|
||||
- `protected_terms`
|
||||
|
||||
LLM-backed:
|
||||
- `spoken_form_plausibility`
|
||||
- `meaning_reversal_review`
|
||||
- `editorial_review`
|
||||
|
||||
## Built-In Chains
|
||||
|
||||
Module chains are defined in `internal/validators/chains.go`.
|
||||
Glossary, homophones, spoken_word, and grammar each resolve a fixed ordered chain.
|
||||
|
||||
## Runtime Execution
|
||||
|
||||
For each module section:
|
||||
1. run deterministic validators;
|
||||
2. run LLM-backed validators;
|
||||
3. record decisions and warnings;
|
||||
4. carry only approved proposals forward.
|
||||
|
||||
Decision cardinality is enforced: each candidate proposal must receive exactly one decision per validator.
|
||||
|
||||
## LLM Validator Batching
|
||||
|
||||
LLM validators:
|
||||
- build canonical validation request payloads;
|
||||
- batch by `validation_max_prompt_tokens`;
|
||||
- call structured LLM client using response schema registry.
|
||||
|
||||
Oversized single proposals are rejected with `validator_input_too_large`.
|
||||
Malformed LLM validator responses are downgraded to warnings and rejected batch decisions.
|
||||
|
||||
## Decision and Rejection Reporting
|
||||
|
||||
Runner records:
|
||||
- `validator_decisions`
|
||||
- `validator_rejected`
|
||||
- warning records (including malformed response warnings)
|
||||
|
||||
Correction ledger classifies deterministic vs LLM validator decisions using canonical metadata classes.
|
||||
|
||||
## Key Tests
|
||||
|
||||
- `internal/validators/*_test.go`
|
||||
- `internal/framework/validators/*_test.go`
|
||||
- `internal/framework/processreport/correction_ledger_test.go`
|
||||
- `internal/cli/run_test.go`
|
||||
Reference in New Issue
Block a user