Refresh architecture and configuration documentation for current runtime behavior
This commit is contained in:
File diff suppressed because it is too large
Load Diff
@@ -1,31 +1,25 @@
|
||||
# Audita Public Contract
|
||||
|
||||
This document defines stability expectations for Audita's external process and data interfaces.
|
||||
|
||||
## Scope
|
||||
This document defines stability expectations for Audita's external runtime interfaces.
|
||||
|
||||
This contract covers:
|
||||
- CLI invocation and behavior
|
||||
- versioned config file behavior
|
||||
- transcript/glossary input forms
|
||||
- transcript output schema selection
|
||||
- process report schema metadata
|
||||
- stable validator key identifiers in report/diagnostics records
|
||||
- prompt metadata identifiers in diagnostics
|
||||
- diagnostics directory behavior
|
||||
- utilization diagnostics and correction-ledger artifact presence/pathing in diagnostics metadata
|
||||
- stdout/stderr and exit-code behavior
|
||||
- secret redaction guarantees
|
||||
- compatibility and deprecation policy
|
||||
|
||||
## CLI stability expectations
|
||||
Covered interfaces:
|
||||
- CLI commands and major flags;
|
||||
- versioned config behavior and precedence;
|
||||
- transcript/glossary input forms;
|
||||
- output schema selection;
|
||||
- report schema metadata;
|
||||
- diagnostics artifact path metadata;
|
||||
- stdout/stderr and exit-code behavior;
|
||||
- redaction guarantees.
|
||||
|
||||
## CLI contract
|
||||
Stable commands:
|
||||
- `audita process`
|
||||
- `audita config validate`
|
||||
- `audita config print-effective`
|
||||
|
||||
For `audita process`, stable high-value flags include:
|
||||
Stable high-value `process` flags:
|
||||
- `--config`
|
||||
- `--glossary`
|
||||
- `--output`
|
||||
@@ -33,136 +27,95 @@ For `audita process`, stable high-value flags include:
|
||||
- `--modules`
|
||||
- `--output-schema`
|
||||
|
||||
Compatibility flags and lower-level tuning flags remain available; they may be narrowed over time with explicit compatibility notes.
|
||||
## Config contract
|
||||
Supported config format:
|
||||
- YAML;
|
||||
- `version: 1`;
|
||||
- strict unknown-field rejection.
|
||||
|
||||
## Config file stability expectations
|
||||
Path resolution for `process` and `config print-effective`:
|
||||
1. `--config`
|
||||
2. `AUDITA_CONFIG`
|
||||
3. `/usr/local/etc/audita/config.yml`
|
||||
4. `/etc/audita/config.yml`
|
||||
|
||||
Supported file format:
|
||||
- YAML
|
||||
- strict unknown-field rejection
|
||||
- explicit `version`
|
||||
Missing explicit path is an error. Missing default paths is non-fatal.
|
||||
|
||||
Supported version:
|
||||
- `version: 1`
|
||||
|
||||
Precedence for `audita process`:
|
||||
1. built-in defaults
|
||||
Precedence for `process`:
|
||||
1. defaults
|
||||
2. file config
|
||||
3. environment overrides
|
||||
4. CLI overrides
|
||||
|
||||
Config source behavior:
|
||||
- `--config <path>`: missing path is a clear failure
|
||||
- `AUDITA_CONFIG`: missing path is a clear failure
|
||||
- defaults `/usr/local/etc/audita/config.yml`, then `/etc/audita/config.yml`: both missing is non-fatal
|
||||
`config validate` remains file-only validation (defaults + file config; no env overrides).
|
||||
|
||||
## Supported transcript input forms
|
||||
Module and output-schema keys are validated against built-in catalogs. Unknown keys fail validation.
|
||||
|
||||
Audita accepts transcript JSON as either:
|
||||
- a top-level array of segments
|
||||
- an object with a `segments` array
|
||||
## Input contract
|
||||
Supported transcript JSON top-level forms:
|
||||
- array of segments
|
||||
- object with `segments` array
|
||||
|
||||
Segments must satisfy the schema and validation rules enforced by `internal/core/schema`.
|
||||
Supported glossary YAML form:
|
||||
- top-level `glossary` list with required entry fields validated by schema parsing.
|
||||
|
||||
## Supported glossary input form
|
||||
|
||||
Audita accepts glossary YAML with a top-level `glossary` entry list and validates required fields per entry.
|
||||
|
||||
## Supported output schema names
|
||||
|
||||
Built-in output schema registry supports:
|
||||
## Output schema contract
|
||||
Supported transcript output schemas:
|
||||
- `bare-segments` (default)
|
||||
- `audita-v1`
|
||||
|
||||
`seriatim-intermediate` is planned but not implemented.
|
||||
Unknown schema keys fail before output write.
|
||||
|
||||
Unknown output schema names fail clearly.
|
||||
|
||||
## Report schema/versioning expectations
|
||||
|
||||
Process report payloads include `report_metadata` with:
|
||||
## Report metadata contract
|
||||
Process reports include stable report metadata fields:
|
||||
- `report_schema_name`
|
||||
- `report_schema_version`
|
||||
- `output_schema`
|
||||
- `config_version` when file config is used
|
||||
- `config_version` (when file config is loaded)
|
||||
|
||||
Current values:
|
||||
- `report_schema_name`: `audita-process-report`
|
||||
- `report_schema_version`: `v1`
|
||||
- `report_schema_name = audita-process-report`
|
||||
- `report_schema_version = v1`
|
||||
|
||||
`--report-json` output and diagnostics run-dir `report.json` use the same report schema metadata.
|
||||
`--report-json` output and run-directory `report.json` use the same report schema metadata.
|
||||
|
||||
Validator decision/rejection records in reports use stable validator keys in `validator_name`.
|
||||
Module results may also include warning records for malformed module-stage LLM payloads.
|
||||
Report diagnostics metadata includes artifact-path fields for utilization diagnostics and correction ledger when diagnostics initialization succeeds.
|
||||
Validator decision/rejection records use stable validator keys via `validator_name`.
|
||||
|
||||
## Diagnostics directory behavior
|
||||
## Diagnostics metadata contract
|
||||
When run-directory initialization succeeds, diagnostics metadata paths reference stable artifacts, including:
|
||||
- transcript and normalization artifacts;
|
||||
- chunking summary;
|
||||
- invocation metadata;
|
||||
- redacted effective config;
|
||||
- utilization diagnostics;
|
||||
- correction ledger;
|
||||
- `error.log` on failures.
|
||||
|
||||
When diagnostics directory creation succeeds, Audita writes run artifacts including:
|
||||
- invocation metadata
|
||||
- redacted effective config
|
||||
- transcript/normalization/chunking artifacts
|
||||
- utilization diagnostics (`utilization-diagnostics.json`)
|
||||
- correction ledger (`correction-ledger.json`)
|
||||
- report and failure error log (when applicable)
|
||||
- module/LLM diagnostics artifacts as available
|
||||
LLM interaction diagnostics include stable prompt and structured-schema identifiers where applicable.
|
||||
|
||||
Retention behavior is controlled by configured retention mode; failed runs are retained.
|
||||
## Stdout/stderr and exit codes
|
||||
Success:
|
||||
- with `--output`, stdout is empty;
|
||||
- without `--output`, stdout contains transcript JSON only;
|
||||
- report JSON is not written to stdout.
|
||||
|
||||
Diagnostics metadata for LLM interactions may include semi-public prompt identifiers:
|
||||
- `prompt_id`
|
||||
- `prompt_version`
|
||||
- `prompt_source`
|
||||
- `embedded_path`
|
||||
- `sha256`
|
||||
Failures:
|
||||
- nonzero exit;
|
||||
- human-readable stderr summary;
|
||||
- diagnostics directory path on stderr when available.
|
||||
|
||||
These are diagnostic identifiers, not user-facing prompt override controls.
|
||||
Exit codes:
|
||||
- `0` success
|
||||
- nonzero failure
|
||||
|
||||
## Stdout/stderr behavior
|
||||
## Redaction contract
|
||||
Configured secrets are redacted from:
|
||||
- effective config outputs;
|
||||
- diagnostics artifacts;
|
||||
- report artifacts;
|
||||
- surfaced adapter/runtime errors.
|
||||
|
||||
Success behavior:
|
||||
- with `--output`, stdout is empty
|
||||
- without `--output`, stdout contains only transcript JSON in selected output schema
|
||||
- report JSON is not written to stdout
|
||||
- success stderr remains empty even when reports/diagnostics contain module warnings
|
||||
## Compatibility policy
|
||||
Stable command behavior, schema names, report metadata keys, diagnostics-path field semantics, and validator key identities are treated as public contract.
|
||||
|
||||
Failure behavior:
|
||||
- stderr contains human-readable error summary
|
||||
- nonzero exit
|
||||
- diagnostics path is printed when available
|
||||
|
||||
## Exit-code behavior
|
||||
|
||||
- `0`: success
|
||||
- nonzero: failure
|
||||
|
||||
Treat any nonzero exit as a failed invocation.
|
||||
|
||||
## Secret redaction guarantees
|
||||
|
||||
Audita redacts API keys and authorization secrets from:
|
||||
- effective config outputs (`audita config print-effective`, diagnostics effective-config artifact)
|
||||
- report artifacts
|
||||
- LLM diagnostics artifacts
|
||||
- surfaced request/response error messages
|
||||
|
||||
Config files should reference secrets via environment variable names (`api_key_env`) rather than embedding secret values.
|
||||
|
||||
## Compatibility and deprecation policy
|
||||
|
||||
- Existing stable schema names, report metadata keys, and top-level command behavior are treated as public contract.
|
||||
- Existing stable validator keys remain public contract values even when validator semantics are refined.
|
||||
- Compatibility inputs (legacy flags/env aliases) may remain during transition windows.
|
||||
- Any planned removal or behavior change should include clear compatibility notes and migration guidance.
|
||||
|
||||
## Breaking changes after 1.0
|
||||
|
||||
After 1.0, breaking changes include, for example:
|
||||
- changing default success/failure exit-code semantics
|
||||
- changing stdout/stderr routing semantics
|
||||
- silently changing default output schema shape
|
||||
- removing supported output schema names without compatibility strategy
|
||||
- changing report schema fields or meanings incompatibly
|
||||
- changing config version semantics incompatibly without version bump
|
||||
|
||||
Additive fields, additive diagnostics, and new optional schema names are generally non-breaking when existing behavior remains intact.
|
||||
Additive fields are acceptable when existing fields and behavior remain compatible.
|
||||
|
||||
@@ -1,90 +1,72 @@
|
||||
# Structured LLM Architecture
|
||||
|
||||
## Purpose
|
||||
|
||||
## Scope
|
||||
This document describes Audita's structured LLM runtime boundary and adapter behavior.
|
||||
|
||||
## Why Audita owns the adapter
|
||||
|
||||
Audita owns a small structured LLM adapter so that core runtime behavior is controlled inside the repository:
|
||||
- request construction and schema handling are explicit and testable;
|
||||
- retries, timeouts, cancellation, and error redaction are consistent across modules and validators;
|
||||
- provider SDK types are not exposed outside the adapter boundary;
|
||||
- dependency weight and transitive provider-specific behavior are reduced.
|
||||
|
||||
At runtime, the rest of Audita depends only on the internal contract:
|
||||
- `StructuredLLMClient`
|
||||
## Runtime boundary
|
||||
Production LLM integration depends on the internal contract only:
|
||||
- `contracts.StructuredLLMClient`
|
||||
- `CompleteStructured(ctx, req, out)`
|
||||
|
||||
## OpenAI-compatible request shape
|
||||
Provider SDK types do not leak past this boundary.
|
||||
|
||||
At a conceptual level, Audita sends chat completion requests with:
|
||||
- `model`
|
||||
- `messages` (role/content pairs)
|
||||
- `response_format`:
|
||||
- `type = "json_schema"`
|
||||
- `json_schema.name` (stable schema name)
|
||||
- `json_schema.strict = true`
|
||||
- `json_schema.schema` (registered JSON Schema payload)
|
||||
## Adapter ownership
|
||||
`internal/framework/llm` owns the OpenAI-compatible HTTP adapter and shared LLM runtime utilities.
|
||||
|
||||
The adapter uses OpenAI-compatible `POST {base_url}/chat/completions` over `net/http`.
|
||||
Key responsibilities:
|
||||
- request assembly;
|
||||
- timeout/cancellation propagation;
|
||||
- bounded retry behavior;
|
||||
- scheduler integration;
|
||||
- provider response decoding;
|
||||
- error redaction.
|
||||
|
||||
## Structured response schema registry
|
||||
## Structured schema registry
|
||||
Structured response schemas are registered in `internal/framework/responseschema` and include stable metadata:
|
||||
- `id`
|
||||
- `version`
|
||||
- `name`
|
||||
- `json_schema`
|
||||
- `sha256`
|
||||
|
||||
Structured response schemas are registered in `internal/framework/responseschema` with stable metadata:
|
||||
- schema key
|
||||
- schema ID
|
||||
- schema version
|
||||
- schema name (OpenAI-compatible `response_format` name)
|
||||
- raw JSON Schema payload
|
||||
- SHA-256 hash
|
||||
Current schema keys:
|
||||
- `correction_set`
|
||||
- `validator_decision_set`
|
||||
|
||||
Current schemas:
|
||||
- `correction_set`:
|
||||
- id `audita.correction_set`
|
||||
- version `v1`
|
||||
- name `audita_correction_set_v1`
|
||||
- `validator_decision_set`:
|
||||
- id `audita.validator_decision_set`
|
||||
- version `v1`
|
||||
- name `audita_validator_decision_set_v1`
|
||||
Schema metadata is attached to diagnostics through `Schema.DiagnosticsMap()`.
|
||||
|
||||
## Provider compatibility assumptions
|
||||
## Request shape assumptions
|
||||
Audita targets OpenAI-compatible chat-completions endpoints and sends structured requests with:
|
||||
- model;
|
||||
- chat messages;
|
||||
- `response_format.type = json_schema`;
|
||||
- schema name and JSON schema payload.
|
||||
|
||||
Audita assumes an OpenAI-compatible chat-completions endpoint that:
|
||||
- accepts message arrays with model selection;
|
||||
- accepts `response_format.type = json_schema`;
|
||||
- returns a completion with assistant message content and optional usage metadata.
|
||||
## Local validation remains mandatory
|
||||
Provider schema enforcement is treated as transport-level guardrails.
|
||||
|
||||
Provider-specific differences are expected in strictness and error payload shapes, so the adapter treats provider output as untrusted until locally decoded.
|
||||
Audita still validates output locally before applying behavior changes:
|
||||
- proposal decoding and proposal invariants;
|
||||
- validator decision decoding and cardinality checks;
|
||||
- deterministic validation and apply-time rules.
|
||||
|
||||
## Local decode and validation remain mandatory
|
||||
## Shared malformed-output policy
|
||||
Malformed structured-output classification is centralized in `internal/framework/structuredoutput`.
|
||||
|
||||
Provider-level structured output is a transport guardrail, not final validation.
|
||||
Proposal generation and validator execution both use this shared classifier so downgrade behavior cannot drift between the two paths.
|
||||
|
||||
After receiving a response, Audita still:
|
||||
- decodes assistant content into typed request-specific structs;
|
||||
- validates proposal and validator payload invariants locally;
|
||||
- enforces deterministic validator/cardinality rules before any transcript application.
|
||||
## Secrets and redaction
|
||||
Secret extraction for LLM redaction is centralized in `llm.ConfiguredSecrets(cfg)` and reused by proposal and validator diagnostics writers.
|
||||
|
||||
This protects runtime correctness even when provider responses are malformed, partial, or semantically inconsistent.
|
||||
Secrets are redacted from:
|
||||
- diagnostics artifacts;
|
||||
- report artifacts;
|
||||
- surfaced adapter/runtime errors.
|
||||
|
||||
## Diagnostics and redaction
|
||||
## Concurrency and scheduling
|
||||
LLM execution is constrained by composed scheduler limits:
|
||||
- total LLM concurrency;
|
||||
- proposal LLM concurrency;
|
||||
- validation LLM concurrency.
|
||||
|
||||
When structured schemas are used, diagnostics metadata records:
|
||||
- schema ID
|
||||
- schema version
|
||||
- schema name
|
||||
- schema hash
|
||||
|
||||
Diagnostics and surfaced errors preserve secret redaction:
|
||||
- API keys and bearer tokens are redacted from request/response/error artifacts;
|
||||
- redaction is applied before diagnostic files are written.
|
||||
|
||||
## Runtime behavior guarantees
|
||||
|
||||
The structured LLM path preserves existing runtime guarantees:
|
||||
- bounded LLM call execution through schedulers;
|
||||
- context-aware cancellation and timeout propagation;
|
||||
- retry behavior for transient failures and retryable malformed structured responses;
|
||||
- deterministic module/chunk/proposal/validator behavior outside provider nondeterminism.
|
||||
The scheduler is FIFO and context-aware so permits are released on success, failure, and cancellation.
|
||||
|
||||
@@ -1,160 +1,96 @@
|
||||
# Audita Validators
|
||||
|
||||
This document describes Audita's built-in validator registry and module validator chains.
|
||||
|
||||
For LLM-backed validator prompt asset details, see [`docs/prompts.md`](prompts.md).
|
||||
|
||||
## Package ownership
|
||||
|
||||
Built-in validator construction is package-owned under `internal/validators/<validator_key>`:
|
||||
- `internal/validators/confidence_threshold`
|
||||
- `internal/validators/proposal_shape`
|
||||
- `internal/validators/original_text_presence`
|
||||
- `internal/validators/non_empty_corrected_text`
|
||||
- `internal/validators/no_effect`
|
||||
- `internal/validators/protected_terms`
|
||||
- `internal/validators/spoken_form_plausibility`
|
||||
- `internal/validators/meaning_reversal_review`
|
||||
- `internal/validators/editorial_review`
|
||||
|
||||
Registry and chain wiring stay in:
|
||||
- `internal/validators/registry.go`
|
||||
- `internal/validators/chains.go`
|
||||
|
||||
Shared validator runtime mechanics stay in `internal/framework/validators`:
|
||||
- request/result/decision models
|
||||
- decision cardinality helpers
|
||||
- protected vocabulary helpers
|
||||
- shared LLM validator runtime, batching, and diagnostics helpers
|
||||
|
||||
Execution classification metadata is defined in `internal/validators/metadata`:
|
||||
- `deterministic`
|
||||
- `llm_backed`
|
||||
|
||||
Runner ordering uses this metadata so deterministic validators run before LLM-backed validators without concrete framework type assertions.
|
||||
|
||||
## Scope
|
||||
This document defines the built-in validator system used by production module runs.
|
||||
|
||||
Validator chains are built-in runtime behavior.
|
||||
## Ownership boundaries
|
||||
Built-in validator keys, constructors, and module chains are owned by `internal/validators`.
|
||||
|
||||
Current 1.0 boundary:
|
||||
- built-in validator keys and built-in module chains are stable runtime identifiers;
|
||||
- thresholds and batching knobs remain configurable where already supported;
|
||||
- arbitrary user-defined validator chains are deferred.
|
||||
Shared runtime execution mechanics are owned by `internal/framework/validators`, including:
|
||||
- validator request/result models;
|
||||
- deterministic proposal checks;
|
||||
- LLM validator batching and execution;
|
||||
- decision-cardinality enforcement;
|
||||
- diagnostics integration.
|
||||
|
||||
## Built-in validator keys
|
||||
|
||||
### Deterministic validators
|
||||
Execution class metadata is owned by `internal/validators/metadata`.
|
||||
|
||||
## Stable validator keys
|
||||
Deterministic:
|
||||
- `proposal_shape`
|
||||
- rejects malformed proposal fields before other validators run.
|
||||
- `confidence_threshold`
|
||||
- checks proposal confidence against module-specific configured threshold.
|
||||
- `original_text_presence`
|
||||
- ensures target segment exists and `original_text` exists in current working segment text.
|
||||
- `non_empty_corrected_text`
|
||||
- rejects proposals whose previewed resulting segment text would be empty or whitespace-only.
|
||||
- `no_effect`
|
||||
- rejects proposals where `original_text == corrected_text`.
|
||||
- `protected_terms`
|
||||
- protects glossary-derived terms from unsafe mutations in non-glossary modules.
|
||||
- glossary stages use glossary-specific protection logic but still report this same stable key.
|
||||
|
||||
### LLM-backed validators
|
||||
|
||||
LLM-backed:
|
||||
- `spoken_form_plausibility`
|
||||
- checks whether proposed spoken-form change remains plausible in transcript context.
|
||||
- `meaning_reversal_review`
|
||||
- checks for likely meaning reversal or semantic contradiction.
|
||||
- `editorial_review`
|
||||
- performs conservative editorial safety review.
|
||||
|
||||
## Built-in module chains
|
||||
`glossary`:
|
||||
- `proposal_shape`
|
||||
- `no_effect`
|
||||
- `original_text_presence`
|
||||
- `confidence_threshold`
|
||||
- `protected_terms`
|
||||
- `non_empty_corrected_text`
|
||||
- `spoken_form_plausibility`
|
||||
- `meaning_reversal_review`
|
||||
|
||||
Current built-in chains resolved from `internal/validators/chains.go`:
|
||||
`homophones`:
|
||||
- `proposal_shape`
|
||||
- `no_effect`
|
||||
- `original_text_presence`
|
||||
- `confidence_threshold`
|
||||
- `protected_terms`
|
||||
- `non_empty_corrected_text`
|
||||
- `spoken_form_plausibility`
|
||||
- `meaning_reversal_review`
|
||||
|
||||
- `glossary`
|
||||
- `proposal_shape`
|
||||
- `no_effect`
|
||||
- `original_text_presence`
|
||||
- `confidence_threshold`
|
||||
- `protected_terms`
|
||||
- `non_empty_corrected_text`
|
||||
- `spoken_form_plausibility`
|
||||
- `meaning_reversal_review`
|
||||
`spoken_word`:
|
||||
- `proposal_shape`
|
||||
- `no_effect`
|
||||
- `original_text_presence`
|
||||
- `confidence_threshold`
|
||||
- `protected_terms`
|
||||
- `non_empty_corrected_text`
|
||||
- `editorial_review`
|
||||
- `meaning_reversal_review`
|
||||
|
||||
- `homophones`
|
||||
- `proposal_shape`
|
||||
- `no_effect`
|
||||
- `original_text_presence`
|
||||
- `confidence_threshold`
|
||||
- `protected_terms`
|
||||
- `non_empty_corrected_text`
|
||||
- `spoken_form_plausibility`
|
||||
- `meaning_reversal_review`
|
||||
`grammar`:
|
||||
- `proposal_shape`
|
||||
- `no_effect`
|
||||
- `original_text_presence`
|
||||
- `confidence_threshold`
|
||||
- `protected_terms`
|
||||
- `non_empty_corrected_text`
|
||||
- `editorial_review`
|
||||
- `meaning_reversal_review`
|
||||
|
||||
- `spoken_word`
|
||||
- `proposal_shape`
|
||||
- `no_effect`
|
||||
- `original_text_presence`
|
||||
- `confidence_threshold`
|
||||
- `protected_terms`
|
||||
- `non_empty_corrected_text`
|
||||
- `editorial_review`
|
||||
- `meaning_reversal_review`
|
||||
## Ordering and execution semantics
|
||||
Validator ordering is based on canonical metadata:
|
||||
- deterministic validators run before LLM-backed validators.
|
||||
|
||||
- `grammar`
|
||||
- `proposal_shape`
|
||||
- `no_effect`
|
||||
- `original_text_presence`
|
||||
- `confidence_threshold`
|
||||
- `protected_terms`
|
||||
- `non_empty_corrected_text`
|
||||
- `editorial_review`
|
||||
- `meaning_reversal_review`
|
||||
Within each module stage:
|
||||
- proposals are generated per section;
|
||||
- validator chains execute on those proposals;
|
||||
- approved proposals are applied once after section work settles.
|
||||
|
||||
## Protected terms construction
|
||||
## Malformed payload behavior
|
||||
Malformed structured-output from proposal generation and LLM validator calls is downgraded, not treated as a process-fatal transport error.
|
||||
|
||||
`protected_terms` has explicit constructors:
|
||||
- general constructor used by non-glossary modules through the built-in registry
|
||||
- glossary-stage constructor used by glossary chain resolution
|
||||
Current outcomes:
|
||||
- malformed proposal-generation payloads produce section/module warnings and zero proposals for the affected section;
|
||||
- malformed validator decision payloads reject the affected validator batch with warnings;
|
||||
- deterministic validator behavior and runner order remain unchanged.
|
||||
|
||||
Both variants preserve existing behavior and report the stable key `protected_terms`.
|
||||
## Reporting identity
|
||||
Reports and diagnostics use stable validator keys as identifiers.
|
||||
|
||||
## Execution semantics
|
||||
Correction-ledger deterministic-vs-LLM classification is derived from canonical validator metadata, not package-local hardcoded maps.
|
||||
|
||||
- modules execute serially;
|
||||
- section proposal work can run concurrently within a module;
|
||||
- deterministic validators run before LLM-backed validators;
|
||||
- malformed module proposal payloads are downgraded to section-scoped module warnings with zero proposals for the affected section rather than module failure;
|
||||
- malformed/missing/duplicate/unknown LLM validator decisions reject the affected validator batch with warnings instead of failing the module;
|
||||
- oversized single-proposal validator inputs reject only the affected proposal under that validator;
|
||||
- approved proposals are applied once per module after section work settles.
|
||||
|
||||
## Validator rejections vs proposal-application skips
|
||||
|
||||
- validator rejection:
|
||||
- proposal is denied by validator-chain review and appears in validator rejection reporting with validator key and reason code.
|
||||
- proposal-application skip:
|
||||
- proposal passed validators but could not be applied under replacement-policy semantics (for example no matching span at apply time).
|
||||
- module warning:
|
||||
- malformed proposal-generation payloads and malformed validator batches are recorded in module warning records and diagnostics without writing success stderr.
|
||||
|
||||
These are separate outcomes and are reported separately.
|
||||
|
||||
## Reporting and diagnostics identity
|
||||
|
||||
- report validator decision/rejection entries use stable validator keys in `validator_name`.
|
||||
- report module results include warning records for malformed module-stage LLM payloads.
|
||||
- validator LLM diagnostics include validator identity in interaction metadata and structured response schema metadata.
|
||||
- correction ledger entries include deterministic and LLM validator decision snapshots keyed by the same stable validator keys, and keep validator rejection distinct from application-level skip.
|
||||
|
||||
Prompt assets are unchanged by the validator package-ownership refactor and remain built-in under `internal/prompts`.
|
||||
|
||||
## Configurable knobs that remain supported
|
||||
|
||||
- per-module confidence thresholds (`thresholds.*` / equivalent env+CLI overrides)
|
||||
- validation batching limits (`validation_max_prompt_tokens` / equivalent env+CLI overrides)
|
||||
- validation LLM model/base URL/timeout/retries/concurrency settings
|
||||
|
||||
These tune validator behavior without exposing arbitrary user-defined chains.
|
||||
## Prompt assets
|
||||
LLM validator prompt assets and prompt metadata are documented in [Prompts](./prompts.md).
|
||||
|
||||
Reference in New Issue
Block a user