Refresh documentation and policies and simplify releases
This commit is contained in:
64
docs/internal/cli-config.md
Normal file
64
docs/internal/cli-config.md
Normal file
@@ -0,0 +1,64 @@
|
||||
# Configuration and CLI Change Recipes
|
||||
|
||||
This document owns contributor editing and validation steps for configuration,
|
||||
CLI wiring, and maintained examples. Definitions of behavior belong to the
|
||||
[configuration](../config.md) and [CLI](../cli.md) contracts.
|
||||
|
||||
## Changing Configuration
|
||||
|
||||
1. Update fields, defaults, loading, and validation in `internal/core/config`.
|
||||
2. Update file, environment, and CLI mapping as applicable, including CLI flag
|
||||
extraction. Preserve the source-specific behavior defined in the
|
||||
[configuration contract](../config.md#effective-precedence), including the
|
||||
distinct validation and effective-config commands.
|
||||
3. Update redaction when secrets are involved.
|
||||
4. Review configuration tests and related CLI tests for precedence, validation,
|
||||
and source-specific behavior. Add or adjust tests for the changed contract
|
||||
according to the [testing policy](../policy/testing.md).
|
||||
5. Update the configuration reference and affected [examples](../../examples/).
|
||||
|
||||
## Changing CLI Behavior
|
||||
|
||||
1. Implement parsing and wiring in `internal/cli`.
|
||||
2. Preserve [stream and exit contracts](../cli.md) unless a change is intentional
|
||||
and documented. Review [subprocess integration](../integrations/subprocess.md)
|
||||
when changing output, error, or report handling.
|
||||
3. Review `internal/cli` tests and `cmd/audita` subprocess tests; update coverage
|
||||
for the changed behavior.
|
||||
4. Update the CLI reference and affected integration guidance.
|
||||
|
||||
## Validating Examples
|
||||
|
||||
Keep complete copyable files in [examples](../../examples/) and link to them from
|
||||
their canonical contract documentation. Keep them valid and secret-free.
|
||||
From the repository root, validate the maintained configuration examples:
|
||||
|
||||
```sh
|
||||
go run ./cmd/audita config validate --config examples/minimal-config.yml
|
||||
go run ./cmd/audita config validate --config examples/production-config.yml
|
||||
go run ./cmd/audita config print-effective --config examples/minimal-config.yml
|
||||
```
|
||||
|
||||
The effective-config command incorporates environment settings; use a controlled
|
||||
environment when comparing its results. Check affected CLI examples against the
|
||||
parser and use schema tests for maintained transcript/glossary inputs. Example
|
||||
validation should not invoke a live LLM merely to check documentation.
|
||||
|
||||
## Runtime Change Checks
|
||||
|
||||
Select checks according to the [testing policy](../policy/testing.md):
|
||||
|
||||
- Configuration or CLI changes: inspect `internal/core/config` tests,
|
||||
`internal/cli` tests, and `cmd/audita` integration tests.
|
||||
- Process/output changes: verify stdout, stderr, exit, output-file, and report
|
||||
behavior against the [CLI contract](../cli.md) and
|
||||
[subprocess guidance](../integrations/subprocess.md), including failures.
|
||||
- Report or diagnostics changes: review [report assembly tests and redaction
|
||||
checks](diagnostics-reporting.md#key-tests), metadata fields, and artifact paths.
|
||||
- Schema or serialization changes: review [output-schema tests](output-schemas.md#key-tests)
|
||||
and `internal/core/schema` tests, including supported formats and rejection of
|
||||
unsupported schemas.
|
||||
- Changes affecting assembled output: inspect CLI parity and release fixtures.
|
||||
|
||||
Use the [contributor guide](../development.md#baseline-validation) for baseline
|
||||
commands. These recipes supplement, rather than duplicate, public contracts.
|
||||
@@ -1,74 +1,69 @@
|
||||
# Audita Diagnostics and Reporting
|
||||
# Diagnostics and Report Assembly
|
||||
|
||||
## Scope
|
||||
|
||||
This document describes diagnostics artifacts, process report mapping, and correction ledger generation.
|
||||
This document owns implementation flow and collaborators. Physical artifacts and
|
||||
retention belong to [operations](../operations.md). Serialized reports, ledger
|
||||
records, and utilization shapes belong to the
|
||||
[output-file contract](../integrations/output-files.md).
|
||||
|
||||
## Run Directory Ownership
|
||||
## Run Directory and Artifact Writers
|
||||
|
||||
`internal/core/diagnostics` owns run-directory creation, artifact writes, and retention decisions.
|
||||
`internal/core/diagnostics` creates run directories, writes snapshots and metadata,
|
||||
and evaluates retention. CLI orchestration supplies the selected configuration,
|
||||
paths, run outcome, and correction-presence information. Artifact names are
|
||||
centralized in diagnostics; their operator-facing inventory is in
|
||||
[operations](../operations.md#diagnostics-directory).
|
||||
|
||||
Stable artifact names include:
|
||||
- `source-transcript.json`
|
||||
- `source-transcript-parsed.json`
|
||||
- `normalized-transcript.json`
|
||||
- `normalization-summary.json`
|
||||
- `chunking-summary.json`
|
||||
- `utilization-diagnostics.json`
|
||||
- `correction-ledger.json`
|
||||
- `invocation.json`
|
||||
- `effective-config.json`
|
||||
- `report.json`
|
||||
- `error.log` (failure)
|
||||
Snapshot writes are attempted at the relevant processing phases. Several diagnostic
|
||||
writes are best-effort, so report metadata can identify a location without proving
|
||||
that a file was written. The CLI distinguishes primary processing errors from
|
||||
secondary diagnostic/report write errors.
|
||||
|
||||
## Process Report Mapping
|
||||
|
||||
`internal/framework/processreport` maps runner/CLI execution facts into `reporting.ProcessReport`.
|
||||
`internal/framework/processreport.Build` maps invocation facts, normalization and
|
||||
chunking summaries, and runner output into `reporting.ProcessReport`. Core reporting
|
||||
owns the Go data structures and JSON writer. The builder attaches diagnostic
|
||||
metadata when a run-directory path is available.
|
||||
|
||||
Report metadata fields include:
|
||||
- `report_schema_name` (`audita-process-report`)
|
||||
- `report_schema_version` (`v1`)
|
||||
- `output_schema`
|
||||
- `config_version` (when file config exists)
|
||||
Module mapping copies outcomes from runner results, collects summary counts, and
|
||||
preserves available partial module results on failure. The CLI failure path does
|
||||
not pass normalization or chunking summaries to report construction even when
|
||||
some earlier processing completed. Consumer interpretation belongs to the
|
||||
[process-report contract](../integrations/output-files.md#process-report).
|
||||
|
||||
The report includes:
|
||||
- top-level status/error phase/error message;
|
||||
- normalization/chunking summaries;
|
||||
- diagnostics metadata paths;
|
||||
- per-module results and module summary.
|
||||
## Correction Ledger and Utilization
|
||||
|
||||
## Correction Ledger
|
||||
`BuildCorrectionLedger` flattens module changes, rejections, and failures into
|
||||
ledger entries. It associates validator records by proposal index within each
|
||||
module, separates execution classes through validator metadata, and sorts entries
|
||||
by module instance, proposal index, and disposition.
|
||||
|
||||
`internal/framework/processreport/BuildCorrectionLedger` flattens run results into `correction-ledger.json` entries.
|
||||
Runner instrumentation collects utilization; CLI orchestration writes it when
|
||||
run output is available. Report assembly does not independently measure calls.
|
||||
For the serialized forms see [ledger](../integrations/output-files.md#correction-ledger)
|
||||
and [utilization](../integrations/output-files.md#utilization-diagnostics).
|
||||
|
||||
Dispositions:
|
||||
- `applied`
|
||||
- `skipped`
|
||||
- `rejected`
|
||||
- `failed`
|
||||
## CLI Coordination and Retention
|
||||
|
||||
Validator decisions are split into deterministic and LLM-backed groups using validator metadata classification.
|
||||
CLI orchestration writes caller-selected reports and run-local artifacts before
|
||||
asking diagnostics to apply retention. `HasSkippedCorrections` derives the
|
||||
retention input from runner outcomes. Keep the exact lifecycle definition in
|
||||
[operations](../operations.md#retention-behavior), not in another internal table.
|
||||
|
||||
## Report Write Paths
|
||||
|
||||
- run directory always attempts to write `report.json` when possible;
|
||||
- optional `--report-json` writes an external report file;
|
||||
- on failure paths, report writing is best-effort and does not mask primary run errors.
|
||||
|
||||
## Retention Interaction
|
||||
|
||||
Current retention behavior:
|
||||
- failed runs are retained;
|
||||
- `always` keeps successful runs;
|
||||
- `auto` removes only clean successful runs;
|
||||
- `never` currently retains successful runs in current implementation.
|
||||
On a processing failure, secondary report-write errors do not replace the original
|
||||
error. On the success path, an external report-write failure returns before the
|
||||
run-local report write; retention errors occur later. This ordering explains why
|
||||
artifacts and process exit status must be assessed together.
|
||||
|
||||
## Redaction
|
||||
|
||||
Redacted data expectations:
|
||||
- effective config artifact uses config redaction;
|
||||
- diagnostics payloads and surfaced errors use LLM secret redaction;
|
||||
- reports should not include raw API key values.
|
||||
Effective configuration uses config redaction. LLM diagnostic and error paths
|
||||
use `llm.ConfiguredSecrets` and the runtime's redaction helpers. Keep these separate
|
||||
mechanisms aligned with the [architecture requirement](../policy/architecture.md#errors-and-secret-handling).
|
||||
Operational handling of transcript-bearing artifacts belongs to
|
||||
[operations](../operations.md#artifact-handling).
|
||||
|
||||
## Key Tests
|
||||
|
||||
|
||||
@@ -11,44 +11,36 @@ All runtime LLM calls go through `contracts.StructuredLLMClient`.
|
||||
Primary adapter:
|
||||
- `internal/framework/llm/OpenAICompatibleClient`
|
||||
|
||||
## Request/Response Behavior
|
||||
## Adapter and Response Schema Registry
|
||||
|
||||
The OpenAI-compatible adapter sends chat completions requests with:
|
||||
- model;
|
||||
- messages;
|
||||
- `response_format.type = json_schema`;
|
||||
- strict schema envelope (`name`, `schema`, `strict=true`).
|
||||
The adapter implements the [LLM wire contract](../integrations/openai-compatible-llm.md).
|
||||
It maps internal structured requests to HTTP messages and decodes provider content
|
||||
into the supplied Go output target. Endpoint, authentication, response shapes, and
|
||||
retry conditions are defined in that contract rather than repeated here.
|
||||
|
||||
The response is decoded into the requested structured output target.
|
||||
`internal/framework/responseschema` stores the per-call response schema definitions
|
||||
and diagnostic metadata. Proposal generation and validation choose their schema
|
||||
before invoking the client. Provider strict mode does not replace local checks.
|
||||
|
||||
## Response Schema Registry
|
||||
## Error Classification
|
||||
|
||||
Structured response schemas are registered in `internal/framework/responseschema`:
|
||||
- `correction_set`
|
||||
- `validator_decision_set`
|
||||
|
||||
Each schema includes stable diagnostics metadata (`id`, `version`, `name`, `sha256`).
|
||||
|
||||
## Retries and Error Handling
|
||||
|
||||
Adapter retries apply to retryable conditions (for example transport/decoding/retryable status classes) up to configured `max_retries`.
|
||||
|
||||
Errors are sanitized to redact configured API-key values before surfacing.
|
||||
|
||||
Malformed structured output detection is shared through `internal/framework/structuredoutput` and is used by:
|
||||
- proposal generation;
|
||||
- LLM-backed validators.
|
||||
The adapter marks retryable errors internally and uses context state and attempt
|
||||
count to decide whether to retry. It sanitizes errors before returning them.
|
||||
`internal/framework/structuredoutput` classifies malformed output for proposal
|
||||
and validator call sites, which can turn those errors into warnings and rejected
|
||||
section/batch work. Other errors propagate through the runner.
|
||||
|
||||
## Scheduling and Concurrency
|
||||
|
||||
`internal/framework/llm/Scheduler` provides FIFO, context-aware permit gating.
|
||||
|
||||
Runner composes scheduler limits across:
|
||||
- total LLM concurrency;
|
||||
- proposal LLM concurrency;
|
||||
- validation LLM concurrency.
|
||||
CLI orchestration composes global and stage schedulers; runner instrumentation
|
||||
measures them. Limit definitions and inheritance belong to
|
||||
[configuration](../config.md#concurrency).
|
||||
|
||||
Scheduler release is guarded to avoid permit leaks on cancellation/error.
|
||||
Scheduler release is guarded to avoid permit leaks on cancellation/error. Runner
|
||||
collects utilization observations; their serialized shape is defined by the
|
||||
[output-file contract](../integrations/output-files.md#utilization-diagnostics).
|
||||
|
||||
## Diagnostics and Redaction
|
||||
|
||||
@@ -58,6 +50,14 @@ Configured secrets are derived from `llm.ConfiguredSecrets(cfg)` and redacted fr
|
||||
- diagnostics payloads;
|
||||
- surfaced runtime/adapter errors.
|
||||
|
||||
## Changing LLM Integration or Scheduling
|
||||
|
||||
Review client, scheduler, redaction, and structured-output malformed-response
|
||||
coverage alongside runner orchestration and utilization tests. Use the
|
||||
[testing policy](../policy/testing.md) to select sufficient checks; preserve the
|
||||
[LLM integration contract](../integrations/openai-compatible-llm.md) and
|
||||
[architecture invariants](../policy/architecture.md#correction-and-concurrency-invariants).
|
||||
|
||||
## Key Tests
|
||||
|
||||
- `internal/framework/llm/openai_compatible_client_test.go`
|
||||
@@ -65,3 +65,4 @@ Configured secrets are derived from `llm.ConfiguredSecrets(cfg)` and redacted fr
|
||||
- `internal/framework/llm/diagnostics_test.go`
|
||||
- `internal/framework/responseschema/registry_test.go`
|
||||
- `internal/framework/structuredoutput/malformed_test.go`
|
||||
- `internal/framework/runner/*_test.go` (orchestration and utilization)
|
||||
|
||||
@@ -2,7 +2,8 @@
|
||||
|
||||
## Scope
|
||||
|
||||
This document covers module contracts and built-in module packages.
|
||||
This document covers internal module contracts and packages. User-selectable keys
|
||||
and the default sequence belong to [configuration](../config.md#pipeline-and-output).
|
||||
|
||||
## Module Contract
|
||||
|
||||
@@ -46,9 +47,26 @@ Module packages call the built-in chain resolver at construction.
|
||||
|
||||
## Failure and Warning Behavior
|
||||
|
||||
- module setup failures surface as `runner_setup` or module setup errors;
|
||||
- module runtime failures surface as `runner_execution` with partial module results preserved;
|
||||
- malformed structured proposal payloads are downgraded to warnings and section-level proposal rejection.
|
||||
Module construction failures propagate to orchestration; execution failures can
|
||||
preserve partial module results. Malformed structured proposal payloads become
|
||||
warnings and section-level rejection. Consumer-visible outcomes are defined by
|
||||
the [report contract](../integrations/output-files.md#module-outcomes).
|
||||
|
||||
## Changing Modules
|
||||
|
||||
1. Add or update the narrowly scoped package under `internal/modules/<module_key>`.
|
||||
2. Keep module-specific prompt ownership in the module and `internal/prompts`;
|
||||
see [prompt rendering and metadata](prompts.md).
|
||||
3. Wire module registration and catalog resolution through framework module
|
||||
registration and the core module catalog. Update selectable keys in the
|
||||
[configuration contract](../config.md) when they change.
|
||||
4. Verify replacement policy and validator-chain selection against the
|
||||
[architecture requirements](../policy/architecture.md#correction-and-concurrency-invariants).
|
||||
5. Review module, prompt-rendering, and proposal-generation tests. Inspect parity
|
||||
and release fixtures when public output changes, choosing sufficient coverage
|
||||
under the [testing policy](../policy/testing.md).
|
||||
6. Update this document for implementation changes and the appropriate public
|
||||
contract owner for externally visible changes.
|
||||
|
||||
## Key Tests
|
||||
|
||||
|
||||
@@ -1,43 +1,31 @@
|
||||
# Audita Output Schemas
|
||||
# Output Encoding Internals
|
||||
|
||||
## Scope
|
||||
## Scope and Ownership
|
||||
|
||||
This document describes the implemented transcript output schema registry.
|
||||
`internal/core/outputschema` owns encoder registration and resolution. External
|
||||
JSON envelopes and segment fields are defined by the
|
||||
[output-file contract](../integrations/output-files.md#corrected-transcript);
|
||||
selectable values and defaults belong to
|
||||
[configuration](../config.md#pipeline-and-output).
|
||||
|
||||
## Registry Ownership
|
||||
## Resolution and Encoding
|
||||
|
||||
Output schema registry is owned by `internal/core/outputschema`.
|
||||
Configuration validation checks registry support. CLI orchestration resolves the
|
||||
selected encoder, passes the final transcript to it, and routes the encoded bytes
|
||||
to the destination defined by the [CLI contract](../cli.md#process-output-and-exit-behavior).
|
||||
An unsupported key fails validation or resolution rather than selecting an
|
||||
implicit fallback.
|
||||
|
||||
Supported schema keys:
|
||||
- `bare-segments`
|
||||
- `audita-v1`
|
||||
The registry delegates bare segment serialization to the transcript schema
|
||||
package and uses an envelope encoder for the versioned form. Report construction
|
||||
receives the selected key separately; see [report assembly](diagnostics-reporting.md).
|
||||
|
||||
## Schemas
|
||||
## Changing an Encoder
|
||||
|
||||
`bare-segments`:
|
||||
- top-level JSON array of transcript segments.
|
||||
|
||||
`audita-v1`:
|
||||
- top-level JSON object with:
|
||||
- `schema: "audita-v1"`
|
||||
- `version: "v1"`
|
||||
- `segments: [...]`
|
||||
|
||||
Segment fields include `id`, `speaker`, `start`, `end`, `text`, and optional `categories`.
|
||||
|
||||
## Validation and Resolution
|
||||
|
||||
Config validation and runtime resolution both reject unsupported schema keys.
|
||||
|
||||
Unknown schema keys fail with `unsupported output schema` before output emission.
|
||||
|
||||
## Output Emission
|
||||
|
||||
The selected schema is used by `audita process` when writing:
|
||||
- output file (`--output`) or
|
||||
- stdout (when no `--output`).
|
||||
|
||||
Report metadata records selected `output_schema`.
|
||||
Update registration, configuration validation, and serialization together. Inspect
|
||||
round-trip and shape tests, then update the external output-file contract if the
|
||||
serialized form changes. Keep internal helper changes separate from decisions
|
||||
about externally visible compatibility.
|
||||
|
||||
## Key Tests
|
||||
|
||||
|
||||
@@ -2,17 +2,15 @@
|
||||
|
||||
## Scope
|
||||
|
||||
This document is the internal architecture entry point for developers and coding agents.
|
||||
|
||||
It summarizes:
|
||||
- package boundaries;
|
||||
- the main `process` execution path;
|
||||
- where to add new code safely.
|
||||
This document owns the implemented component map. For normative boundaries, see
|
||||
[architecture policy](../policy/architecture.md); for task-specific reading and
|
||||
baseline commands, see the [contributor guide](../development.md).
|
||||
|
||||
## Package Map
|
||||
|
||||
CLI and command orchestration:
|
||||
- `internal/cli`
|
||||
Executable and command orchestration:
|
||||
- `cmd/audita`: executable entry point.
|
||||
- `internal/cli`: argument parsing, config commands, streams, filesystem wiring, and process orchestration.
|
||||
|
||||
Core deterministic components:
|
||||
- `internal/core/config`
|
||||
@@ -43,43 +41,47 @@ Domain implementations:
|
||||
- `internal/validators/*`
|
||||
- `internal/prompts`
|
||||
|
||||
Supporting areas:
|
||||
- `docs/`: canonical documentation.
|
||||
- `examples/`: maintained copyable inputs and configuration.
|
||||
|
||||
## Current Scope
|
||||
|
||||
Runtime state is held in memory plus per-run diagnostics artifacts. Audita does
|
||||
not currently implement a plugin framework, generic workflow engine,
|
||||
resume/checkpoint/manifest system, remote storage, or multi-process service mode.
|
||||
Its provider adapter is OpenAI-compatible HTTP behind the structured LLM client
|
||||
contract. These are current implementation boundaries, not bans on future work.
|
||||
|
||||
## Main Execution Path (`audita process`)
|
||||
|
||||
High-level flow:
|
||||
1. CLI loads effective config and validates CLI requirements.
|
||||
2. Run directory is created and invocation/effective config artifacts are written.
|
||||
3. Transcript/glossary files are loaded and parsed.
|
||||
4. Transcript is normalized and chunked.
|
||||
5. `runner.Run` executes configured module instances.
|
||||
6. Proposals are validated, applied deterministically, and serialized in selected output schema.
|
||||
7. Process report, utilization diagnostics, correction ledger, and retention decisions are finalized.
|
||||
The CLI prepares input and configuration, invokes the runner, and writes results.
|
||||
The [pipeline](pipeline.md) owns detailed state transitions; the
|
||||
[operations guide](../operations.md) owns the operator workflow and physical
|
||||
artifact lifecycle.
|
||||
|
||||
## Boundary Summary
|
||||
## Component Responsibilities
|
||||
|
||||
- `internal/core/*` owns deterministic, reusable logic and persistence-independent rules.
|
||||
- `internal/framework/*` owns orchestration contracts and reusable runtime plumbing.
|
||||
- `internal/modules/*` owns module-specific proposal behavior and prompt usage.
|
||||
- `internal/validators/*` owns validator composition and built-in chain assembly.
|
||||
- `internal/prompts` owns embedded prompt assets and metadata registry.
|
||||
Core components implement configuration defaults/loading/validation/redaction,
|
||||
transcript and glossary schemas, normalization and chunking, output encoding,
|
||||
diagnostics artifact helpers, and public report structures.
|
||||
|
||||
## Where To Add New Code
|
||||
Framework components implement module and validator contracts, proposal
|
||||
generation and application, prompt context, runner orchestration, LLM scheduling
|
||||
and transport, response schemas, and process-report/correction-ledger assembly.
|
||||
Domain packages provide correction modules, built-in validator chains and
|
||||
execution metadata, and embedded prompt assets.
|
||||
|
||||
Add config fields:
|
||||
- `internal/core/config`
|
||||
These describe the implementation; dependency and ownership requirements belong
|
||||
to the [architecture policy](../policy/architecture.md#ownership-and-dependency-direction).
|
||||
|
||||
Add module behavior:
|
||||
- one package under `internal/modules/<module_key>`
|
||||
- registration/wiring through `internal/framework/modules` and config module list
|
||||
## Change Recipes
|
||||
|
||||
Add validators:
|
||||
- implementation under `internal/validators/<validator_key>`
|
||||
- registry/chain wiring in `internal/validators`
|
||||
|
||||
Add runtime orchestration behavior:
|
||||
- `internal/framework/*` (runner/proposal/validator/LLM plumbing)
|
||||
|
||||
Add CLI surface:
|
||||
- `internal/cli`
|
||||
- [Configuration, CLI, and example changes](cli-config.md)
|
||||
- [Module changes](modules.md#changing-modules)
|
||||
- [Validator changes](validators.md#changing-validators)
|
||||
- Runtime orchestration changes: use [pipeline](pipeline.md), [LLM runtime](llm-runtime.md),
|
||||
or [diagnostics/reporting](diagnostics-reporting.md) and their test references.
|
||||
|
||||
## Related Internal Docs
|
||||
|
||||
|
||||
@@ -2,7 +2,10 @@
|
||||
|
||||
## Scope
|
||||
|
||||
This document describes the implemented `audita process` pipeline.
|
||||
This document describes processing collaborators and state transitions. External
|
||||
input definitions belong to the [input-file contract](../integrations/transcript-glossary-files.md),
|
||||
settings to [configuration](../config.md), and serialized results to the
|
||||
[output-file contract](../integrations/output-files.md).
|
||||
|
||||
## Inputs
|
||||
|
||||
@@ -13,35 +16,23 @@ Pipeline inputs are:
|
||||
|
||||
## Pipeline Phases
|
||||
|
||||
1. Input loading and schema validation
|
||||
- transcript and glossary files are read and parsed.
|
||||
- schema failures stop the run with `transcript_schema` or `glossary_schema`.
|
||||
|
||||
2. Normalization
|
||||
- canonical transcript segments are normalized by configured gap/duration/token settings.
|
||||
- normalization summary artifacts are written.
|
||||
|
||||
3. Chunking
|
||||
- normalized transcript is chunked with configured max/min tokens and target sections.
|
||||
|
||||
4. Module proposal generation
|
||||
- runner executes configured module instances in sequence.
|
||||
- each module proposes corrections per section.
|
||||
- per-section proposal generation can run concurrently.
|
||||
|
||||
5. Validator filtering
|
||||
- validators run on candidate proposals before apply.
|
||||
- deterministic validators run before LLM-backed validators.
|
||||
- LLM validator inputs are batched by max prompt token limit.
|
||||
|
||||
6. Deterministic apply
|
||||
- approved proposals are applied via replacement policy.
|
||||
- applied/skipped/rejected outcomes are recorded.
|
||||
|
||||
7. Output and reporting
|
||||
- final transcript is serialized with selected output schema.
|
||||
- report, utilization diagnostics, and correction ledger are written.
|
||||
- retention policy is applied to run directory.
|
||||
1. **Input loading:** CLI filesystem adapters read inputs; schema parsing produces
|
||||
source transcript and glossary objects. Source segments are converted to the
|
||||
canonical transcript shape before normalization.
|
||||
2. **Normalization:** the normalizer sorts and merges segments using its configured
|
||||
time/token rules, reassigns IDs, and produces summary counters.
|
||||
3. **Chunking:** the chunker partitions the normalized transcript and returns
|
||||
sections and token estimates, or fails when constraints are impossible.
|
||||
4. **Proposal generation:** the runner executes configured module instances in
|
||||
sequence. Sections can generate proposals concurrently; results are collected
|
||||
for deterministic indexing.
|
||||
5. **Validation:** deterministic validators run before LLM-backed validators.
|
||||
LLM validation is batched by prompt budget; candidate decisions feed application.
|
||||
6. **Application:** `internal/framework/proposals` clones transcript state and
|
||||
applies approved proposals in index order through replacement-policy checks.
|
||||
7. **Serialization and reporting:** the CLI encodes the final transcript, maps
|
||||
runner output into reports, writes artifacts, and delegates retention to
|
||||
diagnostics. See [report assembly](diagnostics-reporting.md).
|
||||
|
||||
## Runner Outputs
|
||||
|
||||
@@ -54,22 +45,18 @@ CLI/reporting then map this into process report and diagnostics artifacts.
|
||||
|
||||
## Failure Behavior
|
||||
|
||||
Representative failure phases include:
|
||||
- `run_dir_creation`
|
||||
- `transcript_read`, `glossary_read`
|
||||
- `transcript_schema`, `glossary_schema`
|
||||
- `chunking`
|
||||
- `runner_setup`, `runner_execution`
|
||||
- `output_schema`, `serialization`, `output_write`, `stdout_write`
|
||||
CLI orchestration labels failures at the phase boundary. Runner execution failures
|
||||
can return completed or partial module results, which report assembly preserves.
|
||||
[Report fields](../integrations/output-files.md#process-report) define how consumers
|
||||
see these outcomes; [CLI semantics](../cli.md#process-output-and-exit-behavior)
|
||||
define streams and exits. Diagnostic write failures do not always abort processing.
|
||||
|
||||
When diagnostics are available, failure stderr includes diagnostics path.
|
||||
## Ordering and State
|
||||
|
||||
## Invariants
|
||||
|
||||
- module execution order follows configured module sequence;
|
||||
- proposal/validator nondeterminism is isolated before deterministic apply;
|
||||
- proposal indices are assigned deterministically by section order;
|
||||
- output/report artifacts are generated from run results, not speculative state.
|
||||
The runner mutates its working transcript between module instances, after collecting
|
||||
section results and validating proposals. Proposal indexes follow section order;
|
||||
reports are derived from actual run results. Global requirements are owned by the
|
||||
[architecture policy](../policy/architecture.md#correction-and-concurrency-invariants).
|
||||
|
||||
## Key Tests
|
||||
|
||||
|
||||
@@ -2,7 +2,9 @@
|
||||
|
||||
## Scope
|
||||
|
||||
This document describes validator composition, execution order, and decision handling.
|
||||
This document describes internal validator composition, execution order, and
|
||||
decision handling. These registry keys identify implementation components; they
|
||||
are not independently selectable through [configuration](../config.md#pipeline-and-output).
|
||||
|
||||
## Ownership
|
||||
|
||||
@@ -49,7 +51,7 @@ Decision cardinality is enforced: each candidate proposal must receive exactly o
|
||||
|
||||
LLM validators:
|
||||
- build canonical validation request payloads;
|
||||
- batch by `validation_max_prompt_tokens`;
|
||||
- batch using the [configured prompt budget](../config.md#concurrency);
|
||||
- call structured LLM client using response schema registry.
|
||||
|
||||
Oversized single proposals are rejected with `validator_input_too_large`.
|
||||
@@ -57,12 +59,25 @@ Malformed LLM validator responses are downgraded to warnings and rejected batch
|
||||
|
||||
## Decision and Rejection Reporting
|
||||
|
||||
Runner records:
|
||||
- `validator_decisions`
|
||||
- `validator_rejected`
|
||||
- warning records (including malformed response warnings)
|
||||
Runner collects decisions, rejected candidates, and warning records. Ledger
|
||||
assembly groups decisions by deterministic versus LLM-backed execution metadata.
|
||||
The serialized names and fields belong to the
|
||||
[report and ledger contract](../integrations/output-files.md); implementation mapping
|
||||
is covered by [report assembly](diagnostics-reporting.md).
|
||||
|
||||
Correction ledger classifies deterministic vs LLM validator decisions using canonical metadata classes.
|
||||
## Changing Validators
|
||||
|
||||
1. Implement validator behavior under `internal/validators/<validator_key>`;
|
||||
update registry and chain wiring in `internal/validators`.
|
||||
2. Put shared runtime mechanics in `internal/framework/validators` only when
|
||||
needed. Keep deterministic versus LLM-backed execution metadata explicit.
|
||||
3. Preserve exposed identities and decision semantics under the
|
||||
[architecture policy](../policy/architecture.md#correction-and-concurrency-invariants).
|
||||
4. Review validator, registry, chain, batching, malformed-output, diagnostics,
|
||||
and decision-cardinality tests; update meaningful coverage under the
|
||||
[testing policy](../policy/testing.md).
|
||||
5. Update implementation guidance here and any affected external contract in its
|
||||
[canonical owner](../policy/documentation.md#canonical-ownership).
|
||||
|
||||
## Key Tests
|
||||
|
||||
|
||||
Reference in New Issue
Block a user