Refresh documentation and policies and simplify releases

This commit is contained in:
2026-09-23 15:03:20 +00:00
parent 588ca36dfc
commit e8b8fbd509
36 changed files with 2143 additions and 2261 deletions

View File

@@ -0,0 +1,64 @@
# Configuration and CLI Change Recipes
This document owns contributor editing and validation steps for configuration,
CLI wiring, and maintained examples. Definitions of behavior belong to the
[configuration](../config.md) and [CLI](../cli.md) contracts.
## Changing Configuration
1. Update fields, defaults, loading, and validation in `internal/core/config`.
2. Update file, environment, and CLI mapping as applicable, including CLI flag
extraction. Preserve the source-specific behavior defined in the
[configuration contract](../config.md#effective-precedence), including the
distinct validation and effective-config commands.
3. Update redaction when secrets are involved.
4. Review configuration tests and related CLI tests for precedence, validation,
and source-specific behavior. Add or adjust tests for the changed contract
according to the [testing policy](../policy/testing.md).
5. Update the configuration reference and affected [examples](../../examples/).
## Changing CLI Behavior
1. Implement parsing and wiring in `internal/cli`.
2. Preserve [stream and exit contracts](../cli.md) unless a change is intentional
and documented. Review [subprocess integration](../integrations/subprocess.md)
when changing output, error, or report handling.
3. Review `internal/cli` tests and `cmd/audita` subprocess tests; update coverage
for the changed behavior.
4. Update the CLI reference and affected integration guidance.
## Validating Examples
Keep complete copyable files in [examples](../../examples/) and link to them from
their canonical contract documentation. Keep them valid and secret-free.
From the repository root, validate the maintained configuration examples:
```sh
go run ./cmd/audita config validate --config examples/minimal-config.yml
go run ./cmd/audita config validate --config examples/production-config.yml
go run ./cmd/audita config print-effective --config examples/minimal-config.yml
```
The effective-config command incorporates environment settings; use a controlled
environment when comparing its results. Check affected CLI examples against the
parser and use schema tests for maintained transcript/glossary inputs. Example
validation should not invoke a live LLM merely to check documentation.
## Runtime Change Checks
Select checks according to the [testing policy](../policy/testing.md):
- Configuration or CLI changes: inspect `internal/core/config` tests,
`internal/cli` tests, and `cmd/audita` integration tests.
- Process/output changes: verify stdout, stderr, exit, output-file, and report
behavior against the [CLI contract](../cli.md) and
[subprocess guidance](../integrations/subprocess.md), including failures.
- Report or diagnostics changes: review [report assembly tests and redaction
checks](diagnostics-reporting.md#key-tests), metadata fields, and artifact paths.
- Schema or serialization changes: review [output-schema tests](output-schemas.md#key-tests)
and `internal/core/schema` tests, including supported formats and rejection of
unsupported schemas.
- Changes affecting assembled output: inspect CLI parity and release fixtures.
Use the [contributor guide](../development.md#baseline-validation) for baseline
commands. These recipes supplement, rather than duplicate, public contracts.

View File

@@ -1,74 +1,69 @@
# Audita Diagnostics and Reporting
# Diagnostics and Report Assembly
## Scope
This document describes diagnostics artifacts, process report mapping, and correction ledger generation.
This document owns implementation flow and collaborators. Physical artifacts and
retention belong to [operations](../operations.md). Serialized reports, ledger
records, and utilization shapes belong to the
[output-file contract](../integrations/output-files.md).
## Run Directory Ownership
## Run Directory and Artifact Writers
`internal/core/diagnostics` owns run-directory creation, artifact writes, and retention decisions.
`internal/core/diagnostics` creates run directories, writes snapshots and metadata,
and evaluates retention. CLI orchestration supplies the selected configuration,
paths, run outcome, and correction-presence information. Artifact names are
centralized in diagnostics; their operator-facing inventory is in
[operations](../operations.md#diagnostics-directory).
Stable artifact names include:
- `source-transcript.json`
- `source-transcript-parsed.json`
- `normalized-transcript.json`
- `normalization-summary.json`
- `chunking-summary.json`
- `utilization-diagnostics.json`
- `correction-ledger.json`
- `invocation.json`
- `effective-config.json`
- `report.json`
- `error.log` (failure)
Snapshot writes are attempted at the relevant processing phases. Several diagnostic
writes are best-effort, so report metadata can identify a location without proving
that a file was written. The CLI distinguishes primary processing errors from
secondary diagnostic/report write errors.
## Process Report Mapping
`internal/framework/processreport` maps runner/CLI execution facts into `reporting.ProcessReport`.
`internal/framework/processreport.Build` maps invocation facts, normalization and
chunking summaries, and runner output into `reporting.ProcessReport`. Core reporting
owns the Go data structures and JSON writer. The builder attaches diagnostic
metadata when a run-directory path is available.
Report metadata fields include:
- `report_schema_name` (`audita-process-report`)
- `report_schema_version` (`v1`)
- `output_schema`
- `config_version` (when file config exists)
Module mapping copies outcomes from runner results, collects summary counts, and
preserves available partial module results on failure. The CLI failure path does
not pass normalization or chunking summaries to report construction even when
some earlier processing completed. Consumer interpretation belongs to the
[process-report contract](../integrations/output-files.md#process-report).
The report includes:
- top-level status/error phase/error message;
- normalization/chunking summaries;
- diagnostics metadata paths;
- per-module results and module summary.
## Correction Ledger and Utilization
## Correction Ledger
`BuildCorrectionLedger` flattens module changes, rejections, and failures into
ledger entries. It associates validator records by proposal index within each
module, separates execution classes through validator metadata, and sorts entries
by module instance, proposal index, and disposition.
`internal/framework/processreport/BuildCorrectionLedger` flattens run results into `correction-ledger.json` entries.
Runner instrumentation collects utilization; CLI orchestration writes it when
run output is available. Report assembly does not independently measure calls.
For the serialized forms see [ledger](../integrations/output-files.md#correction-ledger)
and [utilization](../integrations/output-files.md#utilization-diagnostics).
Dispositions:
- `applied`
- `skipped`
- `rejected`
- `failed`
## CLI Coordination and Retention
Validator decisions are split into deterministic and LLM-backed groups using validator metadata classification.
CLI orchestration writes caller-selected reports and run-local artifacts before
asking diagnostics to apply retention. `HasSkippedCorrections` derives the
retention input from runner outcomes. Keep the exact lifecycle definition in
[operations](../operations.md#retention-behavior), not in another internal table.
## Report Write Paths
- run directory always attempts to write `report.json` when possible;
- optional `--report-json` writes an external report file;
- on failure paths, report writing is best-effort and does not mask primary run errors.
## Retention Interaction
Current retention behavior:
- failed runs are retained;
- `always` keeps successful runs;
- `auto` removes only clean successful runs;
- `never` currently retains successful runs in current implementation.
On a processing failure, secondary report-write errors do not replace the original
error. On the success path, an external report-write failure returns before the
run-local report write; retention errors occur later. This ordering explains why
artifacts and process exit status must be assessed together.
## Redaction
Redacted data expectations:
- effective config artifact uses config redaction;
- diagnostics payloads and surfaced errors use LLM secret redaction;
- reports should not include raw API key values.
Effective configuration uses config redaction. LLM diagnostic and error paths
use `llm.ConfiguredSecrets` and the runtime's redaction helpers. Keep these separate
mechanisms aligned with the [architecture requirement](../policy/architecture.md#errors-and-secret-handling).
Operational handling of transcript-bearing artifacts belongs to
[operations](../operations.md#artifact-handling).
## Key Tests

View File

@@ -11,44 +11,36 @@ All runtime LLM calls go through `contracts.StructuredLLMClient`.
Primary adapter:
- `internal/framework/llm/OpenAICompatibleClient`
## Request/Response Behavior
## Adapter and Response Schema Registry
The OpenAI-compatible adapter sends chat completions requests with:
- model;
- messages;
- `response_format.type = json_schema`;
- strict schema envelope (`name`, `schema`, `strict=true`).
The adapter implements the [LLM wire contract](../integrations/openai-compatible-llm.md).
It maps internal structured requests to HTTP messages and decodes provider content
into the supplied Go output target. Endpoint, authentication, response shapes, and
retry conditions are defined in that contract rather than repeated here.
The response is decoded into the requested structured output target.
`internal/framework/responseschema` stores the per-call response schema definitions
and diagnostic metadata. Proposal generation and validation choose their schema
before invoking the client. Provider strict mode does not replace local checks.
## Response Schema Registry
## Error Classification
Structured response schemas are registered in `internal/framework/responseschema`:
- `correction_set`
- `validator_decision_set`
Each schema includes stable diagnostics metadata (`id`, `version`, `name`, `sha256`).
## Retries and Error Handling
Adapter retries apply to retryable conditions (for example transport/decoding/retryable status classes) up to configured `max_retries`.
Errors are sanitized to redact configured API-key values before surfacing.
Malformed structured output detection is shared through `internal/framework/structuredoutput` and is used by:
- proposal generation;
- LLM-backed validators.
The adapter marks retryable errors internally and uses context state and attempt
count to decide whether to retry. It sanitizes errors before returning them.
`internal/framework/structuredoutput` classifies malformed output for proposal
and validator call sites, which can turn those errors into warnings and rejected
section/batch work. Other errors propagate through the runner.
## Scheduling and Concurrency
`internal/framework/llm/Scheduler` provides FIFO, context-aware permit gating.
Runner composes scheduler limits across:
- total LLM concurrency;
- proposal LLM concurrency;
- validation LLM concurrency.
CLI orchestration composes global and stage schedulers; runner instrumentation
measures them. Limit definitions and inheritance belong to
[configuration](../config.md#concurrency).
Scheduler release is guarded to avoid permit leaks on cancellation/error.
Scheduler release is guarded to avoid permit leaks on cancellation/error. Runner
collects utilization observations; their serialized shape is defined by the
[output-file contract](../integrations/output-files.md#utilization-diagnostics).
## Diagnostics and Redaction
@@ -58,6 +50,14 @@ Configured secrets are derived from `llm.ConfiguredSecrets(cfg)` and redacted fr
- diagnostics payloads;
- surfaced runtime/adapter errors.
## Changing LLM Integration or Scheduling
Review client, scheduler, redaction, and structured-output malformed-response
coverage alongside runner orchestration and utilization tests. Use the
[testing policy](../policy/testing.md) to select sufficient checks; preserve the
[LLM integration contract](../integrations/openai-compatible-llm.md) and
[architecture invariants](../policy/architecture.md#correction-and-concurrency-invariants).
## Key Tests
- `internal/framework/llm/openai_compatible_client_test.go`
@@ -65,3 +65,4 @@ Configured secrets are derived from `llm.ConfiguredSecrets(cfg)` and redacted fr
- `internal/framework/llm/diagnostics_test.go`
- `internal/framework/responseschema/registry_test.go`
- `internal/framework/structuredoutput/malformed_test.go`
- `internal/framework/runner/*_test.go` (orchestration and utilization)

View File

@@ -2,7 +2,8 @@
## Scope
This document covers module contracts and built-in module packages.
This document covers internal module contracts and packages. User-selectable keys
and the default sequence belong to [configuration](../config.md#pipeline-and-output).
## Module Contract
@@ -46,9 +47,26 @@ Module packages call the built-in chain resolver at construction.
## Failure and Warning Behavior
- module setup failures surface as `runner_setup` or module setup errors;
- module runtime failures surface as `runner_execution` with partial module results preserved;
- malformed structured proposal payloads are downgraded to warnings and section-level proposal rejection.
Module construction failures propagate to orchestration; execution failures can
preserve partial module results. Malformed structured proposal payloads become
warnings and section-level rejection. Consumer-visible outcomes are defined by
the [report contract](../integrations/output-files.md#module-outcomes).
## Changing Modules
1. Add or update the narrowly scoped package under `internal/modules/<module_key>`.
2. Keep module-specific prompt ownership in the module and `internal/prompts`;
see [prompt rendering and metadata](prompts.md).
3. Wire module registration and catalog resolution through framework module
registration and the core module catalog. Update selectable keys in the
[configuration contract](../config.md) when they change.
4. Verify replacement policy and validator-chain selection against the
[architecture requirements](../policy/architecture.md#correction-and-concurrency-invariants).
5. Review module, prompt-rendering, and proposal-generation tests. Inspect parity
and release fixtures when public output changes, choosing sufficient coverage
under the [testing policy](../policy/testing.md).
6. Update this document for implementation changes and the appropriate public
contract owner for externally visible changes.
## Key Tests

View File

@@ -1,43 +1,31 @@
# Audita Output Schemas
# Output Encoding Internals
## Scope
## Scope and Ownership
This document describes the implemented transcript output schema registry.
`internal/core/outputschema` owns encoder registration and resolution. External
JSON envelopes and segment fields are defined by the
[output-file contract](../integrations/output-files.md#corrected-transcript);
selectable values and defaults belong to
[configuration](../config.md#pipeline-and-output).
## Registry Ownership
## Resolution and Encoding
Output schema registry is owned by `internal/core/outputschema`.
Configuration validation checks registry support. CLI orchestration resolves the
selected encoder, passes the final transcript to it, and routes the encoded bytes
to the destination defined by the [CLI contract](../cli.md#process-output-and-exit-behavior).
An unsupported key fails validation or resolution rather than selecting an
implicit fallback.
Supported schema keys:
- `bare-segments`
- `audita-v1`
The registry delegates bare segment serialization to the transcript schema
package and uses an envelope encoder for the versioned form. Report construction
receives the selected key separately; see [report assembly](diagnostics-reporting.md).
## Schemas
## Changing an Encoder
`bare-segments`:
- top-level JSON array of transcript segments.
`audita-v1`:
- top-level JSON object with:
- `schema: "audita-v1"`
- `version: "v1"`
- `segments: [...]`
Segment fields include `id`, `speaker`, `start`, `end`, `text`, and optional `categories`.
## Validation and Resolution
Config validation and runtime resolution both reject unsupported schema keys.
Unknown schema keys fail with `unsupported output schema` before output emission.
## Output Emission
The selected schema is used by `audita process` when writing:
- output file (`--output`) or
- stdout (when no `--output`).
Report metadata records selected `output_schema`.
Update registration, configuration validation, and serialization together. Inspect
round-trip and shape tests, then update the external output-file contract if the
serialized form changes. Keep internal helper changes separate from decisions
about externally visible compatibility.
## Key Tests

View File

@@ -2,17 +2,15 @@
## Scope
This document is the internal architecture entry point for developers and coding agents.
It summarizes:
- package boundaries;
- the main `process` execution path;
- where to add new code safely.
This document owns the implemented component map. For normative boundaries, see
[architecture policy](../policy/architecture.md); for task-specific reading and
baseline commands, see the [contributor guide](../development.md).
## Package Map
CLI and command orchestration:
- `internal/cli`
Executable and command orchestration:
- `cmd/audita`: executable entry point.
- `internal/cli`: argument parsing, config commands, streams, filesystem wiring, and process orchestration.
Core deterministic components:
- `internal/core/config`
@@ -43,43 +41,47 @@ Domain implementations:
- `internal/validators/*`
- `internal/prompts`
Supporting areas:
- `docs/`: canonical documentation.
- `examples/`: maintained copyable inputs and configuration.
## Current Scope
Runtime state is held in memory plus per-run diagnostics artifacts. Audita does
not currently implement a plugin framework, generic workflow engine,
resume/checkpoint/manifest system, remote storage, or multi-process service mode.
Its provider adapter is OpenAI-compatible HTTP behind the structured LLM client
contract. These are current implementation boundaries, not bans on future work.
## Main Execution Path (`audita process`)
High-level flow:
1. CLI loads effective config and validates CLI requirements.
2. Run directory is created and invocation/effective config artifacts are written.
3. Transcript/glossary files are loaded and parsed.
4. Transcript is normalized and chunked.
5. `runner.Run` executes configured module instances.
6. Proposals are validated, applied deterministically, and serialized in selected output schema.
7. Process report, utilization diagnostics, correction ledger, and retention decisions are finalized.
The CLI prepares input and configuration, invokes the runner, and writes results.
The [pipeline](pipeline.md) owns detailed state transitions; the
[operations guide](../operations.md) owns the operator workflow and physical
artifact lifecycle.
## Boundary Summary
## Component Responsibilities
- `internal/core/*` owns deterministic, reusable logic and persistence-independent rules.
- `internal/framework/*` owns orchestration contracts and reusable runtime plumbing.
- `internal/modules/*` owns module-specific proposal behavior and prompt usage.
- `internal/validators/*` owns validator composition and built-in chain assembly.
- `internal/prompts` owns embedded prompt assets and metadata registry.
Core components implement configuration defaults/loading/validation/redaction,
transcript and glossary schemas, normalization and chunking, output encoding,
diagnostics artifact helpers, and public report structures.
## Where To Add New Code
Framework components implement module and validator contracts, proposal
generation and application, prompt context, runner orchestration, LLM scheduling
and transport, response schemas, and process-report/correction-ledger assembly.
Domain packages provide correction modules, built-in validator chains and
execution metadata, and embedded prompt assets.
Add config fields:
- `internal/core/config`
These describe the implementation; dependency and ownership requirements belong
to the [architecture policy](../policy/architecture.md#ownership-and-dependency-direction).
Add module behavior:
- one package under `internal/modules/<module_key>`
- registration/wiring through `internal/framework/modules` and config module list
## Change Recipes
Add validators:
- implementation under `internal/validators/<validator_key>`
- registry/chain wiring in `internal/validators`
Add runtime orchestration behavior:
- `internal/framework/*` (runner/proposal/validator/LLM plumbing)
Add CLI surface:
- `internal/cli`
- [Configuration, CLI, and example changes](cli-config.md)
- [Module changes](modules.md#changing-modules)
- [Validator changes](validators.md#changing-validators)
- Runtime orchestration changes: use [pipeline](pipeline.md), [LLM runtime](llm-runtime.md),
or [diagnostics/reporting](diagnostics-reporting.md) and their test references.
## Related Internal Docs

View File

@@ -2,7 +2,10 @@
## Scope
This document describes the implemented `audita process` pipeline.
This document describes processing collaborators and state transitions. External
input definitions belong to the [input-file contract](../integrations/transcript-glossary-files.md),
settings to [configuration](../config.md), and serialized results to the
[output-file contract](../integrations/output-files.md).
## Inputs
@@ -13,35 +16,23 @@ Pipeline inputs are:
## Pipeline Phases
1. Input loading and schema validation
- transcript and glossary files are read and parsed.
- schema failures stop the run with `transcript_schema` or `glossary_schema`.
2. Normalization
- canonical transcript segments are normalized by configured gap/duration/token settings.
- normalization summary artifacts are written.
3. Chunking
- normalized transcript is chunked with configured max/min tokens and target sections.
4. Module proposal generation
- runner executes configured module instances in sequence.
- each module proposes corrections per section.
- per-section proposal generation can run concurrently.
5. Validator filtering
- validators run on candidate proposals before apply.
- deterministic validators run before LLM-backed validators.
- LLM validator inputs are batched by max prompt token limit.
6. Deterministic apply
- approved proposals are applied via replacement policy.
- applied/skipped/rejected outcomes are recorded.
7. Output and reporting
- final transcript is serialized with selected output schema.
- report, utilization diagnostics, and correction ledger are written.
- retention policy is applied to run directory.
1. **Input loading:** CLI filesystem adapters read inputs; schema parsing produces
source transcript and glossary objects. Source segments are converted to the
canonical transcript shape before normalization.
2. **Normalization:** the normalizer sorts and merges segments using its configured
time/token rules, reassigns IDs, and produces summary counters.
3. **Chunking:** the chunker partitions the normalized transcript and returns
sections and token estimates, or fails when constraints are impossible.
4. **Proposal generation:** the runner executes configured module instances in
sequence. Sections can generate proposals concurrently; results are collected
for deterministic indexing.
5. **Validation:** deterministic validators run before LLM-backed validators.
LLM validation is batched by prompt budget; candidate decisions feed application.
6. **Application:** `internal/framework/proposals` clones transcript state and
applies approved proposals in index order through replacement-policy checks.
7. **Serialization and reporting:** the CLI encodes the final transcript, maps
runner output into reports, writes artifacts, and delegates retention to
diagnostics. See [report assembly](diagnostics-reporting.md).
## Runner Outputs
@@ -54,22 +45,18 @@ CLI/reporting then map this into process report and diagnostics artifacts.
## Failure Behavior
Representative failure phases include:
- `run_dir_creation`
- `transcript_read`, `glossary_read`
- `transcript_schema`, `glossary_schema`
- `chunking`
- `runner_setup`, `runner_execution`
- `output_schema`, `serialization`, `output_write`, `stdout_write`
CLI orchestration labels failures at the phase boundary. Runner execution failures
can return completed or partial module results, which report assembly preserves.
[Report fields](../integrations/output-files.md#process-report) define how consumers
see these outcomes; [CLI semantics](../cli.md#process-output-and-exit-behavior)
define streams and exits. Diagnostic write failures do not always abort processing.
When diagnostics are available, failure stderr includes diagnostics path.
## Ordering and State
## Invariants
- module execution order follows configured module sequence;
- proposal/validator nondeterminism is isolated before deterministic apply;
- proposal indices are assigned deterministically by section order;
- output/report artifacts are generated from run results, not speculative state.
The runner mutates its working transcript between module instances, after collecting
section results and validating proposals. Proposal indexes follow section order;
reports are derived from actual run results. Global requirements are owned by the
[architecture policy](../policy/architecture.md#correction-and-concurrency-invariants).
## Key Tests

View File

@@ -2,7 +2,9 @@
## Scope
This document describes validator composition, execution order, and decision handling.
This document describes internal validator composition, execution order, and
decision handling. These registry keys identify implementation components; they
are not independently selectable through [configuration](../config.md#pipeline-and-output).
## Ownership
@@ -49,7 +51,7 @@ Decision cardinality is enforced: each candidate proposal must receive exactly o
LLM validators:
- build canonical validation request payloads;
- batch by `validation_max_prompt_tokens`;
- batch using the [configured prompt budget](../config.md#concurrency);
- call structured LLM client using response schema registry.
Oversized single proposals are rejected with `validator_input_too_large`.
@@ -57,12 +59,25 @@ Malformed LLM validator responses are downgraded to warnings and rejected batch
## Decision and Rejection Reporting
Runner records:
- `validator_decisions`
- `validator_rejected`
- warning records (including malformed response warnings)
Runner collects decisions, rejected candidates, and warning records. Ledger
assembly groups decisions by deterministic versus LLM-backed execution metadata.
The serialized names and fields belong to the
[report and ledger contract](../integrations/output-files.md); implementation mapping
is covered by [report assembly](diagnostics-reporting.md).
Correction ledger classifies deterministic vs LLM validator decisions using canonical metadata classes.
## Changing Validators
1. Implement validator behavior under `internal/validators/<validator_key>`;
update registry and chain wiring in `internal/validators`.
2. Put shared runtime mechanics in `internal/framework/validators` only when
needed. Keep deterministic versus LLM-backed execution metadata explicit.
3. Preserve exposed identities and decision semantics under the
[architecture policy](../policy/architecture.md#correction-and-concurrency-invariants).
4. Review validator, registry, chain, batching, malformed-output, diagnostics,
and decision-cardinality tests; update meaningful coverage under the
[testing policy](../policy/testing.md).
5. Update implementation guidance here and any affected external contract in its
[canonical owner](../policy/documentation.md#canonical-ownership).
## Key Tests