eric/audita
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 2 Wiki Activity

Add integration documentation for subprocess, LLM, and input files

Browse Source
This commit is contained in:
Eric Rakestraw
2026-05-24 01:07:27 +00:00
parent e5944f9875
commit 28d5201a69
7 changed files with 293 additions and 70 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
README.md
View File

@@ -37,6 +37,8 @@ audita config print-effective --config ./audita.yml
- CLI reference: [`docs/cli.md`](docs/cli.md)
- Configuration reference: [`docs/config.md`](docs/config.md)
- Subprocess integration: [`docs/integrations/subprocess.md`](docs/integrations/subprocess.md)
- OpenAI-compatible LLM integration: [`docs/integrations/openai-compatible-llm.md`](docs/integrations/openai-compatible-llm.md)
- Transcript and glossary file integration: [`docs/integrations/transcript-glossary-files.md`](docs/integrations/transcript-glossary-files.md)
- Development workflow: [`docs/policy/development.md`](docs/policy/development.md)
- Architecture policy: [`docs/policy/architecture.md`](docs/policy/architecture.md)
- Documentation policy: [`docs/policy/documentation.md`](docs/policy/documentation.md)

4
docs/cli.md
View File

@@ -106,6 +106,10 @@ Exit behavior:
- `1`: runtime failure during processing/reporting/output paths.
- `2`: CLI usage or configuration input error.
Integration references:
- subprocess contract: [`docs/integrations/subprocess.md`](integrations/subprocess.md)
- transcript/glossary file contract: [`docs/integrations/transcript-glossary-files.md`](integrations/transcript-glossary-files.md)
### `process` Examples
Write corrected transcript to a file:

2
docs/config.md
View File

@@ -13,6 +13,8 @@ It documents:
- validation and secrets behavior.
For CLI command syntax, see [`docs/cli.md`](cli.md).
For OpenAI-compatible endpoint behavior, see [`docs/integrations/openai-compatible-llm.md`](integrations/openai-compatible-llm.md).
For transcript/glossary input file contracts, see [`docs/integrations/transcript-glossary-files.md`](integrations/transcript-glossary-files.md).
## Loading Model

5
docs/integration/subprocess-operations.md
View File

@@ -1,5 +0,0 @@
# Moved: Subprocess Integration
The canonical subprocess integration guide now lives at [`docs/integrations/subprocess.md`](../integrations/subprocess.md).
This file is retained temporarily as a migration shim for older links.

119
docs/integrations/openai-compatible-llm.md Normal file
View File

@@ -0,0 +1,119 @@
# OpenAI-Compatible LLM Integration
## Scope
This document defines the external LLM endpoint contract Audita currently uses.
It covers:
- endpoint and auth expectations;
- structured request and response shape;
- retry and timeout behavior;
- diagnostics and secret redaction.
For user-facing CLI flags and config keys, see [`docs/cli.md`](../cli.md) and [`docs/config.md`](../config.md).
## Endpoint Contract
Audita sends HTTPS `POST` requests to:
- `<base_url>/chat/completions`
`base_url` comes from primary or validation LLM config and is required.
## Authentication Contract
When an API key is configured, Audita sends:
- `Authorization: Bearer <api_key>`
When no API key is configured, the `Authorization` header is omitted.
## Request Shape
Audita sends a chat-completions payload with:
- `model`;
- `messages` (role/content pairs);
- `response_format` using JSON Schema strict mode.
Representative shape:
```json
{
"model": "example-model",
"messages": [
{"role": "system", "content": "..."},
{"role": "user", "content": "..."}
],
"response_format": {
"type": "json_schema",
"json_schema": {
"name": "correction_set",
"strict": true,
"schema": {"type": "object"}
}
}
}
```
Behavioral requirements enforced by Audita:
- `model` must resolve to a non-empty value;
- each message must have non-empty `role` and `content`;
- `response_format.type` is always `json_schema`;
- `response_format.json_schema.name` and `schema` must be present;
- request schema JSON must be valid JSON.
## Response Handling Contract
Audita expects a successful JSON response with at least one choice and assistant content that can be interpreted as JSON.
Supported assistant content forms:
- string containing JSON;
- raw JSON value.
Audita then decodes the JSON against the expected structured output type.
Current structured schema identities used by Audita runtime:
- `correction_set`
- `validator_decision_set`
## Retries and Timeouts
Retry behavior:
- default max retries is `3` when unset;
- retries apply to retryable transport/decode/server-side errors;
- HTTP `429` and `5xx` responses are retryable;
- retry stops immediately when context is canceled or deadline expires.
Timeout behavior:
- request timeout is derived from configured LLM timeout settings;
- timeout/cancellation propagate through HTTP requests and return nonzero process failures.
## Error Behavior
Non-2xx responses fail the request.
Error message extraction behavior:
- if provider JSON includes `error.message`, Audita surfaces that message;
- else if provider JSON includes top-level `message`, Audita surfaces that;
- otherwise Audita surfaces status code plus response body text.
Malformed or incompatible structured responses fail safely and are surfaced as runtime errors or validator/proposal warnings depending on call site.
## Secret Redaction
Configured LLM secrets are redacted from:
- surfaced adapter/runtime errors;
- LLM diagnostics request/response/error artifacts;
- effective config/report artifacts that include LLM configuration material.
Redaction marker:
- `[REDACTED]`
## Compatibility Boundaries
This integration documentation applies only to the implemented OpenAI-compatible chat completions flow.
Not part of current behavior:
- provider SDK integration;
- non-OpenAI-compatible API contracts;
- server-side model routing features beyond explicitly configured model/base URL.

133
docs/integrations/subprocess.md
View File

@@ -1,96 +1,99 @@
# Audita Subprocess Operations
# Subprocess Integration
This document describes how parent processes should invoke `audita process` safely in production orchestration.
## Scope
## Recommended command form
This document describes how a parent process should invoke Audita as a subprocess.
Use explicit file outputs for orchestrated runs:
It covers:
- invocation shape;
- stdout/stderr behavior;
- output/report file behavior;
- diagnostics and exit behavior.
For full CLI and config references, see [`docs/cli.md`](../cli.md) and [`docs/config.md`](../config.md).
## Recommended Invocation
Use explicit output and report paths for machine workflows:
```sh
audita process <transcript.json> \
--transcript-description "Brief context that may help resolve ambiguous terms." \
--glossary <glossary.yaml> \
--output <output-transcript.json> \
--report-json <report.json>
```
Additional flags that may be situationally appropriate:
- `--config <path>` to select an explicit versioned config file.
- `--output-schema <bare-segments|audita-v1>` to select transcript output shape.
- `--work-dir <dir>` to control diagnostics location.
- `--work-dir-retention <always|auto|never>` to control retained run directories.
- `--total-llm-concurrency`, `--proposal-llm-concurrency`, and `--validation-llm-concurrency` when orchestration needs to set explicit LLM throughput controls.
- `--modules ...` only when intentionally overriding the default sequence.
Optional commonly used flags:
- `--config <path>`
- `--output-schema <bare-segments|audita-v1>`
- `--work-dir <dir>`
- `--work-dir-retention <always|auto|never>`
- `--transcript-description <text>`
For config-driven orchestration, validate config files in CI/preflight:
## Stdout Contract
```sh
audita config validate --config <path>
```
On success:
- with `--output`: stdout is expected to be empty;
- without `--output`: stdout contains transcript JSON only.
## Stdout behavior
`--report-json` output is never written to stdout.
- With `--output`: stdout is expected to be empty on success.
- Without `--output`: stdout contains transcript JSON only on success.
- Report JSON is never written to stdout.
## Stderr Contract
## Stderr behavior
Stderr is human-readable status/error output.
- Success path should be quiet or minimal human-readable logs.
- Failure path writes concise human-readable errors.
- When a diagnostics run directory exists, failure stderr includes its path.
- Prompt/response diagnostic payloads are not streamed to stderr.
On failures:
- stderr includes a concise top-level error;
- when diagnostics are initialized, stderr includes diagnostics directory path.
## Output file behavior
Do not treat stderr as a machine-stable JSON channel.
- `--output` writes transcript JSON in the selected output schema to the provided path.
- Output write failures return nonzero and surface actionable errors.
- The command does not silently ignore output write errors.
## Output and Report File Contract
## Report JSON behavior
Transcript output:
- `--output` writes corrected transcript JSON to the provided path;
- output write failures return nonzero.
- `--report-json` writes a machine-readable process report to the requested path.
- Run-directory `report.json` is written independently under diagnostics.
- Best-effort failure reports are emitted when possible without masking the primary failure.
- Report write failures return nonzero with clear stderr messaging.
- Report diagnostics metadata references run-directory artifacts including utilization diagnostics and correction ledger paths when available.
Report output:
- `--report-json` writes machine-readable process report JSON to the provided path;
- run diagnostics also attempt to write their own `report.json`;
- report write failures return nonzero;
- on failure paths, report writing is best-effort and does not mask the primary run error.
## Diagnostics directory behavior
## Diagnostics Contract
- Each run creates (when possible) a per-run diagnostics directory.
- Typical artifacts include transcript, normalization, chunking, invocation, effective config, LLM diagnostics, `utilization-diagnostics.json`, `correction-ledger.json`, `report.json`, and `error.log` on failure.
- Failed runs retain diagnostics.
- Under `auto` retention, successful runs with skipped/rejected corrections are retained; clean successful runs may be removed.
When run-directory initialization succeeds, per-run diagnostics artifacts are written under the configured work directory.
## Exit codes
Typical artifacts include:
- `source-transcript.json`
- `source-transcript-parsed.json`
- `normalized-transcript.json`
- `normalization-summary.json`
- `chunking-summary.json`
- `invocation.json`
- `effective-config.json`
- `utilization-diagnostics.json`
- `correction-ledger.json`
- `report.json`
- `error.log` (failure)
- `0`: success.
- Nonzero: failure (input/schema/config/module/LLM/runtime/output/report/diagnostics errors).
Retention behavior is controlled by `--work-dir-retention` / config.
Treat any nonzero as a failed subprocess invocation.
## Exit Behavior
## Timeout and cancellation
Exit codes:
- `0`: success;
- `1`: runtime processing/output/report failure;
- `2`: CLI usage or configuration input error.
- Runtime operations propagate context cancellation and request timeouts through LLM/scheduler paths.
- On cancellation or timeout, the process exits nonzero and should not hang.
- If diagnostics were initialized before failure, failure artifacts remain available for debugging.
Treat any nonzero as subprocess failure.
## Secret redaction expectations
## Parent-Process Guidance
API keys and configured secret values are redacted from:
- reports (`--report-json` and run-dir `report.json`);
- diagnostics artifacts (including effective config and LLM interaction artifacts);
- surfaced adapter/runtime errors;
- test fixtures and regression outputs.
For reliable orchestration:
- read stdout and stderr concurrently to avoid pipe blocking;
- prefer `--output` and `--report-json` for machine parsing;
- use timeout/cancellation in the parent process;
- inspect diagnostics path and `report.json`/`error.log` on failure.
Parent-process logs should still avoid printing raw environment variables.
## Parent-process pipe guidance
To avoid deadlocks in orchestrators:
- always read both stdout and stderr concurrently when invoking as a subprocess;
- prefer file outputs (`--output`, `--report-json`) for machine workflows;
- treat stderr as human-readable diagnostics, not structured data;
- parse structured results from output/report files.
For Go callers, prefer `exec.CommandContext` with explicit timeout/cancellation and buffered/streamed readers for both pipes.
For input file contracts, see [`docs/integrations/transcript-glossary-files.md`](transcript-glossary-files.md).

98
docs/integrations/transcript-glossary-files.md Normal file
View File

@@ -0,0 +1,98 @@
# Transcript and Glossary File Integration
## Scope
This document defines the input file contracts for:
- transcript JSON;
- glossary YAML.
These files are loaded and validated before processing begins.
## Transcript JSON Contract
Audita accepts either top-level shape:
- JSON array of segments; or
- JSON object with a `segments` array.
Segment fields:
- `id` (optional integer in source form);
- `speaker` (required non-empty string);
- `start` (required finite non-negative number);
- `end` (required finite non-negative number, `>= start`);
- `text` (required non-empty string);
- `categories` (optional string array; entries must be non-empty).
Additional rules:
- transcript must contain at least one segment;
- duplicate segment IDs are rejected when IDs are present.
Example (`examples/tiny-transcript.json`):
```json
[
{
"id": 1,
"speaker": "A",
"start": 0.0,
"end": 1.2,
"text": "hello world"
}
]
```
## Glossary YAML Contract
Audita expects top-level `glossary` list entries.
Entry fields:
- `name` (required non-empty string);
- `category` (required non-empty string);
- `summary` (required non-empty string);
- `aliases` (optional list of strings; entries must be non-empty);
- `plural` (optional string).
Additional rules:
- glossary must contain at least one entry.
Example (`examples/tiny-glossary.yaml`):
```yaml
glossary:
- name: Audita
aliases:
- audita
category: product
summary: The Audita transcript correction CLI.
```
## Validation Failure Behavior
Representative transcript validation failures:
- invalid JSON;
- unsupported top-level shape;
- empty `speaker` or `text`;
- invalid times (`NaN`, `Inf`, negative, or `end < start`);
- duplicate IDs;
- empty transcript array.
Representative glossary validation failures:
- invalid YAML;
- empty or missing glossary entries;
- missing required entry fields;
- empty alias values.
These failures surface as schema errors and the process exits nonzero.
## CLI Usage
Minimal invocation:
```sh
audita process ./transcript.json --glossary ./glossary.yaml --output ./corrected.json
```
See also:
- [`docs/cli.md`](../cli.md)
- [`docs/config.md`](../config.md)
- [`examples/tiny-transcript.json`](../../examples/tiny-transcript.json)
- [`examples/tiny-glossary.yaml`](../../examples/tiny-glossary.yaml)