Migrate examples and guidance and remove legacy test harness

This commit is contained in:
2026-09-24 02:37:20 +00:00
parent 9561e80efb
commit 80804a038f
102 changed files with 328 additions and 8371 deletions

View File

@@ -1,10 +1,9 @@
# Audita CLI Reference
The shared version-2 configuration parser and offline profile inspection are
implemented for `process`, `config validate`, and `config print-effective`.
Process execution is being reconnected in later work, so the CLI is not yet a
usable assembled command.
The [configuration reference](config.md) owns fields, defaults, and validation.
The shared version-2 configuration parser and offline profile inspection serve
`process`, `config validate`, and `config print-effective`.
The [configuration reference](config.md) owns fields, defaults, and validation;
the [migration guide](migrations/v2.md) maps retired inputs.
## Commands and help

View File

@@ -1,9 +1,11 @@
# Audita Configuration
This reference describes the version-2 configuration loader in `internal/core/config`.
CLI command parsing and offline profile inspection are implemented; process
execution is being reconnected in later work. Complete copyable
examples will be migrated with that integration.
Use the maintained [local](../examples/minimal-config.yml) and
[service](../examples/production-config.yml) examples with their PromptKit
[profiles](../examples/profiles/) and optional [testing overlay](../examples/overlays/testing.yml).
Choose real models for your endpoint; a testing profile makes no cost or quality
guarantee. See the [migration guide](migrations/v2.md) for retired inputs.
## Loading and precedence

View File

@@ -18,7 +18,7 @@ backend credential permits an unauthenticated local service.
failures; explicit zero makes one attempt. `llm.repair_attempts` counts
additional generations for schema-invalid model output. The operation timeout
covers preparation, capacity waiting, retries, generation, and repair. Caller
cancellation or an earlier deadline also applies. See [configuration](../config.md#llm-settings)
cancellation or an earlier deadline also applies. See [configuration](../config.md#settings)
for defaults and [PromptKit's consumer contract](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.11.0/docs/consumers/pkg-promptkit.md)
for the pinned protocol and retry details.

View File

@@ -5,7 +5,7 @@ and describes the correction ledger and utilization diagnostics. It documents
current serialized shapes without making every debugging artifact a versioned
compatibility promise. Physical filenames, placement, retention, and permissions
belong to [operations](../operations.md); invocation and output channels belong to
the [CLI](../cli.md#process-output-and-exit-behavior).
the [CLI](../cli.md#streams-and-exit-status).
## Corrected Transcript

View File

@@ -7,10 +7,11 @@ The [CLI reference](../cli.md) owns command syntax, streams, and exit semantics;
## Invocation
Use an argument array with the transcript path and required glossary argument,
following the [process examples](../cli.md#process-examples). For machine workflows,
following the [process examples](../cli.md#commands-and-help). For machine workflows,
select explicit transcript and report destinations rather than depending on the
working directory or run-directory retention. Supplying a configuration path also
makes file selection predictable; inherited environment overrides still apply.
makes file selection predictable; only explicitly referenced credential variables
are read from the environment.
Resolve the executable deliberately and set the child's working directory so
relative input/output paths and reported artifact paths have a known base.
@@ -20,7 +21,7 @@ or cancellation in the parent process when needed.
## Result Handling
Interpret the exit result according to the
[CLI output and exit contract](../cli.md#process-output-and-exit-behavior).
[CLI output and exit contract](../cli.md#streams-and-exit-status).
Consume transcript/report data through the
[output-file contract](output-files.md), and treat stderr as human-readable detail.
The report supplements the exit result: it may be absent after early failures,
@@ -37,4 +38,4 @@ alone establishes successful completion.
- [Transcript and glossary inputs](transcript-glossary-files.md)
- [LLM endpoint compatibility](openai-compatible-llm.md)
- [Configuration inspection commands](../cli.md#configuration-commands)
- [Configuration inspection commands](../cli.md#commands-and-help)

View File

@@ -7,9 +7,9 @@ CLI wiring, and maintained examples. Definitions of behavior belong to the
## Changing Configuration
1. Update fields, defaults, loading, and validation in `internal/core/config`.
2. Update file, environment, and CLI mapping as applicable, including CLI flag
extraction. Preserve the source-specific behavior defined in the
[configuration contract](../config.md#effective-precedence), including the
2. Update file, selected overlay, credential reference, and CLI mapping as
applicable. Preserve the precedence defined in the
[configuration contract](../config.md#loading-and-precedence), including the
distinct validation and effective-config commands.
3. Update redaction when secrets are involved.
4. Review configuration tests and related CLI tests for precedence, validation,
@@ -35,12 +35,13 @@ From the repository root, validate the maintained configuration examples:
```sh
go run ./cmd/audita config validate --config examples/minimal-config.yml
go run ./cmd/audita config validate --config examples/production-config.yml
go run ./cmd/audita config print-effective --config examples/minimal-config.yml
AUDITA_SERVICE_KEY=synthetic-example go run ./cmd/audita config validate --config examples/production-config.yml
AUDITA_SERVICE_KEY=synthetic-example go run ./cmd/audita config validate --config examples/production-config.yml --overlay testing
go run ./cmd/audita config print-effective --config examples/minimal-config.yml --view profiles
```
The effective-config command incorporates environment settings; use a controlled
environment when comparing its results. Check affected CLI examples against the
Only referenced credentials read environment values; use a controlled synthetic
value when validating the service example. Check affected CLI examples against the
parser and use schema tests for maintained transcript/glossary inputs. Example
validation should not invoke a live LLM merely to check documentation.
@@ -58,7 +59,8 @@ Select checks according to the [testing policy](../policy/testing.md):
- Schema or serialization changes: review [output-schema tests](output-schemas.md#key-tests)
and `internal/core/schema` tests, including supported formats and rejection of
unsupported schemas.
- Changes affecting assembled output: inspect CLI parity and release fixtures.
- Changes affecting assembled output: inspect controlled HTTP subprocess workflows
and retained report/artifact assertions.
Use the [contributor guide](../development.md#baseline-validation) for baseline
commands. These recipes supplement, rather than duplicate, public contracts.

View File

@@ -12,7 +12,7 @@ selectable values and defaults belong to
Configuration validation checks registry support. CLI orchestration resolves the
selected encoder, passes the final transcript to it, and routes the encoded bytes
to the destination defined by the [CLI contract](../cli.md#process-output-and-exit-behavior).
to the destination defined by the [CLI contract](../cli.md#streams-and-exit-status).
An unsupported key fails validation or resolution rather than selecting an
implicit fallback.

View File

@@ -50,8 +50,8 @@ Supporting areas:
Runtime state is held in memory plus per-run diagnostics artifacts. Audita does
not currently implement a plugin framework, generic workflow engine,
resume/checkpoint/manifest system, remote storage, or multi-process service mode.
Its provider adapter is OpenAI-compatible HTTP behind the structured LLM client
contract. These are current implementation boundaries, not bans on future work.
Its role-bound PromptKit adapter implements the structured LLM client contract.
These are current implementation boundaries, not bans on future work.
## Main Execution Path (`audita process`)
@@ -67,8 +67,8 @@ transcript and glossary schemas, normalization and chunking, output encoding,
diagnostics artifact helpers, and public report structures.
Framework components implement module and validator contracts, proposal
generation and application, prompt context, runner orchestration, LLM scheduling
and transport, response schemas, and process-report/correction-ledger assembly.
generation and application, prompt context, runner orchestration, PromptKit
execution and observation mapping, response schemas, and process-report/correction-ledger assembly.
Domain packages provide correction modules, built-in validator chains and
execution metadata, and embedded prompt assets.

View File

@@ -49,7 +49,7 @@ boundary for the new report summary.
CLI orchestration labels failures at the phase boundary. Runner execution failures
can return completed or partial module results, which report assembly preserves.
[Report fields](../integrations/output-files.md#process-report) define how consumers
see these outcomes; [CLI semantics](../cli.md#process-output-and-exit-behavior)
see these outcomes; [CLI semantics](../cli.md#streams-and-exit-status)
define streams and exits. Diagnostic write failures do not always abort processing.
## Ordering and State

122
docs/migrations/v2.md Normal file
View File

@@ -0,0 +1,122 @@
# Migrating to Audita v2
Audita v2 adopts ConfigKit for configuration and PromptKit for LLM execution.
Version-1 configuration files, environment overlays, and retired flags are not
accepted. Copy a maintained [local](../../examples/minimal-config.yml) or
[service](../../examples/production-config.yml) example, then select real models
in the accompanying `examples/profiles/` files. A testing overlay is shown at
`examples/overlays/testing.yml`; its model choice carries no cost, speed, or
quality guarantee.
## Configuration and invocation
Use `version: 2` in a YAML main file. `--config` selects that file; otherwise
`process` and `config print-effective` look at
`/usr/local/etc/audita/config.yml`, then `/etc/audita/config.yml`.
`config validate` requires `--config`. `AUDITA_CONFIG` no longer selects a file.
Settings resolve from defaults, main file, optional `--overlay <name>` in
`overlays/<name>.yml`, then explicitly supplied setting flags. General `AUDITA_*`
environment overlays no longer apply. Unknown fields and version-1 files fail;
there is no automatic converter.
Keep transcript as the positional path and `--glossary`, `--output`, and
`--report-json` as invocation paths. Keep `pipeline.modules` / `--modules` and
`output.schema` / `--output-schema`. An explicit `pipeline.modules: []` or
`--modules ''` runs no correction modules; omission uses the default sequence.
Ordered repetitions still matter.
Move `llm.proposal.base_url` and `llm.validation.base_url` to a shared
`backends.<id>.endpoint` when both roles use one service. Move old role models to
PromptKit profile files and select them with `llm.proposal_profile` and
`llm.validation_profile`, or `--proposal-profile` and
`--validation-profile`. Omitted validation selection inherits the final proposal
selection; explicit empty or null is invalid. `profiles_dir` defaults to
`profiles` beside the main config. Complete profiles name a registered backend
and model. Per-role `--base-url`, `--validation-base-url`, `--model`, and
`--validation-model` are retired.
Move old `llm.*.api_key_env` to `backends.<id>.credential`, a scalar name of a
nonempty environment variable, such as `credential: AUDITA_SERVICE_KEY`.
Credentials on every surviving backend must resolve, including unselected
backends. Omit or set null for an unauthenticated local backend. Profiles cannot
perform another key lookup. `--llm-api-key` and `--validation-llm-api-key` are
retired. `AUDITA_LLM_API_KEY`, `OPENROUTER_API_KEY`, and
`AUDITA_VALIDATION_LLM_API_KEY` have no implicit meaning, though any of those
names can be used explicitly as a credential reference.
Replace `concurrency.total_llm`, `.proposal_llm`, and `.validation_llm` plus
`--total-llm-concurrency`, `--llm-concurrency`,
`--proposal-llm-concurrency`, and `--validation-llm-concurrency` with
`backends.<id>.concurrency` (default 1) and optional `queue_capacity` (default
1024; zero means no waiting admission). One backend capacity is shared by both
roles and all retry/repair attempts within this Audita process. It does not
coordinate other processes.
Move old role `timeout` values and `--llm-timeout-seconds` /
`--validation-llm-timeout-seconds` to profile `timeout_seconds`, a per-attempt
limit, and `llm.operation_timeout`, a Go duration such as `15m` covering the
whole logical call. Replace old `max_retries` and `--max-retries` /
`--validation-max-retries` with common `llm.transient_retries` (additional
eligible attempts per generation) and `llm.repair_attempts` (additional
generations for structurally invalid JSON). Explicit zero disables either
feature. Cancellation and the operation deadline cover capacity waiting,
backoff, generation, and repairs. PromptKit owns which failures can retry or
repair; Audita does not retry semantic rejections.
The following domain settings and explicit flags retain their meaning:
| File field | Explicit flag |
| --- | --- |
| `validation.max_prompt_tokens` | `--validation-max-prompt-tokens` |
| `chunking.target_sections` | `--target-sections` |
| `chunking.max_section_tokens`, `.min_section_tokens` | `--max-section-tokens`, `--min-section-tokens` |
| `normalization.max_segment_gap`, `.ellipsis_gap`, `.max_segment_duration`, `.max_segment_tokens` | `--normalize-max-segment-gap`, `--normalize-ellipsis-gap`, `--normalize-max-segment-duration`, `--normalize-max-segment-tokens` |
| `thresholds.glossary`, `.homophones`, `.spoken_word`, `.grammar` | `--glossary-confidence-threshold`, `--homophones-confidence-threshold`, `--spoken-word-confidence-threshold`, `--grammar-confidence-threshold` |
| `context.description` | `--transcript-description` |
| `diagnostics.work_dir`, `.retention` | `--work-dir`, `--work-dir-retention` |
Normalization gaps and durations are **numeric seconds**, so replace `4s`,
`3.5s`, and `60s` with `4.0`, `3.5`, and `60.0`. The operation timeout remains a
Go duration string. `chunking.target_sections` may be omitted or null for
automatic chunking; zero is invalid. Null is not a general synonym for omission:
nonoptional scalar settings reject it. Empty `context.description` clears
context. Retention behavior is unchanged.
Remove old implicit variables `AUDITA_MODULES`, `AUDITA_MODEL`,
`AUDITA_VALIDATION_MODEL`, `AUDITA_BASE_URL`, `AUDITA_VALIDATION_BASE_URL`,
`AUDITA_LLM_TIMEOUT_SECONDS`, `AUDITA_VALIDATION_LLM_TIMEOUT_SECONDS`,
`AUDITA_MAX_RETRIES`, `AUDITA_VALIDATION_MAX_RETRIES`,
`AUDITA_TOTAL_LLM_CONCURRENCY`, `AUDITA_LLM_CONCURRENCY`,
`AUDITA_PROPOSAL_LLM_CONCURRENCY`, `AUDITA_VALIDATION_LLM_CONCURRENCY`,
`AUDITA_VALIDATION_MAX_PROMPT_TOKENS`, `AUDITA_TARGET_SECTIONS`,
`AUDITA_MAX_SECTION_TOKENS`, `AUDITA_MIN_SECTION_TOKENS`,
`AUDITA_NORMALIZE_MAX_SEGMENT_GAP`, `AUDITA_NORMALIZE_ELLIPSIS_GAP`,
`AUDITA_NORMALIZE_MAX_SEGMENT_DURATION`, `AUDITA_NORMALIZE_MAX_SEGMENT_TOKENS`,
`AUDITA_GLOSSARY_CONFIDENCE_THRESHOLD`,
`AUDITA_HOMOPHONES_CONFIDENCE_THRESHOLD`,
`AUDITA_SPOKEN_WORD_CONFIDENCE_THRESHOLD`,
`AUDITA_GRAMMAR_CONFIDENCE_THRESHOLD`, `AUDITA_WORK_DIR`, and
`AUDITA_WORK_DIR_RETENTION`. Put those values in the named fields above or pass
their surviving explicit flags.
## Inspecting and operating
Run `audita config validate --config <file>` before processing. Use
`audita config print-effective --config <file> --view settings|profiles|explain`
to inspect redacted ConfigKit YAML, selected PromptKit profile controls, or
field origins and derivations. Each view resolves both roles offline without
calling a provider. `--overlay testing` selects the same overlay for validation,
inspection, and processing. The old effective-config JSON format is replaced by
YAML; settings YAML alone does not show profile-owned model controls.
The [process report](../integrations/output-files.md#process-report) now uses
schema `v2` and includes an `llm_summary` when execution observations are
available. Logical calls are distinct from provider attempts; usage fields carry
known and complete flags. Utilization is grouped by backend capacity rather than
old scheduler and active-concurrency fields. Retained runs write
`effective-config.redacted.yaml`, `selected-profiles.yaml`,
`config-origins.yaml`, `utilization-diagnostics.json`, and
`llm-execution.json`. Request and response artifacts remain redacted debugging
material. See [operations](../operations.md#diagnostics-directory) for retention
and paths, [configuration](../config.md) for all settings, and the
[CLI reference](../cli.md) for current flags.

View File

@@ -8,9 +8,9 @@ for logical formats.
## Normal Workflow
1. Choose a configuration and validate it using the
[configuration commands](cli.md#configuration-commands).
[configuration commands](cli.md#commands-and-help).
2. Prepare input files conforming to the [input contract](integrations/transcript-glossary-files.md).
3. Run the [processing command](cli.md#process-examples), choosing explicit output
3. Run the [processing command](cli.md#commands-and-help), choosing explicit output
and report destinations when those files must survive run-directory cleanup.
4. Inspect the corrected transcript and process report. Review rejected or skipped
corrections when the report indicates them; successful execution does not mean
@@ -25,7 +25,7 @@ artifact writes can fail, so a run directory need not contain every listed file.
## Output Files
The caller chooses transcript and optional external report destinations using the
[CLI](cli.md#process-output-and-exit-behavior). These are separate from the per-run
[CLI](cli.md#streams-and-exit-status). These are separate from the per-run
diagnostic directory unless the caller deliberately places them inside it.
Choose existing, writable parent directories. Output/report files can be overwritten
by subsequent runs using the same destinations.
@@ -99,7 +99,7 @@ artifact storage facility.
## Failure Inspection and Recovery
1. Read stderr and the exit classification defined by the [CLI](cli.md#process-output-and-exit-behavior).
1. Read stderr and the exit classification defined by the [CLI](cli.md#streams-and-exit-status).
2. Locate the external report, or the run directory identified by the error when
available. If a late error omits the path, inspect the selected work directory.
3. Read the available report and error log. Use

View File

@@ -7,7 +7,7 @@ source/tag push; binary delivery runs asynchronously.
## Prerequisites
Use Bash, Git, and Go 1.25 or newer (CI uses Go 1.25), with dependencies available
Use Bash, Git, and Go 1.25.5 or newer (CI uses Go 1.25.5 or newer), with dependencies available
locally or downloadable. Tests themselves are offline. Work from a
checkout of `main` with permission to push that branch and tags to `origin`.
For this repository, `origin` is `https://gitea.maximumdirect.net/eric/audita.git`.

View File

@@ -535,7 +535,17 @@ JSON golden dumps. Reuse Stage 9 evidence rather than retest every event sequenc
## Stage 11: Finish Examples, Migration Guidance, and Removal
**Status:** Pending. **Depends on:** Stages 1–10.
**Status:** Complete. Maintained schema-2 local and service examples, role profiles,
and a testing overlay validate offline with a synthetic credential. The v2
migration guide maps retired files, flags, environment inputs, timing, secrets,
inspection, and reports. Removed the isolated prototype and obsolete CLI test
fixtures/hooks; production config, runtime, proposal, validator, and runner tests
retain their respective boundaries, with assembled subprocess coverage assigned
to Stage 12. Canonical references, Go requirements, historical links, and the
retry follow-up were updated. `go mod tidy`, `go mod verify`, focused tests,
local-link validation, and `git diff --check` passed.
**Depends on:** Stages 1–10.
**Owns:** maintained examples, affected canonical documentation, obsolete code/tests,
prototype removal, and final dependency cleanup.

View File

@@ -12,12 +12,12 @@ evidence; they do not establish overlay integration.
## Selected Boundary
The prototype lives in [examples/library-adoption](../../examples/library-adoption/README.md),
a separate Go module now using published ConfigKit v0.4.0, PromptKit v0.11.0, and
The former prototype is retained in Git history at `590604b`; it was a separate
Go module using published ConfigKit v0.4.0, PromptKit v0.11.0, and
Audita v1.1.0. The Audita dependency lets this consumer reuse actual internal domain
helpers through its Audita module namespace without a local replacement. It does
not make those internal packages a new public API. Remove this dependency along
with the prototype when equivalent production integration tests exist.
with the prototype after equivalent production integration tests landed.
The initial prototype results below were recorded with ConfigKit v0.3.1. On
2026-09-24, upgrading only its ConfigKit pin and sums to v0.4.0 required no source

View File

@@ -4,10 +4,11 @@
**Status:** Original boundary-design and offline-prototype unit completed;
overlay extension and production-planning handoff completed, 2026-09-24.
Production migration has not started. The work packages below retain the original
scope; [decisions, evidence, limitations, and handoff](library-boundary-findings.md)
record the outcome. The [executable prototype](../../examples/library-adoption/README.md)
is isolated from production. No publication or upstream changes were made.
The work packages below retain the original scope;
[decisions, evidence, limitations, and handoff](library-boundary-findings.md)
record the outcome. The executable prototype was removed after production
integration; Git history at `590604b` retains it. No publication or upstream
changes were made in that historical unit.
## Next Unit: Testing Overlays and Production Migration Plan

View File

@@ -1,6 +1,6 @@
# Runtime Behavior Decisions
**Last updated:** 2026-09-23
**Last updated:** 2026-09-24
The documentation migration identified these existing behaviors for separate
product consideration. Implementation status and any selected direction are noted
@@ -9,10 +9,8 @@ below; this document does not claim that runtime changes have landed.
- Decide whether the `never` retention selection should delete successful-run
diagnostics; current behavior is documented in
[operations](../operations.md#retention-behavior).
- The [library-adoption plan](library-adoption.md) selects explicit zero as disabling
transient retries, alongside separately enabled structural repair. Implement this
as part of the migration; the current adapter behavior documented in
[configuration](../config.md) remains unchanged until then.
- Resolved: [configuration](../config.md) now treats explicit zero as disabling
transient retries or structural repair independently; PromptKit performs both.
- Decide whether missing or null input timestamps should be rejected; current
decoding is documented in the [input contract](../integrations/transcript-glossary-files.md).