From 900ad749589c8b8796bd82606f3fcd5cc6e341ef Mon Sep 17 00:00:00 2001 From: Eric Rakestraw Date: Sun, 24 May 2026 01:10:29 +0000 Subject: [PATCH] Consolidate development policy docs and remove legacy documentation paths --- docs/architecture.md | 16 ---- docs/architecture/architecture.md | 5 - docs/architecture/diagnostics.md | 5 - docs/architecture/output-schemas.md | 3 - docs/architecture/prompts.md | 3 - docs/architecture/public-contract.md | 121 ----------------------- docs/architecture/structured-llm.md | 3 - docs/architecture/validators.md | 3 - docs/development.md | 5 - docs/documentation/policy.md | 27 ------ docs/policy/architecture.md | 16 ++-- docs/policy/development.md | 137 ++++++++++++++++++++++----- docs/release-checklist.md | 128 ------------------------- 13 files changed, 121 insertions(+), 351 deletions(-) delete mode 100644 docs/architecture.md delete mode 100644 docs/architecture/architecture.md delete mode 100644 docs/architecture/diagnostics.md delete mode 100644 docs/architecture/output-schemas.md delete mode 100644 docs/architecture/prompts.md delete mode 100644 docs/architecture/public-contract.md delete mode 100644 docs/architecture/structured-llm.md delete mode 100644 docs/architecture/validators.md delete mode 100644 docs/development.md delete mode 100644 docs/documentation/policy.md delete mode 100644 docs/release-checklist.md diff --git a/docs/architecture.md b/docs/architecture.md deleted file mode 100644 index c538f2c..0000000 --- a/docs/architecture.md +++ /dev/null @@ -1,16 +0,0 @@ -# Audita Architecture Index - -This file is the architecture documentation entrypoint. - -Implemented internals: -- [Internal Overview](./internal/overview.md) -- [Pipeline](./internal/pipeline.md) -- [Modules](./internal/modules.md) -- [Validators](./internal/validators.md) -- [LLM Runtime](./internal/llm-runtime.md) -- [Diagnostics and Reporting](./internal/diagnostics-reporting.md) -- [Prompts](./internal/prompts.md) -- [Output Schemas](./internal/output-schemas.md) - -External/runtime contract summary: -- [Public Contract](./architecture/public-contract.md) diff --git a/docs/architecture/architecture.md b/docs/architecture/architecture.md deleted file mode 100644 index 466473a..0000000 --- a/docs/architecture/architecture.md +++ /dev/null @@ -1,5 +0,0 @@ -# Moved: Internal Overview - -Implemented internal architecture docs now live in [`docs/internal/`](../internal/). - -Start with [`docs/internal/overview.md`](../internal/overview.md). diff --git a/docs/architecture/diagnostics.md b/docs/architecture/diagnostics.md deleted file mode 100644 index 307019f..0000000 --- a/docs/architecture/diagnostics.md +++ /dev/null @@ -1,5 +0,0 @@ -# Moved: Diagnostics and Reporting - -Implemented diagnostics and reporting internals now live at: -- [`docs/internal/diagnostics-reporting.md`](../internal/diagnostics-reporting.md) -- [`docs/operations.md`](../operations.md) diff --git a/docs/architecture/output-schemas.md b/docs/architecture/output-schemas.md deleted file mode 100644 index dbd5f58..0000000 --- a/docs/architecture/output-schemas.md +++ /dev/null @@ -1,3 +0,0 @@ -# Moved: Output Schemas - -Implemented output schema registry docs now live at [`docs/internal/output-schemas.md`](../internal/output-schemas.md). diff --git a/docs/architecture/prompts.md b/docs/architecture/prompts.md deleted file mode 100644 index 1e2b909..0000000 --- a/docs/architecture/prompts.md +++ /dev/null @@ -1,3 +0,0 @@ -# Moved: Prompt Registry - -Implemented prompt registry and prompt metadata docs now live at [`docs/internal/prompts.md`](../internal/prompts.md). diff --git a/docs/architecture/public-contract.md b/docs/architecture/public-contract.md deleted file mode 100644 index 3b8846f..0000000 --- a/docs/architecture/public-contract.md +++ /dev/null @@ -1,121 +0,0 @@ -# Audita Public Contract - -## Scope -This document defines stability expectations for Audita's external runtime interfaces. - -Covered interfaces: -- CLI commands and major flags; -- versioned config behavior and precedence; -- transcript/glossary input forms; -- output schema selection; -- report schema metadata; -- diagnostics artifact path metadata; -- stdout/stderr and exit-code behavior; -- redaction guarantees. - -## CLI contract -Stable commands: -- `audita process` -- `audita config validate` -- `audita config print-effective` - -Stable high-value `process` flags: -- `--config` -- `--glossary` -- `--output` -- `--report-json` -- `--modules` -- `--output-schema` - -## Config contract -Supported config format: -- YAML; -- `version: 1`; -- strict unknown-field rejection. - -Path resolution for `process` and `config print-effective`: -1. `--config` -2. `AUDITA_CONFIG` -3. `/usr/local/etc/audita/config.yml` -4. `/etc/audita/config.yml` - -Missing explicit path is an error. Missing default paths is non-fatal. - -Precedence for `process`: -1. defaults -2. file config -3. environment overrides -4. CLI overrides - -`config validate` remains file-only validation (defaults + file config; no env overrides). - -Module and output-schema keys are validated against built-in catalogs. Unknown keys fail validation. - -## Input contract -Supported transcript JSON top-level forms: -- array of segments -- object with `segments` array - -Supported glossary YAML form: -- top-level `glossary` list with required entry fields validated by schema parsing. - -## Output schema contract -Supported transcript output schemas: -- `bare-segments` (default) -- `audita-v1` - -Unknown schema keys fail before output write. - -## Report metadata contract -Process reports include stable report metadata fields: -- `report_schema_name` -- `report_schema_version` -- `output_schema` -- `config_version` (when file config is loaded) - -Current values: -- `report_schema_name = audita-process-report` -- `report_schema_version = v1` - -`--report-json` output and run-directory `report.json` use the same report schema metadata. - -Validator decision/rejection records use stable validator keys via `validator_name`. - -## Diagnostics metadata contract -When run-directory initialization succeeds, diagnostics metadata paths reference stable artifacts, including: -- transcript and normalization artifacts; -- chunking summary; -- invocation metadata; -- redacted effective config; -- utilization diagnostics; -- correction ledger; -- `error.log` on failures. - -LLM interaction diagnostics include stable prompt and structured-schema identifiers where applicable. - -## Stdout/stderr and exit codes -Success: -- with `--output`, stdout is empty; -- without `--output`, stdout contains transcript JSON only; -- report JSON is not written to stdout. - -Failures: -- nonzero exit; -- human-readable stderr summary; -- diagnostics directory path on stderr when available. - -Exit codes: -- `0` success -- nonzero failure - -## Redaction contract -Configured secrets are redacted from: -- effective config outputs; -- diagnostics artifacts; -- report artifacts; -- surfaced adapter/runtime errors. - -## Compatibility policy -Stable command behavior, schema names, report metadata keys, diagnostics-path field semantics, and validator key identities are treated as public contract. - -Additive fields are acceptable when existing fields and behavior remain compatible. diff --git a/docs/architecture/structured-llm.md b/docs/architecture/structured-llm.md deleted file mode 100644 index 91d1092..0000000 --- a/docs/architecture/structured-llm.md +++ /dev/null @@ -1,3 +0,0 @@ -# Moved: LLM Runtime - -Implemented structured LLM runtime docs now live at [`docs/internal/llm-runtime.md`](../internal/llm-runtime.md). diff --git a/docs/architecture/validators.md b/docs/architecture/validators.md deleted file mode 100644 index b55f6d1..0000000 --- a/docs/architecture/validators.md +++ /dev/null @@ -1,3 +0,0 @@ -# Moved: Validators - -Implemented validator architecture docs now live at [`docs/internal/validators.md`](../internal/validators.md). diff --git a/docs/development.md b/docs/development.md deleted file mode 100644 index 2cf1d87..0000000 --- a/docs/development.md +++ /dev/null @@ -1,5 +0,0 @@ -# Moved: Development Workflow - -The canonical contributor workflow now lives at [`docs/policy/development.md`](policy/development.md). - -This file is retained temporarily as a migration shim for older links. diff --git a/docs/documentation/policy.md b/docs/documentation/policy.md deleted file mode 100644 index bfefc16..0000000 --- a/docs/documentation/policy.md +++ /dev/null @@ -1,27 +0,0 @@ -# Documentation Policy - -## Scope -This policy defines how project documentation should be authored and maintained. - -## Core rules -- Document the current behavior of the codebase. -- Remove stale behavior descriptions promptly when code changes. -- Do not describe development history in architecture or behavior docs unless a document is explicitly historical. -- Do not use architecture or behavior docs as changelogs. -- Prefer rewriting stale sections from scratch when substantial behavior or ownership changes occur. - -## Consistency requirements -- Keep command examples aligned with current CLI surfaces. -- Keep configuration examples aligned with supported fields and precedence. -- Keep architecture package ownership descriptions aligned with current code layout. -- Keep stable contract identifiers accurate (module keys, validator keys, output-schema keys, report metadata fields). - -## Cross-document expectations -- `docs/architecture/*` documents runtime behavior and package ownership. -- `docs/configuration.md` documents config schema and precedence. -- `docs/development.md` documents contributor workflow and engineering conventions. - -## Review expectations for documentation changes -- Verify referenced files and links exist. -- Verify examples match current behavior. -- Prefer concise, direct language and avoid speculative future claims. diff --git a/docs/policy/architecture.md b/docs/policy/architecture.md index 66cf128..474eb29 100644 --- a/docs/policy/architecture.md +++ b/docs/policy/architecture.md @@ -25,14 +25,12 @@ The current built-in modules are `glossary`, `homophones`, `spoken_word`, and `g For external behavior and compatibility details, prefer links to existing behavior docs: -- [Architecture overview](../architecture/architecture.md) -- [Public contract](../architecture/public-contract.md) -- [Diagnostics](../architecture/diagnostics.md) -- [Structured LLM](../architecture/structured-llm.md) -- [Validators](../architecture/validators.md) -- [Prompts](../architecture/prompts.md) -- [Output schemas](../architecture/output-schemas.md) -- [Configuration](../configuration.md) +- [CLI reference](../cli.md) +- [Configuration](../config.md) +- [Operations](../operations.md) +- [Troubleshooting](../troubleshooting.md) +- [Integration docs](../integrations/subprocess.md) +- [Internal docs](../internal/overview.md) ## Core Design Principles @@ -124,7 +122,7 @@ Config behavior is owned by `internal/core/config`; command usage and process wi `audita process` uses implemented precedence: defaults, config file, environment, then CLI flags. `config validate` validates defaults plus a file config and intentionally does not apply environment overrides. `config print-effective` applies defaults, file config, and environment overrides, then prints redacted JSON. -Do not duplicate full CLI or config reference material here. Use [Configuration](../configuration.md), the README, and [Public contract](../architecture/public-contract.md) for current external behavior. +Do not duplicate full CLI or config reference material here. Use [Configuration](../config.md), [CLI reference](../cli.md), [Operations](../operations.md), and integration docs under [`docs/integrations/`](../integrations/subprocess.md) for current external behavior. When adding config fields or CLI flags, update: diff --git a/docs/policy/development.md b/docs/policy/development.md index 614b211..3502b8a 100644 --- a/docs/policy/development.md +++ b/docs/policy/development.md @@ -1,33 +1,124 @@ # Audita Development Workflow ## Scope -This document defines the canonical contributor workflow and engineering conventions for this repository. -## Workflow -1. Start from a clean understanding of scope and constraints. -2. Make focused changes that preserve existing public behavior unless behavior change is explicitly intended. -3. Run targeted tests for touched packages. -4. Run `go test ./...` before finalizing substantial changes. -5. Update affected documentation so it describes current behavior only. +This is the canonical contributor workflow for Audita maintainers and coding agents. -## Engineering conventions -- Keep module packages separate: `glossary`, `homophones`, `spoken_word`, `grammar`. -- Prefer narrow shared helpers and catalogs over broad abstractions. -- Preserve diagnostics artifact naming and report field contracts unless intentionally changed. -- Preserve CLI/config precedence semantics unless intentionally changed. -- Treat stable validator keys, prompt identifiers, and output-schema keys as contract surfaces. +It defines: +- repository layout and boundaries; +- setup and test commands; +- expectations for code changes; +- how to add config, CLI, modules, validators, docs, and examples. -## Configuration and runtime expectations -- `audita process` precedence is defaults -> file -> env -> CLI. -- `audita config validate` validates file config merged onto defaults only. -- `audita config print-effective` includes environment overrides and prints redacted JSON. +## Setup -## Testing expectations -- Add tests for new behavior and for bug fixes. -- Keep deterministic fixtures stable. -- Do not reduce existing parity, release-fixture, subprocess, or module-specific coverage without equivalent replacement. +Prerequisites: +- Go `1.24` or newer. + +Common commands: + +```sh +go test ./... +go build ./cmd/audita +``` + +## Repository Layout + +Top-level areas: +- `cmd/audita`: executable entrypoint. +- `internal/cli`: command parsing and process/config command orchestration. +- `internal/core`: deterministic config/schema/normalization/chunking/output/diagnostics/reporting logic. +- `internal/framework`: runner orchestration, contracts, proposal generation/application, validators runtime, LLM runtime, response schemas. +- `internal/modules/*`: module-specific correction behavior. +- `internal/validators/*`: validator implementations, chains, and metadata. +- `internal/prompts`: embedded prompts and prompt metadata. +- `docs/`: canonical documentation. +- `examples/`: maintained copyable inputs/configs. + +## Change Workflow + +1. Confirm scope and behavior contract before editing. +2. Make focused changes in the appropriate ownership area. +3. Add or update tests for changed behavior. +4. Run targeted package tests for touched areas. +5. Run `go test ./...` for substantial changes. +6. Update docs/examples when external behavior changes. + +## How To Add or Change Configuration + +1. Add fields/defaults/validation under `internal/core/config`. +2. Apply source precedence correctly (defaults, file, env, CLI for `process`). +3. Ensure `config validate` remains file-only and `config print-effective` remains redacted. +4. Update tests in `internal/core/config` and related CLI tests. +5. Update [`docs/config.md`](../config.md) and relevant examples under `examples/`. + +## How To Add or Change CLI Behavior + +1. Implement parsing/wiring in `internal/cli`. +2. Keep stdout/stderr and exit behavior compatible unless intentional and documented. +3. Update CLI tests under `internal/cli` and integration tests under `cmd/audita`. +4. Update [`docs/cli.md`](../cli.md) and related integration docs. + +## How To Add or Change Modules + +1. Add or update one module package under `internal/modules/`. +2. Keep module-specific prompt ownership in the module + `internal/prompts`. +3. Wire module registration/catalog resolution through framework/core module catalog code. +4. Verify replacement policy and validator chain selection. +5. Add/update module tests and proposal-generation tests. +6. Update internal docs when behavior/contracts change. + +## How To Add or Change Validators + +1. Implement validator behavior in `internal/validators` and shared runtime pieces in `internal/framework/validators` only when needed. +2. Preserve stable validator keys and decision semantics where already exposed. +3. Keep deterministic vs LLM-backed execution-class behavior explicit. +4. Add/update validator, chain, batching, and malformed-output tests. +5. Update validator documentation when external or developer-facing behavior changes. + +## Documentation and Examples Expectations + +- Keep one canonical home per topic (see [`docs/policy/documentation.md`](documentation.md)). +- Do not document future/unimplemented behavior outside `docs/roadmap/`. +- Keep command examples and config/examples in sync with current code. +- Keep examples secret-free and copyable. + +## Practical Validation Checklist + +Use this checklist for meaningful runtime-impacting changes: + +1. Run core tests: + +```sh +go test ./... +``` + +2. Verify config commands and examples: + +```sh +go run ./cmd/audita config validate --config examples/minimal-config.yml +go run ./cmd/audita config validate --config examples/production-config.yml +go run ./cmd/audita config print-effective --config examples/minimal-config.yml +``` + +3. Re-check subprocess/runtime contract when touching CLI/process/report paths: +- `--output` success keeps stdout empty; +- no `--output` success writes transcript JSON to stdout; +- `--report-json` writes file output and is not written to stdout; +- failures return nonzero and include diagnostics path when available. + +4. Re-check diagnostics/report/redaction when touching LLM, reporting, or diagnostics code: +- report schema metadata fields remain present; +- diagnostics artifact paths remain valid; +- configured secret values remain redacted in reports/diagnostics/errors. + +5. Re-check output schema behavior when touching serialization/schema code: +- default `bare-segments` behavior remains correct unless intentionally changed; +- `audita-v1` behavior remains correct unless intentionally changed; +- unsupported schemas fail validation/resolve paths clearly. + +## Commit Discipline -## Commit discipline - Keep commits scoped and reviewable. - Avoid mixing unrelated refactors with behavior changes. -- Use clear plain-English commit messages. +- Use concise plain-English commit messages. diff --git a/docs/release-checklist.md b/docs/release-checklist.md deleted file mode 100644 index 23d40f1..0000000 --- a/docs/release-checklist.md +++ /dev/null @@ -1,128 +0,0 @@ -# Audita Release Checklist - -Use this checklist before cutting a pre-1.0 or 1.0 release candidate. - -## Core test pass - -- Run: - - `go test ./...` -- Confirm tests pass without live LLM credentials and without Python dependencies. - -## Config validation and precedence - -- Validate a representative config: - - `audita config validate --config ` -- Inspect redacted effective config: - - `audita config print-effective --config ` -- Confirm precedence behavior: - - defaults -> file config -> environment -> CLI. -- Confirm default config search order: - - `/usr/local/etc/audita/config.yml` first, then `/etc/audita/config.yml`. -- Confirm missing both default-path config files is non-fatal when `--config`/`AUDITA_CONFIG` are unset. - -## Output schema checks - -- Verify default output schema remains `bare-segments`. -- Verify `--output-schema audita-v1` emits object payload with `schema` and `version`. -- Verify unknown schema (for example `seriatim-intermediate`) fails clearly. - -## Subprocess contract checks - -- With `--output`, verify stdout is empty on success. -- Without `--output`, verify stdout contains transcript JSON only. -- Verify `--report-json` writes file output and does not write report JSON to stdout. -- Verify failure stderr remains human-readable and includes diagnostics path when available. -- Verify nonzero exit on failures. - -## Structured LLM checks - -- Verify runtime uses the Audita-owned OpenAI-compatible adapter. -- Verify structured response schemas are attached via `response_format.type=json_schema`. -- Verify diagnostics metadata includes structured schema `id/version/name/sha256`. -- Verify provider output is still locally decoded/validated before use. -- Verify malformed module-stage structured payloads degrade to warnings/rejections instead of failing the run. - -## Report and diagnostics schema checks - -- Verify report metadata fields: - - `report_schema_name` - - `report_schema_version` - - `output_schema` - - `config_version` when file config is used. -- Verify diagnostics artifact references exist in reports: - - transcript/normalization/chunking/invocation/effective-config artifacts - - utilization diagnostics artifact - - correction ledger artifact - - error log on failures. - -## Redaction checks - -- Verify secrets are redacted from: - - `effective-config.json` - - run-dir and `--report-json` reports - - LLM request/response/error diagnostics payloads. -- Verify no API keys/bearer tokens leak into fixtures or outputs. - -## Prompt and validator metadata checks - -- Verify prompt metadata appears in LLM request metadata diagnostics: - - `prompt_id`, `prompt_version`, `prompt_source`, `embedded_path`, `sha256`. -- Verify stable validator keys appear in report decisions/rejections. -- Verify module warning records appear in reports for malformed proposal-generation payloads and malformed validator batches. -- Verify built-in validator chains resolve and execute for default and explicit module runs. - -## Utilization diagnostics checks - -- Verify `utilization-diagnostics.json` exists on successful runs. -- Verify partial utilization artifact behavior on controlled failure paths. -- Verify utilization fields are structurally present and nonnegative: - - effective concurrency - - run timing - - module timing summaries - - per-validator timing summaries. - -## Correction ledger checks - -- Verify `correction-ledger.json` exists on successful runs. -- Verify report references ledger artifact path. -- Verify ledger dispositions include applied/rejected and skipped/failed where exercised. -- Verify validator rejection and proposal-application skip remain distinct. - -## Pipeline behavior checks - -- Verify default full pipeline run remains: - - `glossary`, `homophones`, `glossary`, `spoken_word`, `grammar` - - with deterministic repeated instance naming (`glossary_1`, `glossary_2`). -- Verify explicit module runs (`--modules`) still work. - -## Failure and cancellation checks - -- Verify controlled failure paths retain diagnostics and produce best-effort failure reports. -- Verify malformed proposal-generation payloads keep exit code `0`, keep stderr empty on success, and record warnings in reports/diagnostics. -- Verify malformed validator payloads reject only the affected batch and do not fail the module. -- Verify timeout/cancellation paths exit nonzero, do not hang, and retain failure diagnostics when initialized. - -## Release fixture/idempotence checks - -- Run release fixtures (`internal/cli/testdata/release`) through `go test ./...`. -- Confirm fixture checks cover: - - must-apply and must-not-apply expectations - - protected-term survival - - report and diagnostics contracts - - output-schema checks - - prompt/schema metadata diagnostics - - utilization/ledger artifacts - - idempotence-oriented second pass no-op behavior with deterministic fake responses. - -## Deferred-feature guardrail - -- Confirm release docs do not claim support for deferred items: - - filesystem prompt overrides - - user-configurable validator chains - - arbitrary user-supplied output schemas - - resume/start-at/stop-after execution - - diff/check/propose-only modes - - generated transcript descriptions enabled by default - - interactive review UI - - UI/server wrapper - - provider benchmarking harness.