Complete Phase 18 operational hardening

docs/architecture.md

@@ -43,8 +43,7 @@ Implemented today:
 - Explicit runtime support for `--modules spoken_word` through the production runner path.
 
 Not implemented in CLI runtime path today:
-- Operational hardening tasks beyond current runtime/reporting/diagnostics behavior.
-- Rollout and Python retirement work.
+- Rollout and Python retirement work (Phase 19).
 
 Current reality:
 - all production modules exist and are wired into the default runtime path.
@@ -66,7 +65,8 @@ Phase sequencing note:
 - Phase 15 spoken-word module implementation and explicit runtime wiring are complete;
 - Phase 16 default full pipeline integration is complete;
 - Phase 17 parity fixture suite is complete;
-- next recommended phase is Phase 18 (operational hardening and subprocess integration).
+- Phase 18 operational hardening and subprocess integration are complete;
+- next recommended phase is Phase 19 (documentation, rollout, and Python retirement).
 
 ## Actual Go package layout
 
@@ -492,17 +492,23 @@ Implemented tests currently cover:
 - production homophones module prompt constraints, proposal mapping, validator-chain behavior, confidence-threshold enforcement, diagnostics redaction, protected-term behavior, and explicit CLI/runtime integration (`internal/modules/homophones/*_test.go`, `internal/cli/run_test.go`, `internal/framework/runner/*_test.go`)
 - production spoken_word module prompt constraints, proposal mapping, validator-chain behavior, semantic guardrail behavior, confidence-threshold enforcement, diagnostics redaction, protected-term behavior, and explicit CLI/runtime integration (`internal/modules/spoken_word/*_test.go`, `internal/cli/run_test.go`, `internal/framework/runner/*_test.go`)
 - glossary-derived protected-term extraction and stable behavior (`internal/framework/validators/protected_terms_test.go`)
+- default full-pipeline runtime shape and ordering (`internal/cli/run_test.go`, `cmd/audita/main_integration_test.go`, `internal/cli/parity_test.go`)
+- subprocess operational hardening behavior including large-input, failure-mode, timeout/cancellation, backend-failure, and partial-progress paths (`cmd/audita/main_integration_test.go`)
+- report/diagnostics redaction and artifact-shape behavior across success and failure paths (`internal/cli/run_test.go`, `cmd/audita/main_integration_test.go`)
 
-Not covered yet (because not implemented): full default-sequence transcript-polishing runtime behavior as a single default path.
+## Operational hardening status (Phase 18)
+The runtime now includes hardened subprocess behavior for parent-process callers:
+- deterministic success/failure exit codes;
+- strict stdout/stderr separation suitable for machine orchestration;
+- failure stderr summaries that include diagnostics location when available;
+- retained failure diagnostics (`report.json`, `error.log`, and artifacts written before failure);
+- deterministic timeout/cancellation behavior in tests;
+- redaction coverage for API keys/secrets across reports, diagnostics artifacts, and surfaced errors.
 
-## Intended final architecture (not yet implemented)
-The intended end-state still matches the rewrite plan:
-- sequential module pipeline over a mutable working transcript
-- default full module-sequence integration in the standard runtime path
-- structured LLM proposal generation
-- deterministic and LLM validators
-- validator cardinality enforcement in pipeline execution
-- proposal application integrated per module stage
-- prompt/response diagnostics for LLM/module stages
+Operational caller guidance is documented in [`docs/subprocess-operations.md`](docs/subprocess-operations.md).
 
-Until those phases are implemented, documentation and external descriptions should treat the current Go CLI as deterministic preprocessing/reporting infrastructure, not a full LLM transcript polisher.
+## Remaining work (Phase 19)
+Remaining rewrite work is documentation/rollout/retirement-oriented:
+- repository-level rollout documentation and migration guidance;
+- Python retirement/archival decisions and related documentation cleanup;
+- final release-facing communication of Go-primary operational guidance.

docs/rewrite-notes.md

@@ -31,7 +31,7 @@ The Go rewrite is complete when both of the following are true:
 
 ## Current implementation status
 
-The Go rewrite is currently in a deterministic foundation stage.
+The Go rewrite now has feature-complete runtime behavior for the module pipeline plus Phase 18 operational hardening coverage.
 
 Implemented:
 - CLI command surface for `audita process`.
@@ -97,9 +97,8 @@ Implemented:
   - validation override behavior when validation fields are set.
 - Generic JSON diagnostics primitives for LLM interactions (request metadata, request payload, response payload, optional error payload) with secret redaction.
 
-Not yet implemented in runtime pipeline:
-- Operational hardening beyond current Phase 16 runtime/reporting/diagnostics scope.
-- Rollout/Python retirement work.
+Not yet implemented:
+- Phase 19 rollout and Python retirement/documentation transition work.
 
 ## Completed phases
 
@@ -237,7 +236,7 @@ Not implemented in Phase 8 (by design):
 
 ## Remaining work plan
 
-Next recommended phase: **Phase 18 (operational hardening and subprocess integration)**.
+Next recommended phase: **Phase 19 (documentation, rollout, and Python retirement)**.
 
 ## Phase 9: Structured LLM client and scheduler infrastructure
 
@@ -628,6 +627,38 @@ The Go binary should be safe to call from other Go applications and should not r
 - Operational docs are accurate.
 - `go test ./...` passes.
 
+### Phase 18 completion status
+
+Completed.
+
+Implemented:
+- Expanded subprocess integration coverage in `cmd/audita/main_integration_test.go` for:
+  - successful default full-pipeline runs with `--output` and without `--output`;
+  - successful `--report-json` writes;
+  - large-transcript subprocess behavior;
+  - missing/unreadable transcript and missing/malformed glossary failures (portable handling where required);
+  - unwritable output and unwritable report-json failure behavior (portable handling where required);
+  - malformed structured LLM response failure behavior;
+  - synthetic backend LLM failure behavior;
+  - timeout/cancellation behavior with deterministic context cancellation hooks;
+  - mid-pipeline failure with partial module progress preserved in reports.
+- Hardened subprocess failure stderr output to include diagnostics path when available.
+- Added deterministic test-only subprocess LLM/runtime hooks used only in helper-process tests:
+  - backend failure mode;
+  - malformed structured-output mode;
+  - block-until-cancel mode;
+  - mid-pipeline fail mode.
+- Added regression coverage for secret redaction across subprocess stdout/stderr/report/diagnostics artifacts.
+- Confirmed existing LLM adapter and scheduler tests continue to cover:
+  - timeout and context cancellation propagation;
+  - retry behavior;
+  - malformed output safety;
+  - permit release on error/cancellation.
+- Added focused subprocess-caller operational guidance in `docs/subprocess-operations.md`.
+
+Intentionally deferred:
+- Phase 19 documentation/rollout/Python-retirement transitions.
+
 ## Phase 19: Documentation, rollout, and Python retirement
 
 ### Purpose

docs/subprocess-operations.md

@@ -0,0 +1,85 @@
+# Audita Go Subprocess Operations
+
+This document describes how parent processes should invoke `audita process` safely in production orchestration.
+
+## Recommended command form
+
+Use explicit file outputs for orchestrated runs:
+
+```sh
+audita process <transcript.json> \
+  --glossary <glossary.yaml> \
+  --output <output-transcript.json> \
+  --report-json <report.json>
+```
+
+Recommended additions:
+- `--work-dir <dir>` to control diagnostics location.
+- `--work-dir-retention <always|auto|never>` to control retained run directories.
+- `--modules ...` only when intentionally overriding the default full sequence.
+
+## Stdout behavior
+
+- With `--output`: stdout is expected to be empty on success.
+- Without `--output`: stdout contains transcript JSON only on success.
+- Report JSON is never written to stdout.
+
+## Stderr behavior
+
+- Success path should be quiet or minimal human-readable logs.
+- Failure path writes concise human-readable errors.
+- When a diagnostics run directory exists, failure stderr includes its path.
+- Prompt/response diagnostic payloads are not streamed to stderr.
+
+## Output file behavior
+
+- `--output` writes transcript JSON to the provided path.
+- Output write failures return nonzero and surface actionable errors.
+- The command does not silently ignore output write errors.
+
+## Report JSON behavior
+
+- `--report-json` writes a machine-readable process report to the requested path.
+- Run-directory `report.json` is written independently under diagnostics.
+- Best-effort failure reports are emitted when possible without masking the primary failure.
+- Report write failures return nonzero with clear stderr messaging.
+
+## Diagnostics directory behavior
+
+- Each run creates (when possible) a per-run diagnostics directory.
+- Typical artifacts include transcript, normalization, chunking, invocation, effective config, LLM diagnostics, `report.json`, and `error.log` on failure.
+- Failed runs retain diagnostics.
+- Under `auto` retention, successful runs with skipped/rejected corrections are retained; clean successful runs may be removed.
+
+## Exit codes
+
+- `0`: success.
+- Nonzero: failure (input/schema/config/module/LLM/runtime/output/report/diagnostics errors).
+
+Treat any nonzero as a failed subprocess invocation.
+
+## Timeout and cancellation
+
+- Runtime operations propagate context cancellation and request timeouts through LLM/scheduler paths.
+- On cancellation or timeout, the process exits nonzero and should not hang.
+- If diagnostics were initialized before failure, failure artifacts remain available for debugging.
+
+## Secret redaction expectations
+
+API keys and configured secret values are redacted from:
+- reports (`--report-json` and run-dir `report.json`);
+- diagnostics artifacts (including effective config and LLM interaction artifacts);
+- surfaced adapter/runtime errors;
+- test fixtures and regression outputs.
+
+Parent-process logs should still avoid printing raw environment variables.
+
+## Parent-process pipe guidance
+
+To avoid deadlocks in orchestrators:
+- always read both stdout and stderr concurrently when invoking as a subprocess;
+- prefer file outputs (`--output`, `--report-json`) for machine workflows;
+- treat stderr as human-readable diagnostics, not structured data;
+- parse structured results from output/report files.
+
+For Go callers, prefer `exec.CommandContext` with explicit timeout/cancellation and buffered/streamed readers for both pipes.