Replace structured LLM dependency with Audita adapter

docs/architecture.md

@@ -21,7 +21,7 @@ Implemented today:
 - Runtime validator models and deterministic validators.
 - Deterministic validator-chain execution in the runner with cardinality enforcement.
 - Module-level validator decision/rejection reporting.
-- Internal structured LLM client contract plus an `instructor-go`-backed adapter package.
+- Internal structured LLM client contract plus an Audita-owned OpenAI-compatible structured LLM adapter package.
 - Bounded FIFO LLM scheduler infrastructure with context-aware permit handling.
 - Runtime primary/validation LLM effective-config resolution helpers with validation inheritance.
 - Generic JSON prompt/response diagnostics writer primitives with secret redaction.
@@ -133,7 +133,7 @@ internal/framework/validators/
   llm_validators.go
 
 internal/framework/llm/
-  instructor_client.go
+  openai_compatible_client.go
   scheduler.go
   effective_config.go
   diagnostics.go
@@ -233,14 +233,33 @@ Current caveat:
 `internal/framework/contracts` now defines a typed structured-completion contract:
 - `StructuredLLMClient.CompleteStructured(ctx, req, out)`
 - caller-owned typed decode target via `out` pointer.
+- caller-selected structured response schema metadata via `StructuredCompletionRequest.ResponseSchema`.
 
-`internal/framework/llm` provides `InstructorClient`, an internal adapter over `github.com/jxnl/instructor-go`:
-- configurable `base_url`, model, optional API key, retries, mode, HTTP client, and request timeout;
+`internal/framework/llm` provides `OpenAICompatibleClient`, a direct `net/http` adapter over OpenAI-compatible chat completions:
+- configurable `base_url`, model, optional API key, retries, HTTP client, and request timeout;
 - OpenAI-compatible endpoint behavior (for example OpenAI/OpenRouter/local-compatible base URLs);
-- default mode is JSON mode (`ModeJSON`), with optional tool-call mode (`ModeToolCall`);
 - request message translation from `contracts.LLMMessage` to chat-completions messages;
+- strict `response_format.type = json_schema` with registered structured response schemas (`strict: true`, schema name, and schema body);
 - response metadata mapping (provider/model/token usage) into Audita-owned response types;
-- API-key redaction in adapter-returned errors.
+- API-key redaction in adapter-returned errors;
+- context cancellation and timeout propagation through request contexts and HTTP client timeouts;
+- bounded retry behavior for transient request failures and malformed retryable structured responses.
+
+Structured response schemas are owned by Audita in `internal/framework/responseschema` and currently include:
+- key `correction_set`:
+  - id `audita.correction_set`
+  - version `v1`
+  - name `audita_correction_set_v1`
+  - sha256 `05f8ff3fa04f68115c0cb1859d2656f51aa5c0bae8ff2470b2d4f6f531953195`
+- key `validator_decision_set`:
+  - id `audita.validator_decision_set`
+  - version `v1`
+  - name `audita_validator_decision_set_v1`
+  - sha256 `b73f4790b98fbb955f0aec5496dd8ce9a8fe14aa2f35c700b4b4e5634f106fd5`
+
+Provider-level structured output is treated as a guardrail, not a trust boundary:
+- the adapter decodes assistant message content into caller-owned structs;
+- proposal-generation and validator layers continue local validation (shape, cardinality, confidence bounds, and proposal-index semantics) before changes can be applied.
 
 Current runtime boundary:
 - the default CLI runtime path (without explicit module selection) instantiates the full production module sequence.
@@ -252,6 +271,14 @@ Current runtime boundary:
 - primary/validation effective-config resolution helpers, including validation inheritance fallback to total LLM concurrency settings;
 - generic interaction diagnostics primitives that write machine-readable JSON artifacts for request metadata, request payload, response payload, and optional error payload with secret redaction.
 
+Structured LLM diagnostics behavior:
+- proposal-generation and validator diagnostics include structured response schema metadata (`id`, `version`, `name`, `sha256`) when schema-driven calls are made;
+- API keys and bearer tokens are redacted from request/response/error diagnostics artifacts and surfaced errors.
+
+Dependency posture:
+- the runtime no longer depends on `instructor-go`;
+- structured LLM behavior is implemented through Audita-owned code paths behind `StructuredLLMClient`.
+
 LLM concurrency runtime behavior:
 - `total` concurrency bounds all proposal and validation LLM calls.
 - `proposal` concurrency adds a proposal-only sub-cap, composed with total.

docs/roadmap.md

@@ -67,6 +67,17 @@ The work is best handled in seven phases. The ordering is intentional:
 
 Remove the heavy `instructor-go` dependency and replace it with a small Audita-owned OpenAI-compatible structured-output adapter.
 
+## Implementation status (2026-05-13)
+
+This workstream is now implemented in the repository:
+- production runtime uses an Audita-owned OpenAI-compatible structured LLM adapter (`internal/framework/llm/openai_compatible_client.go`);
+- `StructuredLLMClient` remains the stable internal boundary used by proposal generation and validators;
+- structured response schemas are registered with stable IDs, versions, names, and SHA-256 hashes (`internal/framework/responseschema`);
+- schema metadata is attached to structured completion requests and included in diagnostics metadata;
+- `instructor-go` has been removed from runtime code and module dependencies.
+
+This status update applies only to the structured LLM dependency replacement workstream. Other roadmap workstreams remain planned unless explicitly marked otherwise.
+
 This should happen before 1.0 because structured LLM calls are core runtime infrastructure. Replacing this layer after 1.0 would risk subtle compatibility changes in request construction, schema strictness, retry behavior, error reporting, diagnostics, and provider compatibility.
 
 ## Design direction

docs/structured-llm.md

@@ -0,0 +1,90 @@
+# Structured LLM Architecture
+
+## Purpose
+
+This document describes Audita's structured LLM runtime boundary and adapter behavior.
+
+## Why Audita owns the adapter
+
+Audita owns a small structured LLM adapter so that core runtime behavior is controlled inside the repository:
+- request construction and schema handling are explicit and testable;
+- retries, timeouts, cancellation, and error redaction are consistent across modules and validators;
+- provider SDK types are not exposed outside the adapter boundary;
+- dependency weight and transitive provider-specific behavior are reduced.
+
+At runtime, the rest of Audita depends only on the internal contract:
+- `StructuredLLMClient`
+- `CompleteStructured(ctx, req, out)`
+
+## OpenAI-compatible request shape
+
+At a conceptual level, Audita sends chat completion requests with:
+- `model`
+- `messages` (role/content pairs)
+- `response_format`:
+  - `type = "json_schema"`
+  - `json_schema.name` (stable schema name)
+  - `json_schema.strict = true`
+  - `json_schema.schema` (registered JSON Schema payload)
+
+The adapter uses OpenAI-compatible `POST {base_url}/chat/completions` over `net/http`.
+
+## Structured response schema registry
+
+Structured response schemas are registered in `internal/framework/responseschema` with stable metadata:
+- schema key
+- schema ID
+- schema version
+- schema name (OpenAI-compatible `response_format` name)
+- raw JSON Schema payload
+- SHA-256 hash
+
+Current schemas:
+- `correction_set`:
+  - id `audita.correction_set`
+  - version `v1`
+  - name `audita_correction_set_v1`
+- `validator_decision_set`:
+  - id `audita.validator_decision_set`
+  - version `v1`
+  - name `audita_validator_decision_set_v1`
+
+## Provider compatibility assumptions
+
+Audita assumes an OpenAI-compatible chat-completions endpoint that:
+- accepts message arrays with model selection;
+- accepts `response_format.type = json_schema`;
+- returns a completion with assistant message content and optional usage metadata.
+
+Provider-specific differences are expected in strictness and error payload shapes, so the adapter treats provider output as untrusted until locally decoded.
+
+## Local decode and validation remain mandatory
+
+Provider-level structured output is a transport guardrail, not final validation.
+
+After receiving a response, Audita still:
+- decodes assistant content into typed request-specific structs;
+- validates proposal and validator payload invariants locally;
+- enforces deterministic validator/cardinality rules before any transcript application.
+
+This protects runtime correctness even when provider responses are malformed, partial, or semantically inconsistent.
+
+## Diagnostics and redaction
+
+When structured schemas are used, diagnostics metadata records:
+- schema ID
+- schema version
+- schema name
+- schema hash
+
+Diagnostics and surfaced errors preserve secret redaction:
+- API keys and bearer tokens are redacted from request/response/error artifacts;
+- redaction is applied before diagnostic files are written.
+
+## Runtime behavior guarantees
+
+The structured LLM path preserves existing runtime guarantees:
+- bounded LLM call execution through schedulers;
+- context-aware cancellation and timeout propagation;
+- retry behavior for transient failures and retryable malformed structured responses;
+- deterministic module/chunk/proposal/validator behavior outside provider nondeterminism.