diff --git a/docs/architecture.md b/docs/architecture.md index f3075a1..da03cf4 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -22,6 +22,7 @@ Implemented today: - Runtime validator models and deterministic validators. - Deterministic validator-chain execution in the runner with cardinality enforcement. - Module-level validator decision/rejection reporting. +- Internal structured LLM client contract plus an `instructor-go`-backed adapter package. Not implemented in CLI runtime path today: - Real module execution pipeline (`glossary`, `homophones`, `spoken_word`, `grammar`). @@ -31,7 +32,7 @@ Not implemented in CLI runtime path today: - End-to-end transcript polishing with real module behavior. Phase sequencing note: -- structured LLM client and scheduler infrastructure remain Phase 9 work; +- structured LLM client infrastructure is implemented, but scheduler and runtime wiring remain Phase 9 follow-up work; - LLM-backed validators remain Phase 10 work. ## Actual Go package layout @@ -88,6 +89,9 @@ internal/framework/runner/ internal/framework/validators/ models.go deterministic.go + +internal/framework/llm/ + instructor_client.go ``` ## Current CLI behavior @@ -168,6 +172,23 @@ Implemented config surfaces include: Current caveat: - LLM/module-related settings are mostly infrastructure-only today; runtime path does not execute LLM or modules. +## Implemented structured LLM infrastructure +`internal/framework/contracts` now defines a typed structured-completion contract: +- `StructuredLLMClient.CompleteStructured(ctx, req, out)` +- caller-owned typed decode target via `out` pointer. + +`internal/framework/llm` provides `InstructorClient`, an internal adapter over `github.com/jxnl/instructor-go`: +- configurable `base_url`, model, optional API key, retries, mode, HTTP client, and request timeout; +- OpenAI-compatible endpoint behavior (for example OpenAI/OpenRouter/local-compatible base URLs); +- default mode is JSON mode (`ModeJSON`), with optional tool-call mode (`ModeToolCall`); +- request message translation from `contracts.LLMMessage` to chat-completions messages; +- response metadata mapping (provider/model/token usage) into Audita-owned response types; +- API-key redaction in adapter-returned errors. + +Current runtime boundary: +- the CLI/runner runtime path does not instantiate this adapter yet; +- no production LLM requests are performed by `audita process`. + ## Implemented normalization behavior Normalization (`internal/core/normalization`) currently: - sorts by segment start time; diff --git a/docs/rewrite-notes.md b/docs/rewrite-notes.md index 659c2e7..5ff8aad 100644 --- a/docs/rewrite-notes.md +++ b/docs/rewrite-notes.md @@ -52,10 +52,17 @@ Implemented: - Deterministic validator-chain execution in the production runner. - Module reports including validator decisions and validator rejections. - Broad deterministic and CLI/subprocess test coverage for implemented phases through `go test ./...`. +- Internal typed structured LLM contract (`StructuredLLMClient.CompleteStructured(ctx, req, out)`). +- `internal/framework/llm` instructor-go-backed adapter with: + - configurable base URL/model/retries/mode/timeout + - optional API key support for local-compatible endpoints + - API-key redaction in returned errors + - typed structured decode into caller-provided outputs. Not yet implemented in runtime pipeline: - Real correction modules. -- Structured LLM client integration. +- Structured LLM scheduler/concurrency orchestration. +- Runtime wiring from production runner/modules into the structured LLM adapter. - LLM-backed validators. - Prompt/response diagnostics for LLM calls. - End-to-end transcript polishing behavior. @@ -189,37 +196,44 @@ Implemented: Not implemented in Phase 8 (by design): - LLM-backed validators (Phase 10). -- Structured LLM client implementation or scheduler behavior (Phase 9). +- Structured LLM scheduler behavior or runtime wiring (Phase 9 follow-up). - Real correction modules. - Prompt/response diagnostics. - End-to-end transcript polishing. ## Remaining work plan -Next recommended phase: **Phase 9 (structured LLM client and scheduler infrastructure)**. +Next recommended phase: **Phase 9 follow-up (scheduler + runtime LLM wiring, still no real modules)**. ## Phase 9: Structured LLM client and scheduler infrastructure +### Status + +Partially completed. + +Implemented in this phase so far: +- Added internal structured LLM contract support for caller-provided typed outputs. +- Added `internal/framework/llm` adapter backed by `github.com/jxnl/instructor-go`. +- Confirmed OpenAI-compatible base URL support through the adapter path. +- Added adapter unit tests for model/base URL handling, retries, context cancellation, optional API key behavior, and error redaction. + +Still pending in Phase 9: +- Scheduler/semaphore behavior for bounded concurrency. +- Runtime wiring in runner/module infrastructure (without introducing real modules yet). +- Prompt/response diagnostics writer primitives for LLM call artifacts. +- Full primary vs validation LLM config-resolution plumbing into runtime LLM call sites. + ### Purpose Implement the provider-neutral LLM infrastructure needed by both proposal generation and LLM-backed validators, without yet implementing real modules. ### Scope -Implement: -- `StructuredLLMClient` interface refinement if needed. -- OpenAI-compatible chat completions client. -- Structured JSON response support. -- Request/response types for structured calls. -- Retry handling for malformed structured output. -- Per-request timeout behavior. -- Context cancellation. +Implement (remaining): - Primary LLM config resolution. - Validation LLM config resolution and inheritance from primary settings. -- Optional API key behavior for self-hosted endpoints. - Redaction of credentials in all diagnostics and reports. - LLM scheduler/semaphore for bounded backend concurrency. -- Unit tests using fake HTTP servers or fake client implementations. - Prompt/response diagnostics writer primitives that can later be used by modules and validators. Do not implement: @@ -230,15 +244,13 @@ Do not implement: ### Expected behavior at end of phase -The codebase has a tested OpenAI-compatible structured-output client and scheduler, but the CLI still does not perform real LLM polishing unless later phases wire modules into the runner. +The codebase has a tested OpenAI-compatible structured-output client adapter, but scheduler and runtime wiring remain before this phase is fully complete. The CLI still does not perform real LLM polishing. ### Definition of done -- Structured LLM client is implemented and tested. +Remaining checklist to close Phase 9: - Scheduler enforces configured concurrency. -- Primary and validation LLM settings resolve correctly. -- Timeout and retry behavior are tested. -- Malformed structured responses fail cleanly or retry according to config. +- Primary and validation LLM settings resolve correctly in runtime wiring. - Prompt/response diagnostic primitives exist. - API keys are not leaked. - No real module behavior is introduced. diff --git a/go.mod b/go.mod index f195c17..ef1ac73 100644 --- a/go.mod +++ b/go.mod @@ -1,5 +1,52 @@ module gitea.maximumdirect.net/eric/audita -go 1.22 +go 1.24.0 -require gopkg.in/yaml.v3 v3.0.1 +require ( + github.com/jxnl/instructor-go v0.0.0-20260420201153-d4111c5ef532 + github.com/sashabaranov/go-openai v1.41.2 + gopkg.in/yaml.v3 v3.0.1 +) + +require ( + cloud.google.com/go v0.123.0 // indirect + cloud.google.com/go/auth v0.17.0 // indirect + cloud.google.com/go/compute/metadata v0.9.0 // indirect + github.com/aws/aws-sdk-go-v2 v1.39.2 // indirect + github.com/aws/smithy-go v1.23.0 // indirect + github.com/bahlo/generic-list-go v0.2.0 // indirect + github.com/buger/jsonparser v1.1.2 // indirect + github.com/cespare/xxhash/v2 v2.3.0 // indirect + github.com/cohere-ai/cohere-go/v2 v2.18.0 // indirect + github.com/felixge/httpsnoop v1.0.4 // indirect + github.com/gabriel-vasile/mimetype v1.4.13 // indirect + github.com/go-logr/logr v1.4.3 // indirect + github.com/go-logr/stdr v1.2.2 // indirect + github.com/go-playground/locales v0.14.1 // indirect + github.com/go-playground/universal-translator v0.18.1 // indirect + github.com/go-playground/validator/v10 v10.30.0 // indirect + github.com/google/go-cmp v0.7.0 // indirect + github.com/google/s2a-go v0.1.9 // indirect + github.com/google/uuid v1.6.0 // indirect + github.com/googleapis/enterprise-certificate-proxy v0.3.6 // indirect + github.com/googleapis/gax-go/v2 v2.15.0 // indirect + github.com/gorilla/websocket v1.5.3 // indirect + github.com/invopop/jsonschema v0.13.0 // indirect + github.com/leodido/go-urn v1.4.0 // indirect + github.com/liushuangls/go-anthropic/v2 v2.18.0 // indirect + github.com/mailru/easyjson v0.9.1 // indirect + github.com/wk8/go-ordered-map/v2 v2.1.8 // indirect + go.opentelemetry.io/auto/sdk v1.2.1 // indirect + go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.63.0 // indirect + go.opentelemetry.io/otel v1.39.0 // indirect + go.opentelemetry.io/otel/metric v1.39.0 // indirect + go.opentelemetry.io/otel/trace v1.39.0 // indirect + golang.org/x/crypto v0.48.0 // indirect + golang.org/x/net v0.50.0 // indirect + golang.org/x/sys v0.41.0 // indirect + golang.org/x/text v0.34.0 // indirect + google.golang.org/genai v1.52.1 // indirect + google.golang.org/genproto/googleapis/rpc v0.0.0-20251202230838-ff82c1b0f217 // indirect + google.golang.org/grpc v1.79.3 // indirect + google.golang.org/protobuf v1.36.10 // indirect +) diff --git a/go.sum b/go.sum index a62c313..e642baa 100644 --- a/go.sum +++ b/go.sum @@ -1,4 +1,114 @@ -gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405 h1:yhCVgyC4o1eVCa2tZl7eS0r+SDo693bJlVdllGtEeKM= +cloud.google.com/go v0.123.0 h1:2NAUJwPR47q+E35uaJeYoNhuNEM9kM8SjgRgdeOJUSE= +cloud.google.com/go v0.123.0/go.mod h1:xBoMV08QcqUGuPW65Qfm1o9Y4zKZBpGS+7bImXLTAZU= +cloud.google.com/go/auth v0.17.0 h1:74yCm7hCj2rUyyAocqnFzsAYXgJhrG26XCFimrc/Kz4= +cloud.google.com/go/auth v0.17.0/go.mod h1:6wv/t5/6rOPAX4fJiRjKkJCvswLwdet7G8+UGXt7nCQ= +cloud.google.com/go/compute/metadata v0.9.0 h1:pDUj4QMoPejqq20dK0Pg2N4yG9zIkYGdBtwLoEkH9Zs= +cloud.google.com/go/compute/metadata v0.9.0/go.mod h1:E0bWwX5wTnLPedCKqk3pJmVgCBSM6qQI1yTBdEb3C10= +github.com/aws/aws-sdk-go-v2 v1.39.2 h1:EJLg8IdbzgeD7xgvZ+I8M1e0fL0ptn/M47lianzth0I= +github.com/aws/aws-sdk-go-v2 v1.39.2/go.mod h1:sDioUELIUO9Znk23YVmIk86/9DOpkbyyVb1i/gUNFXY= +github.com/aws/smithy-go v1.23.0 h1:8n6I3gXzWJB2DxBDnfxgBaSX6oe0d/t10qGz7OKqMCE= +github.com/aws/smithy-go v1.23.0/go.mod h1:t1ufH5HMublsJYulve2RKmHDC15xu1f26kHCp/HgceI= +github.com/bahlo/generic-list-go v0.2.0 h1:5sz/EEAK+ls5wF+NeqDpk5+iNdMDXrh3z3nPnH1Wvgk= +github.com/bahlo/generic-list-go v0.2.0/go.mod h1:2KvAjgMlE5NNynlg/5iLrrCCZ2+5xWbdbCW3pNTGyYg= +github.com/buger/jsonparser v1.1.2 h1:frqHqw7otoVbk5M8LlE/L7HTnIq2v9RX6EJ48i9AxJk= +github.com/buger/jsonparser v1.1.2/go.mod h1:6RYKKt7H4d4+iWqouImQ9R2FZql3VbhNgx27UK13J/0= +github.com/cespare/xxhash/v2 v2.3.0 h1:UL815xU9SqsFlibzuggzjXhog7bL6oX9BbNZnL2UFvs= +github.com/cespare/xxhash/v2 v2.3.0/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs= +github.com/cohere-ai/cohere-go/v2 v2.18.0 h1:Y5NyY+YTCN6XrrxHWicgVdpBi4tSBa0bjesD2fJGdvQ= +github.com/cohere-ai/cohere-go/v2 v2.18.0/go.mod h1:MuiJkCxlR18BDV2qQPbz2Yb/OCVphT1y6nD2zYaKeR0= +github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c= +github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= +github.com/felixge/httpsnoop v1.0.4 h1:NFTV2Zj1bL4mc9sqWACXbQFVBBg2W3GPvqp8/ESS2Wg= +github.com/felixge/httpsnoop v1.0.4/go.mod h1:m8KPJKqk1gH5J9DgRY2ASl2lWCfGKXixSwevea8zH2U= +github.com/gabriel-vasile/mimetype v1.4.13 h1:46nXokslUBsAJE/wMsp5gtO500a4F3Nkz9Ufpk2AcUM= +github.com/gabriel-vasile/mimetype v1.4.13/go.mod h1:d+9Oxyo1wTzWdyVUPMmXFvp4F9tea18J8ufA774AB3s= +github.com/go-logr/logr v1.2.2/go.mod h1:jdQByPbusPIv2/zmleS9BjJVeZ6kBagPoEUsqbVz/1A= +github.com/go-logr/logr v1.4.3 h1:CjnDlHq8ikf6E492q6eKboGOC0T8CDaOvkHCIg8idEI= +github.com/go-logr/logr v1.4.3/go.mod h1:9T104GzyrTigFIr8wt5mBrctHMim0Nb2HLGrmQ40KvY= +github.com/go-logr/stdr v1.2.2 h1:hSWxHoqTgW2S2qGc0LTAI563KZ5YKYRhT3MFKZMbjag= +github.com/go-logr/stdr v1.2.2/go.mod h1:mMo/vtBO5dYbehREoey6XUKy/eSumjCCveDpRre4VKE= +github.com/go-playground/assert/v2 v2.2.0 h1:JvknZsQTYeFEAhQwI4qEt9cyV5ONwRHC+lYKSsYSR8s= +github.com/go-playground/assert/v2 v2.2.0/go.mod h1:VDjEfimB/XKnb+ZQfWdccd7VUvScMdVu0Titje2rxJ4= +github.com/go-playground/locales v0.14.1 h1:EWaQ/wswjilfKLTECiXz7Rh+3BjFhfDFKv/oXslEjJA= +github.com/go-playground/locales v0.14.1/go.mod h1:hxrqLVvrK65+Rwrd5Fc6F2O76J/NuW9t0sjnWqG1slY= +github.com/go-playground/universal-translator v0.18.1 h1:Bcnm0ZwsGyWbCzImXv+pAJnYK9S473LQFuzCbDbfSFY= +github.com/go-playground/universal-translator v0.18.1/go.mod h1:xekY+UJKNuX9WP91TpwSH2VMlDf28Uj24BCp08ZFTUY= +github.com/go-playground/validator/v10 v10.30.0 h1:5YBPNs273uzsZJD1I8uiB4Aqg9sN6sMDVX3s6LxmhWU= +github.com/go-playground/validator/v10 v10.30.0/go.mod h1:oSuBIQzuJxL//3MelwSLD5hc2Tu889bF0Idm9Dg26cM= +github.com/golang/protobuf v1.5.4 h1:i7eJL8qZTpSEXOPTxNKhASYpMn+8e5Q6AdndVa1dWek= +github.com/golang/protobuf v1.5.4/go.mod h1:lnTiLA8Wa4RWRcIUkrtSVa5nRhsEGBg48fD6rSs7xps= +github.com/google/go-cmp v0.7.0 h1:wk8382ETsv4JYUZwIsn6YpYiWiBsYLSJiTsyBybVuN8= +github.com/google/go-cmp v0.7.0/go.mod h1:pXiqmnSA92OHEEa9HXL2W4E7lf9JzCmGVUdgjX3N/iU= +github.com/google/s2a-go v0.1.9 h1:LGD7gtMgezd8a/Xak7mEWL0PjoTQFvpRudN895yqKW0= +github.com/google/s2a-go v0.1.9/go.mod h1:YA0Ei2ZQL3acow2O62kdp9UlnvMmU7kA6Eutn0dXayM= +github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0= +github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo= +github.com/googleapis/enterprise-certificate-proxy v0.3.6 h1:GW/XbdyBFQ8Qe+YAmFU9uHLo7OnF5tL52HFAgMmyrf4= +github.com/googleapis/enterprise-certificate-proxy v0.3.6/go.mod h1:MkHOF77EYAE7qfSuSS9PU6g4Nt4e11cnsDUowfwewLA= +github.com/googleapis/gax-go/v2 v2.15.0 h1:SyjDc1mGgZU5LncH8gimWo9lW1DtIfPibOG81vgd/bo= +github.com/googleapis/gax-go/v2 v2.15.0/go.mod h1:zVVkkxAQHa1RQpg9z2AUCMnKhi0Qld9rcmyfL1OZhoc= +github.com/gorilla/websocket v1.5.3 h1:saDtZ6Pbx/0u+bgYQ3q96pZgCzfhKXGPqt7kZ72aNNg= +github.com/gorilla/websocket v1.5.3/go.mod h1:YR8l580nyteQvAITg2hZ9XVh4b55+EU/adAjf1fMHhE= +github.com/invopop/jsonschema v0.13.0 h1:KvpoAJWEjR3uD9Kbm2HWJmqsEaHt8lBUpd0qHcIi21E= +github.com/invopop/jsonschema v0.13.0/go.mod h1:ffZ5Km5SWWRAIN6wbDXItl95euhFz2uON45H2qjYt+0= +github.com/jxnl/instructor-go v0.0.0-20260420201153-d4111c5ef532 h1:9Gd7ONgzQLX+9jKrkAdE9Ol9Tp6/w4tfRH14dayoemI= +github.com/jxnl/instructor-go v0.0.0-20260420201153-d4111c5ef532/go.mod h1:A4cY903g2j4nuOIHvacaHBEf0SwAOh1Yswx/XAgqPgo= +github.com/kr/pretty v0.3.1 h1:flRD4NNwYAUpkphVc1HcthR4KEIFJ65n8Mw5qdRn3LE= +github.com/kr/pretty v0.3.1/go.mod h1:hoEshYVHaxMs3cyo3Yncou5ZscifuDolrwPKZanG3xk= +github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY= +github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE= +github.com/leodido/go-urn v1.4.0 h1:WT9HwE9SGECu3lg4d/dIA+jxlljEa1/ffXKmRjqdmIQ= +github.com/leodido/go-urn v1.4.0/go.mod h1:bvxc+MVxLKB4z00jd1z+Dvzr47oO32F/QSNjSBOlFxI= +github.com/liushuangls/go-anthropic/v2 v2.18.0 h1:q47sqQ1bHsItPTcIj7RVOnpgrP79SgaACsMQ0RW2oYA= +github.com/liushuangls/go-anthropic/v2 v2.18.0/go.mod h1:a550cJXPoTG2FL3DvfKG2zzD5O2vjgvo4tHtoGPzFLU= +github.com/mailru/easyjson v0.9.1 h1:LbtsOm5WAswyWbvTEOqhypdPeZzHavpZx96/n553mR8= +github.com/mailru/easyjson v0.9.1/go.mod h1:1+xMtQp2MRNVL/V1bOzuP3aP8VNwRW55fQUto+XFtTU= +github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM= +github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= +github.com/rogpeppe/go-internal v1.14.1 h1:UQB4HGPB6osV0SQTLymcB4TgvyWu6ZyliaW0tI/otEQ= +github.com/rogpeppe/go-internal v1.14.1/go.mod h1:MaRKkUm5W0goXpeCfT7UZI6fk/L7L7so1lCWt35ZSgc= +github.com/sashabaranov/go-openai v1.41.2 h1:vfPRBZNMpnqu8ELsclWcAvF19lDNgh1t6TVfFFOPiSM= +github.com/sashabaranov/go-openai v1.41.2/go.mod h1:lj5b/K+zjTSFxVLijLSTDZuP7adOgerWeFyZLUhAKRg= +github.com/stretchr/testify v1.11.1 h1:7s2iGBzp5EwR7/aIZr8ao5+dra3wiQyKjjFuvgVKu7U= +github.com/stretchr/testify v1.11.1/go.mod h1:wZwfW3scLgRK+23gO65QZefKpKQRnfz6sD981Nm4B6U= +github.com/wk8/go-ordered-map/v2 v2.1.8 h1:5h/BUHu93oj4gIdvHHHGsScSTMijfx5PeYkE/fJgbpc= +github.com/wk8/go-ordered-map/v2 v2.1.8/go.mod h1:5nJHM5DyteebpVlHnWMV0rPz6Zp7+xBAnxjb1X5vnTw= +go.opentelemetry.io/auto/sdk v1.2.1 h1:jXsnJ4Lmnqd11kwkBV2LgLoFMZKizbCi5fNZ/ipaZ64= +go.opentelemetry.io/auto/sdk v1.2.1/go.mod h1:KRTj+aOaElaLi+wW1kO/DZRXwkF4C5xPbEe3ZiIhN7Y= +go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.63.0 h1:RbKq8BG0FI8OiXhBfcRtqqHcZcka+gU3cskNuf05R18= +go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.63.0/go.mod h1:h06DGIukJOevXaj/xrNjhi/2098RZzcLTbc0jDAUbsg= +go.opentelemetry.io/otel v1.39.0 h1:8yPrr/S0ND9QEfTfdP9V+SiwT4E0G7Y5MO7p85nis48= +go.opentelemetry.io/otel v1.39.0/go.mod h1:kLlFTywNWrFyEdH0oj2xK0bFYZtHRYUdv1NklR/tgc8= +go.opentelemetry.io/otel/metric v1.39.0 h1:d1UzonvEZriVfpNKEVmHXbdf909uGTOQjA0HF0Ls5Q0= +go.opentelemetry.io/otel/metric v1.39.0/go.mod h1:jrZSWL33sD7bBxg1xjrqyDjnuzTUB0x1nBERXd7Ftcs= +go.opentelemetry.io/otel/sdk v1.39.0 h1:nMLYcjVsvdui1B/4FRkwjzoRVsMK8uL/cj0OyhKzt18= +go.opentelemetry.io/otel/sdk v1.39.0/go.mod h1:vDojkC4/jsTJsE+kh+LXYQlbL8CgrEcwmt1ENZszdJE= +go.opentelemetry.io/otel/sdk/metric v1.39.0 h1:cXMVVFVgsIf2YL6QkRF4Urbr/aMInf+2WKg+sEJTtB8= +go.opentelemetry.io/otel/sdk/metric v1.39.0/go.mod h1:xq9HEVH7qeX69/JnwEfp6fVq5wosJsY1mt4lLfYdVew= +go.opentelemetry.io/otel/trace v1.39.0 h1:2d2vfpEDmCJ5zVYz7ijaJdOF59xLomrvj7bjt6/qCJI= +go.opentelemetry.io/otel/trace v1.39.0/go.mod h1:88w4/PnZSazkGzz/w84VHpQafiU4EtqqlVdxWy+rNOA= +golang.org/x/crypto v0.48.0 h1:/VRzVqiRSggnhY7gNRxPauEQ5Drw9haKdM0jqfcCFts= +golang.org/x/crypto v0.48.0/go.mod h1:r0kV5h3qnFPlQnBSrULhlsRfryS2pmewsg+XfMgkVos= +golang.org/x/net v0.50.0 h1:ucWh9eiCGyDR3vtzso0WMQinm2Dnt8cFMuQa9K33J60= +golang.org/x/net v0.50.0/go.mod h1:UgoSli3F/pBgdJBHCTc+tp3gmrU4XswgGRgtnwWTfyM= +golang.org/x/sync v0.19.0 h1:vV+1eWNmZ5geRlYjzm2adRgW2/mcpevXNg50YZtPCE4= +golang.org/x/sync v0.19.0/go.mod h1:9KTHXmSnoGruLpwFjVSX0lNNA75CykiMECbovNTZqGI= +golang.org/x/sys v0.41.0 h1:Ivj+2Cp/ylzLiEU89QhWblYnOE9zerudt9Ftecq2C6k= +golang.org/x/sys v0.41.0/go.mod h1:OgkHotnGiDImocRcuBABYBEXf8A9a87e/uXjp9XT3ks= +golang.org/x/text v0.34.0 h1:oL/Qq0Kdaqxa1KbNeMKwQq0reLCCaFtqu2eNuSeNHbk= +golang.org/x/text v0.34.0/go.mod h1:homfLqTYRFyVYemLBFl5GgL/DWEiH5wcsQ5gSh1yziA= +gonum.org/v1/gonum v0.16.0 h1:5+ul4Swaf3ESvrOnidPp4GZbzf0mxVQpDCYUQE7OJfk= +gonum.org/v1/gonum v0.16.0/go.mod h1:fef3am4MQ93R2HHpKnLk4/Tbh/s0+wqD5nfa6Pnwy4E= +google.golang.org/genai v1.52.1 h1:dYoljKtLDXMiBdVaClSJ/ZPwZ7j1N0lGjMhwOKOQUlk= +google.golang.org/genai v1.52.1/go.mod h1:A3kkl0nyBjyFlNjgxIwKq70julKbIxpSxqKO5gw/gmk= +google.golang.org/genproto/googleapis/rpc v0.0.0-20251202230838-ff82c1b0f217 h1:gRkg/vSppuSQoDjxyiGfN4Upv/h/DQmIR10ZU8dh4Ww= +google.golang.org/genproto/googleapis/rpc v0.0.0-20251202230838-ff82c1b0f217/go.mod h1:7i2o+ce6H/6BluujYR+kqX3GKH+dChPTQU19wjRPiGk= +google.golang.org/grpc v1.79.3 h1:sybAEdRIEtvcD68Gx7dmnwjZKlyfuc61Dyo9pGXXkKE= +google.golang.org/grpc v1.79.3/go.mod h1:KmT0Kjez+0dde/v2j9vzwoAScgEPx/Bw1CYChhHLrHQ= +google.golang.org/protobuf v1.36.10 h1:AYd7cD/uASjIL6Q9LiTjz8JLcrh/88q5UObnmY3aOOE= +google.golang.org/protobuf v1.36.10/go.mod h1:HTf+CrKn2C3g5S8VImy6tdcUvCska2kB7j23XfzDpco= gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= +gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c h1:Hei/4ADfdWqJk1ZMxUNpqntNwaWcugrBjAiHlqqRiVk= +gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c/go.mod h1:JHkPIbrfpd72SG/EVd6muEfDQjcINNoR0C8j2r3qZ4Q= gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA= gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= diff --git a/internal/framework/contracts/contracts.go b/internal/framework/contracts/contracts.go index b904a9e..3fe1472 100644 --- a/internal/framework/contracts/contracts.go +++ b/internal/framework/contracts/contracts.go @@ -14,7 +14,7 @@ import ( // StructuredLLMClient provides provider-agnostic structured completion. type StructuredLLMClient interface { - CompleteStructured(ctx context.Context, req StructuredCompletionRequest) (StructuredCompletionResponse, error) + CompleteStructured(ctx context.Context, req StructuredCompletionRequest, out any) (StructuredCompletionResponse, error) } // TranscriptModule is the minimal contract for framework-integrated modules. @@ -35,12 +35,18 @@ type Validator interface { type StructuredCompletionRequest struct { StageName string `json:"stage_name"` Messages []LLMMessage `json:"messages"` + Model string `json:"model,omitempty"` ResponseSchema json.RawMessage `json:"response_schema,omitempty"` } // StructuredCompletionResponse is a transport-neutral structured completion response payload. type StructuredCompletionResponse struct { - Content json.RawMessage `json:"content"` + Content json.RawMessage `json:"content"` + Provider string `json:"provider,omitempty"` + Model string `json:"model,omitempty"` + PromptTokens int `json:"prompt_tokens,omitempty"` + CompletionTokens int `json:"completion_tokens,omitempty"` + TotalTokens int `json:"total_tokens,omitempty"` } // LLMMessage is a minimal chat message shape for LLM prompts. diff --git a/internal/framework/contracts/contracts_test.go b/internal/framework/contracts/contracts_test.go index c6502ea..2a26d3e 100644 --- a/internal/framework/contracts/contracts_test.go +++ b/internal/framework/contracts/contracts_test.go @@ -13,9 +13,15 @@ import ( type fakeLLMClient struct{} -func (f *fakeLLMClient) CompleteStructured(ctx context.Context, req StructuredCompletionRequest) (StructuredCompletionResponse, error) { +func (f *fakeLLMClient) CompleteStructured(ctx context.Context, req StructuredCompletionRequest, out any) (StructuredCompletionResponse, error) { _ = ctx _ = req + if out != nil { + switch target := out.(type) { + case *map[string]any: + *target = map[string]any{"ok": true} + } + } return StructuredCompletionResponse{Content: json.RawMessage(`{"ok":true}`)}, nil } diff --git a/internal/framework/llm/instructor_client.go b/internal/framework/llm/instructor_client.go new file mode 100644 index 0000000..033c1f6 --- /dev/null +++ b/internal/framework/llm/instructor_client.go @@ -0,0 +1,221 @@ +package llm + +import ( + "context" + "encoding/json" + "fmt" + "net/http" + "reflect" + "strings" + "time" + + "gitea.maximumdirect.net/eric/audita/internal/framework/contracts" + + "github.com/jxnl/instructor-go/pkg/instructor" + openai "github.com/sashabaranov/go-openai" +) + +const ( + defaultMaxRetries = 3 +) + +type Mode string + +const ( + ModeJSON Mode = "json" + ModeToolCall Mode = "tool_call" +) + +type InstructorClientConfig struct { + BaseURL string + Model string + APIKey string + MaxRetries int + Mode Mode + HTTPClient *http.Client + RequestTimeout time.Duration +} + +// chatCompletionClient is intentionally narrow to avoid leaking provider types +// outside this adapter package. +type chatCompletionClient interface { + CreateChatCompletion(ctx context.Context, request openai.ChatCompletionRequest, responseType any) (response openai.ChatCompletionResponse, err error) +} + +// InstructorClient adapts instructor-go behind Audita's internal structured +// LLM interface. +type InstructorClient struct { + cfg InstructorClientConfig + client chatCompletionClient +} + +var _ contracts.StructuredLLMClient = (*InstructorClient)(nil) + +func NewInstructorClient(cfg InstructorClientConfig) (*InstructorClient, error) { + normalized, err := normalizeConfig(cfg) + if err != nil { + return nil, err + } + + openaiConfig := openai.DefaultConfig(normalized.APIKey) + openaiConfig.BaseURL = normalized.BaseURL + openaiConfig.HTTPClient = resolvedHTTPClient(normalized.HTTPClient, normalized.RequestTimeout) + + mode, err := toInstructorMode(normalized.Mode) + if err != nil { + return nil, err + } + + client := instructor.FromOpenAI( + openai.NewClientWithConfig(openaiConfig), + instructor.WithMode(mode), + instructor.WithMaxRetries(normalized.MaxRetries), + ) + + return &InstructorClient{ + cfg: normalized, + client: client, + }, nil +} + +func (c *InstructorClient) CompleteStructured( + ctx context.Context, + req contracts.StructuredCompletionRequest, + out any, +) (contracts.StructuredCompletionResponse, error) { + if err := validateOutputTarget(out); err != nil { + return contracts.StructuredCompletionResponse{}, err + } + + model := strings.TrimSpace(req.Model) + if model == "" { + model = c.cfg.Model + } + if model == "" { + return contracts.StructuredCompletionResponse{}, fmt.Errorf("structured completion model must not be empty") + } + + messages, err := toOpenAIMessages(req.Messages) + if err != nil { + return contracts.StructuredCompletionResponse{}, err + } + + chatRequest := openai.ChatCompletionRequest{ + Model: model, + Messages: messages, + } + + resp, err := c.client.CreateChatCompletion(ctx, chatRequest, out) + if err != nil { + return contracts.StructuredCompletionResponse{}, sanitizeError(err, c.cfg.APIKey) + } + + content, err := json.Marshal(out) + if err != nil { + return contracts.StructuredCompletionResponse{}, fmt.Errorf("marshal structured completion output: %w", err) + } + + return contracts.StructuredCompletionResponse{ + Content: content, + Provider: "openai-compatible", + Model: model, + PromptTokens: resp.Usage.PromptTokens, + CompletionTokens: resp.Usage.CompletionTokens, + TotalTokens: resp.Usage.TotalTokens, + }, nil +} + +// TODO(phase9): integrate bounded scheduler/semaphore in the next Phase 9 prompt. + +func normalizeConfig(cfg InstructorClientConfig) (InstructorClientConfig, error) { + cfg.BaseURL = strings.TrimSpace(cfg.BaseURL) + cfg.Model = strings.TrimSpace(cfg.Model) + cfg.APIKey = strings.TrimSpace(cfg.APIKey) + if cfg.MaxRetries == 0 { + cfg.MaxRetries = defaultMaxRetries + } + if cfg.MaxRetries < 0 { + return InstructorClientConfig{}, fmt.Errorf("max retries must be zero or greater") + } + if cfg.BaseURL == "" { + return InstructorClientConfig{}, fmt.Errorf("base URL must not be empty") + } + if cfg.Model == "" { + return InstructorClientConfig{}, fmt.Errorf("model must not be empty") + } + if cfg.Mode == "" { + cfg.Mode = ModeJSON + } + return cfg, nil +} + +func toInstructorMode(mode Mode) (instructor.Mode, error) { + switch mode { + case ModeJSON: + return instructor.ModeJSON, nil + case ModeToolCall: + return instructor.ModeToolCall, nil + default: + return "", fmt.Errorf("unsupported LLM mode %q", mode) + } +} + +func resolvedHTTPClient(base *http.Client, timeout time.Duration) *http.Client { + if base == nil { + if timeout <= 0 { + return http.DefaultClient + } + return &http.Client{Timeout: timeout} + } + + if timeout <= 0 { + return base + } + + cloned := *base + cloned.Timeout = timeout + return &cloned +} + +func toOpenAIMessages(messages []contracts.LLMMessage) ([]openai.ChatCompletionMessage, error) { + result := make([]openai.ChatCompletionMessage, len(messages)) + for i, message := range messages { + role := strings.TrimSpace(message.Role) + content := strings.TrimSpace(message.Content) + if role == "" { + return nil, fmt.Errorf("message[%d] role must not be empty", i) + } + if content == "" { + return nil, fmt.Errorf("message[%d] content must not be empty", i) + } + result[i] = openai.ChatCompletionMessage{ + Role: role, + Content: content, + } + } + return result, nil +} + +func validateOutputTarget(out any) error { + if out == nil { + return fmt.Errorf("output target must not be nil") + } + value := reflect.ValueOf(out) + if value.Kind() != reflect.Ptr || value.IsNil() { + return fmt.Errorf("output target must be a non-nil pointer") + } + return nil +} + +func sanitizeError(err error, apiKey string) error { + if err == nil { + return nil + } + msg := err.Error() + key := strings.TrimSpace(apiKey) + if key != "" { + msg = strings.ReplaceAll(msg, key, "[REDACTED]") + msg = strings.ReplaceAll(msg, "Bearer "+key, "Bearer [REDACTED]") + } + return fmt.Errorf("%s", msg) +} diff --git a/internal/framework/llm/instructor_client_test.go b/internal/framework/llm/instructor_client_test.go new file mode 100644 index 0000000..5f822c9 --- /dev/null +++ b/internal/framework/llm/instructor_client_test.go @@ -0,0 +1,282 @@ +package llm + +import ( + "context" + "encoding/json" + "fmt" + "io" + "net/http" + "strings" + "sync/atomic" + "testing" + "time" + + "gitea.maximumdirect.net/eric/audita/internal/framework/contracts" + + openai "github.com/sashabaranov/go-openai" +) + +func TestNewInstructorClientValidation(t *testing.T) { + _, err := NewInstructorClient(InstructorClientConfig{ + BaseURL: " ", + Model: "x", + }) + if err == nil || !strings.Contains(err.Error(), "base URL") { + t.Fatalf("expected base URL validation error, got %v", err) + } + + _, err = NewInstructorClient(InstructorClientConfig{ + BaseURL: "http://localhost:1234/v1", + Model: " ", + }) + if err == nil || !strings.Contains(err.Error(), "model") { + t.Fatalf("expected model validation error, got %v", err) + } + + _, err = NewInstructorClient(InstructorClientConfig{ + BaseURL: "http://localhost:1234/v1", + Model: "test-model", + MaxRetries: -1, + }) + if err == nil || !strings.Contains(err.Error(), "max retries") { + t.Fatalf("expected retries validation error, got %v", err) + } + + _, err = NewInstructorClient(InstructorClientConfig{ + BaseURL: "http://localhost:1234/v1", + Model: "test-model", + Mode: "unsupported", + }) + if err == nil || !strings.Contains(err.Error(), "unsupported LLM mode") { + t.Fatalf("expected mode validation error, got %v", err) + } +} + +func TestInstructorClientCompleteStructuredSuccessNoAPIKey(t *testing.T) { + var seenPath string + var seenHost string + var seenAuth string + var seenModel string + client, err := NewInstructorClient(InstructorClientConfig{ + BaseURL: "https://local-compat.example/v1", + Model: "test-model", + HTTPClient: &http.Client{ + Transport: roundTripFunc(func(r *http.Request) (*http.Response, error) { + seenPath = r.URL.Path + seenHost = r.URL.Host + seenAuth = r.Header.Get("Authorization") + + var req map[string]any + if err := json.NewDecoder(r.Body).Decode(&req); err != nil { + t.Fatalf("decode request: %v", err) + } + model, _ := req["model"].(string) + seenModel = model + + return newJSONHTTPResponse(http.StatusOK, chatCompletionResponseBody(`{"name":"Robby","age":22}`)), nil + }), + }, + }) + if err != nil { + t.Fatalf("NewInstructorClient: %v", err) + } + + type person struct { + Name string `json:"name"` + Age int `json:"age"` + } + var out person + resp, err := client.CompleteStructured(context.Background(), contracts.StructuredCompletionRequest{ + StageName: "proposal:test", + Messages: []contracts.LLMMessage{ + {Role: openai.ChatMessageRoleUser, Content: "extract person"}, + }, + }, &out) + if err != nil { + t.Fatalf("CompleteStructured: %v", err) + } + + if seenPath != "/v1/chat/completions" { + t.Fatalf("unexpected request path: %q", seenPath) + } + if seenHost != "local-compat.example" { + t.Fatalf("unexpected request host: %q", seenHost) + } + if seenModel != "test-model" { + t.Fatalf("unexpected model: %q", seenModel) + } + if seenAuth != "" { + t.Fatalf("expected empty Authorization header for empty API key, got %q", seenAuth) + } + if out.Name != "Robby" || out.Age != 22 { + t.Fatalf("unexpected output: %+v", out) + } + if resp.Provider != "openai-compatible" || resp.Model != "test-model" { + t.Fatalf("unexpected response metadata: %+v", resp) + } + if resp.TotalTokens != 18 || resp.PromptTokens != 11 || resp.CompletionTokens != 7 { + t.Fatalf("unexpected usage metadata: %+v", resp) + } +} + +func TestInstructorClientRetriesOnMalformedJSON(t *testing.T) { + var attempts int32 + client, err := NewInstructorClient(InstructorClientConfig{ + BaseURL: "https://retry.example/v1", + Model: "test-model", + MaxRetries: 1, + HTTPClient: &http.Client{ + Transport: roundTripFunc(func(r *http.Request) (*http.Response, error) { + current := atomic.AddInt32(&attempts, 1) + if current == 1 { + return newJSONHTTPResponse(http.StatusOK, chatCompletionResponseBody(`{"name":"broken"`)), nil + } + return newJSONHTTPResponse(http.StatusOK, chatCompletionResponseBody(`{"name":"Recovered","age":30}`)), nil + }), + }, + }) + if err != nil { + t.Fatalf("NewInstructorClient: %v", err) + } + + type person struct { + Name string `json:"name"` + Age int `json:"age"` + } + var out person + _, err = client.CompleteStructured(context.Background(), contracts.StructuredCompletionRequest{ + Messages: []contracts.LLMMessage{ + {Role: openai.ChatMessageRoleUser, Content: "extract person"}, + }, + }, &out) + if err != nil { + t.Fatalf("CompleteStructured: %v", err) + } + if out.Name != "Recovered" || out.Age != 30 { + t.Fatalf("unexpected output after retry: %+v", out) + } + if got := atomic.LoadInt32(&attempts); got != 2 { + t.Fatalf("expected 2 attempts, got %d", got) + } +} + +func TestInstructorClientContextDeadlinePropagates(t *testing.T) { + client, err := NewInstructorClient(InstructorClientConfig{ + BaseURL: "https://slow.example/v1", + Model: "test-model", + HTTPClient: &http.Client{ + Transport: roundTripFunc(func(r *http.Request) (*http.Response, error) { + select { + case <-r.Context().Done(): + return nil, r.Context().Err() + case <-time.After(250 * time.Millisecond): + return newJSONHTTPResponse(http.StatusOK, chatCompletionResponseBody(`{"name":"slow","age":1}`)), nil + } + }), + }, + }) + if err != nil { + t.Fatalf("NewInstructorClient: %v", err) + } + + type person struct { + Name string `json:"name"` + } + var out person + + ctx, cancel := context.WithTimeout(context.Background(), 20*time.Millisecond) + defer cancel() + + _, err = client.CompleteStructured(ctx, contracts.StructuredCompletionRequest{ + Messages: []contracts.LLMMessage{ + {Role: openai.ChatMessageRoleUser, Content: "extract person"}, + }, + }, &out) + if err == nil { + t.Fatalf("expected context deadline error") + } + if !strings.Contains(strings.ToLower(err.Error()), "context deadline") { + t.Fatalf("expected deadline-related error, got %v", err) + } +} + +func TestInstructorClientSanitizesAPIKeyInErrors(t *testing.T) { + apiKey := "super-secret-key" + client := &InstructorClient{ + cfg: InstructorClientConfig{ + BaseURL: "http://localhost:1234/v1", + Model: "test-model", + APIKey: apiKey, + Mode: ModeJSON, + }, + client: fakeChatCompletionClient{ + err: fmt.Errorf("provider failed with Authorization: Bearer %s", apiKey), + }, + } + + var out map[string]any + _, err := client.CompleteStructured(context.Background(), contracts.StructuredCompletionRequest{ + Messages: []contracts.LLMMessage{ + {Role: openai.ChatMessageRoleUser, Content: "extract"}, + }, + }, &out) + if err == nil { + t.Fatalf("expected error") + } + if strings.Contains(err.Error(), apiKey) { + t.Fatalf("error leaked API key: %v", err) + } + if !strings.Contains(err.Error(), "[REDACTED]") { + t.Fatalf("expected redaction marker in error: %v", err) + } +} + +type fakeChatCompletionClient struct { + err error +} + +func (f fakeChatCompletionClient) CreateChatCompletion(ctx context.Context, request openai.ChatCompletionRequest, responseType any) (response openai.ChatCompletionResponse, err error) { + _ = ctx + _ = request + _ = responseType + return openai.ChatCompletionResponse{}, f.err +} + +type roundTripFunc func(*http.Request) (*http.Response, error) + +func (f roundTripFunc) RoundTrip(r *http.Request) (*http.Response, error) { + return f(r) +} + +func newJSONHTTPResponse(status int, body string) *http.Response { + return &http.Response{ + StatusCode: status, + Header: http.Header{"Content-Type": []string{"application/json"}}, + Body: io.NopCloser(strings.NewReader(body)), + } +} + +func chatCompletionResponseBody(content string) string { + body, _ := json.Marshal(map[string]any{ + "id": "chatcmpl-test", + "object": "chat.completion", + "created": 12345, + "model": "test-model", + "choices": []map[string]any{ + { + "index": 0, + "message": map[string]any{ + "role": "assistant", + "content": content, + }, + "finish_reason": "stop", + }, + }, + "usage": map[string]any{ + "prompt_tokens": 11, + "completion_tokens": 7, + "total_tokens": 18, + }, + }) + return string(body) +}