Add explicit LLM concurrency controls

This commit is contained in:
2026-05-12 21:17:35 +00:00
parent a48f6da1f4
commit 509436cc4a
19 changed files with 875 additions and 99 deletions

View File

@@ -121,7 +121,6 @@ Environment:
- `AUDITA_BASE_URL`
- `AUDITA_LLM_TIMEOUT_SECONDS`
- `AUDITA_MAX_RETRIES`
- `AUDITA_LLM_CONCURRENCY`
CLI:
- `--llm-api-key`
@@ -129,7 +128,6 @@ CLI:
- `--base-url`
- `--llm-timeout-seconds`
- `--max-retries`
- `--llm-concurrency`
### Validation LLM
@@ -151,9 +149,26 @@ CLI:
- `--validation-llm-concurrency`
- `--validation-max-prompt-tokens`
Validation concurrency behavior:
- when validation concurrency is unset, it inherits primary `llm-concurrency`
- when explicitly set, validation concurrency must be `<= llm-concurrency`
### LLM Concurrency
Environment:
- `AUDITA_TOTAL_LLM_CONCURRENCY`
- `AUDITA_PROPOSAL_LLM_CONCURRENCY`
- `AUDITA_VALIDATION_LLM_CONCURRENCY`
- `AUDITA_LLM_CONCURRENCY` (legacy alias for `AUDITA_TOTAL_LLM_CONCURRENCY`)
CLI:
- `--total-llm-concurrency`
- `--proposal-llm-concurrency`
- `--validation-llm-concurrency`
- `--llm-concurrency` (legacy alias for `--total-llm-concurrency`)
Behavior:
- all proposal and validation LLM calls are bounded by total LLM concurrency
- proposal LLM calls are additionally bounded by proposal LLM concurrency
- when validation concurrency is unset, it inherits total LLM concurrency
- when explicitly set, proposal and validation concurrency must each be `<= total-llm-concurrency`
- canonical total settings win when both canonical and legacy alias settings are provided at the same precedence layer
### Confidence Thresholds