Add explicit LLM concurrency controls
This commit is contained in:
25
README.md
25
README.md
@@ -121,7 +121,6 @@ Environment:
|
||||
- `AUDITA_BASE_URL`
|
||||
- `AUDITA_LLM_TIMEOUT_SECONDS`
|
||||
- `AUDITA_MAX_RETRIES`
|
||||
- `AUDITA_LLM_CONCURRENCY`
|
||||
|
||||
CLI:
|
||||
- `--llm-api-key`
|
||||
@@ -129,7 +128,6 @@ CLI:
|
||||
- `--base-url`
|
||||
- `--llm-timeout-seconds`
|
||||
- `--max-retries`
|
||||
- `--llm-concurrency`
|
||||
|
||||
### Validation LLM
|
||||
|
||||
@@ -151,9 +149,26 @@ CLI:
|
||||
- `--validation-llm-concurrency`
|
||||
- `--validation-max-prompt-tokens`
|
||||
|
||||
Validation concurrency behavior:
|
||||
- when validation concurrency is unset, it inherits primary `llm-concurrency`
|
||||
- when explicitly set, validation concurrency must be `<= llm-concurrency`
|
||||
### LLM Concurrency
|
||||
|
||||
Environment:
|
||||
- `AUDITA_TOTAL_LLM_CONCURRENCY`
|
||||
- `AUDITA_PROPOSAL_LLM_CONCURRENCY`
|
||||
- `AUDITA_VALIDATION_LLM_CONCURRENCY`
|
||||
- `AUDITA_LLM_CONCURRENCY` (legacy alias for `AUDITA_TOTAL_LLM_CONCURRENCY`)
|
||||
|
||||
CLI:
|
||||
- `--total-llm-concurrency`
|
||||
- `--proposal-llm-concurrency`
|
||||
- `--validation-llm-concurrency`
|
||||
- `--llm-concurrency` (legacy alias for `--total-llm-concurrency`)
|
||||
|
||||
Behavior:
|
||||
- all proposal and validation LLM calls are bounded by total LLM concurrency
|
||||
- proposal LLM calls are additionally bounded by proposal LLM concurrency
|
||||
- when validation concurrency is unset, it inherits total LLM concurrency
|
||||
- when explicitly set, proposal and validation concurrency must each be `<= total-llm-concurrency`
|
||||
- canonical total settings win when both canonical and legacy alias settings are provided at the same precedence layer
|
||||
|
||||
### Confidence Thresholds
|
||||
|
||||
|
||||
Reference in New Issue
Block a user