Review LLM concurrency refactor

This commit is contained in:
2026-05-12 21:23:57 +00:00
parent 509436cc4a
commit a85a7e204e
6 changed files with 276 additions and 39 deletions

View File

@@ -14,11 +14,11 @@ Current implementation already provides:
- deterministic proposal aggregation and deterministic per-module apply ordering,
- subprocess-safe behavior and deterministic test hooks without requiring live LLM credentials.
Main gap versus the requested target architecture:
- there is no separate `--proposal-llm-concurrency` flag,
- there is no separate `--total-llm-concurrency` flag,
- global concurrency is currently represented by existing `--llm-concurrency`,
- scheduler implementation is semaphore-based and does not explicitly guarantee FIFO ordering.
At audit time, the main gaps versus the requested target architecture were:
- there was no separate `--proposal-llm-concurrency` flag,
- there was no separate `--total-llm-concurrency` flag,
- global concurrency was represented by existing `--llm-concurrency`,
- scheduler implementation was semaphore-based and did not explicitly guarantee FIFO ordering.
## Implementation Status (2026-05-12 Update)
@@ -31,6 +31,8 @@ The targeted concurrency gaps identified in this audit have now been addressed:
## Audit Findings (Questions 1-14)
The findings in this section reflect repository state at audit time (before the refactor). See the implementation-status section for current-state behavior.
1. **Does the current runner execute modules serially?**
- Yes. `Runner.Run` loops through `input.ModuleSpecs` sequentially and updates `working` per module.
@@ -95,7 +97,7 @@ The targeted concurrency gaps identified in this audit have now been addressed:
- emit deterministic module result data.
4. Process report and diagnostics are written; subprocess contracts remain stable.
## Gaps vs Desired Target Architecture
## Gaps vs Desired Target Architecture (Audit-Time Snapshot)
Matches target:
- Modules are serial.
@@ -107,13 +109,13 @@ Matches target:
- `go test ./...` does not require real LLM credentials.
- Subprocess-oriented behavior remains intact.
Gaps:
- Missing dedicated `--proposal-llm-concurrency` surface.
- Missing dedicated `--total-llm-concurrency` surface (today this role is played by `--llm-concurrency`).
- Scheduler does not currently provide explicit FIFO semantics/policy abstraction.
- Validation is internally batched and called sequentially within a validator; only scheduler-level sharing enforces global contention, not explicit per-validator parallel fan-out.
Gaps at audit time:
- missing dedicated `--proposal-llm-concurrency` surface,
- missing dedicated `--total-llm-concurrency` surface (at the time this role was played by `--llm-concurrency`),
- scheduler did not provide explicit FIFO semantics,
- validation remained internally batched and sequential within one validator invocation.
## Minimum Implementation Plan
## Minimum Implementation Plan (Completed)
1. **Config/CLI surface**
- Add explicit `total llm concurrency` setting and CLI/env wiring.
@@ -143,5 +145,5 @@ Gaps:
## Notes
- No runtime behavior was changed as part of this audit.
- No prompt/module/validator/report schema changes are proposed in this audit.
- This document is retained as an audit record; see the implementation-status section for current behavior.
- No prompt/module/validator/report schema changes were required to close the identified concurrency gaps.