Complete Phase 17 parity fixture suite
This commit is contained in:
@@ -43,7 +43,6 @@ Implemented today:
|
||||
- Explicit runtime support for `--modules spoken_word` through the production runner path.
|
||||
|
||||
Not implemented in CLI runtime path today:
|
||||
- Python parity fixture suite and parity verification workflow.
|
||||
- Operational hardening tasks beyond current runtime/reporting/diagnostics behavior.
|
||||
- Rollout and Python retirement work.
|
||||
|
||||
@@ -66,7 +65,8 @@ Phase sequencing note:
|
||||
- Phase 14 homophones module implementation and explicit runtime wiring are complete;
|
||||
- Phase 15 spoken-word module implementation and explicit runtime wiring are complete;
|
||||
- Phase 16 default full pipeline integration is complete;
|
||||
- next recommended phase is Phase 17 (Python parity fixture suite).
|
||||
- Phase 17 parity fixture suite is complete;
|
||||
- next recommended phase is Phase 18 (operational hardening and subprocess integration).
|
||||
|
||||
## Actual Go package layout
|
||||
|
||||
@@ -184,6 +184,12 @@ Current runtime flow (`internal/cli/run.go`):
|
||||
15. Optionally write `--report-json`; always write run-dir `report.json`.
|
||||
16. Apply work-dir retention.
|
||||
|
||||
Parity fixture status:
|
||||
- representative Python-parity fixture coverage exists under `internal/cli/testdata/parity`;
|
||||
- parity tests use fake structured LLM responses for deterministic behavior, including default full-pipeline shape assertions;
|
||||
- parity comparisons intentionally ignore nondeterministic metadata (timestamps, run IDs, temp paths, token usage) and remain strict for deterministic contract fields (transcript content, module order/instance naming, applied/skipped/rejected counts, and status).
|
||||
- intentional Python-vs-Go differences and open parity gaps are documented in `docs/python-parity.md`.
|
||||
|
||||
Important behavior details:
|
||||
- Glossary is validated and is used for explicit glossary/grammar/homophones/spoken_word module correction paths.
|
||||
- Default production CLI behavior now executes the full production module sequence unless `--modules` override is supplied.
|
||||
@@ -254,6 +260,7 @@ Current caveat:
|
||||
Current runtime boundary:
|
||||
- the default CLI runtime path (without explicit module selection) instantiates the full production module sequence.
|
||||
- LLM calls are exercised in production in both default full-pipeline runs and explicit `--modules` runs, and in tests when fake/injected clients are used.
|
||||
- normal `go test ./...` does not require real LLM credentials or Python dependencies.
|
||||
|
||||
`internal/framework/llm` also provides:
|
||||
- a bounded `Scheduler` for controlled concurrent LLM calls with reliable permit release;
|
||||
|
||||
Reference in New Issue
Block a user