79 lines
3.2 KiB
Markdown
79 lines
3.2 KiB
Markdown
# Python vs Go Parity Notes
|
|
|
|
This document tracks Phase 17 parity-fixture coverage and differences between the original Python implementation and the Go rewrite.
|
|
|
|
## Scope
|
|
|
|
- Uses deterministic fixture-driven tests in `internal/cli/testdata/parity`.
|
|
- Uses fake structured LLM responses for proposal generation and LLM validators.
|
|
- Verifies functional contract fields (module order, instance naming, applied/skipped counts, report status, diagnostics presence).
|
|
- Does not require real LLM credentials or Python dependencies during `go test ./...`.
|
|
|
|
## Intentional Differences
|
|
|
|
The following differences are expected and treated as intentional unless they break contract behavior:
|
|
|
|
1. JSON formatting and field ordering
|
|
- Serialized JSON whitespace and object key order may differ.
|
|
- Parity tests compare JSON semantically, not byte-for-byte.
|
|
|
|
2. Diagnostics path values
|
|
- Absolute run-directory paths, run IDs, and temp directory roots differ by runtime and platform.
|
|
- Parity checks assert artifact presence/shape, not exact absolute paths.
|
|
|
|
3. Time-variant metadata
|
|
- Timestamps (`started_at`, `completed_at`) and generated run IDs are runtime-specific.
|
|
- Parity checks ignore exact timestamp/run-id values.
|
|
|
|
4. Provider metadata/token accounting
|
|
- Provider/token usage metadata may vary by adapter behavior and is not asserted as strict parity fields.
|
|
|
|
5. Internal adapter implementation details
|
|
- Go uses its own structured LLM adapter implementation details while preserving the same high-level contract semantics.
|
|
|
|
## Current Fixture Coverage
|
|
|
|
Current parity fixtures cover:
|
|
|
|
- Transcript schema handling failure path.
|
|
- Glossary schema handling failure path.
|
|
- Default full module sequence shape:
|
|
- `glossary_1`, `homophones`, `glossary_2`, `spoken_word`, `grammar`.
|
|
- Mutable transcript handoff across default stages.
|
|
- Module-specific behavior inside the default sequence:
|
|
- glossary correction
|
|
- homophone-style correction
|
|
- spoken-word cleanup
|
|
- grammar cleanup
|
|
- Protected glossary-term guardrail behavior.
|
|
- Deterministic validator rejection behavior.
|
|
- LLM validator decision/rejection behavior.
|
|
- Application-level skip behavior (`ambiguous_original_text`).
|
|
- Mid-pipeline failure with partial progress preserved in reports.
|
|
- Diagnostics artifact presence and secret-redaction checks.
|
|
|
|
## Open Parity Gaps (Not Intentional)
|
|
|
|
These are known Phase 17 expansion opportunities and should not be labeled as intentional compatibility differences:
|
|
|
|
1. Broader Python fixture import
|
|
- The current Go parity fixtures are native fixture cases; they do not yet ingest all existing Python test fixtures directly.
|
|
|
|
2. Side-by-side runner command
|
|
- No repository-standard Python+Go side-by-side parity command is required or enforced yet.
|
|
|
|
3. Wider transcript corpus
|
|
- Current fixtures are representative but not exhaustive across all transcript/glossary edge combinations.
|
|
|
|
## How To Extend
|
|
|
|
1. Add a new `*.case.json` file under `internal/cli/testdata/parity`.
|
|
2. Add referenced transcript/glossary/fake-LLM response files.
|
|
3. Encode deterministic expectations in the case:
|
|
- module order and instance names
|
|
- transcript output
|
|
- applied/skipped/rejected counts
|
|
- report status and failure metadata
|
|
- diagnostics artifact presence/redaction markers
|
|
4. Run `go test ./...`.
|