audita/docs/python-parity.md

Python vs Go Parity Notes

This document tracks parity-fixture coverage and intentional differences between historical Python behavior and the current Audita runtime.

Scope

• Uses deterministic fixture-driven tests in internal/cli/testdata/parity.
• Uses fake structured LLM responses for proposal generation and LLM validators.
• Verifies functional contract fields (module order, instance naming, applied/skipped counts, report status, diagnostics presence).
• Does not require real LLM credentials or Python dependencies during go test ./....

Intentional Differences

The following differences are expected and treated as intentional unless they break contract behavior:

1. JSON formatting and field ordering

• Serialized JSON whitespace and object key order may differ.
• Parity tests compare JSON semantically, not byte-for-byte.

2. Diagnostics path values

• Absolute run-directory paths, run IDs, and temp directory roots differ by runtime and platform.
• Parity checks assert artifact presence/shape, not exact absolute paths.

3. Time-variant metadata

• Timestamps (started_at, completed_at) and generated run IDs are runtime-specific.
• Parity checks ignore exact timestamp/run-id values.

4. Provider metadata/token accounting

• Provider/token usage metadata may vary by adapter behavior and is not asserted as strict parity fields.

5. Internal adapter implementation details

• Go uses its own structured LLM adapter implementation details while preserving the same high-level contract semantics.

Current Fixture Coverage

Current parity fixtures cover:

• Transcript schema handling failure path.
• Glossary schema handling failure path.
• Default full module sequence shape:
  • glossary_1, homophones, glossary_2, spoken_word, grammar.
• Mutable transcript handoff across default stages.
• Module-specific behavior inside the default sequence:
  • glossary correction
  • homophone-style correction
  • spoken-word cleanup
  • grammar cleanup
• Protected glossary-term guardrail behavior.
• Deterministic validator rejection behavior.
• LLM validator decision/rejection behavior.
• Application-level skip behavior (ambiguous_original_text).
• Mid-pipeline failure with partial progress preserved in reports.
• Diagnostics artifact presence and secret-redaction checks.

Open Parity Gaps (Not Intentional)

These are known parity expansion opportunities and should not be labeled as intentional compatibility differences:

1. Broader Python fixture import

• The current Go parity fixtures are native fixture cases; they do not yet ingest all existing Python test fixtures directly.

2. Side-by-side runner command

• No repository-standard Python+Go side-by-side parity command is required or enforced yet.

3. Wider transcript corpus

• Current fixtures are representative but not exhaustive across all transcript/glossary edge combinations.

How To Extend

1. Add a new *.case.json file under internal/cli/testdata/parity.
2. Add referenced transcript/glossary/fake-LLM response files.
3. Encode deterministic expectations in the case:

• module order and instance names
• transcript output
• applied/skipped/rejected counts
• report status and failure metadata
• diagnostics artifact presence/redaction markers

4. Run go test ./....