eric/audita
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 2 Wiki Activity

Complete Phase 7 runner orchestration

Browse Source
This commit is contained in:
Eric Rakestraw
2026-05-11 14:22:22 +00:00
parent 30606f5c49
commit 28fe899aa1
8 changed files with 760 additions and 83 deletions
Download Patch File Download Diff File Expand all files Collapse all files

35
docs/architecture.md
View File

@@ -17,14 +17,15 @@ Implemented today:
- Per-run diagnostics directory creation plus Phase 6 process-level artifacts.
- Process report JSON output with diagnostics artifact references.
- Framework foundation packages for contracts and proposal application.
- Production runner orchestration package with deterministic sequential module execution.
- Module-level report structures with applied/skipped change records.
Not implemented in CLI runtime path today:
- Real module execution pipeline (`glossary`, `homophones`, `spoken_word`, `grammar`).
- Structured LLM proposal generation.
- Validator chain execution.
- Pipeline runner orchestration.
- Production LLM scheduler behavior.
- End-to-end transcript polishing beyond normalization.
- End-to-end transcript polishing with real module behavior.
## Actual Go package layout
@@ -73,6 +74,9 @@ internal/framework/proposals/
policy.go
preview.go
apply.go
internal/framework/runner/
runner.go
```
## Current CLI behavior
@@ -94,14 +98,15 @@ Current runtime flow (`internal/cli/run.go`):
9. Write normalized transcript and normalization summary artifacts.
10. Chunk normalized transcript and compute chunk summaries.
11. Write chunking summary artifact.
12. Output normalized transcript to `--output` file or stdout.
13. Build process report (`phase` currently set to `phase3-chunking`).
14. Optionally write `--report-json`; always write run-dir `report.json`.
15. Apply work-dir retention.
12. Optionally execute runner modules sequentially when an injected module registry/factory is available (used by deterministic tests today).
13. Output working transcript to `--output` file or stdout.
14. Build process report (`phase` currently set to `phase7-runner`).
15. Optionally write `--report-json`; always write run-dir `report.json`.
16. Apply work-dir retention.
Important behavior details:
- Glossary is validated but not yet used for correction logic.
- No module execution occurs despite `--modules` config support.
- Glossary is validated but not yet used for real correction module logic.
- Default production CLI behavior remains deterministic normalization/chunking output because no real module implementations are registered yet.
- No LLM calls occur.
- Success path is generally quiet on stderr.
- Source IDs are preserved into a canonical transcript before normalization; normalization then reassigns output IDs sequentially from `1`.
@@ -187,7 +192,7 @@ Current behavior details:
`internal/framework/contracts` provides interfaces and run-spec metadata scaffolding, including deterministic repeated module instance naming (`ResolveModuleRunSpecs`).
These are framework primitives only; they are not yet wired into CLI runtime module execution.
These primitives are wired into the production runner and report model. Real module implementations are still pending.
## Reports and diagnostics (implemented)
Current per-run artifacts include:
@@ -214,6 +219,10 @@ Current process reports include diagnostics metadata references for:
- redacted effective-config artifact path;
- error-log artifact path on failure.
Current process reports also include:
- module-level results (when runner modules execute), including applied/skipped proposal changes;
- run-level module summary totals and failed module instance metadata.
Retention modes implemented in `ApplyRetention`:
- `always`: keep all run directories.
- `never`: keep successful run directories.
@@ -221,7 +230,7 @@ Retention modes implemented in `ApplyRetention`:
- failed runs are always retained.
Current runtime note:
- module execution is not implemented yet, so successful runs generally have no skipped corrections and `auto` typically removes clean successful run directories.
- real module execution is not implemented yet, so normal successful runs generally have no skipped corrections and `auto` typically removes clean successful run directories.
Intentionally deferred to module/LLM phases:
- module prompt/response diagnostics artifacts are not produced yet because module execution and LLM calls are not in the runtime path.
@@ -236,8 +245,10 @@ Implemented tests currently cover:
- deterministic chunking and summaries (`internal/core/chunking/*_test.go`)
- proposal preview/apply semantics (`internal/framework/proposals/*_test.go`)
- contracts/foundation composition tests (`internal/framework/contracts/*_test.go`)
- runner sequencing and failure behavior with deterministic fake modules (`internal/framework/runner/*_test.go`)
- CLI runner integration through injected fake module factories (`internal/cli/run_test.go`)
Not covered yet (because not implemented): end-to-end module runner behavior, validator runtime flow, and real LLM integration.
Not covered yet (because not implemented): validator runtime flow with approvals/rejections and real LLM integration.
## Intended final architecture (not yet implemented)
The intended end-state still matches the rewrite plan:
@@ -247,6 +258,6 @@ The intended end-state still matches the rewrite plan:
- deterministic and LLM validators
- validator cardinality enforcement in pipeline execution
- proposal application integrated per module stage
- richer run reporting with module-level applied/skipped changes
- prompt/response diagnostics for LLM/module stages
Until those phases are implemented, documentation and external descriptions should treat the current Go CLI as deterministic preprocessing/reporting infrastructure, not a full LLM transcript polisher.

76
docs/rewrite-notes.md
View File

@@ -44,10 +44,12 @@ Implemented:
- Process report output through `--report-json` and run-dir `report.json`.
- Report-level diagnostics artifact references.
- Framework foundation packages for contracts and proposal preview/apply semantics.
- Production runner orchestration over a mutable working transcript.
- Module-level report structures and run-level module summaries.
- CLI runner integration point via injectable module factory/registry (used by deterministic tests).
- Broad deterministic and CLI/subprocess test coverage for implemented phases through `go test ./...`.
Not yet implemented in runtime pipeline:
- Module runner orchestration.
- Real correction modules.
- Validator-chain execution.
- Structured LLM client integration.
@@ -135,53 +137,33 @@ Implemented:
Intentionally deferred:
- Module prompt/response diagnostics artifacts are not produced yet because module execution and LLM calls are not implemented in the runtime path.
### Phase 7: Pipeline runner with deterministic test modules
Completed.
Implemented:
- `internal/framework/runner` production package with sequential module orchestration.
- Deterministic module run-spec resolution and repeated instance naming (`glossary_1`, `glossary_2`, etc.).
- Mutable working transcript handoff across module instances.
- Proposal application through `internal/framework/proposals`.
- Per-module applied/skipped change capture and module status/timing metadata.
- Partial-progress return on module failure, with pipeline stop on first failure.
- Process report support for module-level results and run-level module summaries.
- CLI runtime integration point via injectable module factory/registry, exercised by deterministic fake-module tests.
Not implemented in Phase 7 (by design):
- Real `glossary`, `homophones`, `spoken_word`, `grammar` production modules.
- Validators (Phase 8).
- Structured LLM calls or scheduler behavior.
- Prompt/response diagnostics.
- End-to-end transcript polishing.
Current runtime behavior note:
- Default user-facing CLI behavior remains deterministic normalization/chunking output unless test-only module injection is used during tests.
## Remaining work plan
## Phase 7: Pipeline runner with deterministic test modules
### Purpose
Introduce the production pipeline runner that executes configured module instances sequentially over a mutable working transcript, but without real LLM calls or real correction modules yet.
This phase closes the largest architectural gap: the runtime path currently normalizes and chunks, but does not execute module stages.
### Scope
Implement:
- `internal/framework/runner` or equivalent production runner package.
- Module run-spec resolution from configured module keys.
- Sequential execution of module instances.
- Mutable working transcript handoff from one module to the next.
- Integration with existing proposal application package.
- Module-level report structures.
- Applied/skipped changes recorded per module instance.
- Test-only deterministic modules or fake modules to exercise runner behavior.
- CLI wiring from `internal/cli/run.go` into the runner.
Do not implement:
- Real `glossary`, `homophones`, `spoken_word`, or `grammar` modules.
- LLM calls.
- Validator chains.
- LLM scheduler behavior.
- Prompt/response diagnostics.
### Expected behavior at end of phase
The CLI should still not perform real transcript polishing by default unless a test-only module sequence is explicitly used in tests. The production path may resolve configured module names, but unsupported real modules should fail cleanly or remain gated until implemented in later phases.
The runner itself should be production-quality and tested with deterministic fake modules.
### Definition of done
- Configured module keys can be resolved into deterministic run specs.
- Repeated module names are resolved consistently, such as `glossary_1` and `glossary_2`.
- Runner executes module instances sequentially.
- Later modules see transcript changes from earlier modules.
- Applied and skipped changes are recorded per module instance.
- Process report includes module-level report structures.
- Existing stdout/stderr behavior remains unchanged.
- Diagnostics retention behavior remains unchanged.
- `go test ./...` passes.
Next recommended phase: **Phase 8 (runtime validator framework and deterministic validators)**.
## Phase 8: Runtime validator framework and deterministic validators
@@ -716,4 +698,4 @@ Before accepting a phase implementation, verify:
- Are fake LLM tests used instead of requiring real credentials?
- Are new public behaviors documented?
- Does the code follow the architecture in `docs/architecture.md`?
- Does the implementation move Audita closer to Python feature parity?
- Does the implementation move Audita closer to Python feature parity?