Document completed Phase 6 diagnostics
This commit is contained in:
@@ -14,8 +14,8 @@ Implemented today:
|
||||
- Transcript and glossary parsing/validation.
|
||||
- Deterministic transcript normalization.
|
||||
- Deterministic token estimation and transcript chunking.
|
||||
- Per-run diagnostics directory creation plus basic artifacts.
|
||||
- Minimal process report JSON output.
|
||||
- Per-run diagnostics directory creation plus Phase 6 process-level artifacts.
|
||||
- Process report JSON output with diagnostics artifact references.
|
||||
- Framework foundation packages for contracts and proposal application.
|
||||
|
||||
Not implemented in CLI runtime path today:
|
||||
@@ -196,16 +196,35 @@ Current per-run artifacts include:
|
||||
- `normalized-transcript.json`
|
||||
- `normalization-summary.json`
|
||||
- `chunking-summary.json`
|
||||
- `invocation.json`
|
||||
- `effective-config.json` (redacted credentials)
|
||||
- `report.json`
|
||||
- `error.log` on failure
|
||||
|
||||
`--report-json` writes a separate report file when requested.
|
||||
|
||||
Current process reports include diagnostics metadata references for:
|
||||
- diagnostics directory path;
|
||||
- source transcript artifact path;
|
||||
- parsed source transcript artifact path;
|
||||
- normalized transcript artifact path;
|
||||
- normalization summary artifact path;
|
||||
- chunking summary artifact path;
|
||||
- invocation metadata artifact path;
|
||||
- redacted effective-config artifact path;
|
||||
- error-log artifact path on failure.
|
||||
|
||||
Retention modes implemented in `ApplyRetention`:
|
||||
- `always`: keep successful runs
|
||||
- `never`: remove successful runs
|
||||
- `auto`: currently same as `never` for successful runs
|
||||
- failed runs are always retained
|
||||
- `always`: keep all run directories.
|
||||
- `never`: keep successful run directories.
|
||||
- `auto`: keep failed runs and successful runs with skipped corrections.
|
||||
- failed runs are always retained.
|
||||
|
||||
Current runtime note:
|
||||
- module execution is not implemented yet, so successful runs generally have no skipped corrections and `auto` typically removes clean successful run directories.
|
||||
|
||||
Intentionally deferred to module/LLM phases:
|
||||
- module prompt/response diagnostics artifacts are not produced yet because module execution and LLM calls are not in the runtime path.
|
||||
|
||||
## Current tests and quality posture
|
||||
Implemented tests currently cover:
|
||||
|
||||
@@ -9,8 +9,9 @@ Implemented:
|
||||
- Transcript/glossary schema parsing and validation.
|
||||
- Deterministic normalization with summary stats.
|
||||
- Deterministic chunking with summary stats.
|
||||
- Per-run diagnostics directory plus normalization/chunking artifacts.
|
||||
- Minimal process report output (`--report-json` and run-dir `report.json`).
|
||||
- Per-run diagnostics directory plus source/normalization/chunking artifacts.
|
||||
- Redacted invocation/effective-config diagnostics metadata artifacts.
|
||||
- Process report output (`--report-json` and run-dir `report.json`) with diagnostics artifact references.
|
||||
- Framework foundation packages for contracts and proposal preview/apply semantics.
|
||||
- Broad deterministic and CLI/subprocess test coverage for the implemented phases (`go test ./...`).
|
||||
|
||||
@@ -30,29 +31,37 @@ Not yet implemented in runtime pipeline:
|
||||
- Phase 4: Chunking and token estimation (deterministic implementation present).
|
||||
- Phase 5: Proposal model and application semantics (foundation package implemented and tested).
|
||||
|
||||
### Partial
|
||||
- Phase 6: Reports and diagnostics.
|
||||
### Newly completed
|
||||
- Phase 6: Reports and diagnostics (deterministic runtime scope).
|
||||
|
||||
What is complete:
|
||||
- run directory creation
|
||||
- source/normalized/chunking artifacts
|
||||
- report writing
|
||||
Implemented Phase 6 behavior in current runtime:
|
||||
- run directory creation and process-level diagnostics artifact writing
|
||||
- source transcript artifacts
|
||||
- normalized transcript artifact
|
||||
- normalization summary artifact
|
||||
- chunking summary artifact
|
||||
- invocation metadata artifact
|
||||
- redacted effective-config artifact
|
||||
- run-dir `report.json` plus optional external `--report-json`
|
||||
- failure `error.log`
|
||||
- retention mode parsing/validation
|
||||
- report-level diagnostics artifact path references
|
||||
- retention decision model with skip-aware hook:
|
||||
- `always` keeps all runs
|
||||
- `never` keeps successful runs
|
||||
- `auto` keeps failed runs and successful runs with skipped corrections
|
||||
- failed runs are always retained
|
||||
|
||||
What is still incomplete vs original plan:
|
||||
- redacted invocation/config metadata artifacts are not written yet
|
||||
- module prompt/response diagnostics are not present (no module execution yet)
|
||||
- `auto` retention semantics currently behave like `never` for successful runs
|
||||
Current runtime note:
|
||||
- successful runs generally have no skipped corrections because module execution is not implemented yet, so `auto` usually removes clean successful run directories.
|
||||
|
||||
### Pending
|
||||
- Phase 7+: runner orchestration, validators in runtime path, LLM client, bounded concurrency, real modules, full compatibility and side-by-side rollout phases.
|
||||
|
||||
## Deviations from original phase plan
|
||||
## Scope notes for later phases
|
||||
1. Proposal foundation (`internal/framework/proposals`) and framework contracts (`internal/framework/contracts`) were implemented before full runner orchestration.
|
||||
2. Chunking was integrated directly into the current CLI runtime path before module execution exists.
|
||||
3. Reporting currently uses a process-level status report with `phase: "phase3-chunking"`, rather than module-level pipeline reports.
|
||||
4. Work-dir retention `auto` is intentionally deferred and currently simplified.
|
||||
4. Module prompt/response diagnostics are intentionally deferred until module execution and LLM call phases.
|
||||
|
||||
## Next recommended implementation phase
|
||||
Implement **Phase 7 (pipeline runner with fake modules)** next.
|
||||
|
||||
Reference in New Issue
Block a user