3.9 KiB
Diagnostics and Report Assembly
Scope
This document owns implementation flow and collaborators. Physical artifacts and retention belong to operations. Serialized reports, ledger records, and utilization shapes belong to the output-file contract.
Run Directory and Artifact Writers
internal/core/diagnostics creates run directories, writes snapshots and metadata,
and evaluates retention. CLI orchestration supplies the selected configuration,
paths, run outcome, and correction-presence information. Artifact names are
centralized in diagnostics; their operator-facing inventory is in
operations.
Snapshot writes are attempted at the relevant processing phases. Several diagnostic writes are best-effort, so report metadata can identify a location without proving that a file was written. The CLI distinguishes primary processing errors from secondary diagnostic/report write errors.
Process Report Mapping
internal/framework/processreport.Build maps invocation facts, normalization and
chunking summaries, and runner output into reporting.ProcessReport. Core reporting
owns the Go data structures and JSON writer. The builder attaches diagnostic
metadata when a run-directory path is available.
Module mapping copies outcomes from runner results, collects summary counts, and preserves available partial module results on failure. The CLI failure path does not pass normalization or chunking summaries to report construction even when some earlier processing completed. Consumer interpretation belongs to the process-report contract.
Correction Ledger and Utilization
BuildCorrectionLedger flattens module changes, rejections, and failures into
ledger entries. It associates validator records by proposal index within each
module, separates execution classes through validator metadata, and sorts entries
by module instance, proposal index, and disposition.
Runner instrumentation and PromptKit observations collect utilization; CLI
orchestration writes the backend summary and safe call/event artifact when run
output is available. Report assembly maps the same observations to llm_summary
without independently measuring calls. A selected backend remains attributable
when preparation fails before dispatch. Domain module/validator timing stays
separate from provider timing, and those durations can overlap.
For the serialized forms see ledger
and utilization.
CLI Coordination and Retention
CLI orchestration writes caller-selected reports and run-local artifacts before
asking diagnostics to apply retention. HasSkippedCorrections derives the
retention input from runner outcomes. Keep the exact lifecycle definition in
operations, not in another internal table.
On a processing failure, secondary report-write errors do not replace the original error. On the success path, an external report-write failure returns before the run-local report write; retention errors occur later. This ordering explains why artifacts and process exit status must be assessed together.
Redaction
Effective configuration uses ConfigKit redaction. LLM diagnostic and error paths
use llm.ConfiguredSecrets and the runtime's redaction helpers. The CLI also
redacts every resolved backend secret from both report copies. Keep these separate
mechanisms aligned with the architecture requirement.
Operational handling of transcript-bearing artifacts belongs to
operations.
Key Tests
internal/core/diagnostics/*_test.gointernal/framework/processreport/*_test.gointernal/core/reporting/report_test.gocmd/audita/main_integration_test.go