4.7 KiB
Diagnostics and Report Assembly
Scope
This document owns implementation flow and collaborators. Physical artifacts and retention belong to operations. Serialized reports, ledger records, and utilization shapes belong to the output-file contract.
Run Directory and Artifact Writers
internal/core/diagnostics creates run directories, writes snapshots and metadata,
and evaluates retention. CLI orchestration supplies the selected configuration,
paths, run outcome, and correction-presence information. Artifact names are
centralized in diagnostics; their operator-facing inventory is in
operations.
Snapshot writes are attempted at the relevant processing phases. Several diagnostic writes are best-effort, so report metadata can identify a location without proving that a file was written. The CLI distinguishes primary processing errors from secondary diagnostic/report write errors.
Process Report Mapping
internal/framework/processreport.Build maps invocation facts, normalization and
chunking summaries, and runner output into reporting.ProcessReport. Core reporting
owns the Go data structures and JSON writer. The builder attaches diagnostic
metadata when a run-directory path is available.
Module mapping copies outcomes from runner results, collects summary counts, and preserves available partial module results on failure. The CLI failure path does not pass normalization or chunking summaries to report construction even when some earlier processing completed. Consumer interpretation belongs to the process-report contract.
Correction Ledger and Utilization
BuildCorrectionLedger flattens module changes, rejections, and failures into
ledger entries. It associates validator records by proposal index within each
module, separates execution classes through validator metadata, and sorts entries
by module instance, proposal index, and disposition.
Runner instrumentation and PromptKit observations collect utilization; CLI
orchestration writes the backend summary and safe call/event artifact when run
output is available. Report assembly maps the same observations to llm_summary
without independently measuring calls. A selected backend remains attributable
when preparation fails before dispatch. Domain module/validator timing stays
separate from provider timing, and those durations can overlap.
For the serialized forms see ledger
and utilization.
CLI Coordination and Retention
CLI orchestration writes caller-selected reports and run-local artifacts before
asking diagnostics to apply retention. HasSkippedCorrections derives the
retention input from runner outcomes. Keep the exact lifecycle definition in
operations, not in another internal table.
On a processing failure, secondary report-write errors do not replace the original error. On the success path, an external report-write failure returns before the run-local report write; retention errors occur later. This ordering explains why artifacts and process exit status must be assessed together.
Redaction
Effective configuration uses ConfigKit redaction. The CLI supplies all resolved
backend credentials, including unselected backends, to report and diagnostic
writers. core/redaction sanitizes the serialized representation rather than
mutating processing results. JSON string values and keys are redacted after
parsing with number preservation; YAML redaction preserves scalar types.
Serialization failure returns an error without writing the original object.
Interaction artifacts, correction ledgers, execution/utilization diagnostics,
selected profiles, both report copies, and surfaced LLM errors share this
string-redaction implementation. Longer overlapping values take precedence,
and replacement text is not processed again within a redaction pass.
Execution reports index events by call ID. Utilization groups calls directly by backend and actual module instance, keeping module result order and retaining usage knownness/completeness. Section and batch indices remain separate from module identity; section-qualified interaction filenames avoid collisions. These boundaries implement the architecture requirement. Operational handling of transcript-bearing artifacts belongs to operations.
Key Tests
internal/core/redaction/redaction_test.gointernal/core/diagnostics/*_test.gointernal/framework/processreport/*_test.gointernal/core/reporting/report_test.gocmd/audita/main_integration_test.go