eric/audita
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 6 Wiki Activity

Add version two LLM reports and safe execution diagnostics

Browse Source
This commit is contained in:
Eric Rakestraw
2026-09-24 02:31:37 +00:00
parent 4bfc5f3456
commit 9561e80efb
16 changed files with 428 additions and 40 deletions
Download Patch File Download Diff File Expand all files Collapse all files

13
docs/internal/diagnostics-reporting.md
View File

@@ -40,8 +40,12 @@ ledger entries. It associates validator records by proposal index within each
module, separates execution classes through validator metadata, and sorts entries
by module instance, proposal index, and disposition.
Runner instrumentation collects utilization; CLI orchestration writes it when
run output is available. Report assembly does not independently measure calls.
Runner instrumentation and PromptKit observations collect utilization; CLI
orchestration writes the backend summary and safe call/event artifact when run
output is available. Report assembly maps the same observations to `llm_summary`
without independently measuring calls. A selected backend remains attributable
when preparation fails before dispatch. Domain module/validator timing stays
separate from provider timing, and those durations can overlap.
For the serialized forms see [ledger](../integrations/output-files.md#correction-ledger)
and [utilization](../integrations/output-files.md#utilization-diagnostics).
@@ -59,8 +63,9 @@ artifacts and process exit status must be assessed together.
## Redaction
Effective configuration uses config redaction. LLM diagnostic and error paths
use `llm.ConfiguredSecrets` and the runtime's redaction helpers. Keep these separate
Effective configuration uses ConfigKit redaction. LLM diagnostic and error paths
use `llm.ConfiguredSecrets` and the runtime's redaction helpers. The CLI also
redacts every resolved backend secret from both report copies. Keep these separate
mechanisms aligned with the [architecture requirement](../policy/architecture.md#errors-and-secret-handling).
Operational handling of transcript-bearing artifacts belongs to
[operations](../operations.md#artifact-handling).