eric/weatherreporter
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 2 Wiki Activity

Add Promptkit debug capture for generated reports

Browse Source
This commit is contained in:
Eric Rakestraw
2026-07-31 04:48:16 +00:00
parent 06b26d5e88
commit a6d11c01e8
11 changed files with 271 additions and 37 deletions
Download Patch File Download Diff File Expand all files Collapse all files

15
docs/internal/app-orchestration.md
View File

@@ -32,8 +32,9 @@ safe project-owned identity and provenance values.
## Single-Report Workflow
`GenerateDetailed` resolves the requested report using the configured registry
and current time, verifies the exact Promptkit prompt and selected profile, and
only then collects weather data. Inspection failure produces no collection or
and current time, initializes any requested prompt-debug root, verifies the
exact Promptkit prompt and selected profile, and only then collects weather
data. Debug initialization or inspection failure produces no collection or
managed artifacts.
Single-report generation requires a non-nil normalized bundle and then performs this
@@ -46,8 +47,10 @@ ordered work:
3. Serialize and save the prompt data package once, then use those exact bytes
for Promptkit execution.
4. Save preparation provenance and V2 metadata from the preparation callback
before provider execution. Save execution provenance after raw output is
persisted, then save updated metadata.
before provider execution. When requested, save preparation diagnostics in
the isolated debug store before the callback returns. Save execution
diagnostics immediately after a completed execution result, then persist raw
output and execution provenance before saving updated metadata.
5. Validate and save generated text, build and save a render context, and
render the managed Markdown template.
6. Optionally make an output copy, save final metadata, optionally notify
@@ -67,6 +70,10 @@ output, an execution receipt, and metadata before returning. If later report
generation fails, the result retains every reached safe artifact path; output
copies and notification are skipped until rendering succeeds.
The optional debug writer receives sensitive content only when explicitly
enabled. Its path is added to the report result only after a debug artifact is
successfully written; it is never copied into normal state records.
## Batch And Inspection Workflows
`RunBatchDetailed` collects once, asks the report registry to plan the batch

6
docs/internal/cli.md
View File

@@ -17,6 +17,10 @@ factory maps `promptkit` configuration to the Promptkit adapter, while tests can
inject a factory without importing dependency types. Each `generate` request
constructs one executor after configuration loads and passes it to the app.
All four `generate` commands also accept `--llm-debug-dir PATH`. The CLI passes
only this explicit request to the app; the app initializes the secure debug
root before prompt inspection. `run` commands do not accept the flag.
For inspection, it loads configuration, builds the appropriate app inspection
request, and writes the returned value. Inspection is read-only; the inspected
artifact types and user invocation remain owned by the [CLI reference](../cli.md)
@@ -29,7 +33,7 @@ carry report identity, status, relevant artifact paths, and notification
summary data. Batch summaries carry aggregate counts, per-report outcomes, and
the optional batch notification result. The translation deliberately excludes
full module snapshots, prompt packages, raw generated text, Scriptorium output,
and complete Distributor payloads.
complete Distributor payloads, and prompt-debug content.
When an action returns both a result and an error, the CLI writes the failed
summary before returning that error. Parse, configuration-load, and other