Document Promptkit report generation

This commit is contained in:
2026-07-31 05:03:02 +00:00
parent 2c68d0a85f
commit b96f40e5ca
39 changed files with 256 additions and 1741 deletions

View File

@@ -1,126 +1,43 @@
# Application Orchestration Internals
`internal/app` composes top-level generation, batch, collection-save, and
inspection workflows after CLI parsing and configuration loading. It owns
workflow ordering, request composition, partial-result handling, and the
application-facing interfaces used for tests.
`internal/app` owns top-level generation, batch, collection, inspection, and
notification ordering after the CLI has parsed arguments and loaded configuration.
## Inputs And Outputs
## Generation
The package accepts generate, resolved-report, batch, explicit-collection, and
inspection requests. Generation and batch requests may supply collaborators for
tests; single-report generation uses a Promptkit executor, while batch requests
retain a renderer for their compatibility workflow.
`GenerateDetailed` resolves one of the four report definitions, initializes an
optional debug root, and inspects the exact Promptkit prompt/profile before it
collects weather or writes managed state. It then builds facts and modules,
saves the YAML data package, persists preparation metadata from the executor
callback, executes the prepared prompt, saves execution provenance and raw
output, validates generated text, renders Markdown, and optionally copies or
notifies from the managed report.
A report result contains the module snapshot, prompt package, prompt
provenance, generated-text artifacts, report and metadata
paths, prior snapshot, Recent Changes, and notification information. A batch
result contains aggregate counts, per-report outcomes, and an optional batch
notification. Inspection returns persisted values only.
Failure results retain all safe paths reached so far. Validation rejection
persists raw output and execution provenance but does not render a report.
Exact public command syntax, configuration fields, workspace layout, external
protocols, and report definitions belong in [the CLI reference](../cli.md),
[the configuration reference](../config.md), [operations](../operations.md),
and their focused integration and internal documents.
## Batches
`InspectPromptExecution` is a side-effect-free preflight helper for the prompt
workflow. It verifies the exact report prompt version, its required YAML input,
the generated-text JSON Schema contract, the selected profile, and any required
environment credential before collection or persistence begins. It returns only
safe project-owned identity and provenance values.
`RunBatchDetailed` constructs a single debug writer and uses the request's
single executor. Before collection it inspects Today, Tomorrow, and Daily for
morning, or Tomorrow and Daily for evening, deduplicating effective profile
inspection. It then collects once, plans eligible Daily dates, and calls the
same prompt-generation core sequentially for each planned report. Per-report
notification is suppressed; a failed report does not stop later reports.
## Single-Report Workflow
Batch notification is skipped when disabled or when any report failed.
Successful notification uses the completed managed report paths only. Batch
items retain preparation, execution, and optional debug paths when reached.
`GenerateDetailed` resolves the requested report using the configured registry
and current time, initializes any requested prompt-debug root, verifies the
exact Promptkit prompt and selected profile, and only then collects weather
data. Debug initialization or inspection failure produces no collection or
managed artifacts.
## Inspection And Boundaries
Single-report generation requires a non-nil normalized bundle and then performs this
ordered work:
Inspection loads persisted state only. It does not collect weather, invoke
Promptkit, or upload reports. The app coordinates project-owned contracts but
does not parse flags, load YAML, implement transport, construct provider SDKs,
or define report-period policy.
1. Select a state store, determine artifact destinations, and locate a prior
compatible snapshot.
2. Build report facts and deterministic module snapshots, then save the module
snapshot and calculate Recent Changes.
3. Serialize and save the prompt data package once, then use those exact bytes
for Promptkit execution.
4. Save preparation provenance and V2 metadata from the preparation callback
before provider execution. When requested, save preparation diagnostics in
the isolated debug store before the callback returns. Save execution
diagnostics immediately after a completed execution result, then persist raw
output and execution provenance before saving updated metadata.
5. Validate and save generated text, build and save a render context, and
render the managed Markdown template.
6. Optionally make an output copy, save final metadata, optionally notify
Distributor from the managed report path, and save metadata again when a
notification path is produced.
Focused checks:
Every single report looks up its catalog definition, saves raw Promptkit output,
preserves safe preparation and execution provenance separately, validates and
saves generated text, builds and saves a render context, then renders the
embedded Markdown template. Schema and template details remain in their
[generated-text](generatedtext.md) and [report-template](reporttemplate.md)
owners.
Preparation and operational execution failures save classified receipts and
metadata before returning. A completed Promptkit validation rejection saves raw
output, an execution receipt, and metadata before returning. If later report
generation fails, the result retains every reached safe artifact path; output
copies and notification are skipped until rendering succeeds.
The optional debug writer receives sensitive content only when explicitly
enabled. Its path is added to the report result only after a debug artifact is
successfully written; it is never copied into normal state records.
## Batch And Inspection Workflows
`RunBatchDetailed` collects once, asks the report registry to plan the batch
from that collection, and invokes its isolated Scriptorium compatibility helper
independently for every planned report using the same collection and state store. Per-report
notification is suppressed. A failed report is recorded and does not prevent
later planned reports from running.
After report generation, the batch notifier is considered once. It is omitted
when Distributor or batch notification is disabled, skipped when any report
failed, and otherwise receives one multi-file request. A batch notification
failure increments the aggregate failure count but does not rewrite successful
report items. Notification identities, path mappings, polling, and redaction
are owned by the [Distributor adapter](distributor-adapter.md).
Inspection methods create a state store and load existing report records,
metadata, module snapshots, prompt packages, prior snapshots, or source
provenance. They neither collect data nor invoke Scriptorium or Distributor.
## Boundaries And Failure Propagation
The app layer does not parse flags, load configuration files, implement Weather
API transport, invoke provider SDKs, or define report registry policy. It
coordinates the relevant collaborators and preserves their error context.
- Prompt inspection failure stops a single report before collection or durable
writes. Collection failure stops a single report or batch before planning.
- State, fact, module, prompt-input, preparation, or execution failures stop
that report before later report generation.
- A terminal Distributor failure is returned with the saved notification
information when available.
- Batch failures are represented per report and through aggregate batch status.
- Persisted artifact paths are carried in results so callers can inspect work
completed before a later failure.
## Tests And Invariants
Focused tests are in `internal/app/app_test.go` and
`internal/app/batch_plan_test.go`, with collection coverage in
`internal/collect/collect_test.go`.
- Production workflows collect through `internal/collect`.
- A report uses one explicit normalized collection throughout its generation.
- Prompt preparation provenance and metadata are persisted before provider
execution.
- Recent Changes compare structured module snapshots.
- Reports render from a validated typed context, never directly from a raw
prompt package.
- Only managed Markdown reports are notification sources; output copies are
never uploaded.
```sh
go test ./internal/app ./internal/collect
```

View File

@@ -4,7 +4,7 @@
collected facts, and derived facts. It owns the module registry, including
module support, fact requirements, option types, missing-data policy, builders,
and prompt-export hooks. It does not collect data, derive periods, write a
snapshot, construct YAML, invoke Scriptorium, or render a report.
snapshot, construct YAML, invoke Promptkit, or render a report.
## Registry and construction

View File

@@ -1,74 +1,23 @@
# CLI Internals
`internal/cli` turns process arguments into application requests and translates
application results into terminal output. The user-facing command, flag, and
output contract belongs in the [CLI reference](../cli.md).
`internal/cli` parses terminal arguments, loads configuration, constructs app
requests, and translates app results to bounded JSON summaries. The user
contract belongs in the [CLI reference](../cli.md).
## Responsibilities
For each `generate` or `run` action, `Runner` constructs one project-owned
Promptkit executor after configuration loads. It passes the executor and any
`--llm-debug-dir` request into the app. `run` accepts the debug flag as well
as `generate`; the app, not the CLI, secures and initializes the debug root.
`Runner.Run` dispatches the top-level action or inspection request. For actions,
the package parses command-specific and common flags, loads configuration with
CLI overrides, obtains the current time, and constructs either an
`app.GenerateRequest` or an `app.BatchRequest`. It delegates generation and
batch execution to `internal/app`.
Summaries include identity, status, safe artifact paths, and notification
provenance. They intentionally exclude module values, YAML package bodies, raw
generated text, rendered prompts, schemas, endpoints, credentials, and full
Distributor payloads. A failed action with a partial result still emits its
safe summary before its error is returned.
`Runner` also owns a project-owned prompt-executor factory seam. Its production
factory maps `promptkit` configuration to the Promptkit adapter, while tests can
inject a factory without importing dependency types. Each `generate` request
constructs one executor after configuration loads and passes it to the app.
CLI code owns no report policy, weather collection, persistence, provider
execution, or notification policy. Focused checks:
All four `generate` commands also accept `--llm-debug-dir PATH`. The CLI passes
only this explicit request to the app; the app initializes the secure debug
root before prompt inspection. `run` commands do not accept the flag.
For inspection, it loads configuration, builds the appropriate app inspection
request, and writes the returned value. Inspection is read-only; the inspected
artifact types and user invocation remain owned by the [CLI reference](../cli.md)
and [operations guide](../operations.md).
## Result Translation
Action results become CLI-safe JSON summaries in `result.go`. Generate summaries
carry report identity, status, relevant artifact paths, and notification
summary data. Batch summaries carry aggregate counts, per-report outcomes, and
the optional batch notification result. The translation deliberately excludes
full module snapshots, prompt packages, raw generated text, Scriptorium output,
complete Distributor payloads, and prompt-debug content.
When an action returns both a result and an error, the CLI writes the failed
summary before returning that error. Parse, configuration-load, and other
failures that produce no application result return without a summary.
`writeActionResult` writes action status information to stderr first, then JSON
to stdout. Batch execution supplies the status writer; single-report generation
does not emit routine stderr output. Quiet action requests suppress both normal
streams but still return errors. Inspection writes its JSON value to stdout and
does not accept quiet mode because stdout is the inspection result.
## Boundaries
The package owns argument parsing, request adaptation, help text, and terminal
presentation. It does not implement report selection, collection, state
persistence, external transport, subprocess execution, or notification policy.
Those concerns remain in [application orchestration](app-orchestration.md) and
their focused owners.
## Failure Behavior
- Invalid command names, flags, dates, and configuration fail before an app
request is executed.
- Application errors retain their application context; output helpers do not
hide or replace them.
- JSON-encoding errors are returned directly.
- A failed batch summary causes the CLI to return an aggregate batch error even
when the detailed batch call has already returned its result.
## Tests And Invariants
Focused tests are in `internal/cli/root_test.go`, `internal/cli/output_test.go`,
and `internal/cli/result_test.go`.
- CLI summaries are stable, bounded views of app results.
- Routine batch status lines precede the batch JSON summary.
- A quiet action produces no successful or failure summary output.
- Inspection never invokes action-output helpers.
```sh
go test ./internal/cli
```

View File

@@ -13,7 +13,7 @@ calls `FetchBundle`, and returns `Result{Bundle: *weatherdata.Bundle}`.
The package wraps adapter construction failures as weather-collection setup
errors and fetch failures as bundle-collection errors. It does not retry,
persist, select reports, derive facts, build modules, invoke Scriptorium, or
persist, select reports, derive facts, build modules, invoke Promptkit, or
notify Distributor.
## Application Composition

View File

@@ -33,7 +33,7 @@ template iteration rather than maps.
Optional source stanzas become nil or fallback context fields. Missing required
stanzas, type-decoding failures, invalid metadata, or a generated-text type
that does not match the chosen handler fail before template execution. Prompt
packages, raw Scriptorium output, state persistence, and template asset lookup
packages, raw Promptkit output, state persistence, and template asset lookup
remain outside this package.
## Verification and invariants

View File

@@ -2,9 +2,9 @@
`internal/promptinput` converts report metadata, an ordered module snapshot,
Recent Changes, and source warnings into the YAML `data_package` consumed by
Scriptorium. It owns this package's schema, grouping, serialization, loading,
Promptkit. It owns this package's schema, grouping, serialization, loading,
and validation—not weather collection, module construction, path choice, or
subprocess execution.
provider execution.
## Package construction

View File

@@ -0,0 +1,24 @@
# Promptkit Adapter Internals
`internal/adapters/promptkit` maps Weatherreporter's project-owned executor contract to Promptkit.
The CLI maps `promptkit` configuration to a `PromptExecutorConfig` and constructs one executor
per action. Promptkit dependency types do not escape the adapter.
The adapter exposes exact prompt and profile inspection plus prepared execution. It maps Promptkit
inspection values to project-owned prompt input, output-contract, profile, preparation, execution,
validation, and optional debug values. It classifies adapter failures without copying provider secrets
or unbounded response bodies into application errors or normal state.
The app calls the executor's preparation callback before provider execution to persist safe preparation
provenance. Completed executions are then persisted as safe execution provenance and raw generated text
is validated by `internal/generatedtext`. The adapter does not write workspace state, render Markdown,
choose report definitions, or send Distributor notifications.
Focused tests:
```sh
go test ./internal/adapters/promptkit ./internal/cli ./internal/app
```
The public logical prompt/profile/schema contract is owned by the
[Promptkit integration guide](../integrations/promptkit.md).

View File

@@ -53,7 +53,7 @@ are likewise consumed by state and orchestration rather than recomputed there.
Unknown report IDs or batch names return errors. The registry never collects
weather data, builds modules, parses CLI flags, writes state, executes
Scriptorium, or delivers a report.
Promptkit, or delivers a report.
## Verification and invariants

View File

@@ -1,22 +1,21 @@
# Report Template Internals
`internal/reporttemplate` embeds and renders the repository's native Markdown
templates and exposes their companion generated-text schemas. The current asset
IDs are `daily`, `today`, `tomorrow`, and `hourly`. The template files, partials,
and complete render-context field reference are maintained in
templates. The current template IDs are `daily`, `today`, `tomorrow`, and
`hourly`. The template files, partials, and complete render-context field
reference are maintained in
[report templates](../templates.md).
## Assets and lookup
The package embeds top-level templates, shared partials, and JSON schemas from
its asset directories. `Template` and `Schema` return the requested embedded
asset and fail with the requested ID when it is unknown or unreadable.
The package embeds top-level templates and shared partials. `Template` returns
the requested embedded template and fails with the requested ID when it is
unknown or unreadable.
Generated-text catalog handlers obtain schema bytes and template source through
these APIs. Prompt source files are repository assets for prompt registration;
they are not reporttemplate lookup assets. Report definitions select IDs, while
[generated-text internals](generatedtext.md) verifies the supported
schema/template pairing.
Generated-text schemas and Promptkit definitions are owned by
`internal/promptassets`; report-template owns Markdown source only. Report
definitions select IDs, while [generated-text internals](generatedtext.md)
verifies the supported schema/template pairing.
## Rendering
@@ -36,16 +35,17 @@ validation.
This package does not collect weather data, build modules, validate generated
text, construct contexts, resolve report definitions, write state, execute
Scriptorium, or upload reports. It produces Markdown bytes for application
Promptkit, or upload reports. It produces Markdown bytes for application
orchestration to persist.
Focused tests cover asset lookup, schema availability, rendering, partial
Focused tests cover template lookup, rendering, partial
behavior, missing keys, and malformed context:
```sh
go test ./internal/reporttemplate
```
Embedded assets stay as separate files, shared fragments stay under the partial
directory, and generated-text schemas describe prose slots rather than
deterministic weather facts.
Embedded templates stay as separate files and shared fragments stay under the
partial directory. Generated-text schemas are embedded separately by
`internal/promptassets` and describe prose slots rather than deterministic
weather facts.

View File

@@ -1,51 +0,0 @@
# Scriptorium Adapter Internals
`internal/adapters/scriptorium` translates Weatherreporter render requests to
Scriptorium process arguments and translates process results back to local
types. The external CLI and output contract belongs to the
[Scriptorium integration guide](../integrations/scriptorium.md); prompts,
template inputs, and report ownership remain outside this adapter.
## Request-to-command translation
`Runner` accepts a binary, config path, profile, timeout, extra arguments, and
an injectable command executor. Its defaults are the `scriptorium` binary and
the real `ExecRunner`. Optional configuration flags are placed before the
operation-specific arguments, and extra arguments are appended last.
| Local operation | Required values | Translated arguments |
| --- | --- | --- |
| `Render` | prompt ID, data-package path | `render [--config …] [--profile …] --prompt <id> --input data_package=<path> --format json [extra …]` |
| `Run` | prompt ID, data-package path, output path | `run [--config …] [--profile …] --prompt <id> --input data_package=<path> --out <path> [extra …]` |
| `StructuredRun` | prompt ID, data-package path, output path | Same translation as `Run` |
Blank required values fail before a command starts. The adapter does not add
schema flags or interpret a prompt's payload; it only gives Scriptorium the
named `data_package` input.
## Command execution and result translation
`ExecRunner` uses `exec.CommandContext`, never a shell. A positive configured
timeout creates a child context. Standard output and standard error are
captured independently, each with a 1 MiB limit, and the executed command is
retained for diagnostics.
`RenderResult`, `RunResult`, and `StructuredRunResult` expose the command,
captured output, truncation markers, and exit code. Run results also retain the
requested output path. Exit status zero is successful. A nonzero process exit
returns its result and an error, while a start failure, cancellation, or
deadline failure returns no result and the execution error.
The adapter does not parse rendered JSON, validate a generated report, write
state, or upload a report. Those responsibilities sit with
[application orchestration](app-orchestration.md), [state internals](state.md), and the
relevant delivery adapter.
## Verification
Focused tests cover argument order, validation, bounded capture, timeout and
cancellation handling, and exit-status translation:
```sh
go test ./internal/adapters/scriptorium
```

View File

@@ -1,112 +1,44 @@
# State Internals
The `internal/state` package owns filesystem-backed run state: safe path
derivation, metadata persistence, prior-report lookup, and read-only report
inspection. It does not decide which reports to generate or deliver. For the
operator-facing layout and retention procedures, see the
[operations guide](../operations.md).
`internal/state` owns safe workspace paths, atomic artifact writes, metadata,
prior-snapshot lookup, and read-only inspection. Operators should use the
[operations guide](../operations.md) for lifecycle and retention.
## Store construction and artifact paths
## Artifact Paths
`NewFilesystemStore` requires a workspace root and rejects absolute or
escaping values for every configured state directory. `Paths` then validates a
run ID and artifact group before deriving all paths from the report's valid
start date (`YYYY-MM-DD`). This keeps a run's artifacts together while making
the paths safe to use below the configured workspace.
For each run, paths are grouped by artifact group and valid start date:
| Artifact | Derived location |
| Artifact | Location |
| --- | --- |
| Module snapshot | `snapshots/<group>/<date>/modules.<run-id>.json` |
| Metadata | `snapshots/<group>/<date>/metadata.<run-id>.json` |
| Data package | `data-packages/<group>/<date>/data_package.<run-id>.yaml` |
| Prompt preparation | `preflight/<group>/<date>/prompt_preparation.<run-id>.json` |
| Prompt execution | `snapshots/<group>/<date>/prompt_execution.<run-id>.json` |
| Render preflight | `preflight/<group>/<date>/render.<run-id>.json` |
| Notification record | `notifications/<group>/<date>/distributor.<run-id>.json` |
| Raw generated text | `snapshots/<group>/<date>/generated_text_raw.<run-id>.json` |
| Validated generated text | `snapshots/<group>/<date>/generated_text.<run-id>.json` |
| Render context | `snapshots/<group>/<date>/render_context.<run-id>.json` |
| Managed report | `reports/<group>/<date>/report.<run-id>.md` |
| Generated text | `snapshots/<group>/<date>/generated_text.<run-id>.json` |
| Generated-text source and result | `snapshots/<group>/<date>/generated_text_raw.<run-id>.json` and `generated_text_result.<run-id>.json` |
| Generated-text render context | `snapshots/<group>/<date>/render_context.<run-id>.json` |
| Notification | `notifications/<group>/<date>/distributor.<run-id>.json` |
The configured notification root separates notification artifacts from report
artifacts; single-report notification paths use the report's valid date. Report
producers create parent directories as needed and write the report body; state
is responsible for the surrounding paths and saved run artifacts.
Batch notification records are `notifications/batches/<batch>/<local-date>/distributor.<batch-run-id>.json`.
Batch Distributor notifications are derived separately as
`notifications/batches/<batch>/<local-date>/distributor.<batch-run-id>.json`.
Their date is calculated from the batch start in its configured location, and
the batch identity and run ID receive the same path-segment validation as
single-report artifact identifiers.
## Metadata And Debug Storage
## Metadata and durable writes
New metadata is `weatherreporter.metadata.v2` and gains preparation and
execution paths only after those artifacts are saved. Legacy V1 records remain
readable; their historic preflight and generated-text-result fields are mapped
to the corresponding preparation and execution views during inspection. New
runs never write V1 records.
`Metadata` is the durable inventory for a run. It records its schema version,
run identity, generated and valid timestamps, artifact group, source content
and provenance, and the module snapshot, data-package, prompt preparation,
prompt execution, report, generated-artifact, and notification locations when
present. New prompt records use `weatherreporter.metadata.v2`; historic
`weatherreporter.metadata.v1` records remain readable and retain their legacy
JSON field names when inspected.
`PromptDebugWriter` is separate from workspace state. An empty root disables
it. An enabled absolute root is checked for safe directories and symlinks, then
stores `preparation.json` and `execution.json` beneath
`<root>/<report-id>/<valid-date>/<run-id>/`. Directories are `0700`; files are
atomic `0600`. Normal state discovery does not read this root.
`BuildMetadataFromBriefingMetadata` establishes legacy common fields, while
`BuildPromptMetadataFromBriefingMetadata` establishes the V2 record. The
application adds locations only after the corresponding artifacts are
produced. `SaveMetadata` requires the run ID, module snapshot, data package,
metadata path, and the matching preparation reference for its schema. The
package also saves module snapshots, data packages, prompt preparation and
execution records, legacy preflight records, generated-text artifacts, render
contexts, and notifications. JSON writes use atomic replacement, so readers do
not observe a partially written state file.
## Explicit prompt debug storage
`PromptDebugWriter` is a separate, opt-in boundary for content-rich prompt
diagnostics. It is constructed with an explicit absolute operator root, rather
than a workspace-derived path. A blank root produces a disabled writer that
does not access the filesystem.
Enabled debug captures are grouped as
`<root>/<report-id>/<valid-date>/<run-id>/` and contain `preparation.json` and
`execution.json`. The writer rejects symlinks, unsafe path segments, path
escape, and non-directory roots; it creates its directories with `0700` and
writes files atomically with `0600`. Normal state discovery and inspection do
not read this root. Its wire records map only approved project-owned fields;
credentials and dependency objects are not persisted.
The data package itself follows the shared
[prompt-input contract](prompt-input.md). Report text, templates, and external
delivery payloads remain owned by their respective packages and integration
references.
## Prior reports and inspection
`FindPriorSnapshot` searches metadata rather than guessing from filenames. It
only considers an earlier compatible report in the same artifact group and
supports the comparison strategies defined by the report request:
- `same_valid_date` finds an earlier generated report for the same valid day.
The newest eligible metadata record wins; the current run is excluded.
Unreadable or malformed candidate metadata is ignored so a damaged historical
record does not block a new run.
`ListReports` walks saved metadata, returns results ordered newest-first by
generation time, and treats a missing snapshots directory as an empty history.
`LoadMetadataByRunID` builds on that inspection path. These APIs are read-only;
repairing or pruning stored state is an operational concern.
## Boundaries and verification
The package rejects unsafe path components and incomplete metadata before
writing. Callers must provide a valid report request, artifact group, and
store configuration. Its focused tests cover path derivation, atomic
persistence, metadata validation, comparison eligibility, and report listing:
Focused checks:
```sh
go test ./internal/state
```
See [application orchestration](app-orchestration.md) for the order in which
these artifacts are created and [report templates](../templates.md) for the
user-facing report contract.