Update batch collection documentation

This commit is contained in:
2026-06-17 16:11:09 +00:00
parent 3eccafad6b
commit b913194fb4
5 changed files with 178 additions and 86 deletions

View File

@@ -5,11 +5,12 @@ This document describes the workflow coordinator in `internal/app`.
## Purpose
`internal/app` coordinates the top-level use cases after CLI parsing and config
loading are complete. It resolves report definitions, fetches weather data,
builds collected and derived facts, builds module snapshots and prompt-input
artifacts, invokes Scriptorium through the adapter boundary, optionally
notifies distributor through an app-owned notifier boundary, persists managed
state, runs batches, and reads existing artifacts for inspection.
loading are complete. It resolves report definitions, collects weather data
through `internal/collect`, builds collected and derived facts, builds module
snapshots and prompt-input artifacts, invokes Scriptorium through the adapter
boundary, optionally notifies distributor through an app-owned notifier
boundary, persists managed state, runs batches, and reads existing artifacts
for inspection.
## Inputs And Outputs
@@ -20,9 +21,9 @@ Inputs:
- `FetchBundleRequest` for explicit bundle fetch and save workflows
- `ReportRequest` for single-report generation
- resolved report definitions from `internal/report`
- weather data bundles from `internal/adapters/weatherapi`
- collection results from `internal/collect`
- prior snapshots loaded from `internal/state`
- optional renderer, notifier, and state-store fakes for tests
- optional collector, renderer, notifier, and state-store fakes for tests
Outputs:
@@ -45,15 +46,18 @@ construct Scriptorium argv.
Report selection and report identity policy come from `internal/report`.
Collected and derived fact contracts come from `internal/facts`.
Weather API transport stays in `internal/adapters/weatherapi`. Scriptorium
subprocess behavior stays in `internal/adapters/scriptorium`. Distributor
upload behavior stays in `internal/adapters/distributor`. Filesystem layout and
persisted metadata stay in `internal/state`.
Weather API transport stays in `internal/adapters/weatherapi`, and app-facing
upstream collection stays in `internal/collect`. Scriptorium subprocess
behavior stays in `internal/adapters/scriptorium`. Distributor upload behavior
stays in `internal/adapters/distributor`. Filesystem layout and persisted
metadata stay in `internal/state`.
## Data Flow Terms
- `CollectedFacts` are normalized source facts fetched once from Weather API
and made available to derivation and module builders.
- `collect.Result` is the app-facing upstream collection result. It carries the
normalized `weatherdata.Bundle` used by report generation.
- `CollectedFacts` are normalized source facts derived from a collected Weather
API bundle and made available to derivation and module builders.
- `DerivedFacts` are deterministic calculations over collected facts, the
resolved valid period, daypart configuration, and report-specific windows.
- `module.Output` values are ordered deterministic stanzas built from collected
@@ -77,33 +81,35 @@ defaults.
## Generation Workflow
Single-report generation shares this setup:
Single-report commands validate the report command, collect once through
`internal/collect`, resolve the requested report, and pass the resolved report
plus explicit collection into `GenerateReport`.
1. Resolve the command report to a `report.Resolved` value.
2. Create or use a filesystem store.
3. Locate any prior compatible snapshot through `internal/state`.
4. Fetch a Weather API bundle.
5. Build collected and derived facts once.
6. Execute configured modules and save the module snapshot.
7. Compute Recent Changes from structured prior and current module snapshots.
8. Build and save the YAML Scriptorium `data_package`.
9. Run Scriptorium render preflight.
10. Save preflight JSON when a render result is available.
11. Save metadata for inspection.
`GenerateReport` then uses this setup:
1. Create or use a filesystem store.
2. Locate any prior compatible snapshot through `internal/state`.
3. Build collected and derived facts from the supplied collection.
4. Execute configured modules and save the module snapshot.
5. Compute Recent Changes from structured prior and current module snapshots.
6. Build and save the YAML Scriptorium `data_package`.
7. Run Scriptorium render preflight.
8. Save preflight JSON when a render result is available.
9. Save metadata for inspection.
For `scriptorium_markdown` reports, generation then:
12. Runs Scriptorium report generation to the managed report path.
10. Runs Scriptorium report generation to the managed report path.
For `generated_text_template` reports, generation then:
12. Looks up the generated-text catalog entry for the report schema/template
10. Looks up the generated-text catalog entry for the report schema/template
IDs.
13. Runs structured Scriptorium generation to the raw generated-text JSON path.
14. Saves the structured Scriptorium run result.
15. Validates and saves normalized generated text.
16. Builds and saves a typed render context.
17. Renders Markdown from the embedded template to the managed report path.
11. Runs structured Scriptorium generation to the raw generated-text JSON path.
12. Saves the structured Scriptorium run result.
13. Validates and saves normalized generated text.
14. Builds and saves a typed render context.
15. Renders Markdown from the embedded template to the managed report path.
After either mode has produced a managed Markdown report, shared finalization:
@@ -119,7 +125,7 @@ After either mode has produced a managed Markdown report, shared finalization:
If render preflight returns both a result and an error, preflight JSON and
metadata are persisted before the error is returned. If Scriptorium report
generation returns an error after writing output, the managed report and
metadata remain inspectable. Notification is not attempted after Weather API,
metadata remain inspectable. Notification is not attempted after collection,
module snapshot, prompt input, render, Scriptorium run, or metadata-save
failures.
Generated-text report failures are returned with report ID, RunID, and the
@@ -134,17 +140,21 @@ failure context.
## Batch Workflow
`run morning` resolves Today Report, 3-Day Outlook, and Weekend Outlook except
on Sunday. `run evening` resolves Tomorrow Report. Daily Report is generated
only through `generate daily --date YYYY-MM-DD`; it is not part of scheduled
batches. Batch output copy names come from report definitions. Batch generation
continues independent reports after a failure, records each result, writes
compact status lines to stderr, emits a JSON summary to stdout, and returns an
aggregate error when any report failed. When notification is enabled, each
successfully generated report is notified independently. Notification failure
marks that report failed, records notification fields in the batch result, and
does not stop later reports. `--out-dir` copies are never used as notification
source files.
`run morning` collects once, plans Today Report, Tomorrow Report, and eligible
future Daily Reports from the collected hourly forecast, then passes the same
collection into each report generation. `run evening` uses the same collection
and planning rules, but starts with Tomorrow Report. Future Daily reports start
with the day after tomorrow and require complete hourly forecast coverage for
the target local civil day. Dynamic Daily `--out-dir` copies use
`daily-YYYY-MM-DD.md`; other batch copies use report definition output names.
A collection failure stops the batch before planning or report generation.
After planning succeeds, batch generation continues independent reports after a
failure, records each result, writes compact status lines to stderr, emits a
JSON summary to stdout, and returns an aggregate error when any report failed.
When notification is enabled, each successfully generated report is notified
independently. Notification failure marks that report failed, records
notification fields in the batch result, and does not stop later reports.
`--out-dir` copies are never used as notification source files.
## Inspection Workflow
@@ -155,8 +165,8 @@ inspection view.
## Failure Behavior
- Resolve errors stop the requested workflow before fetching weather data.
- Weather API and module execution errors stop that report before Scriptorium
- Resolve errors stop the requested workflow before collection.
- Collection and module execution errors stop that report before Scriptorium
runs.
- Prompt input validation fails before render preflight.
- Render and run errors preserve Scriptorium stderr and exit-code context.
@@ -173,12 +183,17 @@ inspection view.
Inspect:
- `internal/app/app_test.go`
- `internal/app/batch_plan_test.go`
- `internal/collect/collect_test.go`
- `internal/cli/root_test.go`
- `internal/state/filesystem_test.go`
## Invariants
- Report behavior is resolved through `internal/report`.
- Generate and run commands collect once before report generation.
- Batch planning is app-owned because future Daily membership depends on
collected hourly forecast coverage.
- Generated reports use the same app request and result types regardless of
report ID.
- Render preflight precedes Scriptorium report generation.