eric/weatherreporter
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 2 Wiki Activity

Refresh CLI collection and app internals

Browse Source
This commit is contained in:
Eric Rakestraw
2026-07-31 01:21:23 +00:00
parent 1130d807dc
commit c6f8570474
3 changed files with 166 additions and 307 deletions
Download Patch File Download Diff File Expand all files Collapse all files

298
docs/internal/app-orchestration.md
View File

@@ -1,233 +1,109 @@
# App Orchestration Internals # Application Orchestration Internals
This document describes the workflow coordinator in `internal/app`. `internal/app` composes top-level generation, batch, collection-save, and
inspection workflows after CLI parsing and configuration loading. It owns
## Purpose workflow ordering, request composition, partial-result handling, and the
application-facing interfaces used for tests.
`internal/app` coordinates the top-level use cases after CLI parsing and config
loading are complete. It resolves report definitions, collects weather data
through `internal/collect`, builds collected and derived facts, builds module
snapshots and prompt-input artifacts, invokes Scriptorium through the adapter
boundary, optionally notifies distributor through an app-owned notifier
boundary, persists managed state, runs batches, and reads existing artifacts
for inspection.
## Inputs And Outputs ## Inputs And Outputs
Inputs: The package accepts generate, resolved-report, batch, explicit-collection, and
inspection requests. Generation and batch requests may supply collector,
renderer, store, and notifier implementations for tests; production defaults
use the focused packages.
- `GenerateRequest` for one report command A report result contains the module snapshot, prompt package, available
- `BatchRequest` for morning or evening batch commands Scriptorium results, generated-text artifacts when used, report and metadata
- `FetchBundleRequest` for explicit bundle collection and save workflows paths, prior snapshot, Recent Changes, and notification information. A batch
- `ReportRequest` for single-report generation result contains aggregate counts, per-report outcomes, and an optional batch
- resolved report definitions from `internal/report` notification. Inspection returns persisted values only.
- collection results from `internal/collect`
- prior snapshots loaded from `internal/state`
- optional collector, renderer, notifier, and state-store fakes for tests
Outputs: Exact public command syntax, configuration fields, workspace layout, external
protocols, and report definitions belong in [the CLI reference](../cli.md),
[the configuration reference](../config.md), [operations](../operations.md),
and their focused integration and internal documents.
- generated report results with JSON module snapshot, YAML data package, ## Single-Report Workflow
preflight, report, metadata, prior snapshot, Recent Changes, Scriptorium
result details, generated-text artifact paths when applicable, and
notification result when attempted
- batch summaries with per-report status, artifact paths, error text, and
one top-level batch notification result when attempted or skipped
- saved Weather API bundle JSON for explicit bundle collection workflows
- inspection JSON values for reports, metadata, module snapshots, data
packages, prior snapshots, and source provenance
## Boundaries `GenerateDetailed` first collects weather data, then resolves the requested
report using the configured registry and current time, and finally calls
`GenerateReport` with that explicit collection. It returns no result when
collection or resolution fails.
`internal/app` owns workflow order and request composition. It does not parse `GenerateReport` requires a non-nil normalized bundle and then performs this
CLI flags, load YAML files directly, implement HTTP transport, own fact ordered work:
derivation algorithms, define report periods, compare rendered Markdown, or
construct Scriptorium argv.
Report selection and report identity policy come from `internal/report`. 1. Select a state store, determine artifact destinations, and locate a prior
Collected and derived fact contracts come from `internal/facts`. compatible snapshot.
Weather API transport stays in `internal/adapters/weatherapi`, and app-facing 2. Build report facts and deterministic module snapshots, then save the module
upstream collection stays in `internal/collect`. Scriptorium subprocess snapshot and calculate Recent Changes.
behavior stays in `internal/adapters/scriptorium`. Distributor upload behavior 3. Build and save the prompt data package, run Scriptorium render preflight,
stays in `internal/adapters/distributor`. Filesystem layout and persisted save any preflight result, and save initial metadata.
metadata stay in `internal/state`. 4. Produce managed Markdown according to the report generation mode.
5. Finalize metadata, optionally make an output copy, and optionally notify
Distributor from the managed report path.
## Data Flow Terms Direct-Markdown reports prepare the managed report and invoke the Scriptorium
run boundary. Generated-text-template reports look up their catalog definition,
run structured Scriptorium output to the raw artifact, preserve any structured
run result, validate and save generated text, build and save a render context,
then render the embedded Markdown template. Schema, template, and subprocess
details remain in their [generated-text](generatedtext.md),
[report-template](reporttemplate.md), and [Scriptorium adapter](scriptorium-adapter.md)
owners.
- `collect.Result` is the app-facing upstream collection result. It carries the If preflight returns a result with an error, the result and initial metadata are
normalized `weatherdata.Bundle` used by report generation. saved before the error returns. If report generation fails after a managed path
- `CollectedFacts` are normalized source facts derived from a collected Weather is prepared, metadata still records that path; output copies and notification
API bundle and made available to derivation and module builders. are skipped. Generated-text failures preserve the latest artifact reached
- `DerivedFacts` are deterministic calculations over collected facts, the before failure when it was saved.
resolved valid period, daypart configuration, and report-specific windows.
- `module.Output` values are ordered deterministic stanzas built from collected
and derived facts for prompt input and inspection.
- `GeneratedText` is structured prose returned by Scriptorium for
generated-text-template reports and validated by `internal/generatedtext`.
- `RenderContext` is the typed template input built from report metadata,
module outputs, and validated generated text before Markdown rendering.
## Config Fields Used ## Batch And Inspection Workflows
- `weather_api.*` for Weather API client construction and module metadata `RunBatchDetailed` collects once, asks the report registry to plan the batch
- `scriptorium.*` for renderer construction from that collection, and invokes `GenerateReport` independently for every
- `workspace.*` for filesystem state planned report using the same collection and state store. Per-report
- `dayparts` for daily and outlook summarization notification is suppressed. A failed report is recorded and does not prevent
- `recent_change.*` for structured Recent Changes thresholds later planned reports from running.
- `notify.distributor.*` for optional single-report and batch notification
after report generation
Output copy flags are command request fields. They are not configuration After report generation, the batch notifier is considered once. It is omitted
defaults. when Distributor or batch notification is disabled, skipped when any report
failed, and otherwise receives one multi-file request. A batch notification
failure increments the aggregate failure count but does not rewrite successful
report items. Notification identities, path mappings, polling, and redaction
are owned by the [Distributor adapter](distributor-adapter.md).
## Generation Workflow Inspection methods create a state store and load existing report records,
metadata, module snapshots, prompt packages, prior snapshots, or source
provenance. They neither collect data nor invoke Scriptorium or Distributor.
Single-report commands validate the report command, collect once through ## Boundaries And Failure Propagation
`internal/collect`, resolve the requested report, and pass the resolved report
plus explicit collection into `GenerateReport`. `GenerateDetailed` returns the
resulting `ReportResult`; `Generate` wraps the same workflow for error-only
callers.
`GenerateReport` then uses this setup: The app layer does not parse flags, load configuration files, implement Weather
API transport, construct Scriptorium argv, or define report registry policy. It
coordinates the relevant collaborators and preserves their error context.
1. Create or use a filesystem store. - Collection failure stops a single report or batch before resolution or
2. Locate any prior compatible snapshot through `internal/state`. planning completes.
3. Build collected and derived facts from the supplied collection. - State, fact, module, prompt-input, or preflight failures stop that report
4. Execute configured modules and save the module snapshot. before report generation.
5. Compute Recent Changes from structured prior and current module snapshots. - A terminal Distributor failure is returned with the saved notification
6. Build and save the YAML Scriptorium `data_package`. information when available.
7. Run Scriptorium render preflight. - Batch failures are represented per report and through aggregate batch status.
8. Save preflight JSON when a render result is available. - Persisted artifact paths are carried in results so callers can inspect work
9. Save metadata for inspection. completed before a later failure.
For `scriptorium_markdown` reports, generation then: ## Tests And Invariants
10. Runs Scriptorium report generation to the managed report path. Focused tests are in `internal/app/app_test.go` and
`internal/app/batch_plan_test.go`, with collection coverage in
`internal/collect/collect_test.go`.
For `generated_text_template` reports, generation then: - Production workflows collect through `internal/collect`.
- A report uses one explicit normalized collection throughout its generation.
10. Looks up the generated-text catalog entry for the report schema/template - Render preflight precedes report generation.
IDs. - Recent Changes compare structured module snapshots.
11. Runs structured Scriptorium generation to the raw generated-text JSON path. - Generated-text reports render from a validated typed context, never directly
12. Saves the structured Scriptorium run result. from a raw prompt package.
13. Validates and saves normalized generated text. - Only managed Markdown reports are notification sources; output copies are
14. Builds and saves a typed render context. never uploaded.
15. Renders Markdown from the embedded template to the managed report path.
After either mode has produced a managed Markdown report, shared finalization:
1. Copies the managed report to the requested `--out` or `--out-dir` path when
provided.
2. Saves final metadata with the managed report path and any generated-text
artifact paths already produced.
3. If distributor notification is enabled, notifies using the managed report
path as the source file.
4. Saves a distributor notification debug artifact and updates metadata with
its path.
If render preflight returns both a result and an error, preflight JSON and
metadata are persisted before the error is returned. If Scriptorium report
generation returns an error after writing output, the managed report and
metadata remain inspectable. Notification is not attempted after collection,
module snapshot, prompt input, render, Scriptorium run, or metadata-save
failures.
Generated-text report failures are returned with report ID, RunID, and the
failed operation. When available, the app preserves the latest generated-text
artifacts already reached by the workflow: preflight output, structured run
result, raw generated text, validated generated text, and render context.
When notification is attempted, the debug artifact records request identity,
including rendered pipeline ID, bundle paths, accepted upload fields,
distributor status fields, raw status report JSON when available, and redacted
failure context.
`--out` copies are never used as notification source files.
## Batch Workflow
`run morning` collects once, plans Today Report, Tomorrow Report, and eligible
future Daily Reports from the collected hourly forecast, then passes the same
collection into each report generation. `run evening` uses the same collection
and planning rules, but starts with Tomorrow Report. Future Daily reports start
with the day after tomorrow and require complete hourly forecast coverage for
the target local civil day. Dynamic Daily `--out-dir` copies use
`daily-YYYY-MM-DD.md`; other batch copies use report definition output names.
A collection failure stops the batch before planning or report generation.
After planning succeeds, batch generation continues independent reports after a
failure, records each result, writes compact status lines to stderr, emits a
JSON summary to stdout, and returns an aggregate error when any report failed.
Batch report generation suppresses per-report distributor notification. After
all planned reports finish, app orchestration evaluates batch notification:
1. If distributor notification is disabled, the batch notification result is
omitted.
2. If batch notification is disabled, the batch notification result is omitted
and there is no per-report fallback upload.
3. If any planned report failed, the batch notification result is `skipped`
with reason `one or more reports failed`, and distributor is not called.
4. If every report succeeded, app orchestration renders batch pipeline, bundle
ID, and idempotency key templates, renders report-specific distributor
paths for each included report, validates every managed source path and
bundle path, checks duplicate bundle paths across the batch, calls the
notifier once with a multi-file request, and saves a batch notification
debug artifact.
Batch notification failure records a top-level failed notification, increments
the aggregate batch failure count, and returns an aggregate batch error without
marking individual report items failed. `--out-dir` copies are never used as
notification source files.
## Inspection Workflow
Inspection workflows load existing filesystem state only. They do not fetch
weather data or invoke Scriptorium. Run-specific inspect commands share the same
store and metadata lookup path, then load the requested artifact or derived
inspection view.
## Failure Behavior
- Resolve errors stop the requested workflow before collection.
- Collection and module execution errors stop that report before Scriptorium
runs.
- Prompt input validation fails before render preflight.
- Render and run errors preserve Scriptorium stderr and exit-code context.
- Generated-text report errors preserve available intermediate artifacts and do
not create extra output copies.
- Single-report notification errors are wrapped with report ID, RunID, and
managed report path context. Detailed generation returns the inspectable
report, metadata, and notification artifact paths when finalization has
already saved them.
- Batch notification errors are recorded on the top-level batch notification
result and do not change individual report item status.
- Metadata and artifact path errors include filesystem context.
- Batch failures are recorded per report and surfaced through an aggregate
batch error.
## Tests
Inspect:
- `internal/app/app_test.go`
- `internal/app/batch_plan_test.go`
- `internal/collect/collect_test.go`
- `internal/cli/root_test.go`
- `internal/state/filesystem_test.go`
## Invariants
- Report behavior is resolved through `internal/report`.
- Generate and run commands collect once before report generation.
- Batch planning is app-owned because future Daily membership depends on
collected hourly forecast coverage.
- Generated reports use the same app request and result types regardless of
report ID.
- Render preflight precedes Scriptorium report generation.
- Generated-text reports render Markdown from a curated render context, not from
a raw data package.
- Recent Changes are computed from structured module snapshots.
- Metadata links artifacts produced for a run.
- Single-report distributor notification maps the managed Markdown report path
to configured bundle paths.
- Batch distributor notification maps each included managed Markdown report
path to bundle paths rendered for that report and uploads once for the
batch.
- Extra output copies are not upload sources.

100
docs/internal/cli.md
View File

@@ -1,67 +1,65 @@
# CLI Internals # CLI Internals
This document describes command output ownership in `internal/cli`. `internal/cli` turns process arguments into application requests and translates
application results into terminal output. The user-facing command, flag, and
output contract belongs in the [CLI reference](../cli.md).
## Purpose ## Responsibilities
`internal/cli` owns command parsing, app request construction, help text, and `Runner.Run` dispatches the top-level action or inspection request. For actions,
presentation of command results. It converts app-layer results into stable CLI the package parses command-specific and common flags, loads configuration with
summaries and writes stdout/stderr through shared output helpers. CLI overrides, obtains the current time, and constructs either an
`app.GenerateRequest` or an `app.BatchRequest`. It delegates generation and
batch execution to `internal/app`.
## Command Categories For inspection, it loads configuration, builds the appropriate app inspection
request, and writes the returned value. Inspection is read-only; the inspected
artifact types and user invocation remain owned by the [CLI reference](../cli.md)
and [operations guide](../operations.md).
- Action commands: `generate` and `run`. These perform work, write artifacts, ## Result Translation
and return compact summaries.
- Inspection commands: `inspect reports`, `inspect metadata`, `inspect
modules`, `inspect data-package`, `inspect prior`, and `inspect sources`.
These read existing artifacts and return requested data.
Future commands must declare which category they belong to before adding output Action results become CLI-safe JSON summaries in `result.go`. Generate summaries
behavior. carry report identity, status, relevant artifact paths, and notification
summary data. Batch summaries carry aggregate counts, per-report outcomes, and
the optional batch notification result. The translation deliberately excludes
full module snapshots, prompt packages, raw generated text, Scriptorium output,
and complete Distributor payloads.
## Stdout And Stderr When an action returns both a result and an error, the CLI writes the failed
summary before returning that error. Parse, configuration-load, and other
failures that produce no application result return without a summary.
Action commands write JSON summaries to stdout by default. `run` also writes `writeActionResult` writes action status information to stderr first, then JSON
compact status lines to stderr through `writeBatchStatus`. `generate` does not to stdout. Batch execution supplies the status writer; single-report generation
write routine stderr today. Pre-run errors return without partial JSON. does not emit routine stderr output. Quiet action requests suppress both normal
streams but still return errors. Inspection writes its JSON value to stdout and
does not accept quiet mode because stdout is the inspection result.
Inspection commands write requested JSON data to stdout with `writeJSON`. They ## Boundaries
do not use action output helpers and do not support quiet mode.
Returned errors are not hidden by output helpers. The caller remains The package owns argument parsing, request adaptation, help text, and terminal
responsible for displaying command errors. presentation. It does not implement report selection, collection, state
persistence, external transport, subprocess execution, or notification policy.
Those concerns remain in [application orchestration](app-orchestration.md) and
their focused owners.
## Quiet Mode ## Failure Behavior
`--quiet` is supported only by action commands. It suppresses successful stdout - Invalid command names, flags, dates, and configuration fail before an app
and routine stderr by passing `outputOptions{Quiet: true}` to request is executed.
`writeActionResult`. It does not suppress returned errors. - Application errors retain their application context; output helpers do not
hide or replace them.
- JSON-encoding errors are returned directly.
- A failed batch summary causes the CLI to return an aggregate batch error even
when the detailed batch call has already returned its result.
Quiet mode is intentionally not accepted by inspection commands because ## Tests And Invariants
inspection stdout is the command result.
## Summary Ownership Focused tests are in `internal/cli/root_test.go`, `internal/cli/output_test.go`,
and `internal/cli/result_test.go`.
CLI-safe summary structs live in `internal/cli/result.go`. - CLI summaries are stable, bounded views of app results.
- Routine batch status lines precede the batch JSON summary.
- `newGenerateSummary` converts `*app.ReportResult` plus an optional error into - A quiet action produces no successful or failure summary output.
the generate JSON contract. - Inspection never invokes action-output helpers.
- `newBatchSummary` converts `*app.BatchResult` into the run JSON contract and
derives the top-level run status.
Summary types must not expose full app internals, module contents, data package
contents, raw generated text, Scriptorium result bodies, or full distributor
payloads.
## Helper Path
New action commands should:
1. parse command-specific flags into CLI option structs;
2. call the app-layer use case;
3. convert app results into a CLI summary type;
4. write through `writeActionResult`;
5. use a status writer only for routine stderr status lines.
New inspection commands should call the app inspection use case and write the
returned data through `writeJSON`.

75
docs/internal/collect.md
View File

@@ -1,58 +1,43 @@
# Collection Internals # Collection Internals
This document describes the app-facing upstream collection boundary in `internal/collect` is the application-facing boundary for collecting the
`internal/collect`. normalized Weather API bundle. The external HTTP contract belongs in the
[Weather API integration guide](../integrations/weatherapi.md); normalized data
## Purpose semantics belong in [weather-data internals](weather-data.md).
`internal/collect` is the canonical package used by app workflows to collect
upstream Weather API data. It constructs the Weather API adapter, fetches a
normalized bundle, and returns that bundle without applying report selection or
batch policy.
## Contract ## Contract
Inputs: `Run` accepts a `context.Context` and a `Request` containing effective
`config.Config`. It constructs the Weather API adapter from that configuration,
calls `FetchBundle`, and returns `Result{Bundle: *weatherdata.Bundle}`.
- `collect.Request`, containing the effective `config.Config` The package wraps adapter construction failures as weather-collection setup
- `context.Context` for cancellation errors and fetch failures as bundle-collection errors. It does not retry,
persist, select reports, derive facts, build modules, invoke Scriptorium, or
notify Distributor.
Output: ## Application Composition
- `collect.Result`, containing `*weatherdata.Bundle` `internal/app` owns the narrow `Collector` interface used by workflow tests;
the production implementation delegates to `collect.Run`. Generation, batch
execution, and explicit bundle fetching all use this boundary. Application
orchestration rejects a nil collector result or a nil bundle before report work
can continue.
`Run` returns an actionable error when Weather API adapter construction or Single-report generation and a batch each collect once. A batch passes the same
bundle fetch fails. The package does not derive `facts.CollectedFacts`, build normalized collection to planning and to every report it generates. Collection
modules, resolve report periods, select reports, write state, invoke failure prevents later workflow work for that request.
Scriptorium, or notify distributor.
## App Usage ## Boundaries And Invariants
`internal/app` owns a narrow `Collector` interface for orchestration tests. The Collection owns adapter creation and retrieval of one normalized bundle. It
default implementation calls `collect.Run`. must not make report, period, batch, prompt, module, filesystem, or notification
decisions.
Single-report generation collects once, resolves the requested report, and - App-facing Weather API collection always passes through this package.
passes the explicit collection into report generation. Batch generation - The returned value is normalized source data, not facts or prompt input.
collects once before planning and passes the same collection into each planned - Context cancellation is passed to the Weather API adapter.
report. If collection returns no bundle, app orchestration returns an error - Errors retain whether setup or fetching failed.
before report generation.
## Boundaries Focused tests are in `internal/collect/collect_test.go`; orchestration use is
also covered by `internal/app/app_test.go`.
Weather API HTTP details stay in `internal/adapters/weatherapi`. The collection
package returns normalized `weatherdata` only. It must not know about report
IDs, prompt IDs, batch names, Daily eligibility, module composition, Recent
Changes, state paths, or Scriptorium arguments.
## Tests
Inspect:
- `internal/collect/collect_test.go`
- `internal/app/app_test.go`
## Invariants
- App-facing Weather API collection goes through `internal/collect`.
- Collection returns normalized source data, not report facts or prompt input.
- Report and batch policy belongs outside `internal/collect`.