229 lines
10 KiB
Markdown
229 lines
10 KiB
Markdown
# App Orchestration Internals
|
|
|
|
This document describes the workflow coordinator in `internal/app`.
|
|
|
|
## Purpose
|
|
|
|
`internal/app` coordinates the top-level use cases after CLI parsing and config
|
|
loading are complete. It resolves report definitions, collects weather data
|
|
through `internal/collect`, builds collected and derived facts, builds module
|
|
snapshots and prompt-input artifacts, invokes Scriptorium through the adapter
|
|
boundary, optionally notifies distributor through an app-owned notifier
|
|
boundary, persists managed state, runs batches, and reads existing artifacts
|
|
for inspection.
|
|
|
|
## Inputs And Outputs
|
|
|
|
Inputs:
|
|
|
|
- `GenerateRequest` for one report command
|
|
- `BatchRequest` for morning or evening batch commands
|
|
- `FetchBundleRequest` for explicit bundle collection and save workflows
|
|
- `ReportRequest` for single-report generation
|
|
- resolved report definitions from `internal/report`
|
|
- collection results from `internal/collect`
|
|
- prior snapshots loaded from `internal/state`
|
|
- optional collector, renderer, notifier, and state-store fakes for tests
|
|
|
|
Outputs:
|
|
|
|
- generated report results with JSON module snapshot, YAML data package,
|
|
preflight, report, metadata, prior snapshot, Recent Changes, Scriptorium
|
|
result details, generated-text artifact paths when applicable, and
|
|
notification result when attempted
|
|
- batch summaries with per-report status, artifact paths, error text, and
|
|
one top-level batch notification result when attempted or skipped
|
|
- saved Weather API bundle JSON for explicit bundle collection workflows
|
|
- inspection JSON values for reports, metadata, module snapshots, data
|
|
packages, prior snapshots, and source provenance
|
|
|
|
## Boundaries
|
|
|
|
`internal/app` owns workflow order and request composition. It does not parse
|
|
CLI flags, load YAML files directly, implement HTTP transport, own fact
|
|
derivation algorithms, define report periods, compare rendered Markdown, or
|
|
construct Scriptorium argv.
|
|
|
|
Report selection and report identity policy come from `internal/report`.
|
|
Collected and derived fact contracts come from `internal/facts`.
|
|
Weather API transport stays in `internal/adapters/weatherapi`, and app-facing
|
|
upstream collection stays in `internal/collect`. Scriptorium subprocess
|
|
behavior stays in `internal/adapters/scriptorium`. Distributor upload behavior
|
|
stays in `internal/adapters/distributor`. Filesystem layout and persisted
|
|
metadata stay in `internal/state`.
|
|
|
|
## Data Flow Terms
|
|
|
|
- `collect.Result` is the app-facing upstream collection result. It carries the
|
|
normalized `weatherdata.Bundle` used by report generation.
|
|
- `CollectedFacts` are normalized source facts derived from a collected Weather
|
|
API bundle and made available to derivation and module builders.
|
|
- `DerivedFacts` are deterministic calculations over collected facts, the
|
|
resolved valid period, daypart configuration, and report-specific windows.
|
|
- `module.Output` values are ordered deterministic stanzas built from collected
|
|
and derived facts for prompt input and inspection.
|
|
- `GeneratedText` is structured prose returned by Scriptorium for
|
|
generated-text-template reports and validated by `internal/generatedtext`.
|
|
- `RenderContext` is the typed template input built from report metadata,
|
|
module outputs, and validated generated text before Markdown rendering.
|
|
|
|
## Config Fields Used
|
|
|
|
- `weather_api.*` for Weather API client construction and module metadata
|
|
- `scriptorium.*` for renderer construction
|
|
- `workspace.*` for filesystem state
|
|
- `dayparts` for daily and outlook summarization
|
|
- `recent_change.*` for structured Recent Changes thresholds
|
|
- `notify.distributor.*` for optional single-report and batch notification
|
|
after report generation
|
|
|
|
Output copy flags are command request fields. They are not configuration
|
|
defaults.
|
|
|
|
## Generation Workflow
|
|
|
|
Single-report commands validate the report command, collect once through
|
|
`internal/collect`, resolve the requested report, and pass the resolved report
|
|
plus explicit collection into `GenerateReport`.
|
|
|
|
`GenerateReport` then uses this setup:
|
|
|
|
1. Create or use a filesystem store.
|
|
2. Locate any prior compatible snapshot through `internal/state`.
|
|
3. Build collected and derived facts from the supplied collection.
|
|
4. Execute configured modules and save the module snapshot.
|
|
5. Compute Recent Changes from structured prior and current module snapshots.
|
|
6. Build and save the YAML Scriptorium `data_package`.
|
|
7. Run Scriptorium render preflight.
|
|
8. Save preflight JSON when a render result is available.
|
|
9. Save metadata for inspection.
|
|
|
|
For `scriptorium_markdown` reports, generation then:
|
|
|
|
10. Runs Scriptorium report generation to the managed report path.
|
|
|
|
For `generated_text_template` reports, generation then:
|
|
|
|
10. Looks up the generated-text catalog entry for the report schema/template
|
|
IDs.
|
|
11. Runs structured Scriptorium generation to the raw generated-text JSON path.
|
|
12. Saves the structured Scriptorium run result.
|
|
13. Validates and saves normalized generated text.
|
|
14. Builds and saves a typed render context.
|
|
15. Renders Markdown from the embedded template to the managed report path.
|
|
|
|
After either mode has produced a managed Markdown report, shared finalization:
|
|
|
|
1. Copies the managed report to the requested `--out` or `--out-dir` path when
|
|
provided.
|
|
2. Saves final metadata with the managed report path and any generated-text
|
|
artifact paths already produced.
|
|
3. If distributor notification is enabled, notifies using the managed report
|
|
path as the source file.
|
|
4. Saves a distributor notification debug artifact and updates metadata with
|
|
its path.
|
|
|
|
If render preflight returns both a result and an error, preflight JSON and
|
|
metadata are persisted before the error is returned. If Scriptorium report
|
|
generation returns an error after writing output, the managed report and
|
|
metadata remain inspectable. Notification is not attempted after collection,
|
|
module snapshot, prompt input, render, Scriptorium run, or metadata-save
|
|
failures.
|
|
Generated-text report failures are returned with report ID, RunID, and the
|
|
failed operation. When available, the app preserves the latest generated-text
|
|
artifacts already reached by the workflow: preflight output, structured run
|
|
result, raw generated text, validated generated text, and render context.
|
|
When notification is attempted, the debug artifact records request identity,
|
|
including rendered pipeline ID, bundle paths, accepted upload fields,
|
|
distributor status fields, raw status report JSON when available, and redacted
|
|
failure context.
|
|
`--out` copies are never used as notification source files.
|
|
|
|
## Batch Workflow
|
|
|
|
`run morning` collects once, plans Today Report, Tomorrow Report, and eligible
|
|
future Daily Reports from the collected hourly forecast, then passes the same
|
|
collection into each report generation. `run evening` uses the same collection
|
|
and planning rules, but starts with Tomorrow Report. Future Daily reports start
|
|
with the day after tomorrow and require complete hourly forecast coverage for
|
|
the target local civil day. Dynamic Daily `--out-dir` copies use
|
|
`daily-YYYY-MM-DD.md`; other batch copies use report definition output names.
|
|
A collection failure stops the batch before planning or report generation.
|
|
After planning succeeds, batch generation continues independent reports after a
|
|
failure, records each result, writes compact status lines to stderr, emits a
|
|
JSON summary to stdout, and returns an aggregate error when any report failed.
|
|
|
|
Batch report generation suppresses per-report distributor notification. After
|
|
all planned reports finish, app orchestration evaluates batch notification:
|
|
|
|
1. If distributor notification is disabled, the batch notification result is
|
|
omitted.
|
|
2. If batch notification is disabled, the batch notification result is omitted
|
|
and there is no per-report fallback upload.
|
|
3. If any planned report failed, the batch notification result is `skipped`
|
|
with reason `one or more reports failed`, and distributor is not called.
|
|
4. If every report succeeded, app orchestration renders batch pipeline, bundle
|
|
ID, and idempotency key templates, renders `report_path_templates` for each
|
|
included report, validates every managed source path and bundle path, checks
|
|
duplicate bundle paths across the batch, calls the notifier once with a
|
|
multi-file request, and saves a batch notification debug artifact.
|
|
|
|
Batch notification failure records a top-level failed notification, increments
|
|
the aggregate batch failure count, and returns an aggregate batch error without
|
|
marking individual report items failed. `--out-dir` copies are never used as
|
|
notification source files.
|
|
|
|
## Inspection Workflow
|
|
|
|
Inspection workflows load existing filesystem state only. They do not fetch
|
|
weather data or invoke Scriptorium. Run-specific inspect commands share the same
|
|
store and metadata lookup path, then load the requested artifact or derived
|
|
inspection view.
|
|
|
|
## Failure Behavior
|
|
|
|
- Resolve errors stop the requested workflow before collection.
|
|
- Collection and module execution errors stop that report before Scriptorium
|
|
runs.
|
|
- Prompt input validation fails before render preflight.
|
|
- Render and run errors preserve Scriptorium stderr and exit-code context.
|
|
- Generated-text report errors preserve available intermediate artifacts and do
|
|
not create extra output copies.
|
|
- Single-report notification errors are wrapped with report ID, RunID, and
|
|
managed report path context.
|
|
- Batch notification errors are recorded on the top-level batch notification
|
|
result and do not change individual report item status.
|
|
- Metadata and artifact path errors include filesystem context.
|
|
- Batch failures are recorded per report and surfaced through an aggregate
|
|
batch error.
|
|
|
|
## Tests
|
|
|
|
Inspect:
|
|
|
|
- `internal/app/app_test.go`
|
|
- `internal/app/batch_plan_test.go`
|
|
- `internal/collect/collect_test.go`
|
|
- `internal/cli/root_test.go`
|
|
- `internal/state/filesystem_test.go`
|
|
|
|
## Invariants
|
|
|
|
- Report behavior is resolved through `internal/report`.
|
|
- Generate and run commands collect once before report generation.
|
|
- Batch planning is app-owned because future Daily membership depends on
|
|
collected hourly forecast coverage.
|
|
- Generated reports use the same app request and result types regardless of
|
|
report ID.
|
|
- Render preflight precedes Scriptorium report generation.
|
|
- Generated-text reports render Markdown from a curated render context, not from
|
|
a raw data package.
|
|
- Recent Changes are computed from structured module snapshots.
|
|
- Metadata links artifacts produced for a run.
|
|
- Single-report distributor notification maps the managed Markdown report path
|
|
to configured bundle paths.
|
|
- Batch distributor notification maps each included managed Markdown report
|
|
path to bundle paths rendered for that report and uploads once for the
|
|
batch.
|
|
- Extra output copies are not upload sources.
|