Refresh documentation for current collection behavior
This commit is contained in:
@@ -9,7 +9,7 @@ batches, and inspects stored artifacts.
|
|||||||
weatherreporter generate today --out ./today.md
|
weatherreporter generate today --out ./today.md
|
||||||
```
|
```
|
||||||
|
|
||||||
This loads configuration, fetches weather data, writes managed workspace
|
This loads configuration, collects weather data, writes managed workspace
|
||||||
artifacts, runs `scriptorium render` as a preflight check, runs structured
|
artifacts, runs `scriptorium render` as a preflight check, runs structured
|
||||||
`scriptorium run`, validates generated text, renders the embedded Today
|
`scriptorium run`, validates generated text, renders the embedded Today
|
||||||
template, and writes an extra Markdown copy to `./today.md`. If distributor
|
template, and writes an extra Markdown copy to `./today.md`. If distributor
|
||||||
@@ -70,7 +70,7 @@ Hourly Report, 3-Day Outlook, and Weekend Outlook are explicit only; they are
|
|||||||
not included in `run morning` or `run evening`.
|
not included in `run morning` or `run evening`.
|
||||||
|
|
||||||
`inspect` commands read existing workspace artifacts and emit JSON to stdout.
|
`inspect` commands read existing workspace artifacts and emit JSON to stdout.
|
||||||
They do not fetch weather data or invoke `scriptorium`.
|
They do not collect weather data or invoke `scriptorium`.
|
||||||
|
|
||||||
## Flags
|
## Flags
|
||||||
|
|
||||||
|
|||||||
@@ -29,8 +29,8 @@ weather_api:
|
|||||||
base_url: https://weather.api.example.com/
|
base_url: https://weather.api.example.com/
|
||||||
```
|
```
|
||||||
|
|
||||||
`weather_api.base_url` is required for commands that fetch weather data. Other
|
`weather_api.base_url` is required for commands that collect weather data.
|
||||||
fields fall back to defaults.
|
Other fields fall back to defaults.
|
||||||
|
|
||||||
## Production-Oriented Config
|
## Production-Oriented Config
|
||||||
|
|
||||||
@@ -41,7 +41,7 @@ config test suite.
|
|||||||
|
|
||||||
### `weather_api`
|
### `weather_api`
|
||||||
|
|
||||||
- `base_url`: absolute base URL for the Weather API. Required for generation and fetch workflows.
|
- `base_url`: absolute base URL for the Weather API. Required for generation and collection workflows.
|
||||||
- `timeout`: HTTP timeout duration. Default: `10s`.
|
- `timeout`: HTTP timeout duration. Default: `10s`.
|
||||||
- `precision`: numeric precision query value. Default: `1`.
|
- `precision`: numeric precision query value. Default: `1`.
|
||||||
- `units`: Weather API units query value. Default: `us`.
|
- `units`: Weather API units query value. Default: `us`.
|
||||||
|
|||||||
@@ -18,7 +18,7 @@ Inputs:
|
|||||||
|
|
||||||
- `GenerateRequest` for one report command
|
- `GenerateRequest` for one report command
|
||||||
- `BatchRequest` for morning or evening batch commands
|
- `BatchRequest` for morning or evening batch commands
|
||||||
- `FetchBundleRequest` for explicit bundle fetch and save workflows
|
- `FetchBundleRequest` for explicit bundle collection and save workflows
|
||||||
- `ReportRequest` for single-report generation
|
- `ReportRequest` for single-report generation
|
||||||
- resolved report definitions from `internal/report`
|
- resolved report definitions from `internal/report`
|
||||||
- collection results from `internal/collect`
|
- collection results from `internal/collect`
|
||||||
@@ -33,7 +33,7 @@ Outputs:
|
|||||||
notification result when attempted
|
notification result when attempted
|
||||||
- batch summaries with per-report status, artifact paths, error text, and
|
- batch summaries with per-report status, artifact paths, error text, and
|
||||||
notification outcome when attempted
|
notification outcome when attempted
|
||||||
- saved Weather API bundle JSON for fetch workflows
|
- saved Weather API bundle JSON for explicit bundle collection workflows
|
||||||
- inspection JSON values for reports, metadata, module snapshots, data
|
- inspection JSON values for reports, metadata, module snapshots, data
|
||||||
packages, prior snapshots, and source provenance
|
packages, prior snapshots, and source provenance
|
||||||
|
|
||||||
|
|||||||
@@ -77,7 +77,7 @@ remain in report metadata, source provenance, and integration artifacts.
|
|||||||
- This package selects and shapes already-collected weather facts for prompts.
|
- This package selects and shapes already-collected weather facts for prompts.
|
||||||
- It validates module composition against report compatibility and option
|
- It validates module composition against report compatibility and option
|
||||||
types.
|
types.
|
||||||
- It does not fetch weather data, compare prior snapshots, write module
|
- It does not collect weather data, compare prior snapshots, write module
|
||||||
snapshots, build YAML data packages, invoke Scriptorium, or write workflow
|
snapshots, build YAML data packages, invoke Scriptorium, or write workflow
|
||||||
metadata.
|
metadata.
|
||||||
|
|
||||||
|
|||||||
@@ -250,7 +250,7 @@ meet the severity threshold.
|
|||||||
|
|
||||||
- This package owns module identifiers, config item envelopes, output
|
- This package owns module identifiers, config item envelopes, output
|
||||||
envelopes, snapshot validation, and typed stanza lookup.
|
envelopes, snapshot validation, and typed stanza lookup.
|
||||||
- It does not define report IDs, execute builders, fetch weather data, derive
|
- It does not define report IDs, execute builders, collect weather data, derive
|
||||||
forecast facts, write state, or invoke Scriptorium.
|
forecast facts, write state, or invoke Scriptorium.
|
||||||
|
|
||||||
## State Or Manifest Behavior
|
## State Or Manifest Behavior
|
||||||
|
|||||||
@@ -128,7 +128,7 @@ Current categories are:
|
|||||||
|
|
||||||
- This package owns prompt package schema, YAML marshaling, YAML loading, and
|
- This package owns prompt package schema, YAML marshaling, YAML loading, and
|
||||||
validation.
|
validation.
|
||||||
- It does not fetch weather data, derive forecast summaries, execute modules,
|
- It does not collect weather data, derive forecast summaries, execute modules,
|
||||||
choose module prompt export shapes, find prior snapshots, compare changes,
|
choose module prompt export shapes, find prior snapshots, compare changes,
|
||||||
choose artifact paths, or invoke Scriptorium.
|
choose artifact paths, or invoke Scriptorium.
|
||||||
|
|
||||||
|
|||||||
@@ -78,7 +78,7 @@ must be after start time.
|
|||||||
## Boundaries
|
## Boundaries
|
||||||
|
|
||||||
`internal/report` defines report metadata, public report names, batch command
|
`internal/report` defines report metadata, public report names, batch command
|
||||||
names, output naming, and time coverage. It does not fetch weather data, plan
|
names, output naming, and time coverage. It does not collect weather data, plan
|
||||||
batch membership, build module values, compare snapshot contents, write state,
|
batch membership, build module values, compare snapshot contents, write state,
|
||||||
parse CLI flags, or invoke Scriptorium.
|
parse CLI flags, or invoke Scriptorium.
|
||||||
|
|
||||||
|
|||||||
@@ -43,7 +43,7 @@ Generated-text prompt sources are maintained under
|
|||||||
## Boundaries
|
## Boundaries
|
||||||
|
|
||||||
This package owns embedded asset lookup, Go template parsing, and Markdown
|
This package owns embedded asset lookup, Go template parsing, and Markdown
|
||||||
template execution. It does not fetch weather data, build module outputs,
|
template execution. It does not collect weather data, build module outputs,
|
||||||
validate GeneratedText, construct render contexts, choose report definitions,
|
validate GeneratedText, construct render contexts, choose report definitions,
|
||||||
write artifacts, invoke Scriptorium, or notify distributor.
|
write artifacts, invoke Scriptorium, or notify distributor.
|
||||||
|
|
||||||
|
|||||||
@@ -34,7 +34,7 @@ Outputs:
|
|||||||
|
|
||||||
`internal/adapters/scriptorium` owns Scriptorium command construction and
|
`internal/adapters/scriptorium` owns Scriptorium command construction and
|
||||||
subprocess execution. It does not choose report types, build prompt input,
|
subprocess execution. It does not choose report types, build prompt input,
|
||||||
fetch weather data, decide workflow order, or persist workflow metadata.
|
collect weather data, decide workflow order, or persist workflow metadata.
|
||||||
|
|
||||||
The adapter exposes request and result structs for render, Markdown run, and
|
The adapter exposes request and result structs for render, Markdown run, and
|
||||||
structured generated-text run operations. State persistence uses state-owned
|
structured generated-text run operations. State persistence uses state-owned
|
||||||
|
|||||||
@@ -254,8 +254,8 @@ and the default output filename value is `daily.md`.
|
|||||||
For Today, the default report ID and artifact group values are both `today`,
|
For Today, the default report ID and artifact group values are both `today`,
|
||||||
and the batch output filename value is `today.md`.
|
and the batch output filename value is `today.md`.
|
||||||
|
|
||||||
Notification happens after final metadata save for generated reports. Weather
|
Notification happens after final metadata save for generated reports.
|
||||||
API, module snapshot, data-package, render preflight, Scriptorium run,
|
Collection, module snapshot, data-package, render preflight, Scriptorium run,
|
||||||
generated-text validation, template rendering, and metadata-save failures do
|
generated-text validation, template rendering, and metadata-save failures do
|
||||||
not trigger notification. A notification failure fails that report.
|
not trigger notification. A notification failure fails that report.
|
||||||
In a batch, other reports continue, the failed report includes notification
|
In a batch, other reports continue, the failed report includes notification
|
||||||
@@ -281,7 +281,7 @@ bundle, so existing uploaded dated report paths can remain available.
|
|||||||
## Inspection
|
## Inspection
|
||||||
|
|
||||||
Inspection commands read existing workspace artifacts and emit JSON to stdout.
|
Inspection commands read existing workspace artifacts and emit JSON to stdout.
|
||||||
They do not fetch weather data or run `scriptorium`.
|
They do not collect weather data or run `scriptorium`.
|
||||||
|
|
||||||
```text
|
```text
|
||||||
weatherreporter inspect reports --limit 10
|
weatherreporter inspect reports --limit 10
|
||||||
|
|||||||
@@ -1,248 +0,0 @@
|
|||||||
# Batch Collection Roadmap
|
|
||||||
|
|
||||||
## Purpose
|
|
||||||
|
|
||||||
This roadmap defines the planned change to make `weatherreporter run morning`
|
|
||||||
and `weatherreporter run evening` use one canonical upstream collection path
|
|
||||||
and data-aware batch planning.
|
|
||||||
|
|
||||||
The feature has two related goals:
|
|
||||||
|
|
||||||
- move upstream Weather API collection into a single `internal/collect` package
|
|
||||||
used by all generation workflows;
|
|
||||||
- update scheduled batches so they generate Today, Tomorrow, and future Daily
|
|
||||||
reports according to available full-day hourly forecast coverage.
|
|
||||||
|
|
||||||
This document lives under `docs/roadmap/` because the behavior described here
|
|
||||||
is not yet implemented.
|
|
||||||
|
|
||||||
## User Intent
|
|
||||||
|
|
||||||
Batch commands should produce a practical publication set while avoiding
|
|
||||||
partial Daily reports.
|
|
||||||
|
|
||||||
Morning publication should generate:
|
|
||||||
|
|
||||||
- Today report for the current civil day;
|
|
||||||
- Tomorrow report for the next civil day;
|
|
||||||
- one Daily report for each later future civil day where the upstream hourly
|
|
||||||
forecast fully covers the entire target day.
|
|
||||||
|
|
||||||
Evening publication should generate:
|
|
||||||
|
|
||||||
- Tomorrow report for the next civil day;
|
|
||||||
- one Daily report for each later future civil day where the upstream hourly
|
|
||||||
forecast fully covers the entire target day.
|
|
||||||
|
|
||||||
Neither batch should generate Daily reports for partially covered days. That
|
|
||||||
keeps the Daily report format simple and avoids requiring templates or modules
|
|
||||||
to explain incomplete forecast coverage.
|
|
||||||
|
|
||||||
The morning batch should no longer include the legacy 3-Day Outlook or Weekend
|
|
||||||
Outlook. Those reports remain individually generated report types unless
|
|
||||||
separately removed.
|
|
||||||
|
|
||||||
## Locked Decisions
|
|
||||||
|
|
||||||
- Add a new canonical upstream collection package: `internal/collect`.
|
|
||||||
- Keep Weather API HTTP details in `internal/adapters/weatherapi`; `collect`
|
|
||||||
orchestrates collection and returns normalized upstream data.
|
|
||||||
- Use `internal/collect` for all report generation workflows, including
|
|
||||||
`generate <report>`, `run morning`, and `run evening`.
|
|
||||||
- Fetch upstream weather data once per command invocation.
|
|
||||||
- Pass the collected result into report generation; report generation should
|
|
||||||
not fetch directly from Weather API.
|
|
||||||
- Keep batch planning in `internal/app`; do not make `internal/collect` aware of
|
|
||||||
morning, evening, report IDs, or batch membership.
|
|
||||||
- Centralize batch-specific app planning in a single file, expected to be
|
|
||||||
`internal/app/batch_plan.go`, unless implementation shows a clearer local
|
|
||||||
name.
|
|
||||||
- `run morning` should generate `today`, `tomorrow`, then future `daily`
|
|
||||||
reports in ascending local date order.
|
|
||||||
- `run evening` should generate `tomorrow`, then future `daily` reports in
|
|
||||||
ascending local date order.
|
|
||||||
- Future Daily eligibility is based on full hourly forecast coverage, not
|
|
||||||
narrative forecast availability.
|
|
||||||
- Daily expansion starts with the day after tomorrow for both morning and
|
|
||||||
evening batches.
|
|
||||||
- Remove `three-day` and `weekend` from `run morning`.
|
|
||||||
- Dynamic Daily reports in a batch should use date-qualified `--out-dir` copy
|
|
||||||
names, such as `daily-YYYY-MM-DD.md`.
|
|
||||||
- Keep public CLI syntax unchanged: `weatherreporter run morning` and
|
|
||||||
`weatherreporter run evening` remain the commands.
|
|
||||||
|
|
||||||
## Current Repository Shape
|
|
||||||
|
|
||||||
Current code resolves batches statically:
|
|
||||||
|
|
||||||
- `internal/cli/root.go` parses `run morning` or `run evening` and builds
|
|
||||||
`app.BatchRequest`.
|
|
||||||
- `internal/app.RunBatchDetailed` calls `ResolveBatch`.
|
|
||||||
- `internal/app.ResolveBatch` delegates to `report.Registry.BatchReports`.
|
|
||||||
- `internal/report.Registry.BatchReports` currently resolves the morning batch
|
|
||||||
to Today, 3-Day, and Weekend except on Sunday.
|
|
||||||
- `internal/report.Registry.BatchReports` currently resolves the evening batch
|
|
||||||
to Tomorrow.
|
|
||||||
- `internal/app.GenerateReport` fetches the Weather API bundle for each report
|
|
||||||
through `FetchBundle`.
|
|
||||||
- `FetchBundle` directly constructs the Weather API adapter and calls
|
|
||||||
`FetchBundle`.
|
|
||||||
- Daily report resolution already supports explicit target dates through
|
|
||||||
`report.ResolveRequest.Date`.
|
|
||||||
|
|
||||||
This means current batch resolution cannot decide future Daily membership from
|
|
||||||
available upstream data without either fetching during planning or attempting
|
|
||||||
report generation for candidate dates. It also means multiple reports in one
|
|
||||||
batch can fetch different upstream snapshots.
|
|
||||||
|
|
||||||
## Target Architecture
|
|
||||||
|
|
||||||
The target command flow is:
|
|
||||||
|
|
||||||
1. CLI parses the command and loads config.
|
|
||||||
2. App orchestration calls `internal/collect` once.
|
|
||||||
3. `internal/collect` calls the Weather API adapter and returns a collection
|
|
||||||
result containing the normalized `weatherdata.Bundle`.
|
|
||||||
4. App batch planning uses the collected bundle plus report registry metadata
|
|
||||||
to resolve the requested reports.
|
|
||||||
5. App report generation receives the same collected bundle for every report in
|
|
||||||
the command.
|
|
||||||
6. Facts, modules, prompt input, generated text, templates, state, and
|
|
||||||
notifications operate as they do today, but consume already-collected data.
|
|
||||||
|
|
||||||
The intended dependency direction is:
|
|
||||||
|
|
||||||
```text
|
|
||||||
internal/adapters/weatherapi -> internal/collect -> internal/app -> internal/facts -> internal/briefing/internal modules
|
|
||||||
```
|
|
||||||
|
|
||||||
Report definitions still live in `internal/report`. Report definitions should
|
|
||||||
not know how upstream data is fetched or how batch membership is expanded from
|
|
||||||
available data.
|
|
||||||
|
|
||||||
## `internal/collect` Contract
|
|
||||||
|
|
||||||
The initial package should be intentionally narrow. A suitable first contract
|
|
||||||
is:
|
|
||||||
|
|
||||||
```go
|
|
||||||
package collect
|
|
||||||
|
|
||||||
type Request struct {
|
|
||||||
Config config.Config
|
|
||||||
}
|
|
||||||
|
|
||||||
type Result struct {
|
|
||||||
Bundle *weatherdata.Bundle
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
Expected behavior:
|
|
||||||
|
|
||||||
- `collect.Run(ctx, Request)` constructs and uses the Weather API adapter.
|
|
||||||
- It returns the normalized Weather API bundle.
|
|
||||||
- It wraps collection errors with operation context.
|
|
||||||
- It does not derive report facts.
|
|
||||||
- It does not resolve reports or batches.
|
|
||||||
- It does not write state artifacts.
|
|
||||||
- It does not invoke Scriptorium or distributor.
|
|
||||||
|
|
||||||
Future source families, such as radar, observations history, or additional
|
|
||||||
review/report inputs, should be added to this package as upstream collection
|
|
||||||
responsibilities. The package should remain source collection, not report
|
|
||||||
policy.
|
|
||||||
|
|
||||||
## Batch Target Behavior
|
|
||||||
|
|
||||||
### Morning
|
|
||||||
|
|
||||||
`run morning` should resolve reports in this order:
|
|
||||||
|
|
||||||
1. `today`
|
|
||||||
2. `tomorrow`
|
|
||||||
3. `daily` for each eligible future date beginning with the day after tomorrow
|
|
||||||
|
|
||||||
### Evening
|
|
||||||
|
|
||||||
`run evening` should resolve reports in this order:
|
|
||||||
|
|
||||||
1. `tomorrow`
|
|
||||||
2. `daily` for each eligible future date beginning with the day after tomorrow
|
|
||||||
|
|
||||||
### Future Daily Eligibility
|
|
||||||
|
|
||||||
Eligibility for each future Daily report:
|
|
||||||
|
|
||||||
- compute the target civil day in `weather_api.timezone`;
|
|
||||||
- examine the collected hourly forecast periods;
|
|
||||||
- require hourly periods whose local `startTime` values cover every required
|
|
||||||
hourly start inside the civil day;
|
|
||||||
- for ordinary days, required starts are `00:00` through `23:00` local time;
|
|
||||||
- on daylight-saving transitions, generate required starts by stepping through
|
|
||||||
the actual local civil day from `day.Start` to `day.End`, so 23-hour and
|
|
||||||
25-hour days are handled consistently;
|
|
||||||
- require each selected period to have a valid start and end;
|
|
||||||
- stop scanning once no later complete civil day can be found within the
|
|
||||||
available hourly forecast range.
|
|
||||||
|
|
||||||
The planner should use exact local hourly start matching for eligibility, not
|
|
||||||
mere overlap. This keeps the contract clear: a day is eligible only when the
|
|
||||||
hourly source contains a full set of hourly rows for that day.
|
|
||||||
|
|
||||||
## Report Generation Contract
|
|
||||||
|
|
||||||
After the collection refactor, `GenerateReport` should no longer fetch upstream
|
|
||||||
data itself. The preferred final shape is:
|
|
||||||
|
|
||||||
- `Generate` collects once, resolves the single report, then calls
|
|
||||||
`GenerateReport` with the collected result.
|
|
||||||
- `RunBatchDetailed` collects once, plans the batch, then calls
|
|
||||||
`GenerateReport` for each resolved report with the same collected result.
|
|
||||||
- `GenerateReport` requires collected data and returns an actionable error if
|
|
||||||
called without it.
|
|
||||||
|
|
||||||
A transitional optional collected-data field may be used during implementation,
|
|
||||||
but the final state should have one explicit collection path and no hidden
|
|
||||||
Weather API fetch inside report generation.
|
|
||||||
|
|
||||||
## Output And State Policy
|
|
||||||
|
|
||||||
Existing managed workspace artifact paths should remain stable. Multiple Daily
|
|
||||||
reports can appear in one batch, so `--out-dir` copies must not use the same
|
|
||||||
`daily.md` filename for every dynamic Daily report. Dynamic Daily batch copies
|
|
||||||
should use `daily-YYYY-MM-DD.md`. Today and Tomorrow should keep their existing
|
|
||||||
batch copy names.
|
|
||||||
|
|
||||||
Distributor upload paths and managed report paths should continue to be derived
|
|
||||||
from report definitions and resolved report metadata, not from optional
|
|
||||||
`--out-dir` copies.
|
|
||||||
|
|
||||||
## Implementation Planning
|
|
||||||
|
|
||||||
The staged implementation plan belongs in `docs/roadmap/implementation.md`.
|
|
||||||
That document should be treated as the executable plan for coding agents.
|
|
||||||
|
|
||||||
After implementation, update implemented docs only:
|
|
||||||
|
|
||||||
- `docs/cli.md`
|
|
||||||
- `docs/operations.md`
|
|
||||||
- `docs/internal/app-orchestration.md`
|
|
||||||
- `docs/internal/report-registry.md`
|
|
||||||
- relevant troubleshooting entries if new failure modes are surfaced
|
|
||||||
|
|
||||||
Do not document future collection sources outside roadmap docs.
|
|
||||||
|
|
||||||
## Refactors To Avoid
|
|
||||||
|
|
||||||
- Do not create a generic workflow engine.
|
|
||||||
- Do not move report definitions out of `internal/report`.
|
|
||||||
- Do not make `internal/collect` aware of report IDs, prompt IDs, or batch
|
|
||||||
names.
|
|
||||||
- Do not make modules fetch upstream data.
|
|
||||||
- Do not add config knobs for partial Daily coverage in this pass.
|
|
||||||
- Do not remove 3-Day or Weekend report definitions unless separately planned.
|
|
||||||
- Do not change public CLI syntax for `run morning` or `run evening`.
|
|
||||||
|
|
||||||
## Open Questions
|
|
||||||
|
|
||||||
None block the roadmap.
|
|
||||||
@@ -73,8 +73,8 @@ represented in `CollectedFacts`.
|
|||||||
|
|
||||||
Future module work should preserve these boundaries:
|
Future module work should preserve these boundaries:
|
||||||
|
|
||||||
- collect upstream facts once per report run
|
- keep upstream collection in app orchestration
|
||||||
- keep upstream fetching out of modules
|
- keep upstream collection out of modules
|
||||||
- keep broad reusable calculations in `DerivedFacts`
|
- keep broad reusable calculations in `DerivedFacts`
|
||||||
- keep prompt-facing field shape inside module builders
|
- keep prompt-facing field shape inside module builders
|
||||||
- use typed options for configurable module behavior
|
- use typed options for configurable module behavior
|
||||||
|
|||||||
@@ -1,443 +0,0 @@
|
|||||||
# Batch Collection Implementation Roadmap
|
|
||||||
|
|
||||||
## Purpose
|
|
||||||
|
|
||||||
This is the staged implementation plan for `docs/roadmap/batch.md`. Implement
|
|
||||||
the stages in order. The feature has two goals:
|
|
||||||
|
|
||||||
- route all upstream Weather API collection through a canonical
|
|
||||||
`internal/collect` package;
|
|
||||||
- update `run morning` and `run evening` so they use one collected snapshot and
|
|
||||||
dynamically add future Daily reports only for dates with complete hourly
|
|
||||||
coverage.
|
|
||||||
|
|
||||||
This is an implementation roadmap only. Background, user intent, and target
|
|
||||||
policy live in `docs/roadmap/batch.md`.
|
|
||||||
|
|
||||||
## Global Constraints
|
|
||||||
|
|
||||||
- Preserve public CLI syntax.
|
|
||||||
- Preserve managed workspace artifact paths.
|
|
||||||
- Preserve distributor upload behavior and paths.
|
|
||||||
- Keep Weather API HTTP details inside `internal/adapters/weatherapi`.
|
|
||||||
- Keep source collection in `internal/collect`.
|
|
||||||
- Keep batch planning in `internal/app`.
|
|
||||||
- Keep report definitions in `internal/report`.
|
|
||||||
- Do not make `internal/collect` aware of report IDs, prompt IDs, or batch
|
|
||||||
names.
|
|
||||||
- Do not add config fields for partial Daily coverage.
|
|
||||||
- Do not create a workflow engine, scheduler package, or plugin system.
|
|
||||||
- Do not remove 3-Day or Weekend report definitions in this change.
|
|
||||||
|
|
||||||
## Stage 1: Add Canonical Collection Package
|
|
||||||
|
|
||||||
Goal: introduce `internal/collect` as the only app-facing upstream collection
|
|
||||||
API while preserving current behavior.
|
|
||||||
|
|
||||||
Implementation:
|
|
||||||
|
|
||||||
- Add `internal/collect`.
|
|
||||||
- Define:
|
|
||||||
|
|
||||||
```go
|
|
||||||
type Request struct {
|
|
||||||
Config config.Config
|
|
||||||
}
|
|
||||||
|
|
||||||
type Result struct {
|
|
||||||
Bundle *weatherdata.Bundle
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
- Add `Run(ctx context.Context, req Request) (*Result, error)`.
|
|
||||||
- `Run` should construct the Weather API adapter with `weatherapi.New` and call
|
|
||||||
`client.FetchBundle(ctx)`.
|
|
||||||
- Return an actionable error when adapter construction or fetch fails.
|
|
||||||
- Keep the returned data normalized as `*weatherdata.Bundle`; do not derive
|
|
||||||
`facts.CollectedFacts` or `facts.DerivedFacts` in this package.
|
|
||||||
- Update `internal/app.FetchBundle` and `FetchAndSaveBundle`, if retained, to
|
|
||||||
call `collect.Run` rather than constructing `weatherapi.New` directly.
|
|
||||||
|
|
||||||
Tests:
|
|
||||||
|
|
||||||
- Add `internal/collect` tests using a local Weather API test server or a small
|
|
||||||
adapter seam if needed.
|
|
||||||
- Keep existing `FetchAndSaveBundle` app tests passing.
|
|
||||||
- Ensure collection errors include collection/fetch context without leaking
|
|
||||||
secrets.
|
|
||||||
|
|
||||||
Acceptance:
|
|
||||||
|
|
||||||
- `internal/collect` owns app-facing Weather API collection.
|
|
||||||
- `internal/app.FetchBundle` no longer constructs `weatherapi.New` directly.
|
|
||||||
- No report or batch behavior changes yet.
|
|
||||||
|
|
||||||
Validation:
|
|
||||||
|
|
||||||
```sh
|
|
||||||
go test ./internal/collect ./internal/app ./internal/adapters/weatherapi
|
|
||||||
git diff --check
|
|
||||||
```
|
|
||||||
|
|
||||||
## Stage 2: Add App Collection Seam
|
|
||||||
|
|
||||||
Goal: make collection testable from app orchestration without requiring live
|
|
||||||
Weather API calls.
|
|
||||||
|
|
||||||
Implementation:
|
|
||||||
|
|
||||||
- Add a narrow app-owned collector interface, for example:
|
|
||||||
|
|
||||||
```go
|
|
||||||
type Collector interface {
|
|
||||||
Run(context.Context, collect.Request) (*collect.Result, error)
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
- Add a default adapter that calls `collect.Run`.
|
|
||||||
- Add `Collector` fields to `GenerateRequest` and `BatchRequest`.
|
|
||||||
- Default to the real collector when the request does not provide one.
|
|
||||||
- Do not expose collector options through CLI.
|
|
||||||
|
|
||||||
Tests:
|
|
||||||
|
|
||||||
- Add app tests proving `Generate` uses the provided collector.
|
|
||||||
- Add app tests proving collection failure stops generation before report
|
|
||||||
execution.
|
|
||||||
- Add batch tests proving `RunBatchDetailed` can use a fake collector in later
|
|
||||||
stages.
|
|
||||||
|
|
||||||
Acceptance:
|
|
||||||
|
|
||||||
- App orchestration can be tested with a fake collector.
|
|
||||||
- CLI behavior is unchanged.
|
|
||||||
- External adapter types do not leak into app request or result structs.
|
|
||||||
|
|
||||||
Validation:
|
|
||||||
|
|
||||||
```sh
|
|
||||||
go test ./internal/app ./internal/cli
|
|
||||||
git diff --check
|
|
||||||
```
|
|
||||||
|
|
||||||
## Stage 3: Make Report Generation Require Collected Data
|
|
||||||
|
|
||||||
Goal: remove hidden Weather API fetching from `GenerateReport`.
|
|
||||||
|
|
||||||
Implementation:
|
|
||||||
|
|
||||||
- Add a collection field to `ReportRequest`, for example:
|
|
||||||
|
|
||||||
```go
|
|
||||||
Collection collect.Result
|
|
||||||
```
|
|
||||||
|
|
||||||
- Update `Generate` to:
|
|
||||||
- collect once;
|
|
||||||
- resolve the requested report;
|
|
||||||
- pass the collection into `GenerateReport`.
|
|
||||||
- Update `GenerateReport` to use `req.Collection.Bundle`.
|
|
||||||
- If `req.Collection.Bundle` is nil, return an actionable error such as
|
|
||||||
`collected weather bundle is required`.
|
|
||||||
- Remove the direct `FetchBundle` call from `GenerateReport`.
|
|
||||||
- Keep `BuildReportFacts` unchanged: it should still receive a
|
|
||||||
`*weatherdata.Bundle`.
|
|
||||||
- Update all app tests that call `GenerateReport` directly to pass a collected
|
|
||||||
test bundle through `ReportRequest.Collection`.
|
|
||||||
|
|
||||||
Tests:
|
|
||||||
|
|
||||||
- Add or update tests proving single-report generation collects once.
|
|
||||||
- Add or update tests proving `GenerateReport` fails before state writes when
|
|
||||||
no collected bundle is provided.
|
|
||||||
- Keep generated-text, notification, state, and output-copy tests passing with
|
|
||||||
the explicit collection field.
|
|
||||||
|
|
||||||
Acceptance:
|
|
||||||
|
|
||||||
- `GenerateReport` has no direct Weather API fetch path.
|
|
||||||
- Single-report commands still produce the same artifacts.
|
|
||||||
- `GenerateReport` has an explicit data dependency.
|
|
||||||
|
|
||||||
Validation:
|
|
||||||
|
|
||||||
```sh
|
|
||||||
go test ./internal/app ./internal/cli
|
|
||||||
rg -n "weatherapi\\.New|\\.FetchBundle\\(" internal
|
|
||||||
git diff --check
|
|
||||||
```
|
|
||||||
|
|
||||||
Expected grep matches at this stage should be limited to
|
|
||||||
`internal/collect`, `internal/adapters/weatherapi`, and their tests, plus any
|
|
||||||
transitional app fetch helper tests that explicitly verify `FetchBundle`.
|
|
||||||
|
|
||||||
## Stage 4: Add Hourly Full-Day Coverage Helper
|
|
||||||
|
|
||||||
Goal: isolate the Daily eligibility rule before changing batch behavior.
|
|
||||||
|
|
||||||
Implementation:
|
|
||||||
|
|
||||||
- Add helper logic in `internal/app/batch_plan.go`.
|
|
||||||
- The helper should inspect `weatherdata.ForecastRun` hourly periods and return
|
|
||||||
eligible future Daily dates.
|
|
||||||
- Inputs should include:
|
|
||||||
- hourly forecast run;
|
|
||||||
- `now`;
|
|
||||||
- loaded report timezone/location.
|
|
||||||
- The helper should:
|
|
||||||
- start candidate Daily expansion at the day after tomorrow;
|
|
||||||
- compute local civil days with `timeutil.CivilDay`;
|
|
||||||
- build required hourly start instants by stepping from civil-day start to
|
|
||||||
civil-day end in one-hour increments;
|
|
||||||
- require every required start instant to exist as a valid hourly period
|
|
||||||
`StartTime`;
|
|
||||||
- require matching periods to have valid end times after start times;
|
|
||||||
- scan candidate dates through the maximum local date represented by hourly
|
|
||||||
period start times;
|
|
||||||
- return dates in ascending local date order;
|
|
||||||
- return no dates when hourly data is missing.
|
|
||||||
- Use exact start-time matching, not overlap-only matching.
|
|
||||||
|
|
||||||
Tests:
|
|
||||||
|
|
||||||
- Full ordinary local day with starts `00:00` through `23:00` is eligible.
|
|
||||||
- Missing one required hour makes the date ineligible.
|
|
||||||
- Partial final day is skipped.
|
|
||||||
- Today and tomorrow are never returned by expansion.
|
|
||||||
- Multiple eligible future days are returned in order.
|
|
||||||
- Non-hourly or invalid periods are ignored.
|
|
||||||
- DST transition days use actual civil-day hourly instants.
|
|
||||||
- Missing hourly forecast returns no dynamic Daily dates.
|
|
||||||
|
|
||||||
Acceptance:
|
|
||||||
|
|
||||||
- Daily eligibility is tested independently from batch execution.
|
|
||||||
- The helper does not resolve reports, call Weather API, write state, or invoke
|
|
||||||
Scriptorium.
|
|
||||||
|
|
||||||
Validation:
|
|
||||||
|
|
||||||
```sh
|
|
||||||
go test ./internal/app
|
|
||||||
git diff --check
|
|
||||||
```
|
|
||||||
|
|
||||||
## Stage 5: Add Shared Batch Planner
|
|
||||||
|
|
||||||
Goal: make morning and evening batch membership app-owned and data-aware.
|
|
||||||
|
|
||||||
Implementation:
|
|
||||||
|
|
||||||
- Continue using `report.BatchForCommandName` for CLI validation.
|
|
||||||
- Add an internal planned report type in `internal/app`, for example:
|
|
||||||
|
|
||||||
```go
|
|
||||||
type plannedBatchReport struct {
|
|
||||||
Resolved report.Resolved
|
|
||||||
OutputCopyName string
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
- Implement app-owned batch planning in `internal/app/batch_plan.go`.
|
|
||||||
- Morning plan:
|
|
||||||
- resolve `report.Today`;
|
|
||||||
- resolve `report.Tomorrow`;
|
|
||||||
- resolve `report.Daily` for each eligible future date from Stage 4.
|
|
||||||
- Evening plan:
|
|
||||||
- resolve `report.Tomorrow`;
|
|
||||||
- resolve `report.Daily` for each eligible future date from Stage 4.
|
|
||||||
- Resolve dynamic Daily reports by setting `ResolveRequest.Date` to the target
|
|
||||||
local date.
|
|
||||||
- Dynamic Daily planned reports should set `OutputCopyName` to
|
|
||||||
`daily-YYYY-MM-DD.md`.
|
|
||||||
- Today and Tomorrow should use their report definition batch output names.
|
|
||||||
- Remove `three-day` and `weekend` from morning planning.
|
|
||||||
- Stop using `report.Registry.BatchReports` from app batch execution. Either
|
|
||||||
remove that method if it becomes unused, or leave it unused only if tests or
|
|
||||||
docs still need it temporarily. Do not allow app batch execution to use it.
|
|
||||||
|
|
||||||
Tests:
|
|
||||||
|
|
||||||
- Morning order is Today, Tomorrow, then Daily dates.
|
|
||||||
- Evening order is Tomorrow, then Daily dates.
|
|
||||||
- Dynamic Daily dates start day after tomorrow.
|
|
||||||
- 3-Day and Weekend are absent from morning.
|
|
||||||
- Dynamic Daily resolves valid periods for the requested dates.
|
|
||||||
- Planned dynamic Daily output copy names are date-qualified.
|
|
||||||
- Unknown batch names still fail through existing CLI/report validation.
|
|
||||||
|
|
||||||
Acceptance:
|
|
||||||
|
|
||||||
- Batch membership is owned by `internal/app`.
|
|
||||||
- Batch membership is based on one collected bundle.
|
|
||||||
- `internal/collect` has no report or batch policy.
|
|
||||||
|
|
||||||
Validation:
|
|
||||||
|
|
||||||
```sh
|
|
||||||
go test ./internal/app ./internal/report ./internal/cli
|
|
||||||
git diff --check
|
|
||||||
```
|
|
||||||
|
|
||||||
## Stage 6: Use One Collection Result Per Batch
|
|
||||||
|
|
||||||
Goal: make both batch commands collect once and reuse that collection for every
|
|
||||||
report.
|
|
||||||
|
|
||||||
Implementation:
|
|
||||||
|
|
||||||
- Update `RunBatchDetailed` to:
|
|
||||||
- resolve the effective time;
|
|
||||||
- collect once using the request collector/default collector;
|
|
||||||
- plan the batch from that collection;
|
|
||||||
- pass the same `collect.Result` into every `GenerateReport` call.
|
|
||||||
- If collection fails, return before generating any report.
|
|
||||||
- Update batch report result construction to use `plannedBatchReport`.
|
|
||||||
- Update `batchOutputPath` or replace it with planned output-copy path logic:
|
|
||||||
- if `OutputDir` is empty, return empty output path;
|
|
||||||
- if `planned.OutputCopyName` is non-empty, use that;
|
|
||||||
- otherwise use `resolved.Definition.BatchOutputName`;
|
|
||||||
- join with `OutputDir`.
|
|
||||||
- Preserve per-report failure behavior after planning: later reports continue
|
|
||||||
after a report failure.
|
|
||||||
- Preserve notification failure behavior.
|
|
||||||
|
|
||||||
Tests:
|
|
||||||
|
|
||||||
- Counted or fake collector proves one collection call for morning.
|
|
||||||
- Counted or fake collector proves one collection call for evening.
|
|
||||||
- Collection failure prevents any renderer calls.
|
|
||||||
- Report failure still allows later planned reports to run.
|
|
||||||
- Notification failure still marks only that report failed and continues.
|
|
||||||
- `--out-dir` output paths use:
|
|
||||||
- `today.md` for Today;
|
|
||||||
- `tomorrow.md` for Tomorrow;
|
|
||||||
- `daily-YYYY-MM-DD.md` for dynamic Daily.
|
|
||||||
|
|
||||||
Acceptance:
|
|
||||||
|
|
||||||
- Both batch commands use one collection result per invocation.
|
|
||||||
- No report in a batch performs its own upstream fetch.
|
|
||||||
- Batch JSON and stderr behavior remain coherent.
|
|
||||||
|
|
||||||
Validation:
|
|
||||||
|
|
||||||
```sh
|
|
||||||
go test ./internal/app ./internal/cli ./internal/state
|
|
||||||
rg -n "weatherapi\\.New|\\.FetchBundle\\(" internal
|
|
||||||
git diff --check
|
|
||||||
```
|
|
||||||
|
|
||||||
Expected grep matches should be limited to `internal/collect`,
|
|
||||||
`internal/adapters/weatherapi`, and their tests.
|
|
||||||
|
|
||||||
## Stage 7: Remove Or Retire Legacy Static Batch Resolution
|
|
||||||
|
|
||||||
Goal: prevent future code from accidentally using stale batch membership.
|
|
||||||
|
|
||||||
Implementation:
|
|
||||||
|
|
||||||
- Inspect usages of `report.Registry.BatchReports`.
|
|
||||||
- If no longer needed, delete `Registry.BatchReports` and related tests.
|
|
||||||
- If keeping a reduced helper is necessary, document in code comments that app
|
|
||||||
batch planning is authoritative and ensure no production path calls the old
|
|
||||||
static membership helper.
|
|
||||||
- Update report-registry tests that currently assert old morning/evening
|
|
||||||
membership.
|
|
||||||
|
|
||||||
Tests:
|
|
||||||
|
|
||||||
- No production code path calls static registry batch membership.
|
|
||||||
- Report registry tests continue to cover report definitions, generated flags,
|
|
||||||
batch output names, and valid-period resolution as appropriate.
|
|
||||||
|
|
||||||
Acceptance:
|
|
||||||
|
|
||||||
- There is no stale static morning/evening membership path in production code.
|
|
||||||
- Future agents cannot accidentally reintroduce old Today/3-Day/Weekend
|
|
||||||
morning membership by calling a legacy helper.
|
|
||||||
|
|
||||||
Validation:
|
|
||||||
|
|
||||||
```sh
|
|
||||||
go test ./internal/report ./internal/app
|
|
||||||
rg -n "BatchReports\\(" internal
|
|
||||||
git diff --check
|
|
||||||
```
|
|
||||||
|
|
||||||
Expected `BatchReports` grep result should be empty or limited to tests or
|
|
||||||
comments that explicitly document it as non-production.
|
|
||||||
|
|
||||||
## Stage 8: Documentation Updates
|
|
||||||
|
|
||||||
Goal: update implemented documentation after the code behavior changes.
|
|
||||||
|
|
||||||
Implementation:
|
|
||||||
|
|
||||||
- Update `docs/cli.md`:
|
|
||||||
- morning batch now generates Today, Tomorrow, and eligible future Daily
|
|
||||||
reports;
|
|
||||||
- evening batch now generates Tomorrow and eligible future Daily reports;
|
|
||||||
- Daily eligibility is full hourly coverage.
|
|
||||||
- Update `docs/operations.md` with the same operator-facing behavior and
|
|
||||||
collection-failure behavior.
|
|
||||||
- Update `docs/internal/app-orchestration.md`:
|
|
||||||
- `internal/collect` boundary;
|
|
||||||
- one collection per command;
|
|
||||||
- app-owned batch planning;
|
|
||||||
- explicit collected data passed to report generation.
|
|
||||||
- Update `docs/internal/report-registry.md`:
|
|
||||||
- report definitions remain canonical for report metadata;
|
|
||||||
- batch membership is app-owned when data-dependent.
|
|
||||||
- Add a new implemented internal component doc for `internal/collect`, for
|
|
||||||
example `docs/internal/collect.md`.
|
|
||||||
- Do not describe future source families as implemented behavior.
|
|
||||||
|
|
||||||
Tests/checks:
|
|
||||||
|
|
||||||
- Grep docs for stale old batch language: Today/3-Day/Weekend morning,
|
|
||||||
Weekend except Sunday, Daily not part of scheduled batches.
|
|
||||||
- Confirm docs do not imply `internal/collect` owns report or batch policy.
|
|
||||||
|
|
||||||
Acceptance:
|
|
||||||
|
|
||||||
- Non-roadmap docs describe only implemented behavior.
|
|
||||||
- Operator docs and internal docs agree about batch membership.
|
|
||||||
|
|
||||||
Validation:
|
|
||||||
|
|
||||||
```sh
|
|
||||||
rg -n "3-Day|Weekend|scheduled batch|not part of scheduled|except on Sunday" docs README.md
|
|
||||||
git diff --check
|
|
||||||
```
|
|
||||||
|
|
||||||
## Stage 9: Final Validation
|
|
||||||
|
|
||||||
Goal: verify the completed migration and behavior end to end.
|
|
||||||
|
|
||||||
Run:
|
|
||||||
|
|
||||||
```sh
|
|
||||||
go test ./...
|
|
||||||
go run ./cmd/weatherreporter --help
|
|
||||||
rg -n "weatherapi\\.New|\\.FetchBundle\\(" internal
|
|
||||||
rg -n "BatchReports\\(" internal
|
|
||||||
git diff --check
|
|
||||||
```
|
|
||||||
|
|
||||||
Manual review:
|
|
||||||
|
|
||||||
- `weatherapi.New` and `.FetchBundle(` matches are limited to
|
|
||||||
`internal/collect`, `internal/adapters/weatherapi`, and their tests.
|
|
||||||
- `BatchReports(` matches are absent from production paths, or explicitly
|
|
||||||
documented as non-production if retained.
|
|
||||||
- `run morning` output summary can include multiple Daily reports without
|
|
||||||
duplicate `--out-dir` copy paths.
|
|
||||||
- `run evening` uses the same collection and dynamic Daily expansion logic as
|
|
||||||
morning.
|
|
||||||
- Managed workspace artifact paths are unchanged.
|
|
||||||
- Distributor upload paths are unchanged.
|
|
||||||
|
|
||||||
## Open Questions
|
|
||||||
|
|
||||||
None.
|
|
||||||
@@ -6,7 +6,7 @@ fixes. See [CLI reference](cli.md), [Configuration reference](config.md), and
|
|||||||
|
|
||||||
## `weather_api.base_url is required`
|
## `weather_api.base_url is required`
|
||||||
|
|
||||||
Symptom: a generation command fails before fetching weather data.
|
Symptom: a generation command fails before collecting weather data.
|
||||||
|
|
||||||
Likely cause: no Weather API base URL is configured.
|
Likely cause: no Weather API base URL is configured.
|
||||||
|
|
||||||
@@ -235,13 +235,18 @@ Relevant docs: [Report Templates](templates.md),
|
|||||||
|
|
||||||
Symptom: `run morning` or `run evening` returns nonzero.
|
Symptom: `run morning` or `run evening` returns nonzero.
|
||||||
|
|
||||||
Likely cause: at least one report in the batch failed.
|
Likely cause: weather collection failed before planning, or at least one
|
||||||
|
planned report failed after planning succeeded.
|
||||||
|
|
||||||
Diagnostic: inspect stdout for the JSON summary and stderr for compact status
|
Diagnostic: if stdout contains a JSON summary, inspect each failed report item
|
||||||
lines.
|
and stderr status line. If no summary was emitted, inspect the command error;
|
||||||
|
configuration, Weather API collection, or batch validation failed before any
|
||||||
|
report artifacts were created.
|
||||||
|
|
||||||
Safe fix: use the failed report's artifact paths from the summary, then inspect
|
Safe fix: for collection failures, fix the configuration or upstream Weather
|
||||||
metadata, sources, module snapshot, and data package for that RunID.
|
API availability and rerun the batch. For report failures, use the failed
|
||||||
|
report's artifact paths from the summary, then inspect metadata, sources,
|
||||||
|
module snapshot, and data package for that RunID.
|
||||||
|
|
||||||
Relevant docs: [CLI reference](cli.md), [Operations guide](operations.md).
|
Relevant docs: [CLI reference](cli.md), [Operations guide](operations.md).
|
||||||
|
|
||||||
|
|||||||
Reference in New Issue
Block a user