weatherreporter/docs/roadmap/implementation.md

Workspace Layout Implementation Plan

Purpose

This document is the staged implementation plan for workspace.md. It is written for an LLM coding agent that will implement the workspace artifact layout refactor in order.

The feature is complete when every managed local run artifact uses the kind-first, date-scoped layout defined in docs/roadmap/workspace.md, metadata and inspection link those paths correctly, and distributor upload behavior is unchanged.

Ground Rules

• Review docs/policy/architecture.md, docs/policy/development.md, and docs/policy/documentation.md before editing code.
• Keep path construction owned by internal/state.
• Do not add user-visible workspace path template configuration.
• Do not use distributor bundle paths for managed workspace paths.
• Do not implement cleanup, migration tooling, or a local publication mirror in this work.
• Keep optional --out and --out-dir copies separate from managed workspace artifacts.

Target Path Contract

State-managed run artifacts must use these relative path shapes under workspace.root:

reports/<artifact_group>/<valid_start_date>/report.<run_id>.md

snapshots/<artifact_group>/<valid_start_date>/modules.<run_id>.json
snapshots/<artifact_group>/<valid_start_date>/metadata.<run_id>.json
snapshots/<artifact_group>/<valid_start_date>/generated_text_raw.<run_id>.json
snapshots/<artifact_group>/<valid_start_date>/generated_text_result.<run_id>.json
snapshots/<artifact_group>/<valid_start_date>/generated_text.<run_id>.json
snapshots/<artifact_group>/<valid_start_date>/render_context.<run_id>.json

data-packages/<artifact_group>/<valid_start_date>/data_package.<run_id>.yaml
preflight/<artifact_group>/<valid_start_date>/render.<run_id>.json

notifications/<artifact_group>/<valid_start_date>/distributor.<run_id>.json
notifications/batches/<batch>/<batch_start_date>/distributor.<batch_run_id>.json

valid_start_date and batch_start_date are local dates in the effective report timezone already carried by the resolved report or batch reference.

Stage 1: State Path Construction

Goal: switch the canonical managed path builder to the target layout.

Implementation:

• Update internal/state.FilesystemStore.Paths to build all ArtifactPaths using the target path contract above.
• Keep existing validation for workspace subdirectories, RunID, and report artifact group.
• Keep PrepareRenderedReport responsible for creating the parent report directory before Scriptorium or template rendering writes Markdown.
• Update BatchDistributorNotificationPath to use distributor.<batch_run_id>.json.

Tests:

• Update state path tests to assert the exact new relative paths for at least daily, today, tomorrow, and hourly.
• Update batch notification path tests for notifications/batches/<batch>/<date>/distributor.<batch_run_id>.json.
• Run:

go test ./internal/state

Completion criteria:

• FilesystemStore.Paths returns only target-layout managed paths.
• Existing save methods write to the new paths through the existing ArtifactPaths fields.

Stage 2: Metadata Discovery And Prior Lookup

Goal: make inspection and prior lookup discover the new metadata filenames.

Implementation:

• Replace metadata filename checks that currently rely on the old .metadata.json suffix with a helper that matches only metadata.<run_id>.json.
• Use that helper in ListReports and FindPriorSnapshot.
• Do not add legacy metadata filename fallback in this refactor; migration is explicitly deferred by the feature roadmap.
• Keep LoadMetadataByRunID behavior based on ListReports.

Tests:

• Update report listing tests to create/read metadata.<run_id>.json.
• Update prior lookup tests to confirm comparable metadata is found under the new filename.
• Add or adjust one negative test so non-metadata JSON in the snapshot tree is ignored.
• Run:

go test ./internal/state

Completion criteria:

• Inspection and prior lookup work with new metadata names.
• Generated-text, module, and render-context JSON files are not mistaken for metadata.

Stage 3: App Workflow Expectations

Goal: update app-level tests and any path assumptions outside internal/state.

Implementation:

• Do not duplicate path-building logic in internal/app; continue asking the state store for managed paths.
• Update app tests that assert managed report paths, data package paths, generated-text paths, render context paths, notification source paths, batch included report source paths, and result JSON paths.
• Preserve the invariant that distributor uploads use the managed Markdown report path as the upload source and map it to separate distributor bundle paths.
• Preserve --out and --out-dir behavior as extra copies outside the managed report path.

Tests:

• Run:

go test ./internal/app ./internal/cli

Completion criteria:

• App workflows still generate, save metadata, notify distributor, and report paths using state-owned managed paths.
• No app code constructs the new managed path layout manually.

Stage 4: Documentation

Goal: move the implemented layout into maintained docs after code behavior has changed.

Documentation changes:

• Update docs/internal/state.md with the new target managed layout and filename convention.
• Update docs/operations.md filesystem layout examples and artifact wording.
• Update docs/config.md only if workspace field descriptions need clarification.
• Keep cleanup, migration, and local publication mirror details in roadmap docs until those features are implemented.

Validation:

go test ./internal/state ./internal/app ./internal/config
go test ./...
go run ./cmd/weatherreporter --help
git diff --check

Completion criteria:

• Implemented docs match the new behavior.
• Non-roadmap docs do not describe deferred cleanup or local publication mirror behavior as implemented.

Final Verification

Before considering the feature complete, run:

go test ./...
go run ./cmd/weatherreporter --help
git diff --check
rg "\\.metadata\\.json|\\.modules\\.json|\\.data_package\\.yaml|\\.render\\.json|\\.distributor\\.json|generated_text\\.raw|generated_text\\.run" internal docs examples --glob '!docs/roadmap/**'

The final rg should find no old managed filename patterns in implemented code, maintained docs, or examples.

Open Questions

None. The feature roadmap decisions are sufficient for implementation.