Workspace Artifact Layout Roadmap

Purpose

This roadmap defines the target local workspace layout for artifacts generated by weatherreporter.

Weatherreporter currently writes managed local artifacts for each run and also maps the managed Markdown report into distributor bundle paths. Distributor bundle paths are publication paths and may include overwrite-oriented aliases such as hourly/index.md, today/index.md, and daily/<valid_start_date>/index.md. Local workspace paths have a different job: they preserve run-specific source artifacts for inspection, troubleshooting, retry diagnosis, and future cleanup.

Locked Decisions

Keep managed local workspace paths separate from distributor bundle paths.
Do not use distributor-style index.md aliases as the canonical managed local artifact paths.
Store managed local artifacts as immutable run artifacts.
Group every durable run artifact by artifact family, report artifact group, and local valid_start_date.
Use valid_start_date in the effective report timezone as the directory key.
Include the RunID in every durable run artifact filename.
Prefer artifact-kind-first filenames: <kind>.<run_id>.<extension>.
Keep optional --out and --out-dir copies separate from managed workspace artifacts.
Future cleanup should delete old managed artifact date directories by valid_start_date; cleanup should be narrow and opt-in.
If a local publication preview is added later, put it in a separate mirror tree and treat it as rebuildable output, not canonical run state.

Target Managed Layout

The target managed workspace layout is:

<workspace.root>/
  reports/<artifact_group>/<valid_start_date>/report.<run_id>.md

  snapshots/<artifact_group>/<valid_start_date>/modules.<run_id>.json
  snapshots/<artifact_group>/<valid_start_date>/metadata.<run_id>.json
  snapshots/<artifact_group>/<valid_start_date>/generated_text_raw.<run_id>.json
  snapshots/<artifact_group>/<valid_start_date>/generated_text_result.<run_id>.json
  snapshots/<artifact_group>/<valid_start_date>/generated_text.<run_id>.json
  snapshots/<artifact_group>/<valid_start_date>/render_context.<run_id>.json

  data-packages/<artifact_group>/<valid_start_date>/data_package.<run_id>.yaml
  preflight/<artifact_group>/<valid_start_date>/render.<run_id>.json

  notifications/<artifact_group>/<valid_start_date>/distributor.<run_id>.json
  notifications/batches/<batch>/<batch_start_date>/distributor.<batch_run_id>.json

artifact_group comes from the report definition. valid_start_date is the resolved report valid period start date in the effective report timezone. batch_start_date is the batch start date in the effective report timezone.

Generated-text-template reports use the generated-text artifacts. Markdown-path reports omit generated-text artifacts they do not produce.

Cleanup Compatibility

The target layout is designed to support a future cleanup command that removes managed local artifacts where valid_start_date is more than two days before the current local date.

Cleanup can operate by deleting matching date directories across managed artifact families:

reports/<artifact_group>/<old_date>/
snapshots/<artifact_group>/<old_date>/
data-packages/<artifact_group>/<old_date>/
preflight/<artifact_group>/<old_date>/
notifications/<artifact_group>/<old_date>/

The command should not infer dates from RunIDs when the directory date is available. It should use the effective configured report timezone for "current date" and for interpreting valid_start_date directories.

Batch notification cleanup should use batch_start_date under notifications/batches/<batch>/. A future local publication mirror, if added, should be ignored by cleanup or rebuilt after cleanup; it should not be treated as run history.

Intended Final State

internal/state owns the managed workspace layout and path construction. The app layer continues to ask the state store for managed paths and records those paths in metadata and command results.

Metadata links every artifact produced for a run using the new managed paths. Inspection and prior-snapshot lookup continue to read metadata rather than rendered Markdown text. Distributor uploads continue to use the managed Markdown report as the source file and map it to separately resolved distributor bundle paths.

Existing workspace configuration fields remain meaningful:

workspace.root
workspace.snapshots_dir
workspace.reports_dir
workspace.data_packages_dir
workspace.preflight_dir
workspace.notifications_dir

This roadmap does not require a new user-visible workspace path template config surface.

Deferred Work

A cleanup command for managed local artifacts.
Cleanup dry-run output, retention configuration, and deletion confirmation behavior.
A rebuildable local publication mirror that uses distributor-style paths.
Migration tooling for existing pre-refactor workspace directories.

4.9 KiB Raw Blame History