Add a feature roadmap and staged implentation plan to refactor the local workspace layout
This commit is contained in:
194
docs/roadmap/implementation.md
Normal file
194
docs/roadmap/implementation.md
Normal file
@@ -0,0 +1,194 @@
|
||||
# Workspace Layout Implementation Plan
|
||||
|
||||
## Purpose
|
||||
|
||||
This document is the staged implementation plan for
|
||||
[workspace.md](workspace.md). It is written for an LLM coding agent that will
|
||||
implement the workspace artifact layout refactor in order.
|
||||
|
||||
The feature is complete when every managed local run artifact uses the
|
||||
kind-first, date-scoped layout defined in `docs/roadmap/workspace.md`, metadata
|
||||
and inspection link those paths correctly, and distributor upload behavior is
|
||||
unchanged.
|
||||
|
||||
## Ground Rules
|
||||
|
||||
- Review `docs/policy/architecture.md`, `docs/policy/development.md`, and
|
||||
`docs/policy/documentation.md` before editing code.
|
||||
- Keep path construction owned by `internal/state`.
|
||||
- Do not add user-visible workspace path template configuration.
|
||||
- Do not use distributor bundle paths for managed workspace paths.
|
||||
- Do not implement cleanup, migration tooling, or a local publication mirror in
|
||||
this work.
|
||||
- Keep optional `--out` and `--out-dir` copies separate from managed workspace
|
||||
artifacts.
|
||||
|
||||
## Target Path Contract
|
||||
|
||||
State-managed run artifacts must use these relative path shapes under
|
||||
`workspace.root`:
|
||||
|
||||
```text
|
||||
reports/<artifact_group>/<valid_start_date>/report.<run_id>.md
|
||||
|
||||
snapshots/<artifact_group>/<valid_start_date>/modules.<run_id>.json
|
||||
snapshots/<artifact_group>/<valid_start_date>/metadata.<run_id>.json
|
||||
snapshots/<artifact_group>/<valid_start_date>/generated_text_raw.<run_id>.json
|
||||
snapshots/<artifact_group>/<valid_start_date>/generated_text_result.<run_id>.json
|
||||
snapshots/<artifact_group>/<valid_start_date>/generated_text.<run_id>.json
|
||||
snapshots/<artifact_group>/<valid_start_date>/render_context.<run_id>.json
|
||||
|
||||
data-packages/<artifact_group>/<valid_start_date>/data_package.<run_id>.yaml
|
||||
preflight/<artifact_group>/<valid_start_date>/render.<run_id>.json
|
||||
|
||||
notifications/<artifact_group>/<valid_start_date>/distributor.<run_id>.json
|
||||
notifications/batches/<batch>/<batch_start_date>/distributor.<batch_run_id>.json
|
||||
```
|
||||
|
||||
`valid_start_date` and `batch_start_date` are local dates in the effective
|
||||
report timezone already carried by the resolved report or batch reference.
|
||||
|
||||
## Stage 1: State Path Construction
|
||||
|
||||
Goal: switch the canonical managed path builder to the target layout.
|
||||
|
||||
Implementation:
|
||||
|
||||
- Update `internal/state.FilesystemStore.Paths` to build all `ArtifactPaths`
|
||||
using the target path contract above.
|
||||
- Keep existing validation for workspace subdirectories, RunID, and report
|
||||
artifact group.
|
||||
- Keep `PrepareRenderedReport` responsible for creating the parent report
|
||||
directory before Scriptorium or template rendering writes Markdown.
|
||||
- Update `BatchDistributorNotificationPath` to use
|
||||
`distributor.<batch_run_id>.json`.
|
||||
|
||||
Tests:
|
||||
|
||||
- Update state path tests to assert the exact new relative paths for at least
|
||||
`daily`, `today`, `tomorrow`, and `hourly`.
|
||||
- Update batch notification path tests for
|
||||
`notifications/batches/<batch>/<date>/distributor.<batch_run_id>.json`.
|
||||
- Run:
|
||||
|
||||
```bash
|
||||
go test ./internal/state
|
||||
```
|
||||
|
||||
Completion criteria:
|
||||
|
||||
- `FilesystemStore.Paths` returns only target-layout managed paths.
|
||||
- Existing save methods write to the new paths through the existing
|
||||
`ArtifactPaths` fields.
|
||||
|
||||
## Stage 2: Metadata Discovery And Prior Lookup
|
||||
|
||||
Goal: make inspection and prior lookup discover the new metadata filenames.
|
||||
|
||||
Implementation:
|
||||
|
||||
- Replace metadata filename checks that currently rely on the old
|
||||
`.metadata.json` suffix with a helper that matches only
|
||||
`metadata.<run_id>.json`.
|
||||
- Use that helper in `ListReports` and `FindPriorSnapshot`.
|
||||
- Do not add legacy metadata filename fallback in this refactor; migration is
|
||||
explicitly deferred by the feature roadmap.
|
||||
- Keep `LoadMetadataByRunID` behavior based on `ListReports`.
|
||||
|
||||
Tests:
|
||||
|
||||
- Update report listing tests to create/read `metadata.<run_id>.json`.
|
||||
- Update prior lookup tests to confirm comparable metadata is found under the
|
||||
new filename.
|
||||
- Add or adjust one negative test so non-metadata JSON in the snapshot tree is
|
||||
ignored.
|
||||
- Run:
|
||||
|
||||
```bash
|
||||
go test ./internal/state
|
||||
```
|
||||
|
||||
Completion criteria:
|
||||
|
||||
- Inspection and prior lookup work with new metadata names.
|
||||
- Generated-text, module, and render-context JSON files are not mistaken for
|
||||
metadata.
|
||||
|
||||
## Stage 3: App Workflow Expectations
|
||||
|
||||
Goal: update app-level tests and any path assumptions outside `internal/state`.
|
||||
|
||||
Implementation:
|
||||
|
||||
- Do not duplicate path-building logic in `internal/app`; continue asking the
|
||||
state store for managed paths.
|
||||
- Update app tests that assert managed report paths, data package paths,
|
||||
generated-text paths, render context paths, notification source paths, batch
|
||||
included report source paths, and result JSON paths.
|
||||
- Preserve the invariant that distributor uploads use the managed Markdown
|
||||
report path as the upload source and map it to separate distributor bundle
|
||||
paths.
|
||||
- Preserve `--out` and `--out-dir` behavior as extra copies outside the managed
|
||||
report path.
|
||||
|
||||
Tests:
|
||||
|
||||
- Run:
|
||||
|
||||
```bash
|
||||
go test ./internal/app ./internal/cli
|
||||
```
|
||||
|
||||
Completion criteria:
|
||||
|
||||
- App workflows still generate, save metadata, notify distributor, and report
|
||||
paths using state-owned managed paths.
|
||||
- No app code constructs the new managed path layout manually.
|
||||
|
||||
## Stage 4: Documentation
|
||||
|
||||
Goal: move the implemented layout into maintained docs after code behavior has
|
||||
changed.
|
||||
|
||||
Documentation changes:
|
||||
|
||||
- Update `docs/internal/state.md` with the new target managed layout and
|
||||
filename convention.
|
||||
- Update `docs/operations.md` filesystem layout examples and artifact wording.
|
||||
- Update `docs/config.md` only if workspace field descriptions need
|
||||
clarification.
|
||||
- Keep cleanup, migration, and local publication mirror details in roadmap
|
||||
docs until those features are implemented.
|
||||
|
||||
Validation:
|
||||
|
||||
```bash
|
||||
go test ./internal/state ./internal/app ./internal/config
|
||||
go test ./...
|
||||
go run ./cmd/weatherreporter --help
|
||||
git diff --check
|
||||
```
|
||||
|
||||
Completion criteria:
|
||||
|
||||
- Implemented docs match the new behavior.
|
||||
- Non-roadmap docs do not describe deferred cleanup or local publication mirror
|
||||
behavior as implemented.
|
||||
|
||||
## Final Verification
|
||||
|
||||
Before considering the feature complete, run:
|
||||
|
||||
```bash
|
||||
go test ./...
|
||||
go run ./cmd/weatherreporter --help
|
||||
git diff --check
|
||||
rg "\\.metadata\\.json|\\.modules\\.json|\\.data_package\\.yaml|\\.render\\.json|\\.distributor\\.json|generated_text\\.raw|generated_text\\.run" internal docs examples --glob '!docs/roadmap/**'
|
||||
```
|
||||
|
||||
The final `rg` should find no old managed filename patterns in implemented
|
||||
code, maintained docs, or examples.
|
||||
|
||||
## Open Questions
|
||||
|
||||
None. The feature roadmap decisions are sufficient for implementation.
|
||||
Reference in New Issue
Block a user