weatherreporter/docs/roadmap/implementation.md

# Promptkit Migration Implementation Plan

Status: Completed; Stages 1–19 passed their exit gates.

## Purpose And Authority

This document records the completed implementation of the
[Promptkit migration roadmap](promptkit.md) and its post-implementation audit
remediation. The feature roadmap records scope, user intent, policy choices,
and the implemented end state. This plan records implementation sequence,
tests, and completion gates.

Stages 12–19 were completed in order. They fixed additional defects exposed by
their required tests only when those defects were within the same stated
contract; they did not add new product behavior or reinterpret roadmap
decisions.

This plan follows the repository's
[architecture](../policy/architecture.md),
[documentation](../policy/documentation.md), and
[testing](../policy/testing.md) policies.

## Continuing Invariants

- Keep `gitea.maximumdirect.net/eric/promptkit` pinned at exactly `v0.4.0`.
- Keep Promptkit types inside `internal/adapters/promptkit`, its tests, and the
  external prompt-asset contract test.
- Preserve one Promptkit engine per `generate` or `run` invocation and one
  shared engine for every sequential report in a batch.
- Preserve exact prompt version `1.0.0`, the exact persisted YAML data-package
  bytes, prepared execution, and preparation persistence before provider work.
- Do not add retries, repair attempts, concurrent batch generation, direct
  Markdown generation, arbitrary backend registration, or live-provider
  tests.
- Keep ordinary artifacts, errors, logs, and summaries free of credentials,
  rendered messages, schemas, input bodies, generated bodies, endpoints, and
  full effective parameter maps.
- Keep sensitive debug artifacts opt-in, outside normal state, owner-only,
  atomic, and free of credentials.
- Treat an artifact path as reached only after the corresponding write or copy
  succeeds. Never persist or summarize a merely derivable future path.
- Preserve every safe reached path in partial app and CLI results even when a
  later persistence, validation, rendering, copy, or notification step fails.
- Keep v1 metadata read compatibility and write only v2 metadata for new runs.
- Keep the default test suite deterministic, offline, and credential-free.
- Run `git diff --check` before completing every stage. Run the full repository
  gate in Stage 19.

## Completed Migration Summary

Stages 1–11 are implemented and committed. They remain summarized here to
preserve the history and dependencies of the follow-up work.

| Stage | Completed outcome |
| --- | --- |
| 1 | Removed the unfinished three-day, weekend, and storm product surfaces and retained Daily, Today, Tomorrow, and Hourly with exact prompt version `1.0.0`. |
| 2 | Promoted the four operational prompts and canonical schemas into the embedded `internal/promptassets` source used by Promptkit and generated-text validation. |
| 3 | Added the project-owned `internal/promptexec` inspection, preparation, execution, validation, debug, and error contract. |
| 4 | Added the Promptkit v0.4.0 adapter with prepared execution, explicit value mapping, safe error classification, and offline model-client tests. |
| 5 | Added Promptkit-era preparation and execution artifacts, metadata v2, new paths, and v1 decoding support. |
| 6 | Added explicitly rooted, permission-restricted, atomic LLM debug persistence. |
| 7 | Added Promptkit configuration, executor composition, and pre-collection prompt/profile/credential inspection. |
| 8 | Cut single-report generation over to prepared Promptkit execution and v2 persistence. |
| 9 | Added `--llm-debug-dir` and Promptkit-era single-report summary fields. |
| 10 | Cut morning and evening batches over to one shared Promptkit executor and removed Scriptorium code, configuration, and dependency metadata. |
| 11 | Updated canonical Promptkit documentation, removed the temporary Scriptorium corpus, and ran the available repository checks. |

The post-implementation audit confirmed the principal dependency and package
boundaries, but found incorrect reached-path bookkeeping, incomplete execution
artifact updates, insufficient artifact validation, extensive loss of
behavioral tests during the final cutover, and roadmap lifecycle text that was
not finalized. The completed remediation addressed those findings without
changing the intended feature scope.

| Stage | Completed outcome |
| --- | --- |
| 12 | Corrected reached-path bookkeeping across metadata, app results, batch items, and CLI summaries. |
| 13 | Hardened Promptkit-era durable state validation and restored v1/v2 state coverage. |
| 14 | Recorded every downstream path reached after completed prompt execution. |
| 15 | Restored assembled single-report behavioral and failure coverage. |
| 16 | Simplified prompt-generation orchestration while preserving behavior. |
| 17 | Restored assembled batch, planning, artifact, and notification coverage. |
| 18 | Restored supported CLI, summary, safety, and historical inspection coverage. |
| 19 | Reconciled canonical documentation and passed the complete repository verification gate. |

## Stage 12: Correct Reached-Artifact Bookkeeping

Status: Completed.

### Goal

Make metadata, app results, batch items, and CLI summaries truthful at every
failure boundary: a nonblank path means that artifact was successfully
created.

### Work

1. Change `state.BuildPromptMetadataFromBriefingMetadata` so it initializes
   identity, schema, metadata destination, and only artifacts already saved at
   the call site. It must not prepopulate raw-output, normalized-text,
   render-context, managed-report, preparation, execution, notification, or
   output-copy paths.
2. In `generatePromptReport`, assign each metadata and `ReportResult` path
   immediately after that artifact write succeeds and before attempting the
   next write. In particular:

   - do not initialize `ReportResult.ReportPath` from `Store.Paths`;
   - record a saved failed-preparation receipt in the result before saving
     metadata;
   - record a saved failed or completed execution receipt before saving
     metadata;
   - retain raw, normalized, context, report, copy, and notification paths
     when a later step fails; and
   - keep `MetadataPath` unchanged when a metadata rewrite fails, because the
     prior successfully written metadata record remains the reached version.

3. Remove batch-item prepopulation from derived `Store.Paths` values.
   `BatchReportResult` receives paths only from the returned `ReportResult` or
   from a write that the batch itself successfully completed.
4. Preserve current CLI field names and omission behavior. Human and JSON
   summaries must omit every unreached path.
5. Do not change artifact locations, filenames, schemas, report output, or
   notification policy in this stage.

### Tests

- Add focused app tests for one representative report using real temporary
  state plus a narrow failure-injecting store wrapper.
- Fail the next persistence step immediately after a successful preparation
  receipt, execution receipt, raw output, normalized output, render context,
  managed report, output copy, and notification artifact; assert that the
  returned result contains every reached path and no future path.
- Include one preparation failure, one operational execution failure, and one
  completed validation rejection to cover the three execution outcome shapes.
- Add batch and CLI summary assertions proving unreached paths are omitted.
- Run:

  ```sh
  go test ./internal/state ./internal/app ./internal/cli
  git diff --check
  ```

### Exit Gate

Every nonblank path in newly written metadata, app results, batch items, and
CLI summaries names an artifact that exists. Every safe artifact successfully
written before a later failure remains discoverable from the returned partial
result.

## Stage 13: Harden Durable State Contracts And Restore State Coverage

Status: Completed.

### Goal

Make the v1/v2 wire boundary and Promptkit-era artifact validation explicit,
strict, and durably tested.

### Work

1. Strengthen `PromptPreparationArtifact.Validate`:

   - require report ID, Weatherreporter RunID, prompt ID, exact prompt version,
     data-package path, nonzero start/end times, nonnegative duration, and an
     end not earlier than the start;
   - for success, require preparation provenance, prohibit an error, and
     require its prompt ID/version and data-package path to match the top-level
     artifact;
   - for failure, require a classified bounded error and prohibit fabricated
     preparation provenance.

2. Strengthen `PromptExecutionArtifact.Validate`:

   - require report ID, Weatherreporter RunID, prompt ID, exact prompt version,
     nonzero start/end times, nonnegative duration, and an end not earlier than
     the start;
   - for success and validation rejection, require provenance and completed
     validation, prohibit an operational error, and require the provenance
     prompt ID/version to match the artifact;
   - do not compare the provenance RunID with the Weatherreporter RunID because
     the provenance value is Promptkit's run identity;
   - for operational failure, require a classified bounded error and prohibit
     invented provenance or completed validation.

3. Validate required provenance fields for completed executions, including
   Promptkit RunID, prompt and rendered hashes, selected profile/backend/model,
   and data-package path. Permit usage counters and generated hash to be zero
   when the provider legitimately reports no value.
4. Restore focused filesystem and metadata tests for:

   - exact v2 paths and filenames;
   - preparation/execution round trips and required fields;
   - metadata v2 round trips without legacy aliases;
   - v1 decoding, normalized internal aliases, and v1-preserving re-marshaling;
   - unknown schema rejection;
   - report listing, RunID lookup, source/module/data-package inspection, and
     retained v1 behavior for historical report IDs;
   - atomic writes and unsafe workspace/path rejection; and
   - prior-snapshot behavior for the four supported report IDs.

5. Adapt useful tests from the deleted filesystem suite rather than recreating
   redundant low-value cases. Do not restore Scriptorium writes or retired
   report behavior.

### Tests

Run:

```sh
go test ./internal/state ./internal/app
git diff --check
```

### Exit Gate

The state package rejects incomplete or contradictory Promptkit-era artifacts,
reads historical v1 records, writes only valid v2 records, and has focused
offline coverage for its durable compatibility and filesystem contracts.

## Stage 14: Complete Execution-Artifact Path Tracking

Status: Completed.

### Goal

Make `PromptExecutionArtifact.Paths` accurately record every downstream
artifact reached after a completed Promptkit run.

### Work

1. Treat the execution artifact as an atomically updated durable record of the
   completed Promptkit execution and subsequent artifact destinations. Its
   status, provenance, validation, usage, and timing remain the provider-run
   outcome; later application failures do not change a successful Promptkit
   status into an execution failure.
2. Save the initial execution artifact after raw output is persisted, with
   `RawOutputPath` populated.
3. After each later successful write, update and atomically resave the same
   execution artifact with the corresponding reached path:

   - normalized generated text;
   - render context;
   - managed Markdown report;
   - an explicitly requested extra output copy, only after the copy succeeds;
     and
   - a Distributor notification artifact, including a persisted failure or
     status artifact when notification produced one.

4. Keep metadata and execution-artifact path values consistent after every
   successful checkpoint. Save the execution artifact before metadata so a
   metadata failure does not erase knowledge of a reached downstream artifact.
   Failure to update the execution artifact is terminal and returns a partial
   result containing the downstream artifact that was already written.
5. Refactor finalization return values only as needed to tell the orchestration
   layer which copy and notification paths were actually written. Distributor
   must continue uploading only the managed Markdown report.
6. A validation-rejected execution ends after raw output and therefore records
   only the raw-output path. An operational execution failure has no completed
   provenance and records only safe paths reached before that failure.

### Tests

- Add table-driven execution-artifact lifecycle tests for success and every
  downstream failure point.
- Load the persisted execution artifact after normalized-text, context,
  template, copy, metadata, and notification failures and assert its status and
  exact reached paths.
- Assert that execution artifacts never contain generated bodies, rendered
  prompts, schemas, endpoints, parameters, or credentials.
- Run:

  ```sh
  go test ./internal/state ./internal/app
  git diff --check
  ```

### Exit Gate

For every completed Promptkit run, its execution artifact contains exactly the
safe downstream paths reached by the workflow and remains semantically correct
when a later application stage fails.

## Stage 15: Restore Single-Report Behavioral Coverage

Status: Completed.

### Goal

Restore the risk-based application coverage removed during final cutover and
prove the complete single-report Promptkit workflow through project-owned
boundaries.

### Work

1. Reintroduce a focused app test harness using real state, prompt-input,
   generated-text validation, render contexts, and templates with deterministic
   collector, executor, notifier, clock, and filesystem boundaries.
2. Add representative successful workflows for Daily, Today, Tomorrow, and
   Hourly. Verify report identity, exact prompt version, one collection, exact
   persisted YAML bytes passed to the executor, expected template output,
   optional copy behavior, and managed-report notification source.
3. Cover the required failure matrix:

   - inspection and missing credentials before collection;
   - preparation failure and callback persistence failure before provider work;
   - execution-time credential disappearance;
   - capacity rejection without retry;
   - cancellation and deadline;
   - generation and operational-validation failure;
   - completed Promptkit schema rejection with retained raw output;
   - generated-text decode/domain rejection;
   - render-context and template failure;
   - output-copy failure; and
   - notification failure.

4. Verify preparation persistence precedes provider execution, debug-write
   failure prevents provider execution, and execution-debug failure preserves
   previously reached normal and debug artifacts.
5. Verify Recent Changes, prior-snapshot selection, output naming, and
   Distributor template values for all four retained reports.
6. Adapt useful tests from the deleted app suite. Omit Scriptorium mechanics,
   subprocess interaction assertions, and retired report products.
7. Fix defects exposed by these tests only when the expected behavior is
   already decided by the roadmap or canonical policy. Record any new product
   question instead of silently choosing it.

### Tests

Run:

```sh
go test ./internal/app
go test -race ./internal/app ./internal/adapters/promptkit
git diff --check
```

### Exit Gate

The single-report workflow has deterministic behavioral coverage for all four
reports, all consequential failure stages, artifact ordering, partial results,
debug isolation, output copying, and notification behavior.

## Stage 16: Refactor Prompt Generation Orchestration

Status: Completed.

### Goal

Reduce the complexity and duplicated persistence logic in
`generatePromptReport` without changing observable behavior.

### Work

1. Use the Stage 12–15 tests as the refactoring safety boundary. Do not weaken
   assertions to accommodate structural changes.
2. Split the current orchestration into small app-owned operations with clear
   inputs and outcomes for:

   - deterministic input and initial state construction;
   - preparation callback persistence;
   - preparation-failure persistence;
   - operational-execution-failure persistence;
   - completed execution and raw-output persistence;
   - normalized text and render-context persistence;
   - managed report, optional copy, metadata, and notification finalization;
     and
   - reached-path updates shared by success and failure paths.

3. Keep workflow order visible in one coordinator. Do not introduce a generic
   workflow engine, hidden retry loop, provider-specific app type, or mutable
   global state.
4. Centralize the repeated rule that a successful artifact write updates the
   result before any following write can fail.
5. Preserve error identities, safe text, atomic writes, exact bytes, debug
   ordering, partial results, and notification behavior.

### Tests

Run:

```sh
gofmt -w internal/app/*.go
go test ./internal/app ./internal/state ./internal/cli
go test -race ./internal/app
git diff --check
```

### Exit Gate

The top-level coordinator communicates the workflow order without containing
the full persistence implementation, duplicate failure branches are reduced,
and every Stage 12–15 behavioral test passes unchanged.

## Stage 17: Restore Batch Behavioral Coverage

Status: Completed.

### Goal

Re-establish confidence that morning and evening batches preserve their
pre-migration behavior while sharing one Promptkit executor.

### Work

1. Add assembled batch tests proving:

   - one executor factory call and one executor per CLI invocation;
   - inspection of the full candidate set before collection;
   - one weather collection;
   - existing morning/evening planning and ordering;
   - sequential execution through the shared executor;
   - continuation after an independent report failure;
   - no retry after capacity rejection;
   - distinct identities and debug directories for multiple Daily dates; and
   - exact reached paths on successful and failed batch items.

2. Restore notification coverage for disabled notification, suppressed
   per-report notification, all-success batch notification, skipped
   notification after report failure, and persisted notification failure/status
   artifacts.
3. Restore output-directory, Today/Tomorrow naming, dynamic Daily planning,
   prior-snapshot, and managed-Markdown upload-source coverage.
4. Adapt useful tests from the deleted batch portions of the app and CLI suites.
   Do not restore retired report cases or Scriptorium fakes.
5. Fix only roadmap-defined batch regressions exposed by the restored tests.

### Tests

Run:

```sh
go test ./internal/app ./internal/cli
go test -race ./internal/app
git diff --check
```

### Exit Gate

Morning and evening batches are covered as assembled sequential workflows and
demonstrably preserve collection, planning, continuation, output, debug,
artifact, and notification contracts with one Promptkit executor.

## Stage 18: Restore CLI And Inspection Coverage

Status: Completed.

### Goal

Restore the user-facing command, summary, and historical inspection contracts
removed with the old root test suite.

### Work

1. Add parser and resolver tests for all four generate commands, both batch
   commands, shared flags, report-specific date rules, malformed input,
   `--llm-debug-dir`, `--quiet`, output paths, and rejection of retired report
   names.
2. Add assembled CLI tests for representative successful single and batch
   invocations using injected offline boundaries. Verify exactly one executor
   construction per action.
3. Cover pre-run errors with no invented run summary, successful and failed
   JSON summaries, quiet-mode behavior, safe human status output, partial paths,
   and omission of absent notification/debug fields.
4. Restore inspection tests for report listing and v1/v2 metadata, modules,
   data packages, prior snapshots, and sources. Include failed v2 runs and v1
   fixtures using historical report IDs.
5. Assert that routine output never contains rendered prompts, schema bodies,
   data packages, generated bodies, endpoints, full parameters, credentials, or
   secret-like dependency errors.
6. Keep tests at stable CLI/app boundaries; do not restore assertions about
   private parser formatting or Scriptorium subprocess mechanics.

### Tests

Run:

```sh
go test ./internal/cli ./internal/app ./internal/state
go run ./cmd/weatherreporter --help
git diff --check
```

### Exit Gate

The supported CLI surface, summaries, quiet mode, partial failures, executor
composition, and v1/v2 inspection behavior have deterministic offline coverage.

## Stage 19: Finalize Documentation And Repository Verification

Status: Completed.

### Goal

Close the audit remediation, make roadmap lifecycle state truthful, and verify
the repository against the complete target contract.

### Work

1. Update `docs/roadmap/promptkit.md` from future tense and “unimplemented”
   statuses to a completed roadmap record. Describe its old seven-report and
   Scriptorium material explicitly as the pre-migration baseline rather than
   current behavior.
2. Mark Stages 12–19 and this implementation plan complete only after their
   exit gates pass. Retain the concise completed-stage history unless the
   documentation policy calls for archival in the same change.
3. Review canonical architecture, app, state, CLI, Promptkit integration,
   operations, troubleshooting, configuration, and testing documentation
   against the corrected implementation. Update only actual current-state
   discrepancies; do not duplicate the roadmap.
4. Search current-state code, tests, examples, help, and non-roadmap
   documentation for stale Scriptorium terms, retired reports, old artifact
   fields, speculative-path descriptions, or claims of missing Promptkit
   implementation.
5. Confirm examples contain no credentials or private infrastructure values
   and load through config tests.

### Final Verification

Run:

```sh
gofmt -w <all changed Go files>
go mod tidy
go vet ./...
go test -count=1 ./...
go test -race ./...
go run ./cmd/weatherreporter --help
git diff --check
```

Then verify explicitly:

- `go list -m gitea.maximumdirect.net/eric/promptkit` reports `v0.4.0`;
- no committed `go.work`, `replace`, secret fixture, or live-provider test
  exists;
- all four prompts inspect at exact version `1.0.0`;
- no runtime prompt requests repair attempts;
- v1 fixtures remain inspectable and new runs write only v2;
- normal artifacts and output contain no sensitive prompt/debug content;
- failed-run metadata, execution artifacts, app results, batch items, and CLI
  summaries contain exactly the paths actually reached;
- help exposes only Daily, Today, Tomorrow, Hourly, morning, and evening; and
- managed Markdown remains the only Distributor upload source.

### Exit Gate

Every migration and audit-remediation criterion is demonstrably satisfied,
the restored tests protect the consequential contracts, canonical
documentation describes the corrected implementation, and both roadmap
documents are marked complete.

## Open Questions

None. The roadmap and this completed plan record the decisions used for the
audit remediation.