Add embedded weather profile catalog
This commit is contained in:
@@ -1,373 +0,0 @@
|
||||
# Domain-Specific Prompt Profiles Implementation Plan
|
||||
|
||||
Status: Ready for implementation.
|
||||
|
||||
## Purpose And Authority
|
||||
|
||||
This document gives a coding agent the ordered work needed to implement the
|
||||
[domain-specific prompt profiles roadmap](domain-profiles.md). The roadmap is
|
||||
authoritative for scope, user intent, policy choices, and the desired end
|
||||
state. This plan is authoritative for implementation sequence, verification,
|
||||
and stage exit gates.
|
||||
|
||||
The target implementer is a `gpt-5.6-terra` coding agent using high reasoning.
|
||||
Complete the stages in order. Each stage is scoped for one implementation
|
||||
prompt and must leave the repository coherent, tested, and reviewable before
|
||||
the next stage begins.
|
||||
|
||||
This plan follows the repository's
|
||||
[architecture](../policy/architecture.md),
|
||||
[documentation](../policy/documentation.md), and
|
||||
[testing](../policy/testing.md) policies.
|
||||
|
||||
## Completed Prerequisite
|
||||
|
||||
Weatherreporter is already pinned to Promptkit v0.5.0. That release provides
|
||||
the public `WithFallbackProfileFS` option and the required precedence across
|
||||
inspection, preparation, and execution. The dependency upgrade passed
|
||||
Weatherreporter's full offline test suite, race-enabled suite, CLI help check,
|
||||
and an operator `generate hourly` smoke test. Do not repeat or replace the
|
||||
dependency upgrade as part of these stages.
|
||||
|
||||
## Locked Product Decisions
|
||||
|
||||
Implement these exact Weatherreporter-owned profiles:
|
||||
|
||||
| Profile ID | Backend | Model | Reasoning effort | Timeout | Service tier |
|
||||
| --- | --- | --- | --- | --- | --- |
|
||||
| `weather-light` | `openrouter` | `deepseek/deepseek-v4-flash` | Omitted | 180 seconds | `flex` |
|
||||
| `weather-balanced` | `openrouter` | `~google/gemini-flash-latest` | `high` | 240 seconds | `flex` |
|
||||
| `weather-deep` | `openrouter` | `~anthropic/claude-sonnet-latest` | `high` | 240 seconds | `flex` |
|
||||
|
||||
Assign Hourly to `weather-light`; assign Daily, Today, and Tomorrow to
|
||||
`weather-balanced`; assign no report to `weather-deep` initially. Advance all
|
||||
four prompt definitions and matching report-registry entries from `1.0.1` to
|
||||
`1.1.0` when their defaults change.
|
||||
|
||||
The leading `~` in the Gemini and Claude model IDs is required and denotes an
|
||||
OpenRouter rolling alias. Do not substitute the unavailable non-tilde IDs or a
|
||||
dated model version. Do not add temperature, `top_p`, maximum-token, endpoint,
|
||||
or credential fields to the embedded definitions.
|
||||
|
||||
Prompt preparation and execution artifacts written at `1.0.1` are not required
|
||||
to remain readable after the transition to `1.1.0`. Do not add a migration,
|
||||
compatibility shim, or weaker historical-artifact validation for this feature.
|
||||
|
||||
Definition lookup must remain:
|
||||
|
||||
1. explicit Promptkit in-memory profiles used by tests or an embedding
|
||||
consumer;
|
||||
2. Weatherreporter's configured `profile_file` or `profile_dir` source;
|
||||
3. Weatherreporter's embedded fallback profiles; and
|
||||
4. Promptkit's built-in catalog.
|
||||
|
||||
Selection remains a separate concern: a nonblank global `promptkit.profile`
|
||||
selects the profile for every report in the invocation; otherwise the exact
|
||||
prompt definition's `default_profile` selects it. A malformed matching
|
||||
higher-precedence profile is an error and never falls through.
|
||||
|
||||
## Continuing Invariants
|
||||
|
||||
- Keep all Promptkit types and mechanics inside
|
||||
`internal/adapters/promptkit`, its focused tests, and asset contract tests.
|
||||
- Keep prompt inspection before weather collection and provider work.
|
||||
- Keep one Promptkit engine per command action and one shared engine across a
|
||||
sequential batch.
|
||||
- Preserve logical profile ID and effective backend/model information through
|
||||
active inspection and execution where the project-owned contract already
|
||||
exposes it. Do not add new durable-provenance fields or compatibility
|
||||
guarantees.
|
||||
- Leave existing workspace persistence behavior otherwise unchanged. The
|
||||
accepted [ephemeral-state roadmap](ephemeral-state.md) owns its future
|
||||
removal and must not be partially implemented here.
|
||||
- Do not expose endpoints, credentials, rendered messages, schemas, request
|
||||
bodies, response bodies, or complete parameter maps through ordinary errors,
|
||||
logs, summaries, or state.
|
||||
- Keep the default suite deterministic, offline, and credential-free.
|
||||
- Do not add endpoint discovery, health probing, provider failover, retries at
|
||||
a more expensive tier, profile merging, per-report configuration fields, or
|
||||
severity-driven model selection.
|
||||
- Update canonical current-state documentation only in the stage where the
|
||||
corresponding behavior becomes implemented.
|
||||
- Run `git diff --check` before completing every stage.
|
||||
|
||||
## Stage 1: Add The Embedded Weather Profile Catalog
|
||||
|
||||
### Goal
|
||||
|
||||
Create one repository-owned, embedded profile source containing exactly the
|
||||
three locked logical profiles.
|
||||
|
||||
### Work
|
||||
|
||||
1. Add strict YAML profile assets beneath `internal/promptassets` using the
|
||||
exact IDs and definitions in this plan.
|
||||
2. Extend `internal/promptassets` with a narrowly named accessor that returns
|
||||
the embedded profile `fs.FS`. Follow the existing prompt and schema asset
|
||||
pattern without exposing Promptkit types from the package.
|
||||
3. Keep profile filenames and embed layout simple and deterministic. Do not
|
||||
duplicate Promptkit's built-in directory taxonomy unless the application
|
||||
assets require it.
|
||||
4. Validate the assets through Promptkit's public engine/profile inspection
|
||||
surface rather than adding a second YAML parser or a Weatherreporter-owned
|
||||
profile representation.
|
||||
|
||||
### Tests
|
||||
|
||||
- Extend the asset contract tests to assert exactly the three logical IDs,
|
||||
their exact effective model IDs, and the intentional parameters.
|
||||
- Prove all three profiles inspect successfully offline when supplied as a
|
||||
fallback source and no operator source is present.
|
||||
- Assert that the catalog contains no endpoints, credentials, temperature,
|
||||
`top_p`, or maximum-token settings.
|
||||
- Run:
|
||||
|
||||
```sh
|
||||
go test ./internal/promptassets
|
||||
git diff --check
|
||||
```
|
||||
|
||||
### Exit Gate
|
||||
|
||||
The embedded catalog is complete, strictly valid, safe, and independently
|
||||
inspectable through Promptkit v0.5.0's public API.
|
||||
|
||||
## Stage 2: Wire Fallback Resolution And Protect Precedence
|
||||
|
||||
### Goal
|
||||
|
||||
Supply the embedded catalog through Promptkit's application fallback layer
|
||||
without changing existing operator configuration or application boundaries.
|
||||
|
||||
### Work
|
||||
|
||||
1. Add `promptkit.WithFallbackProfileFS(promptassets.ProfileFS(), ".")` to
|
||||
normal adapter engine construction.
|
||||
2. Preserve existing `profile_file`, `profile_dir`, configured local backend,
|
||||
timeout, prompt filesystem, schema filesystem, and test-option behavior.
|
||||
3. Ensure ordinary production construction and the adapter's test
|
||||
construction path exercise the same fallback wiring. Test-only explicit
|
||||
profiles may retain Promptkit's documented highest precedence.
|
||||
4. Keep all fallback resolution in Promptkit. Do not add filesystem overlays,
|
||||
existence checks, YAML parsing, or merge behavior to Weatherreporter.
|
||||
|
||||
### Tests
|
||||
|
||||
- At the adapter boundary, prove fallback-only inspection of all three
|
||||
Weatherreporter profiles.
|
||||
- Prove same-ID overrides through both configured `profile_file` and
|
||||
`profile_dir`, including resolution of the override's effective backend and
|
||||
model.
|
||||
- Prove an absent operator match falls through, while a malformed matching
|
||||
operator definition fails without using the embedded profile.
|
||||
- Prove a selected Promptkit built-in that is absent from both higher layers
|
||||
still resolves.
|
||||
- Prove an explicit in-memory test profile retains highest precedence.
|
||||
- Cover both local override forms required by the roadmap: an endpoint-only
|
||||
OpenAI-compatible `weather-light` profile and a `backend: local` profile
|
||||
using the configured local endpoint. No test may contact either endpoint.
|
||||
- Run:
|
||||
|
||||
```sh
|
||||
go test ./internal/adapters/promptkit
|
||||
git diff --check
|
||||
```
|
||||
|
||||
### Exit Gate
|
||||
|
||||
Inspection and prepared execution use Promptkit's exact four-layer precedence,
|
||||
operator errors remain visible, and local overrides require no prompt or code
|
||||
changes.
|
||||
|
||||
## Stage 3: Adopt Logical Defaults And Prompt Version 1.1.0
|
||||
|
||||
### Goal
|
||||
|
||||
Move operational prompts from provider-oriented defaults to the three-tier
|
||||
Weatherreporter policy with an exact, synchronized version transition.
|
||||
|
||||
### Work
|
||||
|
||||
1. Change Hourly's `default_profile` to `weather-light`.
|
||||
2. Change Daily, Today, and Tomorrow to `weather-balanced`.
|
||||
3. Advance the exact version in all four prompt YAML assets from `1.0.1` to
|
||||
`1.1.0` without changing prompt text or generated-text schemas solely for
|
||||
this feature.
|
||||
4. Advance the four matching report-registry prompt versions to `1.1.0` in the
|
||||
same change. Keep prompt IDs, report IDs, modules, periods, templates, and
|
||||
output contracts unchanged.
|
||||
5. Update fixtures and expectations that intentionally assert the current
|
||||
prompt contract. Do not rewrite historical fixture versions or weaken tests
|
||||
that protect actual compatibility.
|
||||
|
||||
### Tests
|
||||
|
||||
- Update asset and report-registry contract tests to require exact version
|
||||
`1.1.0` and the report-to-profile assignments locked in this plan.
|
||||
- Inspect every exact prompt version through the real embedded prompt, schema,
|
||||
and fallback-profile filesystems.
|
||||
- Prove Hourly resolves DeepSeek V4 Flash, the three day-scale reports resolve
|
||||
Gemini Flash Latest, and `weather-deep` remains inspectable but unassigned.
|
||||
- Run:
|
||||
|
||||
```sh
|
||||
go test ./internal/promptassets ./internal/report ./internal/adapters/promptkit
|
||||
git diff --check
|
||||
```
|
||||
|
||||
### Exit Gate
|
||||
|
||||
Every operational prompt and registry definition agrees on exact version
|
||||
`1.1.0`, selects its intended logical tier, and resolves its expected effective
|
||||
model offline.
|
||||
|
||||
## Stage 4: Verify Application Selection And Batch Reuse
|
||||
|
||||
### Goal
|
||||
|
||||
Protect the assembled application behavior created by the new defaults and
|
||||
confirm that logical identity is not lost during active effective-model
|
||||
resolution.
|
||||
|
||||
### Work
|
||||
|
||||
1. Preserve the current pre-collection inspection order and fail-fast behavior
|
||||
for missing credentials, unknown profiles, malformed profiles, and unusable
|
||||
backends.
|
||||
2. Preserve the global `promptkit.profile` all-report override. Do not add a
|
||||
second override mechanism or report-specific configuration fields.
|
||||
3. Preserve batch preflight deduplication by selected effective profile ID:
|
||||
Today and Tomorrow in the same batch should inspect their shared
|
||||
`weather-balanced` selection once.
|
||||
4. Preserve the selected logical profile ID and resolved backend/model through
|
||||
active inspection, preparation, and execution using the existing
|
||||
project-owned contract. Do not add state fields, expand persisted parameter
|
||||
detail, or create a new historical compatibility guarantee.
|
||||
|
||||
### Tests
|
||||
|
||||
- Add or update representative app tests for default Hourly and day-scale
|
||||
selection, a global-profile override, and a morning/evening batch sharing
|
||||
`weather-balanced`.
|
||||
- Assert inspection completes before weather collection and provider
|
||||
generation, including malformed same-ID operator overrides.
|
||||
- Assert active inspection and execution expose the logical profile ID and
|
||||
effective model for both embedded and overridden profiles.
|
||||
- Assert endpoints and credentials remain absent from errors, summaries,
|
||||
normal logs, and ordinary state.
|
||||
- Use project-owned executor fakes or Promptkit provider fakes; do not make live
|
||||
provider calls.
|
||||
- Run:
|
||||
|
||||
```sh
|
||||
go test ./internal/app ./internal/cli
|
||||
git diff --check
|
||||
```
|
||||
|
||||
### Exit Gate
|
||||
|
||||
Single-report and batch workflows select the intended tier, retain existing
|
||||
override and preflight behavior, deduplicate shared batch inspection, and
|
||||
preserve safe logical and effective model information during active execution
|
||||
without adding a durable-provenance contract.
|
||||
|
||||
## Stage 5: Publish Canonical Operator And Maintainer Documentation
|
||||
|
||||
### Goal
|
||||
|
||||
Document the implemented feature once in each canonical owner and provide one
|
||||
maintained, copyable local override example.
|
||||
|
||||
### Work
|
||||
|
||||
1. Update `docs/config.md` to explain global profile selection versus
|
||||
`profile_file`/`profile_dir` definition lookup and link to the maintained
|
||||
example. Keep the field reference in this canonical document.
|
||||
2. Update the Promptkit integration document with the logical profile catalog,
|
||||
source precedence, exact prompt-version relationship, and safe active
|
||||
inspection and execution contract. Avoid restating complete configuration
|
||||
syntax or presenting transitional persistence as the target architecture.
|
||||
3. Update the report-registry, Promptkit adapter, app-orchestration, and state
|
||||
internal documents only where their implemented contracts changed.
|
||||
4. Update `docs/operations.md` with the normal local-override workflow and
|
||||
`docs/troubleshooting.md` with malformed override, unavailable local
|
||||
endpoint, missing credential, and unexpected effective-model diagnostics.
|
||||
5. Add or update one secret-free file under `examples/` showing a
|
||||
`weather-light` override for a local OpenAI-compatible endpoint. Choose one
|
||||
supported form as the complete example and mention the other form only in
|
||||
its canonical reference.
|
||||
6. Update the architecture policy only if implementation changed a normative
|
||||
boundary or invariant. Do not add future behavior to current-state docs.
|
||||
7. Keep the feature roadmap and this plan in their pre-implementation statuses
|
||||
until the final repository gate passes. Do not create release notes before
|
||||
a release version is chosen.
|
||||
|
||||
### Tests
|
||||
|
||||
- Verify every changed repository-relative link and every profile/model ID.
|
||||
- Validate maintained YAML examples through the same strict configuration or
|
||||
Promptkit profile path used by production where practical.
|
||||
- Run the focused tests that own any executable examples, followed by:
|
||||
|
||||
```sh
|
||||
git diff --check
|
||||
```
|
||||
|
||||
### Exit Gate
|
||||
|
||||
Users, operators, and maintainers can discover the tier defaults, precedence,
|
||||
global override, local override, and failure behavior without duplicated or
|
||||
future-state documentation.
|
||||
|
||||
## Stage 6: Complete Repository Verification And Roadmap Handoff
|
||||
|
||||
### Goal
|
||||
|
||||
Demonstrate that the complete feature is coherent, offline-testable, and ready
|
||||
for review and a later release decision.
|
||||
|
||||
### Work
|
||||
|
||||
1. Review the complete diff against the roadmap, this plan, and all three
|
||||
policy documents. Remove stale identifiers, temporary helpers, redundant
|
||||
tests, and documentation duplication.
|
||||
2. Confirm `go.mod` and `go.sum` retain tagged Promptkit v0.5.0 without a local
|
||||
replacement or dependency drift.
|
||||
3. Confirm only the four supported report products exist and no retired report
|
||||
surfaces were reintroduced.
|
||||
4. Confirm the roadmap's completion criteria one by one. Change its status to
|
||||
implemented and this plan's status to completed only after every criterion
|
||||
and command below passes.
|
||||
5. Do not require a live provider for completion. If credentials and network
|
||||
access are deliberately supplied by an operator, record live smoke results
|
||||
separately as release-candidate evidence rather than adding them to the
|
||||
default suite.
|
||||
|
||||
### Verification
|
||||
|
||||
Run `gofmt -w` on every changed Go file, then run:
|
||||
|
||||
```sh
|
||||
go test ./...
|
||||
go test -race ./...
|
||||
go run ./cmd/weatherreporter --help
|
||||
git diff --check
|
||||
git status --short
|
||||
```
|
||||
|
||||
Also inspect all three logical profiles through the application's normal
|
||||
preflight path using offline provider doubles, including one same-ID local
|
||||
override and one explicit global override.
|
||||
|
||||
### Exit Gate
|
||||
|
||||
All roadmap completion criteria are satisfied, all verification commands pass,
|
||||
the working tree contains only intentional changes, and the canonical
|
||||
documentation describes the implemented state. The feature is ready for code
|
||||
review and release preparation.
|
||||
|
||||
## Open Questions
|
||||
|
||||
None. The model identifiers, profile settings, report assignments, version
|
||||
transition, precedence, compatibility behavior, test boundaries, and
|
||||
documentation ownership are decision-complete.
|
||||
@@ -1,555 +1,373 @@
|
||||
# Promptkit Migration Implementation Plan
|
||||
# Domain-Specific Prompt Profiles Implementation Plan
|
||||
|
||||
Status: Completed; Stages 1–19 passed their exit gates.
|
||||
Status: Ready for implementation.
|
||||
|
||||
## Purpose And Authority
|
||||
|
||||
This document records the completed implementation of the
|
||||
[Promptkit migration roadmap](promptkit.md) and its post-implementation audit
|
||||
remediation. The feature roadmap records scope, user intent, policy choices,
|
||||
and the implemented end state. This plan records implementation sequence,
|
||||
tests, and completion gates.
|
||||
This document gives a coding agent the ordered work needed to implement the
|
||||
[domain-specific prompt profiles roadmap](domain-profiles.md). The roadmap is
|
||||
authoritative for scope, user intent, policy choices, and the desired end
|
||||
state. This plan is authoritative for implementation sequence, verification,
|
||||
and stage exit gates.
|
||||
|
||||
Stages 12–19 were completed in order. They fixed additional defects exposed by
|
||||
their required tests only when those defects were within the same stated
|
||||
contract; they did not add new product behavior or reinterpret roadmap
|
||||
decisions.
|
||||
The target implementer is a `gpt-5.6-terra` coding agent using high reasoning.
|
||||
Complete the stages in order. Each stage is scoped for one implementation
|
||||
prompt and must leave the repository coherent, tested, and reviewable before
|
||||
the next stage begins.
|
||||
|
||||
This plan follows the repository's
|
||||
[architecture](../policy/architecture.md),
|
||||
[documentation](../policy/documentation.md), and
|
||||
[testing](../policy/testing.md) policies.
|
||||
|
||||
## Completed Prerequisite
|
||||
|
||||
Weatherreporter is already pinned to Promptkit v0.5.0. That release provides
|
||||
the public `WithFallbackProfileFS` option and the required precedence across
|
||||
inspection, preparation, and execution. The dependency upgrade passed
|
||||
Weatherreporter's full offline test suite, race-enabled suite, CLI help check,
|
||||
and an operator `generate hourly` smoke test. Do not repeat or replace the
|
||||
dependency upgrade as part of these stages.
|
||||
|
||||
## Locked Product Decisions
|
||||
|
||||
Implement these exact Weatherreporter-owned profiles:
|
||||
|
||||
| Profile ID | Backend | Model | Reasoning effort | Timeout | Service tier |
|
||||
| --- | --- | --- | --- | --- | --- |
|
||||
| `weather-light` | `openrouter` | `deepseek/deepseek-v4-flash` | Omitted | 180 seconds | `flex` |
|
||||
| `weather-balanced` | `openrouter` | `~google/gemini-flash-latest` | `high` | 240 seconds | `flex` |
|
||||
| `weather-deep` | `openrouter` | `~anthropic/claude-sonnet-latest` | `high` | 240 seconds | `flex` |
|
||||
|
||||
Assign Hourly to `weather-light`; assign Daily, Today, and Tomorrow to
|
||||
`weather-balanced`; assign no report to `weather-deep` initially. Advance all
|
||||
four prompt definitions and matching report-registry entries from `1.0.1` to
|
||||
`1.1.0` when their defaults change.
|
||||
|
||||
The leading `~` in the Gemini and Claude model IDs is required and denotes an
|
||||
OpenRouter rolling alias. Do not substitute the unavailable non-tilde IDs or a
|
||||
dated model version. Do not add temperature, `top_p`, maximum-token, endpoint,
|
||||
or credential fields to the embedded definitions.
|
||||
|
||||
Prompt preparation and execution artifacts written at `1.0.1` are not required
|
||||
to remain readable after the transition to `1.1.0`. Do not add a migration,
|
||||
compatibility shim, or weaker historical-artifact validation for this feature.
|
||||
|
||||
Definition lookup must remain:
|
||||
|
||||
1. explicit Promptkit in-memory profiles used by tests or an embedding
|
||||
consumer;
|
||||
2. Weatherreporter's configured `profile_file` or `profile_dir` source;
|
||||
3. Weatherreporter's embedded fallback profiles; and
|
||||
4. Promptkit's built-in catalog.
|
||||
|
||||
Selection remains a separate concern: a nonblank global `promptkit.profile`
|
||||
selects the profile for every report in the invocation; otherwise the exact
|
||||
prompt definition's `default_profile` selects it. A malformed matching
|
||||
higher-precedence profile is an error and never falls through.
|
||||
|
||||
## Continuing Invariants
|
||||
|
||||
- Keep `gitea.maximumdirect.net/eric/promptkit` pinned at exactly `v0.4.0`.
|
||||
- Keep Promptkit types inside `internal/adapters/promptkit`, its tests, and the
|
||||
external prompt-asset contract test.
|
||||
- Preserve one Promptkit engine per `generate` or `run` invocation and one
|
||||
shared engine for every sequential report in a batch.
|
||||
- Preserve exact prompt version `1.0.0`, the exact persisted YAML data-package
|
||||
bytes, prepared execution, and preparation persistence before provider work.
|
||||
- Do not add retries, repair attempts, concurrent batch generation, direct
|
||||
Markdown generation, arbitrary backend registration, or live-provider
|
||||
tests.
|
||||
- Keep ordinary artifacts, errors, logs, and summaries free of credentials,
|
||||
rendered messages, schemas, input bodies, generated bodies, endpoints, and
|
||||
full effective parameter maps.
|
||||
- Keep sensitive debug artifacts opt-in, outside normal state, owner-only,
|
||||
atomic, and free of credentials.
|
||||
- Treat an artifact path as reached only after the corresponding write or copy
|
||||
succeeds. Never persist or summarize a merely derivable future path.
|
||||
- Preserve every safe reached path in partial app and CLI results even when a
|
||||
later persistence, validation, rendering, copy, or notification step fails.
|
||||
- Keep v1 metadata read compatibility and write only v2 metadata for new runs.
|
||||
- Keep the default test suite deterministic, offline, and credential-free.
|
||||
- Run `git diff --check` before completing every stage. Run the full repository
|
||||
gate in Stage 19.
|
||||
- Keep all Promptkit types and mechanics inside
|
||||
`internal/adapters/promptkit`, its focused tests, and asset contract tests.
|
||||
- Keep prompt inspection before weather collection and provider work.
|
||||
- Keep one Promptkit engine per command action and one shared engine across a
|
||||
sequential batch.
|
||||
- Preserve logical profile ID and effective backend/model information through
|
||||
active inspection and execution where the project-owned contract already
|
||||
exposes it. Do not add new durable-provenance fields or compatibility
|
||||
guarantees.
|
||||
- Leave existing workspace persistence behavior otherwise unchanged. The
|
||||
accepted [ephemeral-state roadmap](ephemeral-state.md) owns its future
|
||||
removal and must not be partially implemented here.
|
||||
- Do not expose endpoints, credentials, rendered messages, schemas, request
|
||||
bodies, response bodies, or complete parameter maps through ordinary errors,
|
||||
logs, summaries, or state.
|
||||
- Keep the default suite deterministic, offline, and credential-free.
|
||||
- Do not add endpoint discovery, health probing, provider failover, retries at
|
||||
a more expensive tier, profile merging, per-report configuration fields, or
|
||||
severity-driven model selection.
|
||||
- Update canonical current-state documentation only in the stage where the
|
||||
corresponding behavior becomes implemented.
|
||||
- Run `git diff --check` before completing every stage.
|
||||
|
||||
## Completed Migration Summary
|
||||
|
||||
Stages 1–11 are implemented and committed. They remain summarized here to
|
||||
preserve the history and dependencies of the follow-up work.
|
||||
|
||||
| Stage | Completed outcome |
|
||||
| --- | --- |
|
||||
| 1 | Removed the unfinished three-day, weekend, and storm product surfaces and retained Daily, Today, Tomorrow, and Hourly with exact prompt version `1.0.0`. |
|
||||
| 2 | Promoted the four operational prompts and canonical schemas into the embedded `internal/promptassets` source used by Promptkit and generated-text validation. |
|
||||
| 3 | Added the project-owned `internal/promptexec` inspection, preparation, execution, validation, debug, and error contract. |
|
||||
| 4 | Added the Promptkit v0.4.0 adapter with prepared execution, explicit value mapping, safe error classification, and offline model-client tests. |
|
||||
| 5 | Added Promptkit-era preparation and execution artifacts, metadata v2, new paths, and v1 decoding support. |
|
||||
| 6 | Added explicitly rooted, permission-restricted, atomic LLM debug persistence. |
|
||||
| 7 | Added Promptkit configuration, executor composition, and pre-collection prompt/profile/credential inspection. |
|
||||
| 8 | Cut single-report generation over to prepared Promptkit execution and v2 persistence. |
|
||||
| 9 | Added `--llm-debug-dir` and Promptkit-era single-report summary fields. |
|
||||
| 10 | Cut morning and evening batches over to one shared Promptkit executor and removed Scriptorium code, configuration, and dependency metadata. |
|
||||
| 11 | Updated canonical Promptkit documentation, removed the temporary Scriptorium corpus, and ran the available repository checks. |
|
||||
|
||||
The post-implementation audit confirmed the principal dependency and package
|
||||
boundaries, but found incorrect reached-path bookkeeping, incomplete execution
|
||||
artifact updates, insufficient artifact validation, extensive loss of
|
||||
behavioral tests during the final cutover, and roadmap lifecycle text that was
|
||||
not finalized. The completed remediation addressed those findings without
|
||||
changing the intended feature scope.
|
||||
|
||||
| Stage | Completed outcome |
|
||||
| --- | --- |
|
||||
| 12 | Corrected reached-path bookkeeping across metadata, app results, batch items, and CLI summaries. |
|
||||
| 13 | Hardened Promptkit-era durable state validation and restored v1/v2 state coverage. |
|
||||
| 14 | Recorded every downstream path reached after completed prompt execution. |
|
||||
| 15 | Restored assembled single-report behavioral and failure coverage. |
|
||||
| 16 | Simplified prompt-generation orchestration while preserving behavior. |
|
||||
| 17 | Restored assembled batch, planning, artifact, and notification coverage. |
|
||||
| 18 | Restored supported CLI, summary, safety, and historical inspection coverage. |
|
||||
| 19 | Reconciled canonical documentation and passed the complete repository verification gate. |
|
||||
|
||||
## Stage 12: Correct Reached-Artifact Bookkeeping
|
||||
|
||||
Status: Completed.
|
||||
## Stage 1: Add The Embedded Weather Profile Catalog
|
||||
|
||||
### Goal
|
||||
|
||||
Make metadata, app results, batch items, and CLI summaries truthful at every
|
||||
failure boundary: a nonblank path means that artifact was successfully
|
||||
created.
|
||||
Create one repository-owned, embedded profile source containing exactly the
|
||||
three locked logical profiles.
|
||||
|
||||
### Work
|
||||
|
||||
1. Change `state.BuildPromptMetadataFromBriefingMetadata` so it initializes
|
||||
identity, schema, metadata destination, and only artifacts already saved at
|
||||
the call site. It must not prepopulate raw-output, normalized-text,
|
||||
render-context, managed-report, preparation, execution, notification, or
|
||||
output-copy paths.
|
||||
2. In `generatePromptReport`, assign each metadata and `ReportResult` path
|
||||
immediately after that artifact write succeeds and before attempting the
|
||||
next write. In particular:
|
||||
|
||||
- do not initialize `ReportResult.ReportPath` from `Store.Paths`;
|
||||
- record a saved failed-preparation receipt in the result before saving
|
||||
metadata;
|
||||
- record a saved failed or completed execution receipt before saving
|
||||
metadata;
|
||||
- retain raw, normalized, context, report, copy, and notification paths
|
||||
when a later step fails; and
|
||||
- keep `MetadataPath` unchanged when a metadata rewrite fails, because the
|
||||
prior successfully written metadata record remains the reached version.
|
||||
|
||||
3. Remove batch-item prepopulation from derived `Store.Paths` values.
|
||||
`BatchReportResult` receives paths only from the returned `ReportResult` or
|
||||
from a write that the batch itself successfully completed.
|
||||
4. Preserve current CLI field names and omission behavior. Human and JSON
|
||||
summaries must omit every unreached path.
|
||||
5. Do not change artifact locations, filenames, schemas, report output, or
|
||||
notification policy in this stage.
|
||||
1. Add strict YAML profile assets beneath `internal/promptassets` using the
|
||||
exact IDs and definitions in this plan.
|
||||
2. Extend `internal/promptassets` with a narrowly named accessor that returns
|
||||
the embedded profile `fs.FS`. Follow the existing prompt and schema asset
|
||||
pattern without exposing Promptkit types from the package.
|
||||
3. Keep profile filenames and embed layout simple and deterministic. Do not
|
||||
duplicate Promptkit's built-in directory taxonomy unless the application
|
||||
assets require it.
|
||||
4. Validate the assets through Promptkit's public engine/profile inspection
|
||||
surface rather than adding a second YAML parser or a Weatherreporter-owned
|
||||
profile representation.
|
||||
|
||||
### Tests
|
||||
|
||||
- Add focused app tests for one representative report using real temporary
|
||||
state plus a narrow failure-injecting store wrapper.
|
||||
- Fail the next persistence step immediately after a successful preparation
|
||||
receipt, execution receipt, raw output, normalized output, render context,
|
||||
managed report, output copy, and notification artifact; assert that the
|
||||
returned result contains every reached path and no future path.
|
||||
- Include one preparation failure, one operational execution failure, and one
|
||||
completed validation rejection to cover the three execution outcome shapes.
|
||||
- Add batch and CLI summary assertions proving unreached paths are omitted.
|
||||
- Extend the asset contract tests to assert exactly the three logical IDs,
|
||||
their exact effective model IDs, and the intentional parameters.
|
||||
- Prove all three profiles inspect successfully offline when supplied as a
|
||||
fallback source and no operator source is present.
|
||||
- Assert that the catalog contains no endpoints, credentials, temperature,
|
||||
`top_p`, or maximum-token settings.
|
||||
- Run:
|
||||
|
||||
```sh
|
||||
go test ./internal/state ./internal/app ./internal/cli
|
||||
go test ./internal/promptassets
|
||||
git diff --check
|
||||
```
|
||||
|
||||
### Exit Gate
|
||||
|
||||
Every nonblank path in newly written metadata, app results, batch items, and
|
||||
CLI summaries names an artifact that exists. Every safe artifact successfully
|
||||
written before a later failure remains discoverable from the returned partial
|
||||
result.
|
||||
The embedded catalog is complete, strictly valid, safe, and independently
|
||||
inspectable through Promptkit v0.5.0's public API.
|
||||
|
||||
## Stage 13: Harden Durable State Contracts And Restore State Coverage
|
||||
|
||||
Status: Completed.
|
||||
## Stage 2: Wire Fallback Resolution And Protect Precedence
|
||||
|
||||
### Goal
|
||||
|
||||
Make the v1/v2 wire boundary and Promptkit-era artifact validation explicit,
|
||||
strict, and durably tested.
|
||||
Supply the embedded catalog through Promptkit's application fallback layer
|
||||
without changing existing operator configuration or application boundaries.
|
||||
|
||||
### Work
|
||||
|
||||
1. Strengthen `PromptPreparationArtifact.Validate`:
|
||||
|
||||
- require report ID, Weatherreporter RunID, prompt ID, exact prompt version,
|
||||
data-package path, nonzero start/end times, nonnegative duration, and an
|
||||
end not earlier than the start;
|
||||
- for success, require preparation provenance, prohibit an error, and
|
||||
require its prompt ID/version and data-package path to match the top-level
|
||||
artifact;
|
||||
- for failure, require a classified bounded error and prohibit fabricated
|
||||
preparation provenance.
|
||||
|
||||
2. Strengthen `PromptExecutionArtifact.Validate`:
|
||||
|
||||
- require report ID, Weatherreporter RunID, prompt ID, exact prompt version,
|
||||
nonzero start/end times, nonnegative duration, and an end not earlier than
|
||||
the start;
|
||||
- for success and validation rejection, require provenance and completed
|
||||
validation, prohibit an operational error, and require the provenance
|
||||
prompt ID/version to match the artifact;
|
||||
- do not compare the provenance RunID with the Weatherreporter RunID because
|
||||
the provenance value is Promptkit's run identity;
|
||||
- for operational failure, require a classified bounded error and prohibit
|
||||
invented provenance or completed validation.
|
||||
|
||||
3. Validate required provenance fields for completed executions, including
|
||||
Promptkit RunID, prompt and rendered hashes, selected profile/backend/model,
|
||||
and data-package path. Permit usage counters and generated hash to be zero
|
||||
when the provider legitimately reports no value.
|
||||
4. Restore focused filesystem and metadata tests for:
|
||||
|
||||
- exact v2 paths and filenames;
|
||||
- preparation/execution round trips and required fields;
|
||||
- metadata v2 round trips without legacy aliases;
|
||||
- v1 decoding, normalized internal aliases, and v1-preserving re-marshaling;
|
||||
- unknown schema rejection;
|
||||
- report listing, RunID lookup, source/module/data-package inspection, and
|
||||
retained v1 behavior for historical report IDs;
|
||||
- atomic writes and unsafe workspace/path rejection; and
|
||||
- prior-snapshot behavior for the four supported report IDs.
|
||||
|
||||
5. Adapt useful tests from the deleted filesystem suite rather than recreating
|
||||
redundant low-value cases. Do not restore Scriptorium writes or retired
|
||||
report behavior.
|
||||
1. Add `promptkit.WithFallbackProfileFS(promptassets.ProfileFS(), ".")` to
|
||||
normal adapter engine construction.
|
||||
2. Preserve existing `profile_file`, `profile_dir`, configured local backend,
|
||||
timeout, prompt filesystem, schema filesystem, and test-option behavior.
|
||||
3. Ensure ordinary production construction and the adapter's test
|
||||
construction path exercise the same fallback wiring. Test-only explicit
|
||||
profiles may retain Promptkit's documented highest precedence.
|
||||
4. Keep all fallback resolution in Promptkit. Do not add filesystem overlays,
|
||||
existence checks, YAML parsing, or merge behavior to Weatherreporter.
|
||||
|
||||
### Tests
|
||||
|
||||
Run:
|
||||
|
||||
```sh
|
||||
go test ./internal/state ./internal/app
|
||||
git diff --check
|
||||
```
|
||||
|
||||
### Exit Gate
|
||||
|
||||
The state package rejects incomplete or contradictory Promptkit-era artifacts,
|
||||
reads historical v1 records, writes only valid v2 records, and has focused
|
||||
offline coverage for its durable compatibility and filesystem contracts.
|
||||
|
||||
## Stage 14: Complete Execution-Artifact Path Tracking
|
||||
|
||||
Status: Completed.
|
||||
|
||||
### Goal
|
||||
|
||||
Make `PromptExecutionArtifact.Paths` accurately record every downstream
|
||||
artifact reached after a completed Promptkit run.
|
||||
|
||||
### Work
|
||||
|
||||
1. Treat the execution artifact as an atomically updated durable record of the
|
||||
completed Promptkit execution and subsequent artifact destinations. Its
|
||||
status, provenance, validation, usage, and timing remain the provider-run
|
||||
outcome; later application failures do not change a successful Promptkit
|
||||
status into an execution failure.
|
||||
2. Save the initial execution artifact after raw output is persisted, with
|
||||
`RawOutputPath` populated.
|
||||
3. After each later successful write, update and atomically resave the same
|
||||
execution artifact with the corresponding reached path:
|
||||
|
||||
- normalized generated text;
|
||||
- render context;
|
||||
- managed Markdown report;
|
||||
- an explicitly requested extra output copy, only after the copy succeeds;
|
||||
and
|
||||
- a Distributor notification artifact, including a persisted failure or
|
||||
status artifact when notification produced one.
|
||||
|
||||
4. Keep metadata and execution-artifact path values consistent after every
|
||||
successful checkpoint. Save the execution artifact before metadata so a
|
||||
metadata failure does not erase knowledge of a reached downstream artifact.
|
||||
Failure to update the execution artifact is terminal and returns a partial
|
||||
result containing the downstream artifact that was already written.
|
||||
5. Refactor finalization return values only as needed to tell the orchestration
|
||||
layer which copy and notification paths were actually written. Distributor
|
||||
must continue uploading only the managed Markdown report.
|
||||
6. A validation-rejected execution ends after raw output and therefore records
|
||||
only the raw-output path. An operational execution failure has no completed
|
||||
provenance and records only safe paths reached before that failure.
|
||||
|
||||
### Tests
|
||||
|
||||
- Add table-driven execution-artifact lifecycle tests for success and every
|
||||
downstream failure point.
|
||||
- Load the persisted execution artifact after normalized-text, context,
|
||||
template, copy, metadata, and notification failures and assert its status and
|
||||
exact reached paths.
|
||||
- Assert that execution artifacts never contain generated bodies, rendered
|
||||
prompts, schemas, endpoints, parameters, or credentials.
|
||||
- At the adapter boundary, prove fallback-only inspection of all three
|
||||
Weatherreporter profiles.
|
||||
- Prove same-ID overrides through both configured `profile_file` and
|
||||
`profile_dir`, including resolution of the override's effective backend and
|
||||
model.
|
||||
- Prove an absent operator match falls through, while a malformed matching
|
||||
operator definition fails without using the embedded profile.
|
||||
- Prove a selected Promptkit built-in that is absent from both higher layers
|
||||
still resolves.
|
||||
- Prove an explicit in-memory test profile retains highest precedence.
|
||||
- Cover both local override forms required by the roadmap: an endpoint-only
|
||||
OpenAI-compatible `weather-light` profile and a `backend: local` profile
|
||||
using the configured local endpoint. No test may contact either endpoint.
|
||||
- Run:
|
||||
|
||||
```sh
|
||||
go test ./internal/state ./internal/app
|
||||
go test ./internal/adapters/promptkit
|
||||
git diff --check
|
||||
```
|
||||
|
||||
### Exit Gate
|
||||
|
||||
For every completed Promptkit run, its execution artifact contains exactly the
|
||||
safe downstream paths reached by the workflow and remains semantically correct
|
||||
when a later application stage fails.
|
||||
Inspection and prepared execution use Promptkit's exact four-layer precedence,
|
||||
operator errors remain visible, and local overrides require no prompt or code
|
||||
changes.
|
||||
|
||||
## Stage 15: Restore Single-Report Behavioral Coverage
|
||||
|
||||
Status: Completed.
|
||||
## Stage 3: Adopt Logical Defaults And Prompt Version 1.1.0
|
||||
|
||||
### Goal
|
||||
|
||||
Restore the risk-based application coverage removed during final cutover and
|
||||
prove the complete single-report Promptkit workflow through project-owned
|
||||
boundaries.
|
||||
Move operational prompts from provider-oriented defaults to the three-tier
|
||||
Weatherreporter policy with an exact, synchronized version transition.
|
||||
|
||||
### Work
|
||||
|
||||
1. Reintroduce a focused app test harness using real state, prompt-input,
|
||||
generated-text validation, render contexts, and templates with deterministic
|
||||
collector, executor, notifier, clock, and filesystem boundaries.
|
||||
2. Add representative successful workflows for Daily, Today, Tomorrow, and
|
||||
Hourly. Verify report identity, exact prompt version, one collection, exact
|
||||
persisted YAML bytes passed to the executor, expected template output,
|
||||
optional copy behavior, and managed-report notification source.
|
||||
3. Cover the required failure matrix:
|
||||
|
||||
- inspection and missing credentials before collection;
|
||||
- preparation failure and callback persistence failure before provider work;
|
||||
- execution-time credential disappearance;
|
||||
- capacity rejection without retry;
|
||||
- cancellation and deadline;
|
||||
- generation and operational-validation failure;
|
||||
- completed Promptkit schema rejection with retained raw output;
|
||||
- generated-text decode/domain rejection;
|
||||
- render-context and template failure;
|
||||
- output-copy failure; and
|
||||
- notification failure.
|
||||
|
||||
4. Verify preparation persistence precedes provider execution, debug-write
|
||||
failure prevents provider execution, and execution-debug failure preserves
|
||||
previously reached normal and debug artifacts.
|
||||
5. Verify Recent Changes, prior-snapshot selection, output naming, and
|
||||
Distributor template values for all four retained reports.
|
||||
6. Adapt useful tests from the deleted app suite. Omit Scriptorium mechanics,
|
||||
subprocess interaction assertions, and retired report products.
|
||||
7. Fix defects exposed by these tests only when the expected behavior is
|
||||
already decided by the roadmap or canonical policy. Record any new product
|
||||
question instead of silently choosing it.
|
||||
1. Change Hourly's `default_profile` to `weather-light`.
|
||||
2. Change Daily, Today, and Tomorrow to `weather-balanced`.
|
||||
3. Advance the exact version in all four prompt YAML assets from `1.0.1` to
|
||||
`1.1.0` without changing prompt text or generated-text schemas solely for
|
||||
this feature.
|
||||
4. Advance the four matching report-registry prompt versions to `1.1.0` in the
|
||||
same change. Keep prompt IDs, report IDs, modules, periods, templates, and
|
||||
output contracts unchanged.
|
||||
5. Update fixtures and expectations that intentionally assert the current
|
||||
prompt contract. Do not rewrite historical fixture versions or weaken tests
|
||||
that protect actual compatibility.
|
||||
|
||||
### Tests
|
||||
|
||||
Run:
|
||||
- Update asset and report-registry contract tests to require exact version
|
||||
`1.1.0` and the report-to-profile assignments locked in this plan.
|
||||
- Inspect every exact prompt version through the real embedded prompt, schema,
|
||||
and fallback-profile filesystems.
|
||||
- Prove Hourly resolves DeepSeek V4 Flash, the three day-scale reports resolve
|
||||
Gemini Flash Latest, and `weather-deep` remains inspectable but unassigned.
|
||||
- Run:
|
||||
|
||||
```sh
|
||||
go test ./internal/app
|
||||
go test -race ./internal/app ./internal/adapters/promptkit
|
||||
git diff --check
|
||||
```
|
||||
```sh
|
||||
go test ./internal/promptassets ./internal/report ./internal/adapters/promptkit
|
||||
git diff --check
|
||||
```
|
||||
|
||||
### Exit Gate
|
||||
|
||||
The single-report workflow has deterministic behavioral coverage for all four
|
||||
reports, all consequential failure stages, artifact ordering, partial results,
|
||||
debug isolation, output copying, and notification behavior.
|
||||
Every operational prompt and registry definition agrees on exact version
|
||||
`1.1.0`, selects its intended logical tier, and resolves its expected effective
|
||||
model offline.
|
||||
|
||||
## Stage 16: Refactor Prompt Generation Orchestration
|
||||
|
||||
Status: Completed.
|
||||
## Stage 4: Verify Application Selection And Batch Reuse
|
||||
|
||||
### Goal
|
||||
|
||||
Reduce the complexity and duplicated persistence logic in
|
||||
`generatePromptReport` without changing observable behavior.
|
||||
Protect the assembled application behavior created by the new defaults and
|
||||
confirm that logical identity is not lost during active effective-model
|
||||
resolution.
|
||||
|
||||
### Work
|
||||
|
||||
1. Use the Stage 12–15 tests as the refactoring safety boundary. Do not weaken
|
||||
assertions to accommodate structural changes.
|
||||
2. Split the current orchestration into small app-owned operations with clear
|
||||
inputs and outcomes for:
|
||||
|
||||
- deterministic input and initial state construction;
|
||||
- preparation callback persistence;
|
||||
- preparation-failure persistence;
|
||||
- operational-execution-failure persistence;
|
||||
- completed execution and raw-output persistence;
|
||||
- normalized text and render-context persistence;
|
||||
- managed report, optional copy, metadata, and notification finalization;
|
||||
and
|
||||
- reached-path updates shared by success and failure paths.
|
||||
|
||||
3. Keep workflow order visible in one coordinator. Do not introduce a generic
|
||||
workflow engine, hidden retry loop, provider-specific app type, or mutable
|
||||
global state.
|
||||
4. Centralize the repeated rule that a successful artifact write updates the
|
||||
result before any following write can fail.
|
||||
5. Preserve error identities, safe text, atomic writes, exact bytes, debug
|
||||
ordering, partial results, and notification behavior.
|
||||
1. Preserve the current pre-collection inspection order and fail-fast behavior
|
||||
for missing credentials, unknown profiles, malformed profiles, and unusable
|
||||
backends.
|
||||
2. Preserve the global `promptkit.profile` all-report override. Do not add a
|
||||
second override mechanism or report-specific configuration fields.
|
||||
3. Preserve batch preflight deduplication by selected effective profile ID:
|
||||
Today and Tomorrow in the same batch should inspect their shared
|
||||
`weather-balanced` selection once.
|
||||
4. Preserve the selected logical profile ID and resolved backend/model through
|
||||
active inspection, preparation, and execution using the existing
|
||||
project-owned contract. Do not add state fields, expand persisted parameter
|
||||
detail, or create a new historical compatibility guarantee.
|
||||
|
||||
### Tests
|
||||
|
||||
Run:
|
||||
- Add or update representative app tests for default Hourly and day-scale
|
||||
selection, a global-profile override, and a morning/evening batch sharing
|
||||
`weather-balanced`.
|
||||
- Assert inspection completes before weather collection and provider
|
||||
generation, including malformed same-ID operator overrides.
|
||||
- Assert active inspection and execution expose the logical profile ID and
|
||||
effective model for both embedded and overridden profiles.
|
||||
- Assert endpoints and credentials remain absent from errors, summaries,
|
||||
normal logs, and ordinary state.
|
||||
- Use project-owned executor fakes or Promptkit provider fakes; do not make live
|
||||
provider calls.
|
||||
- Run:
|
||||
|
||||
```sh
|
||||
gofmt -w internal/app/*.go
|
||||
go test ./internal/app ./internal/state ./internal/cli
|
||||
go test -race ./internal/app
|
||||
git diff --check
|
||||
```
|
||||
```sh
|
||||
go test ./internal/app ./internal/cli
|
||||
git diff --check
|
||||
```
|
||||
|
||||
### Exit Gate
|
||||
|
||||
The top-level coordinator communicates the workflow order without containing
|
||||
the full persistence implementation, duplicate failure branches are reduced,
|
||||
and every Stage 12–15 behavioral test passes unchanged.
|
||||
Single-report and batch workflows select the intended tier, retain existing
|
||||
override and preflight behavior, deduplicate shared batch inspection, and
|
||||
preserve safe logical and effective model information during active execution
|
||||
without adding a durable-provenance contract.
|
||||
|
||||
## Stage 17: Restore Batch Behavioral Coverage
|
||||
|
||||
Status: Completed.
|
||||
## Stage 5: Publish Canonical Operator And Maintainer Documentation
|
||||
|
||||
### Goal
|
||||
|
||||
Re-establish confidence that morning and evening batches preserve their
|
||||
pre-migration behavior while sharing one Promptkit executor.
|
||||
Document the implemented feature once in each canonical owner and provide one
|
||||
maintained, copyable local override example.
|
||||
|
||||
### Work
|
||||
|
||||
1. Add assembled batch tests proving:
|
||||
|
||||
- one executor factory call and one executor per CLI invocation;
|
||||
- inspection of the full candidate set before collection;
|
||||
- one weather collection;
|
||||
- existing morning/evening planning and ordering;
|
||||
- sequential execution through the shared executor;
|
||||
- continuation after an independent report failure;
|
||||
- no retry after capacity rejection;
|
||||
- distinct identities and debug directories for multiple Daily dates; and
|
||||
- exact reached paths on successful and failed batch items.
|
||||
|
||||
2. Restore notification coverage for disabled notification, suppressed
|
||||
per-report notification, all-success batch notification, skipped
|
||||
notification after report failure, and persisted notification failure/status
|
||||
artifacts.
|
||||
3. Restore output-directory, Today/Tomorrow naming, dynamic Daily planning,
|
||||
prior-snapshot, and managed-Markdown upload-source coverage.
|
||||
4. Adapt useful tests from the deleted batch portions of the app and CLI suites.
|
||||
Do not restore retired report cases or Scriptorium fakes.
|
||||
5. Fix only roadmap-defined batch regressions exposed by the restored tests.
|
||||
1. Update `docs/config.md` to explain global profile selection versus
|
||||
`profile_file`/`profile_dir` definition lookup and link to the maintained
|
||||
example. Keep the field reference in this canonical document.
|
||||
2. Update the Promptkit integration document with the logical profile catalog,
|
||||
source precedence, exact prompt-version relationship, and safe active
|
||||
inspection and execution contract. Avoid restating complete configuration
|
||||
syntax or presenting transitional persistence as the target architecture.
|
||||
3. Update the report-registry, Promptkit adapter, app-orchestration, and state
|
||||
internal documents only where their implemented contracts changed.
|
||||
4. Update `docs/operations.md` with the normal local-override workflow and
|
||||
`docs/troubleshooting.md` with malformed override, unavailable local
|
||||
endpoint, missing credential, and unexpected effective-model diagnostics.
|
||||
5. Add or update one secret-free file under `examples/` showing a
|
||||
`weather-light` override for a local OpenAI-compatible endpoint. Choose one
|
||||
supported form as the complete example and mention the other form only in
|
||||
its canonical reference.
|
||||
6. Update the architecture policy only if implementation changed a normative
|
||||
boundary or invariant. Do not add future behavior to current-state docs.
|
||||
7. Keep the feature roadmap and this plan in their pre-implementation statuses
|
||||
until the final repository gate passes. Do not create release notes before
|
||||
a release version is chosen.
|
||||
|
||||
### Tests
|
||||
|
||||
Run:
|
||||
- Verify every changed repository-relative link and every profile/model ID.
|
||||
- Validate maintained YAML examples through the same strict configuration or
|
||||
Promptkit profile path used by production where practical.
|
||||
- Run the focused tests that own any executable examples, followed by:
|
||||
|
||||
```sh
|
||||
go test ./internal/app ./internal/cli
|
||||
go test -race ./internal/app
|
||||
git diff --check
|
||||
```
|
||||
```sh
|
||||
git diff --check
|
||||
```
|
||||
|
||||
### Exit Gate
|
||||
|
||||
Morning and evening batches are covered as assembled sequential workflows and
|
||||
demonstrably preserve collection, planning, continuation, output, debug,
|
||||
artifact, and notification contracts with one Promptkit executor.
|
||||
Users, operators, and maintainers can discover the tier defaults, precedence,
|
||||
global override, local override, and failure behavior without duplicated or
|
||||
future-state documentation.
|
||||
|
||||
## Stage 18: Restore CLI And Inspection Coverage
|
||||
|
||||
Status: Completed.
|
||||
## Stage 6: Complete Repository Verification And Roadmap Handoff
|
||||
|
||||
### Goal
|
||||
|
||||
Restore the user-facing command, summary, and historical inspection contracts
|
||||
removed with the old root test suite.
|
||||
Demonstrate that the complete feature is coherent, offline-testable, and ready
|
||||
for review and a later release decision.
|
||||
|
||||
### Work
|
||||
|
||||
1. Add parser and resolver tests for all four generate commands, both batch
|
||||
commands, shared flags, report-specific date rules, malformed input,
|
||||
`--llm-debug-dir`, `--quiet`, output paths, and rejection of retired report
|
||||
names.
|
||||
2. Add assembled CLI tests for representative successful single and batch
|
||||
invocations using injected offline boundaries. Verify exactly one executor
|
||||
construction per action.
|
||||
3. Cover pre-run errors with no invented run summary, successful and failed
|
||||
JSON summaries, quiet-mode behavior, safe human status output, partial paths,
|
||||
and omission of absent notification/debug fields.
|
||||
4. Restore inspection tests for report listing and v1/v2 metadata, modules,
|
||||
data packages, prior snapshots, and sources. Include failed v2 runs and v1
|
||||
fixtures using historical report IDs.
|
||||
5. Assert that routine output never contains rendered prompts, schema bodies,
|
||||
data packages, generated bodies, endpoints, full parameters, credentials, or
|
||||
secret-like dependency errors.
|
||||
6. Keep tests at stable CLI/app boundaries; do not restore assertions about
|
||||
private parser formatting or Scriptorium subprocess mechanics.
|
||||
1. Review the complete diff against the roadmap, this plan, and all three
|
||||
policy documents. Remove stale identifiers, temporary helpers, redundant
|
||||
tests, and documentation duplication.
|
||||
2. Confirm `go.mod` and `go.sum` retain tagged Promptkit v0.5.0 without a local
|
||||
replacement or dependency drift.
|
||||
3. Confirm only the four supported report products exist and no retired report
|
||||
surfaces were reintroduced.
|
||||
4. Confirm the roadmap's completion criteria one by one. Change its status to
|
||||
implemented and this plan's status to completed only after every criterion
|
||||
and command below passes.
|
||||
5. Do not require a live provider for completion. If credentials and network
|
||||
access are deliberately supplied by an operator, record live smoke results
|
||||
separately as release-candidate evidence rather than adding them to the
|
||||
default suite.
|
||||
|
||||
### Tests
|
||||
### Verification
|
||||
|
||||
Run:
|
||||
Run `gofmt -w` on every changed Go file, then run:
|
||||
|
||||
```sh
|
||||
go test ./internal/cli ./internal/app ./internal/state
|
||||
go run ./cmd/weatherreporter --help
|
||||
git diff --check
|
||||
```
|
||||
|
||||
### Exit Gate
|
||||
|
||||
The supported CLI surface, summaries, quiet mode, partial failures, executor
|
||||
composition, and v1/v2 inspection behavior have deterministic offline coverage.
|
||||
|
||||
## Stage 19: Finalize Documentation And Repository Verification
|
||||
|
||||
Status: Completed.
|
||||
|
||||
### Goal
|
||||
|
||||
Close the audit remediation, make roadmap lifecycle state truthful, and verify
|
||||
the repository against the complete target contract.
|
||||
|
||||
### Work
|
||||
|
||||
1. Update `docs/roadmap/promptkit.md` from future tense and “unimplemented”
|
||||
statuses to a completed roadmap record. Describe its old seven-report and
|
||||
Scriptorium material explicitly as the pre-migration baseline rather than
|
||||
current behavior.
|
||||
2. Mark Stages 12–19 and this implementation plan complete only after their
|
||||
exit gates pass. Retain the concise completed-stage history unless the
|
||||
documentation policy calls for archival in the same change.
|
||||
3. Review canonical architecture, app, state, CLI, Promptkit integration,
|
||||
operations, troubleshooting, configuration, and testing documentation
|
||||
against the corrected implementation. Update only actual current-state
|
||||
discrepancies; do not duplicate the roadmap.
|
||||
4. Search current-state code, tests, examples, help, and non-roadmap
|
||||
documentation for stale Scriptorium terms, retired reports, old artifact
|
||||
fields, speculative-path descriptions, or claims of missing Promptkit
|
||||
implementation.
|
||||
5. Confirm examples contain no credentials or private infrastructure values
|
||||
and load through config tests.
|
||||
|
||||
### Final Verification
|
||||
|
||||
Run:
|
||||
|
||||
```sh
|
||||
gofmt -w <all changed Go files>
|
||||
go mod tidy
|
||||
go vet ./...
|
||||
go test -count=1 ./...
|
||||
go test ./...
|
||||
go test -race ./...
|
||||
go run ./cmd/weatherreporter --help
|
||||
git diff --check
|
||||
git status --short
|
||||
```
|
||||
|
||||
Then verify explicitly:
|
||||
|
||||
- `go list -m gitea.maximumdirect.net/eric/promptkit` reports `v0.4.0`;
|
||||
- no committed `go.work`, `replace`, secret fixture, or live-provider test
|
||||
exists;
|
||||
- all four prompts inspect at exact version `1.0.0`;
|
||||
- no runtime prompt requests repair attempts;
|
||||
- v1 fixtures remain inspectable and new runs write only v2;
|
||||
- normal artifacts and output contain no sensitive prompt/debug content;
|
||||
- failed-run metadata, execution artifacts, app results, batch items, and CLI
|
||||
summaries contain exactly the paths actually reached;
|
||||
- help exposes only Daily, Today, Tomorrow, Hourly, morning, and evening; and
|
||||
- managed Markdown remains the only Distributor upload source.
|
||||
Also inspect all three logical profiles through the application's normal
|
||||
preflight path using offline provider doubles, including one same-ID local
|
||||
override and one explicit global override.
|
||||
|
||||
### Exit Gate
|
||||
|
||||
Every migration and audit-remediation criterion is demonstrably satisfied,
|
||||
the restored tests protect the consequential contracts, canonical
|
||||
documentation describes the corrected implementation, and both roadmap
|
||||
documents are marked complete.
|
||||
All roadmap completion criteria are satisfied, all verification commands pass,
|
||||
the working tree contains only intentional changes, and the canonical
|
||||
documentation describes the implemented state. The feature is ready for code
|
||||
review and release preparation.
|
||||
|
||||
## Open Questions
|
||||
|
||||
None. The roadmap and this completed plan record the decisions used for the
|
||||
audit remediation.
|
||||
None. The model identifiers, profile settings, report assignments, version
|
||||
transition, precedence, compatibility behavior, test boundaries, and
|
||||
documentation ownership are decision-complete.
|
||||
|
||||
Reference in New Issue
Block a user