Finalize Promptkit migration documentation
This commit is contained in:
@@ -1,21 +1,19 @@
|
|||||||
# Promptkit Migration Implementation Plan
|
# Promptkit Migration Implementation Plan
|
||||||
|
|
||||||
Status: Stages 1–11 completed; audit remediation in Stages 12–19 remains.
|
Status: Completed; Stages 1–19 passed their exit gates.
|
||||||
|
|
||||||
## Purpose And Authority
|
## Purpose And Authority
|
||||||
|
|
||||||
This document records the completed implementation of the
|
This document records the completed implementation of the
|
||||||
[Promptkit migration roadmap](promptkit.md) and defines the ordered follow-up
|
[Promptkit migration roadmap](promptkit.md) and its post-implementation audit
|
||||||
work required by the post-implementation audit. The feature roadmap remains
|
remediation. The feature roadmap records scope, user intent, policy choices,
|
||||||
authoritative for scope, user intent, policy choices, and the desired end
|
and the implemented end state. This plan records implementation sequence,
|
||||||
state. This plan is authoritative for implementation sequence, tests, and
|
tests, and completion gates.
|
||||||
completion gates.
|
|
||||||
|
|
||||||
Follow Stages 12–19 in order, using one implementation prompt per stage. A
|
Stages 12–19 were completed in order. They fixed additional defects exposed by
|
||||||
stage may fix additional defects exposed by its required tests when those
|
their required tests only when those defects were within the same stated
|
||||||
defects are within the same stated contract. Do not expand a stage into new
|
contract; they did not add new product behavior or reinterpret roadmap
|
||||||
product behavior or reinterpret a roadmap decision to accommodate the current
|
decisions.
|
||||||
implementation.
|
|
||||||
|
|
||||||
This plan follows the repository's
|
This plan follows the repository's
|
||||||
[architecture](../policy/architecture.md),
|
[architecture](../policy/architecture.md),
|
||||||
@@ -71,11 +69,24 @@ The post-implementation audit confirmed the principal dependency and package
|
|||||||
boundaries, but found incorrect reached-path bookkeeping, incomplete execution
|
boundaries, but found incorrect reached-path bookkeeping, incomplete execution
|
||||||
artifact updates, insufficient artifact validation, extensive loss of
|
artifact updates, insufficient artifact validation, extensive loss of
|
||||||
behavioral tests during the final cutover, and roadmap lifecycle text that was
|
behavioral tests during the final cutover, and roadmap lifecycle text that was
|
||||||
not finalized. The remaining stages address those findings without changing
|
not finalized. The completed remediation addressed those findings without
|
||||||
the intended feature scope.
|
changing the intended feature scope.
|
||||||
|
|
||||||
|
| Stage | Completed outcome |
|
||||||
|
| --- | --- |
|
||||||
|
| 12 | Corrected reached-path bookkeeping across metadata, app results, batch items, and CLI summaries. |
|
||||||
|
| 13 | Hardened Promptkit-era durable state validation and restored v1/v2 state coverage. |
|
||||||
|
| 14 | Recorded every downstream path reached after completed prompt execution. |
|
||||||
|
| 15 | Restored assembled single-report behavioral and failure coverage. |
|
||||||
|
| 16 | Simplified prompt-generation orchestration while preserving behavior. |
|
||||||
|
| 17 | Restored assembled batch, planning, artifact, and notification coverage. |
|
||||||
|
| 18 | Restored supported CLI, summary, safety, and historical inspection coverage. |
|
||||||
|
| 19 | Reconciled canonical documentation and passed the complete repository verification gate. |
|
||||||
|
|
||||||
## Stage 12: Correct Reached-Artifact Bookkeeping
|
## Stage 12: Correct Reached-Artifact Bookkeeping
|
||||||
|
|
||||||
|
Status: Completed.
|
||||||
|
|
||||||
### Goal
|
### Goal
|
||||||
|
|
||||||
Make metadata, app results, batch items, and CLI summaries truthful at every
|
Make metadata, app results, batch items, and CLI summaries truthful at every
|
||||||
@@ -138,6 +149,8 @@ result.
|
|||||||
|
|
||||||
## Stage 13: Harden Durable State Contracts And Restore State Coverage
|
## Stage 13: Harden Durable State Contracts And Restore State Coverage
|
||||||
|
|
||||||
|
Status: Completed.
|
||||||
|
|
||||||
### Goal
|
### Goal
|
||||||
|
|
||||||
Make the v1/v2 wire boundary and Promptkit-era artifact validation explicit,
|
Make the v1/v2 wire boundary and Promptkit-era artifact validation explicit,
|
||||||
@@ -206,6 +219,8 @@ offline coverage for its durable compatibility and filesystem contracts.
|
|||||||
|
|
||||||
## Stage 14: Complete Execution-Artifact Path Tracking
|
## Stage 14: Complete Execution-Artifact Path Tracking
|
||||||
|
|
||||||
|
Status: Completed.
|
||||||
|
|
||||||
### Goal
|
### Goal
|
||||||
|
|
||||||
Make `PromptExecutionArtifact.Paths` accurately record every downstream
|
Make `PromptExecutionArtifact.Paths` accurately record every downstream
|
||||||
@@ -267,6 +282,8 @@ when a later application stage fails.
|
|||||||
|
|
||||||
## Stage 15: Restore Single-Report Behavioral Coverage
|
## Stage 15: Restore Single-Report Behavioral Coverage
|
||||||
|
|
||||||
|
Status: Completed.
|
||||||
|
|
||||||
### Goal
|
### Goal
|
||||||
|
|
||||||
Restore the risk-based application coverage removed during final cutover and
|
Restore the risk-based application coverage removed during final cutover and
|
||||||
@@ -325,6 +342,8 @@ debug isolation, output copying, and notification behavior.
|
|||||||
|
|
||||||
## Stage 16: Refactor Prompt Generation Orchestration
|
## Stage 16: Refactor Prompt Generation Orchestration
|
||||||
|
|
||||||
|
Status: Completed.
|
||||||
|
|
||||||
### Goal
|
### Goal
|
||||||
|
|
||||||
Reduce the complexity and duplicated persistence logic in
|
Reduce the complexity and duplicated persistence logic in
|
||||||
@@ -374,6 +393,8 @@ and every Stage 12–15 behavioral test passes unchanged.
|
|||||||
|
|
||||||
## Stage 17: Restore Batch Behavioral Coverage
|
## Stage 17: Restore Batch Behavioral Coverage
|
||||||
|
|
||||||
|
Status: Completed.
|
||||||
|
|
||||||
### Goal
|
### Goal
|
||||||
|
|
||||||
Re-establish confidence that morning and evening batches preserve their
|
Re-establish confidence that morning and evening batches preserve their
|
||||||
@@ -421,6 +442,8 @@ artifact, and notification contracts with one Promptkit executor.
|
|||||||
|
|
||||||
## Stage 18: Restore CLI And Inspection Coverage
|
## Stage 18: Restore CLI And Inspection Coverage
|
||||||
|
|
||||||
|
Status: Completed.
|
||||||
|
|
||||||
### Goal
|
### Goal
|
||||||
|
|
||||||
Restore the user-facing command, summary, and historical inspection contracts
|
Restore the user-facing command, summary, and historical inspection contracts
|
||||||
@@ -464,6 +487,8 @@ composition, and v1/v2 inspection behavior have deterministic offline coverage.
|
|||||||
|
|
||||||
## Stage 19: Finalize Documentation And Repository Verification
|
## Stage 19: Finalize Documentation And Repository Verification
|
||||||
|
|
||||||
|
Status: Completed.
|
||||||
|
|
||||||
### Goal
|
### Goal
|
||||||
|
|
||||||
Close the audit remediation, make roadmap lifecycle state truthful, and verify
|
Close the audit remediation, make roadmap lifecycle state truthful, and verify
|
||||||
@@ -526,5 +551,5 @@ documents are marked complete.
|
|||||||
|
|
||||||
## Open Questions
|
## Open Questions
|
||||||
|
|
||||||
None. The roadmap and this follow-up plan contain the decisions required to
|
None. The roadmap and this completed plan record the decisions used for the
|
||||||
complete the audit remediation.
|
audit remediation.
|
||||||
|
|||||||
@@ -1,39 +1,40 @@
|
|||||||
# Promptkit Migration Roadmap
|
# Promptkit Migration Roadmap
|
||||||
|
|
||||||
Status: Accepted feature roadmap; the target state is unimplemented.
|
Status: Completed roadmap record.
|
||||||
|
|
||||||
## Purpose
|
## Purpose
|
||||||
|
|
||||||
This roadmap defines the scope and desired end state for replacing the
|
This roadmap records the scope, decisions, and completed outcome of replacing
|
||||||
external Scriptorium CLI integration with Promptkit. Current implemented
|
the external Scriptorium CLI integration with Promptkit. Canonical
|
||||||
behavior remains documented outside `docs/roadmap/` until the replacement is
|
documentation outside `docs/roadmap/` owns the implemented behavior.
|
||||||
complete.
|
|
||||||
|
|
||||||
## Current Baseline
|
## Pre-Migration Baseline
|
||||||
|
|
||||||
Status: Current migration input.
|
Status: Historical migration input.
|
||||||
|
|
||||||
Weatherreporter currently exposes seven report definitions, but only four have
|
Before the migration, Weatherreporter exposed seven report definitions, but
|
||||||
complete prompt-backed report implementations:
|
only four had complete prompt-backed report implementations:
|
||||||
|
|
||||||
- Daily Report: `weather.daily_generated_text`
|
- Daily Report: `weather.daily_generated_text`
|
||||||
- Today Report: `weather.today_generated_text`
|
- Today Report: `weather.today_generated_text`
|
||||||
- Tomorrow Report: `weather.tomorrow_generated_text`
|
- Tomorrow Report: `weather.tomorrow_generated_text`
|
||||||
- Hourly Report: `weather.hourly_generated_text`
|
- Hourly Report: `weather.hourly_generated_text`
|
||||||
|
|
||||||
The three-day, weekend, and storm commands and registry definitions have no
|
The three-day, weekend, and storm commands and registry definitions had no
|
||||||
corresponding Scriptorium prompt or schema and never formed complete
|
corresponding Scriptorium prompt or schema and never formed complete
|
||||||
operational report products. The `weather.daily_report` Scriptorium prompt is
|
operational report products. The `weather.daily_report` Scriptorium prompt was
|
||||||
legacy source material and is not selected by the current registry.
|
legacy source material and was not selected by the registry.
|
||||||
|
|
||||||
The available Scriptorium source corpus is retained temporarily under
|
The Scriptorium source corpus was retained temporarily under
|
||||||
`docs/roadmap/scriptorium/`. It contains the four operational generated-text
|
`docs/roadmap/scriptorium/` as migration input. It contained the four
|
||||||
prompt definitions, their referenced content, their private response schemas,
|
operational generated-text prompt definitions, their referenced content,
|
||||||
shared instructions, and the unused legacy Daily Markdown prompt.
|
private response schemas, shared instructions, and the unused legacy Daily
|
||||||
|
Markdown prompt. The temporary corpus was removed after the runtime assets
|
||||||
|
were reconciled and embedded.
|
||||||
|
|
||||||
## Desired End State
|
## Implemented End State
|
||||||
|
|
||||||
Status: Accepted target state; unimplemented.
|
Status: Completed.
|
||||||
|
|
||||||
Weatherreporter pins
|
Weatherreporter pins
|
||||||
`gitea.maximumdirect.net/eric/promptkit` at `v0.4.0` and uses it as the
|
`gitea.maximumdirect.net/eric/promptkit` at `v0.4.0` and uses it as the
|
||||||
@@ -71,33 +72,33 @@ packages, CLI summaries, durable state contracts, or Distributor behavior.
|
|||||||
|
|
||||||
## Goals
|
## Goals
|
||||||
|
|
||||||
Status: Accepted migration scope; unimplemented.
|
Status: Completed migration outcomes.
|
||||||
|
|
||||||
- Remove the Scriptorium runtime dependency and subprocess boundary.
|
- Removed the Scriptorium runtime dependency and subprocess boundary.
|
||||||
- Migrate the four operational report prompts to Promptkit `v0.4.0`.
|
- Migrated the four operational report prompts to Promptkit `v0.4.0`.
|
||||||
- Use prepared execution to persist preparation provenance before provider work
|
- Used prepared execution to persist preparation provenance before provider work
|
||||||
while executing the exact frozen snapshot.
|
while executing the exact frozen snapshot.
|
||||||
- Validate report prompt and profile selections before weather collection when
|
- Validated report prompt and profile selections before weather collection when
|
||||||
the required information is available.
|
the required information is available.
|
||||||
- Preserve deterministic module snapshots and structured Recent Changes.
|
- Preserved deterministic module snapshots and structured Recent Changes.
|
||||||
- Preserve generated-text domain validation and repository-owned Markdown
|
- Preserved generated-text domain validation and repository-owned Markdown
|
||||||
rendering.
|
rendering.
|
||||||
- Preserve context cancellation, actionable errors, secret redaction, and
|
- Preserved context cancellation, actionable errors, secret redaction, and
|
||||||
inspectable failures.
|
inspectable failures.
|
||||||
- Improve durable prompt provenance with prompt, input, profile, model,
|
- Improved durable prompt provenance with prompt, input, profile, model,
|
||||||
validation, usage, and timing metadata.
|
validation, usage, and timing metadata.
|
||||||
- Keep content-rich prompt and response diagnostics separate from routine
|
- Kept content-rich prompt and response diagnostics separate from routine
|
||||||
metadata and CLI output.
|
metadata and CLI output.
|
||||||
- Keep tests offline and deterministic through injected Promptkit model
|
- Kept tests offline and deterministic through injected Promptkit model
|
||||||
clients and fixtures.
|
clients and fixtures.
|
||||||
- Remove incomplete report declarations from the implemented product surface
|
- Removed incomplete report declarations from the implemented product surface
|
||||||
rather than creating new report products during an integration migration.
|
rather than creating new report products during an integration migration.
|
||||||
|
|
||||||
## Non-Goals
|
## Non-Goals
|
||||||
|
|
||||||
Status: Accepted migration scope; unimplemented.
|
Status: Completed migration constraints.
|
||||||
|
|
||||||
The migration will not:
|
The completed migration did not:
|
||||||
|
|
||||||
- create prompts, schemas, templates, or completed products for three-day,
|
- create prompts, schemas, templates, or completed products for three-day,
|
||||||
weekend, or storm reports;
|
weekend, or storm reports;
|
||||||
@@ -122,7 +123,7 @@ The migration will not:
|
|||||||
|
|
||||||
## Locked Decisions
|
## Locked Decisions
|
||||||
|
|
||||||
Status: Accepted decisions for the unimplemented migration.
|
Status: Implemented migration decisions.
|
||||||
|
|
||||||
### Dependency And Upgrade Boundary
|
### Dependency And Upgrade Boundary
|
||||||
|
|
||||||
@@ -329,7 +330,7 @@ promptkit:
|
|||||||
|
|
||||||
## Durable Artifacts And Observability
|
## Durable Artifacts And Observability
|
||||||
|
|
||||||
Status: Accepted design constraints; unimplemented.
|
Status: Implemented design constraints.
|
||||||
|
|
||||||
Routine durable state retains useful non-secret provenance without persisting
|
Routine durable state retains useful non-secret provenance without persisting
|
||||||
full rendered prompts.
|
full rendered prompts.
|
||||||
@@ -367,7 +368,7 @@ Rendered messages and other content-rich preparation or response diagnostics
|
|||||||
are available only when the operator supplies
|
are available only when the operator supplies
|
||||||
`--llm-debug-dir <path>` to a single-report or batch command.
|
`--llm-debug-dir <path>` to a single-report or batch command.
|
||||||
|
|
||||||
- There is initially no persistent YAML setting for debug capture.
|
- There is no persistent YAML setting for debug capture.
|
||||||
- The debug root is validated or created before weather collection or provider
|
- The debug root is validated or created before weather collection or provider
|
||||||
work. A requested destination that cannot be secured or written is an error.
|
work. A requested destination that cannot be secured or written is an error.
|
||||||
- Artifacts are grouped beneath
|
- Artifacts are grouped beneath
|
||||||
@@ -411,7 +412,7 @@ dual-write deprecated aliases. CLI summary fields adopt `preparationPath` and
|
|||||||
|
|
||||||
## Failure Contract
|
## Failure Contract
|
||||||
|
|
||||||
Status: Accepted design constraints; unimplemented.
|
Status: Implemented design constraints.
|
||||||
|
|
||||||
- A preparation failure produces a redacted weatherreporter-owned failure
|
- A preparation failure produces a redacted weatherreporter-owned failure
|
||||||
receipt with report, RunID, prompt, stage, timing, and classified error
|
receipt with report, RunID, prompt, stage, timing, and classified error
|
||||||
@@ -441,7 +442,7 @@ Status: Accepted design constraints; unimplemented.
|
|||||||
|
|
||||||
## Compatibility Requirements
|
## Compatibility Requirements
|
||||||
|
|
||||||
Status: Accepted design constraints; unimplemented.
|
Status: Implemented design constraints.
|
||||||
|
|
||||||
- Daily, Today, Tomorrow, and Hourly report IDs, prompt IDs, valid periods,
|
- Daily, Today, Tomorrow, and Hourly report IDs, prompt IDs, valid periods,
|
||||||
artifact grouping, output names, and Distributor bundle behavior remain
|
artifact grouping, output names, and Distributor bundle behavior remain
|
||||||
@@ -468,9 +469,9 @@ Status: Accepted design constraints; unimplemented.
|
|||||||
|
|
||||||
## Verification And Completion Criteria
|
## Verification And Completion Criteria
|
||||||
|
|
||||||
Status: Accepted completion criteria for the unimplemented migration.
|
Status: Completed and verified.
|
||||||
|
|
||||||
The migration is complete when:
|
Completion was verified by the following outcomes:
|
||||||
|
|
||||||
- the four operational reports inspect, prepare, and execute through Promptkit
|
- the four operational reports inspect, prepare, and execute through Promptkit
|
||||||
`v0.4.0` using embedded report-owned assets;
|
`v0.4.0` using embedded report-owned assets;
|
||||||
@@ -508,8 +509,8 @@ costly, and difficult to compare meaningfully.
|
|||||||
|
|
||||||
## Decision Status
|
## Decision Status
|
||||||
|
|
||||||
Status: Decision-complete.
|
Status: Completed.
|
||||||
|
|
||||||
The roadmap has no remaining open product or architecture questions. Any later
|
The roadmap has no remaining open product or architecture questions. Later
|
||||||
question that changes migration scope, policy, or the desired end state must be
|
changes to this completed scope require new roadmap or decision-record scope
|
||||||
resolved here rather than implicitly in code.
|
rather than implicit changes to this historical record.
|
||||||
|
|||||||
Reference in New Issue
Block a user