From 87b5ec3d7546f79e65ed3dd7b5bb8fee0fb04296 Mon Sep 17 00:00:00 2001 From: Eric Rakestraw Date: Thu, 30 Jul 2026 17:13:26 +0000 Subject: [PATCH] Organize downstream feature requests in the future roadmap --- docs/roadmap/future.md | 65 ++++++++++++++++++- docs/roadmap/notarius-promptkit-wishlist.md | 16 ++++- .../weatherreporter-promptkit-wishlist.md | 23 +++++++ 3 files changed, 100 insertions(+), 4 deletions(-) diff --git a/docs/roadmap/future.md b/docs/roadmap/future.md index 2fa6f82..8787ed2 100644 --- a/docs/roadmap/future.md +++ b/docs/roadmap/future.md @@ -33,9 +33,68 @@ consumers. ## Ideas -No ideas are currently cataloged. Backend-specific concurrency management has -been selected for active planning in the -[focused concurrency roadmap](concurrency.md). +### Executable preparation handles + +Allow a consumer to prepare one exact execution, inspect and retain its +caller-owned preparation details, and later execute that already-prepared work +without reloading or rerendering prompt, profile, schema, or input sources. +This would remove the duplicate `Prepare`-then-`Run` workaround reported by +[Notarius](notarius-promptkit-wishlist.md#priority-1-atomic-execution-with-prepared-details) +and +[Weatherreporter](weatherreporter-promptkit-wishlist.md#priority-1-executable-preparation-handles). + +- Keep the handle opaque, engine-bound, non-serializable, and initially + one-shot. +- Keep existing `Prepare` and `Run` workflows available. +- Return public preparation details under existing caller-ownership and + credential-redaction rules. +- Perform backend admission when execution begins rather than reserving + capacity while a consumer retains the prepared handle. +- Treat a separate atomic detailed-run method as possible later convenience + rather than a second initial execution model. + +### Prompt-independent profile inspection + +Provide exact profile lookup and structural resolution without requiring a +synthetic prompt, placeholder inputs, or model generation. This shared need is +described by +[Notarius](notarius-promptkit-wishlist.md#priority-2-prompt-independent-profile-inspection) +and +[Weatherreporter](weatherreporter-promptkit-wishlist.md#priority-3-prompt-independent-profile-inspection). + +- Apply ordinary built-in, file-backed, and programmatic profile precedence. +- Validate referenced backend membership and the structurally resolved + execution target. +- Report credential requirements and environment-variable names without + exposing credential values or requiring current credential availability. +- Support exact lookup by profile ID; enumeration is not required initially. + +### Prompt-definition inspection + +Provide exact prompt-definition lookup without rendering, placeholder inputs, +profile resolution, or model generation, as requested by +[Weatherreporter](weatherreporter-promptkit-wishlist.md#priority-2-prompt-definition-inspection). + +- Return caller-owned identity, version, input definitions, default-profile, + output-contract, and opaque definition-equality information. +- Apply ordinary prompt-source precedence and exact ID/version selection. +- Validate the selected definition and referenced prompt content + structurally, without returning source bodies or rendered messages. +- Leave complete cross-source corpus validation and enumeration outside the + initial inspection contract. + +### Structured capacity errors + +Add safe structured context to backend admission rejection, as requested by +[Notarius](notarius-promptkit-wishlist.md#priority-4-structured-capacity-errors) +and +[Weatherreporter](weatherreporter-promptkit-wishlist.md#structured-capacity-errors). + +- Preserve compatibility with `errors.Is(err, ErrCapacityExceeded)`. +- Support `errors.As` to obtain the stable backend ID. +- Do not expose endpoints, credential configuration or values, request + content, or speculative retry timing. +- Keep retry and backoff policy with downstream consumers. ## Entry Format diff --git a/docs/roadmap/notarius-promptkit-wishlist.md b/docs/roadmap/notarius-promptkit-wishlist.md index 1e4679a..493f4e1 100644 --- a/docs/roadmap/notarius-promptkit-wishlist.md +++ b/docs/roadmap/notarius-promptkit-wishlist.md @@ -17,6 +17,12 @@ not prescriptive names or finalized Go contracts. ## Priority 1: Atomic Execution With Prepared Details +**Disposition:** Covered by the accepted +[executable preparation handles](future.md#executable-preparation-handles) +catalog entry. The shared two-phase capability should provide the required +single-preparation consistency; a separate `RunDetailed` method is not +cataloged initially. + ### Downstream need Notarius needs both: @@ -83,6 +89,9 @@ corresponds atomically to the actual execution. ## Priority 2: Prompt-Independent Profile Inspection +**Disposition:** Accepted into the +[future catalog](future.md#prompt-independent-profile-inspection). + ### Downstream need Notarius validates configured pipeline profile IDs before beginning a run. It @@ -150,6 +159,9 @@ validation. ## Priority 3: Semantic Execution-Target Fingerprints +**Disposition:** Deferred until prompt-independent profile inspection defines +the resolved target whose configuration identity would be fingerprinted. + ### Downstream need Notarius checkpoints model-backed pipeline stages. A checkpoint must not be @@ -233,6 +245,9 @@ resolution. ## Priority 4: Structured Capacity Errors +**Disposition:** Accepted into the +[future catalog](future.md#structured-capacity-errors). + ### Downstream need Notarius translates PromptKit backend-capacity rejection into a @@ -342,4 +357,3 @@ Notarius would be: The first two address concrete workarounds in current Notarius code. The third would improve checkpoint correctness and reduce coupling. The fourth is operational polish. - diff --git a/docs/roadmap/weatherreporter-promptkit-wishlist.md b/docs/roadmap/weatherreporter-promptkit-wishlist.md index 6b6e3e7..58cb58d 100644 --- a/docs/roadmap/weatherreporter-promptkit-wishlist.md +++ b/docs/roadmap/weatherreporter-promptkit-wishlist.md @@ -19,6 +19,9 @@ overlapping features from another downstream consumer's perspective. ## Priority 1: Executable Preparation Handles +**Disposition:** Accepted into the +[future catalog](future.md#executable-preparation-handles). + ### Downstream need Weatherreporter treats prompt preparation as a durable preflight boundary. It @@ -103,6 +106,9 @@ describes the actual execution. ## Priority 2: Prompt-Definition Inspection +**Disposition:** Accepted into the +[future catalog](future.md#prompt-definition-inspection). + ### Downstream need Weatherreporter has a fixed registry of seven report definitions. Each report @@ -179,6 +185,9 @@ and move failures ahead of weather collection. ## Priority 3: Prompt-Independent Profile Inspection +**Disposition:** Accepted into the +[future catalog](future.md#prompt-independent-profile-inspection). + ### Downstream need Weatherreporter will allow operators to select an external PromptKit profile @@ -224,6 +233,10 @@ the initial migration. ## Priority 4: Eager Source Validation +**Disposition:** Deferred until prompt and profile inspection have been used +to determine whether a broader engine-wide validation operation is still +needed. + ### Downstream need PromptKit deliberately defers reading and validating filesystem and `fs.FS` @@ -279,6 +292,10 @@ prompt and profile inspection. ## Priority 5: Structured Generation Errors +**Disposition:** Deferred pending stronger downstream demand and a narrower +design that does not duplicate prepared provenance or impose HTTP-specific +fields on injected model clients. + ### Downstream need Weatherreporter preserves redacted, inspectable failure receipts for report @@ -335,6 +352,9 @@ the existing sentinel remain available. ### Structured Capacity Errors +**Disposition:** Accepted into the +[future catalog](future.md#structured-capacity-errors). + The typed capacity error proposed by the [Notarius wishlist](notarius-promptkit-wishlist.md#priority-4-structured-capacity-errors) would improve Weatherreporter diagnostics by exposing the stable backend ID @@ -348,6 +368,9 @@ It should not block adoption. ### Semantic Execution-Target Fingerprints +**Disposition:** Deferred until prompt-independent profile inspection defines +the resolved target whose configuration identity would be fingerprinted. + The semantic target digest proposed by the [Notarius wishlist](notarius-promptkit-wishlist.md#priority-3-semantic-execution-target-fingerprints) would provide a compact equality signal for audit metadata.