Finalize domain prompt profile roadmap
This commit is contained in:
@@ -1,6 +1,6 @@
|
|||||||
# Domain-Specific Prompt Profiles Roadmap
|
# Domain-Specific Prompt Profiles Roadmap
|
||||||
|
|
||||||
Status: Accepted; implementation plan ready.
|
Status: Implemented.
|
||||||
|
|
||||||
## Purpose
|
## Purpose
|
||||||
|
|
||||||
@@ -10,9 +10,9 @@ These logical profiles should give each report an appropriate default while
|
|||||||
allowing operators to replace any definition through the existing configured
|
allowing operators to replace any definition through the existing configured
|
||||||
profile source.
|
profile source.
|
||||||
|
|
||||||
This roadmap defines the scope, policy, and intended end state. The companion
|
This roadmap records the scope, policy, and implemented end state. The
|
||||||
[implementation plan](domain-profiles-implementation.md) owns the ordered work
|
companion [implementation plan](implementation.md) records the ordered work
|
||||||
needed to reach that state.
|
and verification used to reach it.
|
||||||
|
|
||||||
## User Intent
|
## User Intent
|
||||||
|
|
||||||
@@ -29,22 +29,22 @@ The feature is intended to provide three related benefits:
|
|||||||
guarantee. A locally hosted lightweight model may still generate slowly on the
|
guarantee. A locally hosted lightweight model may still generate slowly on the
|
||||||
available hardware.
|
available hardware.
|
||||||
|
|
||||||
## Current State
|
## Pre-Implementation Baseline
|
||||||
|
|
||||||
Daily, Today, Tomorrow, and Hourly each declare Promptkit's
|
Before implementation, Daily, Today, Tomorrow, and Hourly each declared
|
||||||
`gemini-flash-latest` profile as their prompt default. The optional
|
Promptkit's `gemini-flash-latest` profile as their prompt default. The optional
|
||||||
`promptkit.profile` setting overrides that default for every selected report in
|
`promptkit.profile` setting overrode that default for every selected report in
|
||||||
an invocation.
|
an invocation.
|
||||||
|
|
||||||
Weatherreporter accepts either `promptkit.profile_file` or
|
Weatherreporter accepted either `promptkit.profile_file` or
|
||||||
`promptkit.profile_dir` and passes that source to Promptkit. A matching external
|
`promptkit.profile_dir` and passed that source to Promptkit. A matching external
|
||||||
profile can override a Promptkit built-in profile, and the configured local
|
profile could override a Promptkit built-in profile, and the configured local
|
||||||
backend can support profiles that select `backend: local`. Endpoint-only
|
backend could support profiles that select `backend: local`. Endpoint-only
|
||||||
OpenAI-compatible profiles can also provide their own endpoint.
|
OpenAI-compatible profiles could also provide their own endpoint.
|
||||||
|
|
||||||
Weatherreporter does not currently own or embed execution profiles. Promptkit
|
Weatherreporter did not own or embed execution profiles. Promptkit v0.5.0
|
||||||
v0.5.0 now provides the fallback-profile layer needed to add them without
|
provided the fallback-profile layer used to add them without changing the
|
||||||
changing the existing operator-source precedence.
|
existing operator-source precedence.
|
||||||
|
|
||||||
## Prerequisite
|
## Prerequisite
|
||||||
|
|
||||||
@@ -56,7 +56,7 @@ passed the repository test suite and an operator smoke test. Weatherreporter
|
|||||||
must continue to use only Promptkit's public API rather than depending on its
|
must continue to use only Promptkit's public API rather than depending on its
|
||||||
internal packages or reproducing its profile repository behavior.
|
internal packages or reproducing its profile repository behavior.
|
||||||
|
|
||||||
## Desired End State
|
## Implemented End State
|
||||||
|
|
||||||
Weatherreporter embeds usable definitions for these exact logical profile IDs:
|
Weatherreporter embeds usable definitions for these exact logical profile IDs:
|
||||||
|
|
||||||
@@ -204,7 +204,7 @@ The purpose is to choose an appropriate default for each tier, not to add a
|
|||||||
permanent benchmark framework or live-provider requirement to the ordinary
|
permanent benchmark framework or live-provider requirement to the ordinary
|
||||||
test suite. Repository tests remain offline and deterministic.
|
test suite. Repository tests remain offline and deterministic.
|
||||||
|
|
||||||
## Scope
|
## Implemented Scope
|
||||||
|
|
||||||
The completed feature includes:
|
The completed feature includes:
|
||||||
|
|
||||||
@@ -260,9 +260,9 @@ An external same-ID override is an operator-owned compatibility commitment.
|
|||||||
Weatherreporter may evolve its embedded definitions, but it must not rewrite or
|
Weatherreporter may evolve its embedded definitions, but it must not rewrite or
|
||||||
silently merge an operator file.
|
silently merge an operator file.
|
||||||
|
|
||||||
## Completion Criteria
|
## Completion Record
|
||||||
|
|
||||||
The roadmap is complete when:
|
The following conditions are satisfied:
|
||||||
|
|
||||||
- a tagged Promptkit dependency supports the required fallback layer;
|
- a tagged Promptkit dependency supports the required fallback layer;
|
||||||
- every operational prompt selects its assigned logical profile at exact
|
- every operational prompt selects its assigned logical profile at exact
|
||||||
|
|||||||
@@ -1,19 +1,14 @@
|
|||||||
# Domain-Specific Prompt Profiles Implementation Plan
|
# Domain-Specific Prompt Profiles Implementation Plan
|
||||||
|
|
||||||
Status: Ready for implementation.
|
Status: Completed.
|
||||||
|
|
||||||
## Purpose And Authority
|
## Purpose And Authority
|
||||||
|
|
||||||
This document gives a coding agent the ordered work needed to implement the
|
This document records the completed implementation of the
|
||||||
[domain-specific prompt profiles roadmap](domain-profiles.md). The roadmap is
|
[domain-specific prompt profiles roadmap](domain-profiles.md). The roadmap is
|
||||||
authoritative for scope, user intent, policy choices, and the desired end
|
authoritative for scope, user intent, policy choices, and the implemented end
|
||||||
state. This plan is authoritative for implementation sequence, verification,
|
state. This plan records the implementation sequence, verification, and exit
|
||||||
and stage exit gates.
|
gates used to reach it.
|
||||||
|
|
||||||
The target implementer is a `gpt-5.6-terra` coding agent using high reasoning.
|
|
||||||
Complete the stages in order. Each stage is scoped for one implementation
|
|
||||||
prompt and must leave the repository coherent, tested, and reviewable before
|
|
||||||
the next stage begins.
|
|
||||||
|
|
||||||
This plan follows the repository's
|
This plan follows the repository's
|
||||||
[architecture](../policy/architecture.md),
|
[architecture](../policy/architecture.md),
|
||||||
|
|||||||
@@ -12,7 +12,10 @@ import (
|
|||||||
"time"
|
"time"
|
||||||
|
|
||||||
promptkit "gitea.maximumdirect.net/eric/promptkit"
|
promptkit "gitea.maximumdirect.net/eric/promptkit"
|
||||||
|
"gitea.maximumdirect.net/eric/weatherreporter/internal/app"
|
||||||
|
appconfig "gitea.maximumdirect.net/eric/weatherreporter/internal/config"
|
||||||
"gitea.maximumdirect.net/eric/weatherreporter/internal/promptexec"
|
"gitea.maximumdirect.net/eric/weatherreporter/internal/promptexec"
|
||||||
|
"gitea.maximumdirect.net/eric/weatherreporter/internal/report"
|
||||||
)
|
)
|
||||||
|
|
||||||
type fakeClient struct {
|
type fakeClient struct {
|
||||||
@@ -158,6 +161,44 @@ func TestMaintainedWeatherLightLocalProfileExampleInspectsOffline(t *testing.T)
|
|||||||
assertProfile(t, adapter, "weather-light", "", "weather-local")
|
assertProfile(t, adapter, "weather-light", "", "weather-local")
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func TestApplicationPreflightResolvesEmbeddedAndOverriddenProfilesOffline(t *testing.T) {
|
||||||
|
lookupEnv := func(string) (string, bool) { return "test-key", true }
|
||||||
|
inspect := func(t *testing.T, adapter *Adapter, id report.ID, profile string, wantID string, wantBackend string, wantModel string) {
|
||||||
|
t.Helper()
|
||||||
|
result, err := app.InspectPromptExecution(context.Background(), app.PromptInspectionRequest{
|
||||||
|
Resolved: resolvedPromptProfile(t, id),
|
||||||
|
Executor: adapter,
|
||||||
|
Promptkit: appconfig.PromptkitConfig{Profile: profile},
|
||||||
|
LookupEnv: lookupEnv,
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("InspectPromptExecution() error = %v", err)
|
||||||
|
}
|
||||||
|
if result.ProfileID != wantID || result.BackendID != wantBackend || result.ModelName != wantModel {
|
||||||
|
t.Fatalf("inspection = %#v, want profile/backend/model %q/%q/%q", result, wantID, wantBackend, wantModel)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
embedded, err := newAdapterForTest(Config{}, &fakeClient{})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("newAdapterForTest(embedded) error = %v", err)
|
||||||
|
}
|
||||||
|
inspect(t, embedded, report.Hourly, "", "weather-light", "openrouter", "deepseek/deepseek-v4-flash")
|
||||||
|
inspect(t, embedded, report.Daily, "", "weather-balanced", "openrouter", "~google/gemini-flash-latest")
|
||||||
|
inspect(t, embedded, report.Today, "", "weather-balanced", "openrouter", "~google/gemini-flash-latest")
|
||||||
|
inspect(t, embedded, report.Tomorrow, "", "weather-balanced", "openrouter", "~google/gemini-flash-latest")
|
||||||
|
inspect(t, embedded, report.Daily, "weather-deep", "weather-deep", "openrouter", "~anthropic/claude-sonnet-latest")
|
||||||
|
|
||||||
|
override, err := newAdapterForTest(Config{ProfileFile: writeProfileFile(t, `id: weather-light
|
||||||
|
endpoint: https://local.example/v1
|
||||||
|
model: local-weather
|
||||||
|
`)}, &fakeClient{})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("newAdapterForTest(override) error = %v", err)
|
||||||
|
}
|
||||||
|
inspect(t, override, report.Hourly, "", "weather-light", "", "local-weather")
|
||||||
|
}
|
||||||
|
|
||||||
func TestProfileResolutionFallsThroughOnlyWhenTheConfiguredIDIsAbsent(t *testing.T) {
|
func TestProfileResolutionFallsThroughOnlyWhenTheConfiguredIDIsAbsent(t *testing.T) {
|
||||||
absentAdapter, err := New(Config{ProfileDirectory: testProfileDirectory(t, `id: other-profile
|
absentAdapter, err := New(Config{ProfileDirectory: testProfileDirectory(t, `id: other-profile
|
||||||
backend: openrouter
|
backend: openrouter
|
||||||
@@ -487,6 +528,20 @@ func writeProfileFile(t *testing.T, profile string) string {
|
|||||||
return path
|
return path
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func resolvedPromptProfile(t *testing.T, id report.ID) report.Resolved {
|
||||||
|
t.Helper()
|
||||||
|
now := time.Date(2026, 5, 29, 12, 0, 0, 0, time.UTC)
|
||||||
|
request := report.ResolveRequest{Now: now, Location: time.UTC}
|
||||||
|
if id == report.Daily {
|
||||||
|
request.Date = now
|
||||||
|
}
|
||||||
|
resolved, err := report.DefaultRegistry().Resolve(id, request)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("Resolve(%q) error = %v", id, err)
|
||||||
|
}
|
||||||
|
return resolved
|
||||||
|
}
|
||||||
|
|
||||||
func testExecuteRequest() promptexec.ExecuteRequest {
|
func testExecuteRequest() promptexec.ExecuteRequest {
|
||||||
return promptexec.ExecuteRequest{
|
return promptexec.ExecuteRequest{
|
||||||
PromptID: "weather.daily_generated_text",
|
PromptID: "weather.daily_generated_text",
|
||||||
|
|||||||
Reference in New Issue
Block a user