Document logical prompt profile configuration
This commit is contained in:
@@ -22,8 +22,12 @@ extra report copies for a command and do not change configuration.
|
||||
collection and generation configuration.
|
||||
- [config.yml](../examples/config.yml) is a representative production-oriented
|
||||
configuration using synthetic endpoints and no credentials.
|
||||
- [weather-light-local-profile.yml](../examples/weather-light-local-profile.yml)
|
||||
is a complete endpoint-only override for the embedded `weather-light`
|
||||
profile.
|
||||
|
||||
Both files are loaded by the configuration test suite.
|
||||
The configuration examples are loaded by the configuration test suite. The
|
||||
profile example is inspected through the Promptkit adapter test suite.
|
||||
|
||||
## Minimal Configuration
|
||||
|
||||
@@ -146,13 +150,35 @@ individual `generate` or `run` command when explicitly needed.
|
||||
|
||||
| Field | Default | Rules |
|
||||
| --- | --- | --- |
|
||||
| `profile` | empty | Optional explicit execution profile. Otherwise the prompt's declared default is used. |
|
||||
| `profile_file` | empty | Optional external profile file. Cannot be combined with `profile_dir`. |
|
||||
| `profile_dir` | empty | Optional external profile directory. Cannot be combined with `profile_file`. |
|
||||
| `profile` | empty | Optional global profile selection for every report in one command. When empty, each exact prompt version selects its declared default. |
|
||||
| `profile_file` | empty | Optional external Promptkit profile file. It cannot be combined with `profile_dir`. A same-ID profile completely replaces Weatherreporter's embedded definition. |
|
||||
| `profile_dir` | empty | Optional external Promptkit profile directory. It cannot be combined with `profile_file`. A same-ID profile completely replaces Weatherreporter's embedded definition. |
|
||||
| `timeout` | `2m` | Must be greater than zero. |
|
||||
| `local.endpoint` | empty | Optional absolute URL for the conventional local backend. A blank endpoint leaves it unregistered. |
|
||||
| `local.concurrency_limit` | `1` | Maximum local backend concurrency. `0` is unlimited; negative values are invalid. |
|
||||
|
||||
`profile` selects an ID; `profile_file` and `profile_dir` supply definitions.
|
||||
They are separate decisions. An explicit `profile` applies to every selected
|
||||
report. Otherwise Hourly selects `weather-light`, while Daily, Today, and
|
||||
Tomorrow select `weather-balanced` through their exact `1.1.0` prompt
|
||||
definitions.
|
||||
|
||||
Promptkit resolves a selected profile definition from a test or embedding
|
||||
consumer's explicit in-memory profile, then the configured `profile_file` or
|
||||
`profile_dir`, then Weatherreporter's embedded catalog, and finally Promptkit's
|
||||
built-in catalog. Sources provide complete definitions; fields are never
|
||||
merged. A matching malformed external profile fails rather than using the
|
||||
embedded definition. The [Promptkit integration guide](integrations/promptkit.md)
|
||||
owns the catalog and precedence details.
|
||||
|
||||
To replace the default Hourly definition with a local OpenAI-compatible
|
||||
endpoint, set `profile_file` to a copy of
|
||||
[weather-light-local-profile.yml](../examples/weather-light-local-profile.yml).
|
||||
The example has no credential and should be edited for the local endpoint and
|
||||
model before use. An alternative profile may use `backend: local`; in that
|
||||
case `promptkit.local.endpoint` supplies the conventional local backend
|
||||
endpoint.
|
||||
|
||||
### `workspace`
|
||||
|
||||
| Field | Default |
|
||||
|
||||
@@ -1,16 +1,49 @@
|
||||
# Promptkit Integration
|
||||
|
||||
Weatherreporter uses Promptkit for all generated-text reports. The four logical prompts are
|
||||
`weather.daily_generated_text`, `weather.today_generated_text`,
|
||||
`weather.tomorrow_generated_text`, and `weather.hourly_generated_text`, each at version
|
||||
`1.0.1`. Their prompt assets and generated-text JSON Schemas are embedded by
|
||||
`internal/promptassets`.
|
||||
Weatherreporter uses Promptkit for all generated-text reports. The four logical
|
||||
prompts are `weather.daily_generated_text`, `weather.today_generated_text`,
|
||||
`weather.tomorrow_generated_text`, and `weather.hourly_generated_text`, each at
|
||||
version `1.1.0`. Their prompt assets, generated-text JSON Schemas, and
|
||||
Weatherreporter profile catalog are embedded by `internal/promptassets`.
|
||||
|
||||
Before collection, Weatherreporter inspects the exact prompt version, requires one required
|
||||
`data_package` input with content type `application/yaml`, and requires the report's JSON
|
||||
Schema output contract. It selects `promptkit.profile` when configured, otherwise the
|
||||
prompt's declared default profile. Profiles that require a direct API key are unsupported; a
|
||||
profile that reports `APIKeyEnv` requires a nonblank value in that environment variable.
|
||||
## Logical profile catalog
|
||||
|
||||
Prompt definitions select a stable Weatherreporter profile ID. The embedded
|
||||
definitions currently use Promptkit's `openrouter` backend:
|
||||
|
||||
| Profile ID | Model | Reasoning effort | Timeout | Service tier | Default reports |
|
||||
| --- | --- | --- | --- | --- | --- |
|
||||
| `weather-light` | `deepseek/deepseek-v4-flash` | Provider default | 180 seconds | `flex` | Hourly |
|
||||
| `weather-balanced` | `~google/gemini-flash-latest` | `high` | 240 seconds | `flex` | Daily, Today, Tomorrow |
|
||||
| `weather-deep` | `~anthropic/claude-sonnet-latest` | `high` | 240 seconds | `flex` | None |
|
||||
|
||||
The `~` prefix is part of each OpenRouter rolling-alias model ID. The embedded
|
||||
profiles intentionally omit endpoints, credentials, temperature, `top_p`, and
|
||||
output-token limits.
|
||||
|
||||
## Selection, lookup, and active execution
|
||||
|
||||
Before collection, Weatherreporter inspects the exact prompt version and output
|
||||
contract. A nonblank `promptkit.profile` selects one profile ID for every
|
||||
report in the command; otherwise the prompt's declared default selects it.
|
||||
Promptkit resolves the selected definition in this order:
|
||||
|
||||
1. explicit in-memory profiles used by an embedding consumer or test;
|
||||
2. the configured `profile_file` or `profile_dir`;
|
||||
3. Weatherreporter's embedded fallback profiles; and
|
||||
4. Promptkit's built-in catalog.
|
||||
|
||||
A source falls through only when the selected ID is absent. Each source
|
||||
supplies a complete definition, so profile fields are not merged. A malformed
|
||||
matching operator definition is an error and does not fall back.
|
||||
|
||||
Profiles that require a direct API key are unsupported; a profile that reports
|
||||
`APIKeyEnv` requires a nonblank value in that environment variable. Inspection,
|
||||
preparation, and execution retain the selected logical profile ID and resolved
|
||||
backend and model through Weatherreporter's project-owned contract. Ordinary
|
||||
errors, summaries, logs, and workspace state exclude endpoints, credentials,
|
||||
rendered messages, schemas, request bodies, response bodies, and complete
|
||||
parameter maps.
|
||||
|
||||
Execution receives the already-persisted YAML package, prepares it once, and returns structured
|
||||
JSON that Weatherreporter validates before rendering its own Markdown template. Preparation and
|
||||
@@ -21,6 +54,7 @@ The generated-text schemas require `summary`, `forecast_discussion`, and
|
||||
`precipitation_timing`, and reject additional properties. Prompts return an empty string for
|
||||
`precipitation_timing` when the deterministic package contains no precipitation windows.
|
||||
|
||||
Prompt/profile configuration is owned by the [configuration reference](../config.md). Adapter
|
||||
construction and mapping are documented in the [Promptkit adapter internals](../internal/promptkit-adapter.md).
|
||||
Prompt/profile configuration and the maintained local override example are
|
||||
owned by the [configuration reference](../config.md). Adapter construction and
|
||||
mapping are documented in the [Promptkit adapter internals](../internal/promptkit-adapter.md).
|
||||
Durable metadata compatibility is described in [state internals](../internal/state.md).
|
||||
|
||||
@@ -7,7 +7,10 @@ notification ordering after the CLI has parsed arguments and loaded configuratio
|
||||
|
||||
`GenerateDetailed` resolves one of the four report definitions, initializes an
|
||||
optional debug root, and inspects the exact Promptkit prompt/profile before it
|
||||
collects weather or writes managed state. It then builds facts and modules,
|
||||
collects weather or writes managed state. A configured global profile selects
|
||||
every report in the action; otherwise the exact prompt selects its default
|
||||
logical profile. Inspection keeps only the selected profile ID and effective
|
||||
backend/model needed by the project-owned execution contract. It then builds facts and modules,
|
||||
saves the YAML data package, persists preparation metadata from the executor
|
||||
callback, executes the prepared prompt, saves execution provenance and raw
|
||||
output, validates generated text, renders Markdown, and optionally copies or
|
||||
@@ -25,8 +28,8 @@ persists raw output and execution provenance but does not render a report.
|
||||
|
||||
`RunBatchDetailed` constructs a single debug writer and uses the request's
|
||||
single executor. Before collection it inspects Today, Tomorrow, and Daily for
|
||||
morning, or Tomorrow and Daily for evening, deduplicating effective profile
|
||||
inspection. It then collects once, plans eligible Daily dates, and calls the
|
||||
morning, or Tomorrow and Daily for evening, deduplicating inspection of a
|
||||
shared selected profile. It then collects once, plans eligible Daily dates, and calls the
|
||||
same prompt-generation core sequentially for each planned report. Per-report
|
||||
notification is suppressed; a failed report does not stop later reports.
|
||||
|
||||
|
||||
@@ -4,10 +4,18 @@
|
||||
The CLI maps `promptkit` configuration to a `PromptExecutorConfig` and constructs one executor
|
||||
per action. Promptkit dependency types do not escape the adapter.
|
||||
|
||||
The adapter exposes exact prompt and profile inspection plus prepared execution. It maps Promptkit
|
||||
inspection values to project-owned prompt input, output-contract, profile, preparation, execution,
|
||||
validation, and optional debug values. It classifies adapter failures without copying provider secrets
|
||||
or unbounded response bodies into application errors or normal state.
|
||||
The adapter supplies Weatherreporter's embedded prompt, schema, and fallback
|
||||
profile filesystems to each engine. Promptkit remains responsible for resolving
|
||||
the configured operator profile source, application fallback catalog, and its
|
||||
built-in catalog; the adapter does not parse profile YAML, merge sources, or
|
||||
probe endpoints.
|
||||
|
||||
The adapter exposes exact prompt and profile inspection plus prepared execution.
|
||||
It maps Promptkit inspection values to project-owned prompt input,
|
||||
output-contract, logical profile identity, effective backend/model,
|
||||
preparation, execution, validation, and optional debug values. It classifies
|
||||
adapter failures without copying provider secrets or unbounded response bodies
|
||||
into application errors or normal state.
|
||||
|
||||
The app calls the executor's preparation callback before provider execution to persist safe preparation
|
||||
provenance. Completed executions are then persisted as safe execution provenance and raw generated text
|
||||
|
||||
@@ -16,16 +16,19 @@ templates, generation eligibility, compatible prior IDs, default modules, and
|
||||
batch eligibility flags. `Resolved` combines that definition with the valid
|
||||
period and run metadata for one invocation.
|
||||
|
||||
| Report ID | Prompt version | Period policy | Comparison | Registry batch flag | Output copy |
|
||||
| --- | --- | --- | --- | --- | --- |
|
||||
| `daily` | `1.0.1` | Explicit local civil day | Same valid date | Dynamic Daily inclusion is app-owned | `daily.md` |
|
||||
| `today` | `1.0.1` | Selected or current local civil day | Same valid date | Morning | `today.md` |
|
||||
| `tomorrow` | `1.0.1` | Next local civil day | Same valid date | Evening | `tomorrow.md` |
|
||||
| `hourly` | `1.0.1` | Rolling six-hour interval | Rolling window | — | `hourly.md` |
|
||||
| Report ID | Prompt version | Default profile | Period policy | Comparison | Registry batch flag | Output copy |
|
||||
| --- | --- | --- | --- | --- | --- | --- |
|
||||
| `daily` | `1.1.0` | `weather-balanced` | Explicit local civil day | Same valid date | Dynamic Daily inclusion is app-owned | `daily.md` |
|
||||
| `today` | `1.1.0` | `weather-balanced` | Selected or current local civil day | Same valid date | Morning | `today.md` |
|
||||
| `tomorrow` | `1.1.0` | `weather-balanced` | Next local civil day | Same valid date | Evening | `tomorrow.md` |
|
||||
| `hourly` | `1.1.0` | `weather-light` | Rolling six-hour interval | Rolling window | — | `hourly.md` |
|
||||
|
||||
Each report pairs its ID and prompt version with matching template and schema
|
||||
IDs. Exact template fields and schema assets belong to [report templates](../templates.md)
|
||||
and [generated-text internals](generatedtext.md).
|
||||
and [generated-text internals](generatedtext.md). Prompt assets own default
|
||||
profile selection; the registry deliberately stores no provider setting. The
|
||||
[Promptkit integration guide](../integrations/promptkit.md) owns profile
|
||||
definitions and resolution.
|
||||
|
||||
All valid periods are half-open.
|
||||
|
||||
|
||||
@@ -34,8 +34,10 @@ runs never write V1 records.
|
||||
Prompt preparation and execution records are validated on both save and load.
|
||||
They require exact report/prompt identity, complete timing, internally
|
||||
consistent provenance, and status-appropriate validation or bounded classified
|
||||
errors. Completed execution provenance keeps Promptkit's run identity distinct
|
||||
from the Weatherreporter run identity.
|
||||
errors. Completed preparation and execution provenance retain the selected
|
||||
logical profile ID and resolved backend/model, but never profile endpoints or
|
||||
credentials. Completed execution provenance keeps Promptkit's run identity
|
||||
distinct from the Weatherreporter run identity.
|
||||
|
||||
For a completed prompt run, the execution record is atomically replaced after
|
||||
each downstream artifact is saved. Its path set therefore records the raw and
|
||||
|
||||
@@ -25,6 +25,30 @@ Distributor notification is attempted. `--out` writes an extra operator copy;
|
||||
it never changes the managed report or upload source. A successful generate
|
||||
command prints its summary to stdout unless `--quiet` is used.
|
||||
|
||||
## Local Prompt Profile Override
|
||||
|
||||
Hourly normally selects the embedded `weather-light` profile. To use a local
|
||||
OpenAI-compatible model without changing prompts or application code, copy
|
||||
[weather-light-local-profile.yml](../examples/weather-light-local-profile.yml),
|
||||
set its `endpoint` and `model` for the local server, and configure the copy as
|
||||
`promptkit.profile_file`. The profile file's `weather-light` definition
|
||||
completely replaces the embedded definition; it does not affect a report that
|
||||
selects another profile ID.
|
||||
|
||||
For example, install the profile file at a known absolute path and set:
|
||||
|
||||
```yaml
|
||||
promptkit:
|
||||
profile_file: /etc/weatherreporter/weather-light-local-profile.yml
|
||||
```
|
||||
|
||||
Prompt inspection occurs before weather collection. A malformed profile file,
|
||||
missing required credential, or unsupported selected backend stops the command
|
||||
before collection. A reachable profile can still fail later if its local model
|
||||
endpoint is unavailable; Weatherreporter does not switch to a remote profile.
|
||||
See the [configuration reference](config.md) for field definitions and the
|
||||
[troubleshooting guide](troubleshooting.md) for recovery.
|
||||
|
||||
## Optional Prompt Debug Capture
|
||||
|
||||
Use `--llm-debug-dir` only when content-rich prompt diagnostics are required:
|
||||
@@ -174,5 +198,7 @@ first response; retain it until the failure is understood.
|
||||
not publish them unintentionally.
|
||||
- Weatherreporter uses one configured Weather API endpoint and local workspace
|
||||
state.
|
||||
- Promptkit profile resolution does not discover local endpoints or fail over
|
||||
between local and remote profiles.
|
||||
- It does not provide automatic resume, cleanup, archival, remote state, daemon
|
||||
operation, or automatic storm monitoring.
|
||||
|
||||
@@ -11,6 +11,28 @@ the configured `promptkit` profile or profile source, confirm the exact
|
||||
Promptkit asset is available, and supply any reported environment credential.
|
||||
Do not add provider keys to YAML. See [configuration](config.md).
|
||||
|
||||
## Local profile override is malformed or selects an unexpected model
|
||||
|
||||
`promptkit.profile_file` and `promptkit.profile_dir` supply complete profile
|
||||
definitions. A same-ID definition replaces the embedded profile, and a malformed
|
||||
matching definition fails before collection instead of falling back. Validate
|
||||
the selected profile's YAML, ID, backend or endpoint, and model. If the model
|
||||
is unexpected, first check the global `promptkit.profile` selection and then
|
||||
look for a same-ID definition in the configured file or directory.
|
||||
|
||||
The preparation and execution receipts named by run metadata retain the
|
||||
selected profile ID and effective backend/model for diagnosis, but not an
|
||||
endpoint or credential. See the maintained
|
||||
[local `weather-light` profile example](../examples/weather-light-local-profile.yml).
|
||||
|
||||
## Local model endpoint is unavailable
|
||||
|
||||
An endpoint-only `weather-light` override can pass preflight and still fail
|
||||
during provider preparation or execution when the local server is unavailable
|
||||
or does not accept the configured model. Start the local server, correct the
|
||||
endpoint or model in the profile, and run the command again. Weatherreporter
|
||||
does not probe endpoints or automatically use a remote profile instead.
|
||||
|
||||
## Preparation, capacity, or execution fails
|
||||
|
||||
A preparation failure occurs before provider work; an execution failure occurs
|
||||
|
||||
4
examples/weather-light-local-profile.yml
Normal file
4
examples/weather-light-local-profile.yml
Normal file
@@ -0,0 +1,4 @@
|
||||
id: weather-light
|
||||
endpoint: http://127.0.0.1:11434/v1
|
||||
model: weather-local
|
||||
timeout_seconds: 180
|
||||
@@ -150,6 +150,14 @@ model: directory-light
|
||||
assertProfile(t, directoryAdapter, "weather-light", promptkit.BackendLocal, "directory-light")
|
||||
}
|
||||
|
||||
func TestMaintainedWeatherLightLocalProfileExampleInspectsOffline(t *testing.T) {
|
||||
adapter, err := New(Config{ProfileFile: filepath.Join("..", "..", "..", "examples", "weather-light-local-profile.yml")})
|
||||
if err != nil {
|
||||
t.Fatalf("New() error = %v", err)
|
||||
}
|
||||
assertProfile(t, adapter, "weather-light", "", "weather-local")
|
||||
}
|
||||
|
||||
func TestProfileResolutionFallsThroughOnlyWhenTheConfiguredIDIsAbsent(t *testing.T) {
|
||||
absentAdapter, err := New(Config{ProfileDirectory: testProfileDirectory(t, `id: other-profile
|
||||
backend: openrouter
|
||||
|
||||
Reference in New Issue
Block a user