Document logical prompt profile configuration
This commit is contained in:
@@ -22,8 +22,12 @@ extra report copies for a command and do not change configuration.
|
|||||||
collection and generation configuration.
|
collection and generation configuration.
|
||||||
- [config.yml](../examples/config.yml) is a representative production-oriented
|
- [config.yml](../examples/config.yml) is a representative production-oriented
|
||||||
configuration using synthetic endpoints and no credentials.
|
configuration using synthetic endpoints and no credentials.
|
||||||
|
- [weather-light-local-profile.yml](../examples/weather-light-local-profile.yml)
|
||||||
|
is a complete endpoint-only override for the embedded `weather-light`
|
||||||
|
profile.
|
||||||
|
|
||||||
Both files are loaded by the configuration test suite.
|
The configuration examples are loaded by the configuration test suite. The
|
||||||
|
profile example is inspected through the Promptkit adapter test suite.
|
||||||
|
|
||||||
## Minimal Configuration
|
## Minimal Configuration
|
||||||
|
|
||||||
@@ -146,13 +150,35 @@ individual `generate` or `run` command when explicitly needed.
|
|||||||
|
|
||||||
| Field | Default | Rules |
|
| Field | Default | Rules |
|
||||||
| --- | --- | --- |
|
| --- | --- | --- |
|
||||||
| `profile` | empty | Optional explicit execution profile. Otherwise the prompt's declared default is used. |
|
| `profile` | empty | Optional global profile selection for every report in one command. When empty, each exact prompt version selects its declared default. |
|
||||||
| `profile_file` | empty | Optional external profile file. Cannot be combined with `profile_dir`. |
|
| `profile_file` | empty | Optional external Promptkit profile file. It cannot be combined with `profile_dir`. A same-ID profile completely replaces Weatherreporter's embedded definition. |
|
||||||
| `profile_dir` | empty | Optional external profile directory. Cannot be combined with `profile_file`. |
|
| `profile_dir` | empty | Optional external Promptkit profile directory. It cannot be combined with `profile_file`. A same-ID profile completely replaces Weatherreporter's embedded definition. |
|
||||||
| `timeout` | `2m` | Must be greater than zero. |
|
| `timeout` | `2m` | Must be greater than zero. |
|
||||||
| `local.endpoint` | empty | Optional absolute URL for the conventional local backend. A blank endpoint leaves it unregistered. |
|
| `local.endpoint` | empty | Optional absolute URL for the conventional local backend. A blank endpoint leaves it unregistered. |
|
||||||
| `local.concurrency_limit` | `1` | Maximum local backend concurrency. `0` is unlimited; negative values are invalid. |
|
| `local.concurrency_limit` | `1` | Maximum local backend concurrency. `0` is unlimited; negative values are invalid. |
|
||||||
|
|
||||||
|
`profile` selects an ID; `profile_file` and `profile_dir` supply definitions.
|
||||||
|
They are separate decisions. An explicit `profile` applies to every selected
|
||||||
|
report. Otherwise Hourly selects `weather-light`, while Daily, Today, and
|
||||||
|
Tomorrow select `weather-balanced` through their exact `1.1.0` prompt
|
||||||
|
definitions.
|
||||||
|
|
||||||
|
Promptkit resolves a selected profile definition from a test or embedding
|
||||||
|
consumer's explicit in-memory profile, then the configured `profile_file` or
|
||||||
|
`profile_dir`, then Weatherreporter's embedded catalog, and finally Promptkit's
|
||||||
|
built-in catalog. Sources provide complete definitions; fields are never
|
||||||
|
merged. A matching malformed external profile fails rather than using the
|
||||||
|
embedded definition. The [Promptkit integration guide](integrations/promptkit.md)
|
||||||
|
owns the catalog and precedence details.
|
||||||
|
|
||||||
|
To replace the default Hourly definition with a local OpenAI-compatible
|
||||||
|
endpoint, set `profile_file` to a copy of
|
||||||
|
[weather-light-local-profile.yml](../examples/weather-light-local-profile.yml).
|
||||||
|
The example has no credential and should be edited for the local endpoint and
|
||||||
|
model before use. An alternative profile may use `backend: local`; in that
|
||||||
|
case `promptkit.local.endpoint` supplies the conventional local backend
|
||||||
|
endpoint.
|
||||||
|
|
||||||
### `workspace`
|
### `workspace`
|
||||||
|
|
||||||
| Field | Default |
|
| Field | Default |
|
||||||
|
|||||||
@@ -1,16 +1,49 @@
|
|||||||
# Promptkit Integration
|
# Promptkit Integration
|
||||||
|
|
||||||
Weatherreporter uses Promptkit for all generated-text reports. The four logical prompts are
|
Weatherreporter uses Promptkit for all generated-text reports. The four logical
|
||||||
`weather.daily_generated_text`, `weather.today_generated_text`,
|
prompts are `weather.daily_generated_text`, `weather.today_generated_text`,
|
||||||
`weather.tomorrow_generated_text`, and `weather.hourly_generated_text`, each at version
|
`weather.tomorrow_generated_text`, and `weather.hourly_generated_text`, each at
|
||||||
`1.0.1`. Their prompt assets and generated-text JSON Schemas are embedded by
|
version `1.1.0`. Their prompt assets, generated-text JSON Schemas, and
|
||||||
`internal/promptassets`.
|
Weatherreporter profile catalog are embedded by `internal/promptassets`.
|
||||||
|
|
||||||
Before collection, Weatherreporter inspects the exact prompt version, requires one required
|
## Logical profile catalog
|
||||||
`data_package` input with content type `application/yaml`, and requires the report's JSON
|
|
||||||
Schema output contract. It selects `promptkit.profile` when configured, otherwise the
|
Prompt definitions select a stable Weatherreporter profile ID. The embedded
|
||||||
prompt's declared default profile. Profiles that require a direct API key are unsupported; a
|
definitions currently use Promptkit's `openrouter` backend:
|
||||||
profile that reports `APIKeyEnv` requires a nonblank value in that environment variable.
|
|
||||||
|
| Profile ID | Model | Reasoning effort | Timeout | Service tier | Default reports |
|
||||||
|
| --- | --- | --- | --- | --- | --- |
|
||||||
|
| `weather-light` | `deepseek/deepseek-v4-flash` | Provider default | 180 seconds | `flex` | Hourly |
|
||||||
|
| `weather-balanced` | `~google/gemini-flash-latest` | `high` | 240 seconds | `flex` | Daily, Today, Tomorrow |
|
||||||
|
| `weather-deep` | `~anthropic/claude-sonnet-latest` | `high` | 240 seconds | `flex` | None |
|
||||||
|
|
||||||
|
The `~` prefix is part of each OpenRouter rolling-alias model ID. The embedded
|
||||||
|
profiles intentionally omit endpoints, credentials, temperature, `top_p`, and
|
||||||
|
output-token limits.
|
||||||
|
|
||||||
|
## Selection, lookup, and active execution
|
||||||
|
|
||||||
|
Before collection, Weatherreporter inspects the exact prompt version and output
|
||||||
|
contract. A nonblank `promptkit.profile` selects one profile ID for every
|
||||||
|
report in the command; otherwise the prompt's declared default selects it.
|
||||||
|
Promptkit resolves the selected definition in this order:
|
||||||
|
|
||||||
|
1. explicit in-memory profiles used by an embedding consumer or test;
|
||||||
|
2. the configured `profile_file` or `profile_dir`;
|
||||||
|
3. Weatherreporter's embedded fallback profiles; and
|
||||||
|
4. Promptkit's built-in catalog.
|
||||||
|
|
||||||
|
A source falls through only when the selected ID is absent. Each source
|
||||||
|
supplies a complete definition, so profile fields are not merged. A malformed
|
||||||
|
matching operator definition is an error and does not fall back.
|
||||||
|
|
||||||
|
Profiles that require a direct API key are unsupported; a profile that reports
|
||||||
|
`APIKeyEnv` requires a nonblank value in that environment variable. Inspection,
|
||||||
|
preparation, and execution retain the selected logical profile ID and resolved
|
||||||
|
backend and model through Weatherreporter's project-owned contract. Ordinary
|
||||||
|
errors, summaries, logs, and workspace state exclude endpoints, credentials,
|
||||||
|
rendered messages, schemas, request bodies, response bodies, and complete
|
||||||
|
parameter maps.
|
||||||
|
|
||||||
Execution receives the already-persisted YAML package, prepares it once, and returns structured
|
Execution receives the already-persisted YAML package, prepares it once, and returns structured
|
||||||
JSON that Weatherreporter validates before rendering its own Markdown template. Preparation and
|
JSON that Weatherreporter validates before rendering its own Markdown template. Preparation and
|
||||||
@@ -21,6 +54,7 @@ The generated-text schemas require `summary`, `forecast_discussion`, and
|
|||||||
`precipitation_timing`, and reject additional properties. Prompts return an empty string for
|
`precipitation_timing`, and reject additional properties. Prompts return an empty string for
|
||||||
`precipitation_timing` when the deterministic package contains no precipitation windows.
|
`precipitation_timing` when the deterministic package contains no precipitation windows.
|
||||||
|
|
||||||
Prompt/profile configuration is owned by the [configuration reference](../config.md). Adapter
|
Prompt/profile configuration and the maintained local override example are
|
||||||
construction and mapping are documented in the [Promptkit adapter internals](../internal/promptkit-adapter.md).
|
owned by the [configuration reference](../config.md). Adapter construction and
|
||||||
|
mapping are documented in the [Promptkit adapter internals](../internal/promptkit-adapter.md).
|
||||||
Durable metadata compatibility is described in [state internals](../internal/state.md).
|
Durable metadata compatibility is described in [state internals](../internal/state.md).
|
||||||
|
|||||||
@@ -7,7 +7,10 @@ notification ordering after the CLI has parsed arguments and loaded configuratio
|
|||||||
|
|
||||||
`GenerateDetailed` resolves one of the four report definitions, initializes an
|
`GenerateDetailed` resolves one of the four report definitions, initializes an
|
||||||
optional debug root, and inspects the exact Promptkit prompt/profile before it
|
optional debug root, and inspects the exact Promptkit prompt/profile before it
|
||||||
collects weather or writes managed state. It then builds facts and modules,
|
collects weather or writes managed state. A configured global profile selects
|
||||||
|
every report in the action; otherwise the exact prompt selects its default
|
||||||
|
logical profile. Inspection keeps only the selected profile ID and effective
|
||||||
|
backend/model needed by the project-owned execution contract. It then builds facts and modules,
|
||||||
saves the YAML data package, persists preparation metadata from the executor
|
saves the YAML data package, persists preparation metadata from the executor
|
||||||
callback, executes the prepared prompt, saves execution provenance and raw
|
callback, executes the prepared prompt, saves execution provenance and raw
|
||||||
output, validates generated text, renders Markdown, and optionally copies or
|
output, validates generated text, renders Markdown, and optionally copies or
|
||||||
@@ -25,8 +28,8 @@ persists raw output and execution provenance but does not render a report.
|
|||||||
|
|
||||||
`RunBatchDetailed` constructs a single debug writer and uses the request's
|
`RunBatchDetailed` constructs a single debug writer and uses the request's
|
||||||
single executor. Before collection it inspects Today, Tomorrow, and Daily for
|
single executor. Before collection it inspects Today, Tomorrow, and Daily for
|
||||||
morning, or Tomorrow and Daily for evening, deduplicating effective profile
|
morning, or Tomorrow and Daily for evening, deduplicating inspection of a
|
||||||
inspection. It then collects once, plans eligible Daily dates, and calls the
|
shared selected profile. It then collects once, plans eligible Daily dates, and calls the
|
||||||
same prompt-generation core sequentially for each planned report. Per-report
|
same prompt-generation core sequentially for each planned report. Per-report
|
||||||
notification is suppressed; a failed report does not stop later reports.
|
notification is suppressed; a failed report does not stop later reports.
|
||||||
|
|
||||||
|
|||||||
@@ -4,10 +4,18 @@
|
|||||||
The CLI maps `promptkit` configuration to a `PromptExecutorConfig` and constructs one executor
|
The CLI maps `promptkit` configuration to a `PromptExecutorConfig` and constructs one executor
|
||||||
per action. Promptkit dependency types do not escape the adapter.
|
per action. Promptkit dependency types do not escape the adapter.
|
||||||
|
|
||||||
The adapter exposes exact prompt and profile inspection plus prepared execution. It maps Promptkit
|
The adapter supplies Weatherreporter's embedded prompt, schema, and fallback
|
||||||
inspection values to project-owned prompt input, output-contract, profile, preparation, execution,
|
profile filesystems to each engine. Promptkit remains responsible for resolving
|
||||||
validation, and optional debug values. It classifies adapter failures without copying provider secrets
|
the configured operator profile source, application fallback catalog, and its
|
||||||
or unbounded response bodies into application errors or normal state.
|
built-in catalog; the adapter does not parse profile YAML, merge sources, or
|
||||||
|
probe endpoints.
|
||||||
|
|
||||||
|
The adapter exposes exact prompt and profile inspection plus prepared execution.
|
||||||
|
It maps Promptkit inspection values to project-owned prompt input,
|
||||||
|
output-contract, logical profile identity, effective backend/model,
|
||||||
|
preparation, execution, validation, and optional debug values. It classifies
|
||||||
|
adapter failures without copying provider secrets or unbounded response bodies
|
||||||
|
into application errors or normal state.
|
||||||
|
|
||||||
The app calls the executor's preparation callback before provider execution to persist safe preparation
|
The app calls the executor's preparation callback before provider execution to persist safe preparation
|
||||||
provenance. Completed executions are then persisted as safe execution provenance and raw generated text
|
provenance. Completed executions are then persisted as safe execution provenance and raw generated text
|
||||||
|
|||||||
@@ -16,16 +16,19 @@ templates, generation eligibility, compatible prior IDs, default modules, and
|
|||||||
batch eligibility flags. `Resolved` combines that definition with the valid
|
batch eligibility flags. `Resolved` combines that definition with the valid
|
||||||
period and run metadata for one invocation.
|
period and run metadata for one invocation.
|
||||||
|
|
||||||
| Report ID | Prompt version | Period policy | Comparison | Registry batch flag | Output copy |
|
| Report ID | Prompt version | Default profile | Period policy | Comparison | Registry batch flag | Output copy |
|
||||||
| --- | --- | --- | --- | --- | --- |
|
| --- | --- | --- | --- | --- | --- | --- |
|
||||||
| `daily` | `1.0.1` | Explicit local civil day | Same valid date | Dynamic Daily inclusion is app-owned | `daily.md` |
|
| `daily` | `1.1.0` | `weather-balanced` | Explicit local civil day | Same valid date | Dynamic Daily inclusion is app-owned | `daily.md` |
|
||||||
| `today` | `1.0.1` | Selected or current local civil day | Same valid date | Morning | `today.md` |
|
| `today` | `1.1.0` | `weather-balanced` | Selected or current local civil day | Same valid date | Morning | `today.md` |
|
||||||
| `tomorrow` | `1.0.1` | Next local civil day | Same valid date | Evening | `tomorrow.md` |
|
| `tomorrow` | `1.1.0` | `weather-balanced` | Next local civil day | Same valid date | Evening | `tomorrow.md` |
|
||||||
| `hourly` | `1.0.1` | Rolling six-hour interval | Rolling window | — | `hourly.md` |
|
| `hourly` | `1.1.0` | `weather-light` | Rolling six-hour interval | Rolling window | — | `hourly.md` |
|
||||||
|
|
||||||
Each report pairs its ID and prompt version with matching template and schema
|
Each report pairs its ID and prompt version with matching template and schema
|
||||||
IDs. Exact template fields and schema assets belong to [report templates](../templates.md)
|
IDs. Exact template fields and schema assets belong to [report templates](../templates.md)
|
||||||
and [generated-text internals](generatedtext.md).
|
and [generated-text internals](generatedtext.md). Prompt assets own default
|
||||||
|
profile selection; the registry deliberately stores no provider setting. The
|
||||||
|
[Promptkit integration guide](../integrations/promptkit.md) owns profile
|
||||||
|
definitions and resolution.
|
||||||
|
|
||||||
All valid periods are half-open.
|
All valid periods are half-open.
|
||||||
|
|
||||||
|
|||||||
@@ -34,8 +34,10 @@ runs never write V1 records.
|
|||||||
Prompt preparation and execution records are validated on both save and load.
|
Prompt preparation and execution records are validated on both save and load.
|
||||||
They require exact report/prompt identity, complete timing, internally
|
They require exact report/prompt identity, complete timing, internally
|
||||||
consistent provenance, and status-appropriate validation or bounded classified
|
consistent provenance, and status-appropriate validation or bounded classified
|
||||||
errors. Completed execution provenance keeps Promptkit's run identity distinct
|
errors. Completed preparation and execution provenance retain the selected
|
||||||
from the Weatherreporter run identity.
|
logical profile ID and resolved backend/model, but never profile endpoints or
|
||||||
|
credentials. Completed execution provenance keeps Promptkit's run identity
|
||||||
|
distinct from the Weatherreporter run identity.
|
||||||
|
|
||||||
For a completed prompt run, the execution record is atomically replaced after
|
For a completed prompt run, the execution record is atomically replaced after
|
||||||
each downstream artifact is saved. Its path set therefore records the raw and
|
each downstream artifact is saved. Its path set therefore records the raw and
|
||||||
|
|||||||
@@ -25,6 +25,30 @@ Distributor notification is attempted. `--out` writes an extra operator copy;
|
|||||||
it never changes the managed report or upload source. A successful generate
|
it never changes the managed report or upload source. A successful generate
|
||||||
command prints its summary to stdout unless `--quiet` is used.
|
command prints its summary to stdout unless `--quiet` is used.
|
||||||
|
|
||||||
|
## Local Prompt Profile Override
|
||||||
|
|
||||||
|
Hourly normally selects the embedded `weather-light` profile. To use a local
|
||||||
|
OpenAI-compatible model without changing prompts or application code, copy
|
||||||
|
[weather-light-local-profile.yml](../examples/weather-light-local-profile.yml),
|
||||||
|
set its `endpoint` and `model` for the local server, and configure the copy as
|
||||||
|
`promptkit.profile_file`. The profile file's `weather-light` definition
|
||||||
|
completely replaces the embedded definition; it does not affect a report that
|
||||||
|
selects another profile ID.
|
||||||
|
|
||||||
|
For example, install the profile file at a known absolute path and set:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
promptkit:
|
||||||
|
profile_file: /etc/weatherreporter/weather-light-local-profile.yml
|
||||||
|
```
|
||||||
|
|
||||||
|
Prompt inspection occurs before weather collection. A malformed profile file,
|
||||||
|
missing required credential, or unsupported selected backend stops the command
|
||||||
|
before collection. A reachable profile can still fail later if its local model
|
||||||
|
endpoint is unavailable; Weatherreporter does not switch to a remote profile.
|
||||||
|
See the [configuration reference](config.md) for field definitions and the
|
||||||
|
[troubleshooting guide](troubleshooting.md) for recovery.
|
||||||
|
|
||||||
## Optional Prompt Debug Capture
|
## Optional Prompt Debug Capture
|
||||||
|
|
||||||
Use `--llm-debug-dir` only when content-rich prompt diagnostics are required:
|
Use `--llm-debug-dir` only when content-rich prompt diagnostics are required:
|
||||||
@@ -174,5 +198,7 @@ first response; retain it until the failure is understood.
|
|||||||
not publish them unintentionally.
|
not publish them unintentionally.
|
||||||
- Weatherreporter uses one configured Weather API endpoint and local workspace
|
- Weatherreporter uses one configured Weather API endpoint and local workspace
|
||||||
state.
|
state.
|
||||||
|
- Promptkit profile resolution does not discover local endpoints or fail over
|
||||||
|
between local and remote profiles.
|
||||||
- It does not provide automatic resume, cleanup, archival, remote state, daemon
|
- It does not provide automatic resume, cleanup, archival, remote state, daemon
|
||||||
operation, or automatic storm monitoring.
|
operation, or automatic storm monitoring.
|
||||||
|
|||||||
@@ -11,6 +11,28 @@ the configured `promptkit` profile or profile source, confirm the exact
|
|||||||
Promptkit asset is available, and supply any reported environment credential.
|
Promptkit asset is available, and supply any reported environment credential.
|
||||||
Do not add provider keys to YAML. See [configuration](config.md).
|
Do not add provider keys to YAML. See [configuration](config.md).
|
||||||
|
|
||||||
|
## Local profile override is malformed or selects an unexpected model
|
||||||
|
|
||||||
|
`promptkit.profile_file` and `promptkit.profile_dir` supply complete profile
|
||||||
|
definitions. A same-ID definition replaces the embedded profile, and a malformed
|
||||||
|
matching definition fails before collection instead of falling back. Validate
|
||||||
|
the selected profile's YAML, ID, backend or endpoint, and model. If the model
|
||||||
|
is unexpected, first check the global `promptkit.profile` selection and then
|
||||||
|
look for a same-ID definition in the configured file or directory.
|
||||||
|
|
||||||
|
The preparation and execution receipts named by run metadata retain the
|
||||||
|
selected profile ID and effective backend/model for diagnosis, but not an
|
||||||
|
endpoint or credential. See the maintained
|
||||||
|
[local `weather-light` profile example](../examples/weather-light-local-profile.yml).
|
||||||
|
|
||||||
|
## Local model endpoint is unavailable
|
||||||
|
|
||||||
|
An endpoint-only `weather-light` override can pass preflight and still fail
|
||||||
|
during provider preparation or execution when the local server is unavailable
|
||||||
|
or does not accept the configured model. Start the local server, correct the
|
||||||
|
endpoint or model in the profile, and run the command again. Weatherreporter
|
||||||
|
does not probe endpoints or automatically use a remote profile instead.
|
||||||
|
|
||||||
## Preparation, capacity, or execution fails
|
## Preparation, capacity, or execution fails
|
||||||
|
|
||||||
A preparation failure occurs before provider work; an execution failure occurs
|
A preparation failure occurs before provider work; an execution failure occurs
|
||||||
|
|||||||
4
examples/weather-light-local-profile.yml
Normal file
4
examples/weather-light-local-profile.yml
Normal file
@@ -0,0 +1,4 @@
|
|||||||
|
id: weather-light
|
||||||
|
endpoint: http://127.0.0.1:11434/v1
|
||||||
|
model: weather-local
|
||||||
|
timeout_seconds: 180
|
||||||
@@ -150,6 +150,14 @@ model: directory-light
|
|||||||
assertProfile(t, directoryAdapter, "weather-light", promptkit.BackendLocal, "directory-light")
|
assertProfile(t, directoryAdapter, "weather-light", promptkit.BackendLocal, "directory-light")
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func TestMaintainedWeatherLightLocalProfileExampleInspectsOffline(t *testing.T) {
|
||||||
|
adapter, err := New(Config{ProfileFile: filepath.Join("..", "..", "..", "examples", "weather-light-local-profile.yml")})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("New() error = %v", err)
|
||||||
|
}
|
||||||
|
assertProfile(t, adapter, "weather-light", "", "weather-local")
|
||||||
|
}
|
||||||
|
|
||||||
func TestProfileResolutionFallsThroughOnlyWhenTheConfiguredIDIsAbsent(t *testing.T) {
|
func TestProfileResolutionFallsThroughOnlyWhenTheConfiguredIDIsAbsent(t *testing.T) {
|
||||||
absentAdapter, err := New(Config{ProfileDirectory: testProfileDirectory(t, `id: other-profile
|
absentAdapter, err := New(Config{ProfileDirectory: testProfileDirectory(t, `id: other-profile
|
||||||
backend: openrouter
|
backend: openrouter
|
||||||
|
|||||||
Reference in New Issue
Block a user