Reconcile prompt execution provenance
This commit is contained in:
@@ -16,7 +16,7 @@ The `~` prefix is part of each OpenRouter rolling-alias model ID. The embedded p
|
||||
|
||||
## Selection And Active Execution
|
||||
|
||||
Before weather collection, Weatherreporter validates the report's exact generated-text report/schema/template catalog binding, prompt version, output contract, and selected profile. A nonblank `promptkit.profile` selects one profile ID for every report in the command; otherwise the prompt's declared default selects it. Promptkit resolves the selected definition in this order:
|
||||
Before weather collection, Weatherreporter validates the report's exact generated-text report/schema/template catalog binding, prompt version and hash, output contract, and selected profile. Active profiles must resolve a nonblank backend and model identity. A nonblank `promptkit.profile` selects one profile ID for every report in the command; otherwise the prompt's declared default selects it. Promptkit resolves the selected definition in this order:
|
||||
|
||||
1. explicit in-memory profiles used by an embedding consumer or test;
|
||||
2. the configured `profile_file` or `profile_dir`;
|
||||
@@ -27,7 +27,7 @@ A source falls through only when the selected ID is absent. Each source supplies
|
||||
|
||||
Profiles that require a direct API key are unsupported; a profile that reports `APIKeyEnv` requires a nonblank value in that environment variable. Active results retain the selected logical profile ID and resolved backend and model. Ordinary errors, summaries, logs, and outputs exclude endpoints, credentials, rendered messages, schemas, request bodies, response bodies, and complete parameter maps.
|
||||
|
||||
Promptkit receives the YAML data package as an inline input and returns structured JSON that Weatherreporter validates before rendering its own Markdown template. The package contains only reviewed prompt-facing warning summaries, never source transport or provenance details. Safe active provenance remains in memory. Content-rich diagnostics are opt-in through `--llm-debug-dir`; see [operations](../operations.md) for retention and permissions.
|
||||
Promptkit receives the YAML data package as an inline input and returns structured JSON that Weatherreporter validates before rendering its own Markdown template. Before accepting that JSON, Weatherreporter requires exactly one preparation callback and reconciles its prompt/profile/backend/model and rendered/input hashes with the inspected identity and completed result. The callback output contract and completed validation must use the report's expected JSON Schema mode and path. The package contains only reviewed prompt-facing warning summaries, never source transport or provenance details. Safe active provenance remains in memory. Content-rich diagnostics are opt-in through `--llm-debug-dir`; see [operations](../operations.md) for retention and permissions.
|
||||
|
||||
When capture is enabled, its preparation artifact projects a provider endpoint
|
||||
to its scheme and host and retains only reviewed execution settings. Provider
|
||||
@@ -42,7 +42,7 @@ binding, one exact prompt, and every explicitly selected profile before weather
|
||||
collection. It prepares one deterministic YAML
|
||||
data package, retains immutable copies of the report inputs, and executes every
|
||||
profile against the same exact data-package bytes. Each profile remains an
|
||||
independent Promptkit execution: one provider or validation failure does not
|
||||
independent Promptkit execution: one provider, provenance, or validation failure does not
|
||||
stop its peers, while caller cancellation applies to every in-flight execution.
|
||||
|
||||
Weatherreporter starts selected profile executions concurrently and does not
|
||||
|
||||
@@ -9,7 +9,7 @@ is owned by the [CLI reference](../cli.md) and [operations guide](../operations.
|
||||
|
||||
`GenerateDetailed` resolves the requested report and output destination before initializing an optional explicit debug writer. An explicit output file wins; otherwise the configured output directory is used, falling back to the captured working directory. It validates the report's generated-text catalog binding, exact Promptkit prompt, and selected profile before collecting weather data. The resolved profile, backend, and model are carried in the active result.
|
||||
|
||||
The workflow builds facts, a module snapshot, briefing metadata, and the YAML prompt package in memory. It executes Promptkit, validates the returned generated text, builds a render context, and renders Markdown. `fileutil` atomically writes the completed Markdown to the selected output path. Only after that write succeeds does single-report notification run.
|
||||
The workflow builds facts, a module snapshot, briefing metadata, and the YAML prompt package in memory. It executes Promptkit only against the inspected prompt and profile, reconciles the preparation callback and completed result with that identity and the prepared report schema, validates the returned generated text, builds a render context, and renders Markdown. `fileutil` atomically writes the completed Markdown to the selected output path. Only after that write succeeds does single-report notification run.
|
||||
|
||||
Failures return an active partial result with safe identity, profile, warning, validation, debug, and output information when available. After rendering and immediately before publication, the workflow checks for cancellation or deadline expiry. Any failure before publication leaves an existing destination unchanged. A notification failure retains the newly published output.
|
||||
|
||||
@@ -36,10 +36,11 @@ inspection fails, the partial result retains the resolved prompt ID, version,
|
||||
and hash. Artifact paths are added only after publication commits.
|
||||
|
||||
The comparison execution core starts each inspected profile independently,
|
||||
keeps results in selection order, and waits for all started work. Independent
|
||||
profile failures are recorded and do not stop peers. Context cancellation marks
|
||||
unfinished work and prevents publication. Details of prepared values, execution
|
||||
and debugging, and publication are documented in [prepared report
|
||||
keeps results in selection order, and waits for all started work. Every profile
|
||||
reconciles its callback and completion provenance before its JSON can be
|
||||
rendered. Independent profile failures are recorded and do not stop peers.
|
||||
Context cancellation marks unfinished work and prevents publication. Details of
|
||||
prepared values, execution and debugging, and publication are documented in [prepared report
|
||||
internals](prepared-report.md), [comparison execution
|
||||
internals](comparison-execution.md), and [comparison publication
|
||||
internals](comparison-publication.md).
|
||||
|
||||
@@ -17,6 +17,14 @@ Preparation deep-copies mutable facts, snapshots, identity, and data-package
|
||||
bytes before returning them. Consumers receive independent copies so one
|
||||
execution cannot change another's input or rendering context.
|
||||
|
||||
Before accepting generated JSON, the execution boundary reconciles the prepared
|
||||
report definition, inspected prompt hash and selected profile identity, the one
|
||||
preparation callback, and the completed Promptkit result. The callback and
|
||||
completion must agree on prompt, profile, backend, model, and rendered/input
|
||||
hashes; the callback output and completed validation must name the prepared
|
||||
report's JSON Schema. A mismatch produces no rendered Markdown and leaves
|
||||
results with only the inspected safe identity.
|
||||
|
||||
Single-report generation executes one prepared profile and publishes its
|
||||
Markdown. Comparison prepares once, gives every selected profile the same YAML
|
||||
bytes, and only then assembles the resulting logical bundle. The prompt-input
|
||||
|
||||
Reference in New Issue
Block a user