21 Commits

Author SHA1 Message Date
f33af20524 Prepare the v0.13.0 release
All checks were successful
ci/woodpecker/tag/release Pipeline was successful
2026-08-29 15:36:01 +00:00
85d14784ca Mark Promptkit adoption roadmap complete 2026-08-29 15:16:40 +00:00
edca9fbbc1 Document inspection architecture 2026-08-29 15:16:25 +00:00
6c2541c1ee Unify inspection configuration handling 2026-08-29 15:16:04 +00:00
b9b98b2aec List profile inspection in CLI usage 2026-08-29 15:12:45 +00:00
8936ca7c18 Add Promptkit definition conformance coverage 2026-08-29 15:12:27 +00:00
3e5f5c5198 Complete Promptkit v0.9 adoption 2026-08-29 14:31:50 +00:00
68ebe9ee50 Add profile inspection command 2026-08-29 14:31:13 +00:00
36c7c5a358 Add prompt inspection command 2026-08-29 14:29:14 +00:00
157209f097 Expose backend identity and capacity errors 2026-08-29 14:26:04 +00:00
06b37c2bae Add session and reasoning controls 2026-08-29 14:22:45 +00:00
1a0f15e210 Add custom backend configuration 2026-08-29 14:18:29 +00:00
5f946a5a1f Allow prompt version selection without inputs 2026-08-29 14:12:51 +00:00
1806df9888 Adopt the Promptkit v0.9 compatibility baseline 2026-08-29 14:08:35 +00:00
c35ac5a0cb Document the Promptkit v0.9 adoption plan 2026-08-29 13:54:37 +00:00
40f8a1c628 Identify v0.12.0 as the first published application-only release 2026-07-28 15:59:23 -05:00
27d7ad5057 Retire the completed migration roadmaps 2026-07-28 20:04:24 +00:00
1f08a1a94c Record completion of the Promptkit migration 2026-07-28 20:03:11 +00:00
2f42bdde39 Publish Promptkit migration guidance and release notes
All checks were successful
ci/woodpecker/tag/release Pipeline was successful
2026-07-28 19:43:57 +00:00
9a00f30c7b Establish the Scriptorium release procedure 2026-07-28 19:38:11 +00:00
f71d2bbb73 Add an implementation plan and roadmap for Step 9 of the migration plan 2026-07-28 14:25:56 -05:00
53 changed files with 3890 additions and 604 deletions

3
.gitignore vendored
View File

@@ -56,6 +56,8 @@ mono_crash.*
[Dd]ebugPublic/
[Rr]elease/
[Rr]eleases/
!docs/releases/
!docs/releases/*.md
x64/
x86/
[Ww][Ii][Nn]32/
@@ -433,4 +435,3 @@ FodyWeavers.xsd
# JetBrains Rider
*.sln.iml

View File

@@ -11,9 +11,16 @@ steps:
version="$CI_COMMIT_TAG"
dist="dist"
pkg="gitea.maximumdirect.net/eric/scriptorium/cmd/scriptorium"
notes="docs/releases/$version.md"
if [ ! -f "$notes" ]; then
printf 'release notes not found: %s\n' "$notes" >&2
exit 1
fi
rm -rf "$dist"
mkdir -p "$dist"
cp "$notes" "$dist/RELEASE_NOTES.md"
build_binary() {
goos="$1"
@@ -22,7 +29,7 @@ steps:
output="$dist/scriptorium-$version-$goos-$goarch$suffix"
CGO_ENABLED=0 GOOS="$goos" GOARCH="$goarch" \
go build -trimpath -ldflags "-s -w -X gitea.maximumdirect.net/eric/scriptorium/internal/buildinfo.Version=$version" \
go build -trimpath -ldflags "-s -w" \
-o "$output" "$pkg"
}
@@ -38,6 +45,7 @@ steps:
from_secret: GITEA_RELEASE_TOKEN
files:
- dist/scriptorium-*
note: dist/RELEASE_NOTES.md
checksum: sha256
checksum-file: SHA256SUMS
checksum-flatten: true

View File

@@ -6,7 +6,7 @@ OpenAI-compatible model endpoints, and returns generated output with validation
metadata.
The application uses
[Promptkit v0.1.0](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.1.0/)
[Promptkit v0.9.0](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.9.0/)
for prompt, profile, schema, preparation, generation, and validation behavior.
Scriptorium owns executable configuration, CLI and HTTP mapping, process
behavior, output presentation, and HTTP artifact-containment policy.
@@ -35,10 +35,11 @@ a model. For complete invocation and output behavior, see the
- [HTTP API reference](docs/api.md)
- [Operations guide](docs/operations.md)
- [Consumer integration overview](docs/consumers/api.md)
- [Migration from the former Go package](docs/consumers/migrating-to-promptkit.md)
- [Subprocess integration](docs/integrations/subprocess.md)
- [Architecture policy](docs/policy/architecture.md)
- [Promptkit framework formats](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.1.0/docs/formats.md)
- [Promptkit Go consumer guide](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.1.0/docs/consumers/pkg-promptkit.md)
- [Promptkit framework formats](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.9.0/docs/formats.md)
- [Promptkit Go consumer guide](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.9.0/docs/consumers/pkg-promptkit.md)
## Examples

View File

@@ -0,0 +1,112 @@
# Preserve Definition Compatibility And A Simple Execution Model
## Status
Accepted.
## Date
2026-08-29.
## Context
Scriptorium is a user-facing application built on Promptkit. Promptkit is a Go
library with both declarative framework formats and public APIs intended for
applications that need to assemble specialized integrations or workflows.
Scriptorium needs a durable rule for deciding which upstream capabilities it
must support. Treating every new Promptkit public API as an application feature
would steadily expand Scriptorium into a workflow framework. Selecting an
arbitrary subset of prompt or profile fields would instead make valid Promptkit
definitions unexpectedly unusable through Scriptorium.
The intended product is narrower: Scriptorium provides simple executable
interfaces that select and execute predefined prompts and return their results
in a consistent, repeatable form. It is not intended to own conversations,
modify prompts dynamically, coordinate follow-up messages, or manage complex
multi-step workflows.
## Decision
Scriptorium will aim to support the complete set of features expressible in
valid Promptkit prompt and profile definitions for the Promptkit version it
selects. This includes the schema, backend, source, and request plumbing needed
to prepare and execute those definitions without Scriptorium imposing a
narrower field-level format contract.
Promptkit remains the canonical parser and validator for its definitions.
Scriptorium will pass its configured framework sources and mapped request
values through the supported Promptkit public facade rather than copying
Promptkit formats, defaults, or validation rules.
Promptkit library features that are not required to use valid prompt or profile
definitions are considered individually. They may be incorporated when they
serve Scriptorium's simple execution model and fit its application-owned CLI,
HTTP, configuration, presentation, security, and process boundaries. A public
Promptkit declaration does not by itself require a corresponding Scriptorium
interface.
Each execution request will remain self-contained from the caller's
perspective. A request must not depend on conversation or workflow state
retained from a completed request, and Scriptorium will not become a general
workflow or conversation system. Application-managed prompt modification,
appended or follow-up message flows, conversation state, prepared-handle
coordination, multi-step orchestration, and similar higher-level workflows are
outside its intended scope.
Scriptorium may, and when necessary must, maintain process-scoped operational
state shared by concurrent requests. This includes the configured Promptkit
engine and backend registry, capacity admission and queue accounting,
in-flight request coordination, cancellation, and ordinary server lifecycle
resources. The state must be scoped so configured backend limits apply across
all relevant in-flight requests. But this is operational implementation state,
not durable user workflow or conversation state.
## Alternatives Considered
### Expose Every Promptkit Public Feature
This would make the application surface track the library API closely, but it
would blur the boundary between a reusable Go library and a simple executable
consumer. Library-oriented lifecycle, source-construction, extension, and
workflow primitives would add commands, wire contracts, configuration, and
state without necessarily improving predefined prompt execution.
### Support A Curated Subset Of Definition Fields
This would keep the application small in the short term, but valid Promptkit
definitions could fail or behave differently solely because they were invoked
through Scriptorium. Maintaining a parallel field-level compatibility list
would also duplicate upstream format ownership and create recurring drift.
### Expand Scriptorium Into A Workflow Service
Scriptorium could own conversations, prompt changes, follow-up messages,
retries, checkpoints, and multi-step execution. That is a different product
with durable state, lifecycle, recovery, privacy, and operational requirements
that are not justified by Scriptorium's current purpose.
## Consequences
- Promptkit upgrades require an audit of the complete prompt and profile format
contracts, not only a source-compatibility build.
- Scriptorium adapters must not add unconditional request requirements that
reject a definition Promptkit can validly prepare or execute.
- Application configuration may need new plumbing, such as custom backend
registration, when it is necessary to use a valid profile feature.
- Definition parsing, validation, precedence, inheritance, defaults, and
built-in catalogs remain Promptkit responsibilities.
- A running HTTP server must share its appropriately scoped Promptkit engine
across requests so the engine's backend admission and concurrency limits
cannot be bypassed by per-request construction.
- Process-scoped counters, queues, in-flight request records, cancellation, and
lifecycle resources are permitted implementation state. They must be
concurrency-safe and do not create a durable resume or conversation
contract.
- Library-only source constructors, injected collaborators, opaque lifecycle
handles, and request-composition primitives remain optional Scriptorium
features rather than automatic requirements.
- New application features must be evaluated against the simple,
request-independent predefined-prompt execution model.
- Work that implements this decision remains tracked in roadmaps until it is
delivered and incorporated into current-state contracts.

View File

@@ -32,14 +32,11 @@ returns `413 response_too_large`.
### Request Body
The maintained [request example](../examples/http-run.json) is a complete
copyable shape. The smallest valid shape is:
copyable shape. At the HTTP adapter boundary, the smallest valid shape is:
```json
{
"prompt_id": "generic.markdown_summary",
"inputs": {
"transcript": {"type": "inline", "body": "Source text"}
}
"prompt_id": "generic.markdown_summary"
}
```
@@ -48,7 +45,8 @@ copyable shape. The smallest valid shape is:
| `prompt_id` | yes | Non-blank prompt ID. |
| `prompt_version` | no | Prompt version filter. |
| `profile_id` | no | Execution-profile ID; otherwise the prompt must set `default_profile`. |
| `inputs` | yes | Non-empty object mapping input names to references. |
| `session_id` | no | Optional direct, non-secret session identifier. |
| `inputs` | no | Optional object mapping input names to references. Promptkit decides whether the selected definition needs them. |
| `vars` | no | Object mapping template-variable names to strings. |
| `model` | no | Runtime model-override object. |
| `include_raw_output` | no | Include `raw_model_output` when true. |
@@ -67,20 +65,31 @@ The optional `model` object accepts `endpoint`, `model`, `temperature`,
`max_tokens`, `top_p`, `timeout_seconds`, `service_tier`,
`reasoning_effort`, `api_key_env`, and `extra_params`. Numeric ranges and
framework credential semantics are defined by the
[Promptkit format reference](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.1.0/docs/formats.md).
[Promptkit format reference](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.9.0/docs/formats.md).
Explicit zero values for the numeric fields are overrides; zero
`timeout_seconds` disables the per-generation deadline only, retaining the
request context and configured transport cap. The timeout layers are defined in
the [Promptkit outbound integration contract](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.1.0/docs/integrations/openai-compatible-chat.md#timeout-and-cancellation).
the [Promptkit outbound integration contract](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.9.0/docs/integrations/openai-compatible-chat.md#timeout-and-cancellation).
Raw API-key values are not accepted. `api_key` and any other unknown model
field cause `400 invalid_json`.
`model.reasoning_effort` is an optional JSON string with three states: omission
inherits the selected profile, a non-empty string replaces its value, and an
empty string explicitly clears it. JSON `null` is treated as omission.
`session_id` is passed directly to Promptkit. A nonblank value replaces a
definition-rendered session ID; omission or a blank value lets the definition
provide one. Promptkit trims direct values and limits them to 256 Unicode code
points. Session IDs are not credentials and may be included in prepared data,
results, and provider-facing requests, so use stable non-sensitive identifiers.
### Strict JSON
Request decoding rejects malformed JSON, unknown fields at every request level,
and trailing JSON tokens with `400 invalid_json`. A blank `prompt_id` or
empty `inputs` object returns `400 invalid_request`.
and trailing JSON tokens with `400 invalid_json`. A blank `prompt_id` returns
`400 invalid_request`. Omitted or empty `inputs` are passed to Promptkit, which
reports any definition-required or template-referenced inputs.
### Success Response
@@ -91,17 +100,26 @@ validation contract. The response contains:
optional `uri`;
- `validation`: `status`, `mode`, `repair_attempts`, `is_valid`, plus
optional `errors` and `schema_path`;
- `metadata`: run, prompt, rendered-prompt, profile, model, input-hash, usage,
- `metadata`: run, prompt, rendered-prompt, profile, optional backend identity, model, input-hash, usage,
timing, validation, and repair-attempt metadata; and
- optional `raw_model_output` when requested.
`metadata.model_params` has `endpoint`, `model`, `temperature`,
`max_tokens`, `top_p`, and `timeout_seconds`, plus optional
`service_tier`, `reasoning_effort`, `api_key_env`, and `extra_params`.
`backend_id`, `service_tier`, `reasoning_effort`, `api_key_env`, and `extra_params`.
`metadata.usage` always includes `prompt_tokens`, `completion_tokens`,
`total_tokens`, `cached_tokens`, and `cache_write_tokens`; unavailable
cache usage is reported as zero.
When Promptkit resolves a direct or definition-rendered session ID,
`metadata.session_id` contains that effective result value. It is omitted when
no effective session ID exists.
`metadata.selected_backend_id` and `metadata.model_params.backend_id` report
the corresponding Promptkit result fields independently when present. Both are
omitted for an endpoint-only profile; Scriptorium does not infer backend
identity from an endpoint.
A validation failure has `validation.status: "failed"`, `is_valid: false`,
and any available diagnostic errors, while still returning the artifact and
metadata.
@@ -137,6 +155,7 @@ Messages are concise and do not expose wrapped internal causes.
| `500` | `validation_runtime_failed` | Schema or validator runtime failure. |
| `500` | `internal_error` | Unclassified server failure. |
| `502` | `llm_failed` | Outbound model request failed. |
| `503` | `capacity_exceeded` | The selected model backend has no admission capacity. No retry timing is supplied. |
## Retry And Idempotency

View File

@@ -23,6 +23,8 @@ go run ./cmd/scriptorium render \
generated artifact.
- `scriptorium render`: prepare a prompt and write prepared-run output.
- `scriptorium serve`: start the HTTP server.
- `scriptorium inspect prompt`: inspect one prompt definition without model execution.
- `scriptorium inspect profile`: inspect one effective profile without model execution.
All commands accept `--config <path>` and reject positional arguments. An
effective `prompt_dir` is required for every command. Supply it through the
@@ -39,7 +41,6 @@ Required flags:
| Flag | Meaning |
| --- | --- |
| `--prompt <id>` | Prompt ID to execute. |
| `--input name=path` | Input file mapping; repeat or use comma-separated mappings. |
Optional flags:
@@ -49,7 +50,10 @@ Optional flags:
| `--prompt-dir <dir>` | Prompt-definition directory override. |
| `--profile-dir <dir>` | Custom profile-directory override. |
| `--schema-dir <dir>` | Schema base-directory override. |
| `--prompt-version <version>` | Optional prompt-definition version selector. |
| `--profile <id>` | Execution-profile override. |
| `--session-id <id>` | Optional direct session identifier. |
| `--input name=path` | Optional input file mapping; repeat or use comma-separated mappings. |
| `--var name=value` | Template-variable mapping; repeat or use comma-separated mappings. |
| `--out <path>` | Write generated content to this file instead of stdout. |
| `--llm-base-url <url>` | Runtime endpoint override. |
@@ -58,6 +62,7 @@ Optional flags:
| `--temperature <float>` | Runtime temperature override. |
| `--max-tokens <int>` | Runtime maximum-token override. |
| `--top-p <float>` | Runtime top-p override. |
| `--reasoning-effort <value>` | Runtime reasoning-effort override. |
| `--timeout <duration>` | Runtime timeout override using Go duration syntax. |
Deprecated aliases: `--prompt-id` for `--prompt`, and `--profile-id` for
@@ -69,25 +74,44 @@ only; the caller context and configured transport cap remain active. CLI
durations are converted to whole seconds by truncation toward zero, so any
duration whose absolute value is below one second becomes an explicit
zero-second override. The timeout layers are defined in the
[Promptkit outbound integration contract](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.1.0/docs/integrations/openai-compatible-chat.md#timeout-and-cancellation).
[Promptkit outbound integration contract](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.9.0/docs/integrations/openai-compatible-chat.md#timeout-and-cancellation).
There is no raw API-key flag. Use `--api-key-env`.
`--prompt-version` is passed directly to Promptkit. When it is omitted, the
selected prompt ID must have exactly one available version. `--input` is
optional at the CLI boundary: Promptkit decides whether the selected definition
requires declared inputs or template-referenced values.
`--session-id` supplies a direct, non-secret session identifier. A nonblank
value replaces a definition-rendered session ID; an omitted or blank value lets
the definition supply one. Promptkit trims direct values and limits them to 256
Unicode code points. Use stable, non-sensitive identifiers because effective
session IDs may appear in prepared output, run metadata, and provider-facing
requests.
`--reasoning-effort` is presence-aware: omitting it inherits the selected
profile value, a nonblank value replaces that value, and
`--reasoning-effort=` explicitly clears inherited reasoning. Promptkit treats
nonblank values as provider-specific opaque strings.
## `scriptorium render`
```text
scriptorium render [flags]
```
`--prompt <id>` and at least one `--input name=path` are required. The
following optional flags are supported: `--config`, `--prompt-dir`,
`--profile-dir`, `--profile`, `--var`, `--out`, `--llm-base-url`,
`--prompt <id>` is required. The following optional flags are supported:
`--config`, `--prompt-dir`, `--profile-dir`, `--prompt-version`, `--profile`,
`--input`, `--var`, `--out`, `--llm-base-url`,
`--model`, `--api-key-env`, `--temperature`, `--max-tokens`, `--top-p`,
`--timeout`, and `--format text|json`. Their meanings match the corresponding
`--reasoning-effort`, `--session-id`, `--timeout`, and `--format text|json`. Their meanings match the corresponding
`run` flags; `--format` selects prepared-run output and otherwise uses
`defaults.render_format`.
The same deprecated aliases and numeric/timeout behavior as `run` apply.
The same session and reasoning inheritance, replacement, and clearing behavior
also applies.
`render` does not accept `--schema-dir`; configure `schema_dir` through the
configuration file. It resolves profiles and schemas as part of preparation but
does not call an LLM.
@@ -115,6 +139,31 @@ Optional flags:
`serve` accepts no runtime model override flags. HTTP request fields, response
schemas, and error codes are defined in the [HTTP API reference](api.md).
## `scriptorium inspect prompt`
```text
scriptorium inspect prompt --prompt ID [--prompt-version VERSION]
[--config PATH] [--prompt-dir DIR] [--format text|json] [--out PATH]
```
`--prompt` is required. Inspection uses normal configuration discovery and a
`--prompt-dir` override, defaults to text regardless of `defaults.render_format`,
and writes to stdout unless `--out` is supplied. It loads and normalizes the
selected definition but does not resolve a profile, load a schema, render a
template, reserve backend capacity, or call a model.
## `scriptorium inspect profile`
```text
scriptorium inspect profile --profile ID
[--config PATH] [--profile-dir DIR] [--format text|json] [--out PATH]
```
`--profile` is required; built-in profiles need no prompt directory. Inspection
resolves profile inheritance and backend defaults, but never reads a credential
value, loads a prompt, reserves capacity, or calls a model. Unset provider
controls are shown as zero values when Promptkit leaves them unspecified.
## Input And Variable Syntax
`--input name=path` maps an input name to a local file; `--var name=value`
@@ -128,9 +177,12 @@ CLI inputs are file references. HTTP inline inputs are defined by the
## Output And Exit Behavior
- `run` writes generated content to stdout, or to `--out` when supplied, and
writes a concise summary to stderr.
writes a concise summary to stderr. The summary includes `backend=<id>` when
Promptkit selected a backend; endpoint-only profiles omit it.
- `render` writes prepared-run output to stdout, or to `--out` when supplied,
without a success summary.
without a success summary. Text output includes `selected_backend_id` after
`selected_profile_id` when Promptkit selected one; endpoint-only profiles
omit it.
- `serve` writes startup and server errors to stderr.
Exit statuses:
@@ -141,6 +193,10 @@ Exit statuses:
| `1` | Parse, configuration, loading, rendering, generation, output-write, or other runtime error. |
| `2` | `run` generated and wrote output, but validation failed. |
A backend admission rejection is a runtime error and prints `run error: model
backend capacity is exhausted`. The HTTP capacity response is defined in the
[HTTP API reference](api.md).
## Workflows And Examples
The [maintained render script](../examples/render-markdown-summary.sh) is a

View File

@@ -3,7 +3,7 @@
This is the canonical reference for Scriptorium application settings. Prompt,
profile, schema, execution-setting, built-in profile, and framework credential
semantics are defined by the
[Promptkit v0.1.0 format reference](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.1.0/docs/formats.md).
[Promptkit v0.9.0 format reference](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.9.0/docs/formats.md).
For command syntax, see the [CLI reference](cli.md); for HTTP request shapes and
outcomes, see the [HTTP API reference](api.md).
@@ -39,11 +39,39 @@ do not override a prior value. Raw API-key fields are not accepted.
| `server.max_artifact_bytes` | `16777216` | Maximum HTTP file-input artifact bytes; `0` disables the limit. |
| `server.max_response_bytes` | `16777216` | Maximum encoded HTTP response bytes; `0` disables the limit. |
| `defaults.render_format` | `text` | Default prepared-run output format: `text` or `json`. |
| `backends` | unset | Optional mapping of custom Promptkit backend IDs to engine-scoped connection and capacity settings. |
The size fields must be zero or greater. The [HTTP API](api.md) defines how
each limit is enforced and reported. `server.artifact_root` configures an HTTP
deployment boundary; see [operations](operations.md) for deployment handling.
## Custom Backends
Use `backends` when a profile selects an application-defined backend ID:
```yaml
backends:
local-gpu:
endpoint: http://localhost:11434/v1
api_key_env: LOCAL_GPU_API_KEY
extra_params:
provider_option: enabled
concurrency_limit: 2
queue_capacity: 0
```
Each mapping key is the case-sensitive backend ID. `endpoint` is required;
`api_key_env`, `extra_params`, `concurrency_limit`, and `queue_capacity` are
optional. `concurrency_limit: 0` leaves the backend unlimited. Omitting
`queue_capacity` lets Promptkit use its default for a limited backend, while
an explicit `queue_capacity: 0` disables queueing.
Configuration strictly owns the YAML shape and rejects unknown fields. Promptkit
validates backend IDs, endpoints, environment-variable names, extra parameters,
and capacity relationships when Scriptorium constructs its engine. There are no
backend command-line overrides. Store only an environment-variable name in
`api_key_env`; raw API-key fields are not accepted.
## Framework Source Mapping
Scriptorium passes `prompt_dir`, `profile_dir`, and `schema_dir` to Promptkit
@@ -58,7 +86,7 @@ parse those framework file formats.
contracts.
See the
[tagged Promptkit format reference](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.1.0/docs/formats.md)
[tagged Promptkit format reference](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.9.0/docs/formats.md)
for all of those definitions. The files under
[`examples/prompts`](../examples/prompts/),
[`examples/profiles`](../examples/profiles/), and
@@ -73,8 +101,20 @@ their names in configuration or framework source files. Do not place raw keys
in configuration, prompts, profiles, CLI arguments, examples, or HTTP
payloads.
An optional `api_key_env` whose environment value is absent or empty can result
in an unauthenticated provider request. A profile that declares credentials
required still fails as an invalid request when no credential source is
selected, and fails with `ErrAPIKeyEnvMissing` when its selected environment
source is absent or empty. Scriptorium never reads or emits the environment
value itself.
Unset optional provider controls are omitted from compatible provider requests.
A positive Promptkit `repair_attempts` budget can add provider calls, latency,
token use, and cost; see the tagged format reference for its permitted values
and validation-mode requirements.
Promptkit's
[OpenAI-compatible integration contract](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.1.0/docs/integrations/openai-compatible-chat.md)
[OpenAI-compatible integration contract](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.9.0/docs/integrations/openai-compatible-chat.md)
defines outbound authentication, provider request mapping, transport limits,
and timeout layering.
@@ -83,4 +123,4 @@ and timeout layering.
- [CLI reference](cli.md)
- [HTTP API reference](api.md)
- [Operations guide](operations.md)
- [Promptkit framework formats](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.1.0/docs/formats.md)
- [Promptkit framework formats](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.9.0/docs/formats.md)

View File

@@ -14,8 +14,10 @@ and the HTTP service according to the boundary your application needs.
Go applications that need an in-process prompt framework should import
Promptkit directly. The tagged
[Promptkit Go consumer guide](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.1.0/docs/consumers/pkg-promptkit.md)
[Promptkit Go consumer guide](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.9.0/docs/consumers/pkg-promptkit.md)
owns that interface; Scriptorium does not provide a Go library package.
Consumers arriving from the former Scriptorium Go API should follow the
[migration guide](migrating-to-promptkit.md).
## Consumer Responsibilities
@@ -33,4 +35,4 @@ Scriptorium does not persist run state. A retry can produce different output
and can incur another provider request. CLI exits belong to the
[CLI reference](../cli.md), HTTP status behavior belongs to the
[HTTP API reference](../api.md), and framework semantics belong to
[Promptkit v0.1.0](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.1.0/docs/formats.md).
[Promptkit v0.9.0](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.9.0/docs/formats.md).

View File

@@ -0,0 +1,109 @@
# Migrate From Scriptorium To Promptkit
## Supported Migration Boundary
Scriptorium `v0.11.1` at
`gitea.maximumdirect.net/eric/scriptorium` is the final release that provides
the former in-process Go framework. Promptkit `v0.9.0` at
`gitea.maximumdirect.net/eric/promptkit` is the destination for that framework
API. Scriptorium `v0.12.0` and later provide the CLI and HTTP application only.
There is no Scriptorium compatibility facade, alias package, forwarding
package, or deprecated wrapper. A consumer that cannot migrate may remain
pinned to Scriptorium `v0.11.1`, but that framework-bearing line does not
provide the slim application release.
## Update A Go Consumer
Start from a clean consumer checkout and review the pending diff before
committing it. Add the published Promptkit module:
```sh
go get gitea.maximumdirect.net/eric/promptkit@v0.9.0
```
For an ordinary consumer that imports the former root package under its
default name, replace the exact import and package qualifier, then format the
changed Go files:
```sh
git grep -l \
'"gitea.maximumdirect.net/eric/scriptorium"' \
-- '*.go' |
while IFS= read -r go_file
do
perl -pi -e \
's{"gitea.maximumdirect.net/eric/scriptorium"}{"gitea.maximumdirect.net/eric/promptkit"}g; s{\bscriptorium\.}{promptkit.}g' \
"$go_file"
gofmt -w "$go_file"
done
```
Inspect the resulting diff. Consumers that used an import alias should retain
or deliberately rename that alias instead of applying the qualifier
replacement mechanically.
Remove the now-unused Scriptorium requirement through module tidiness and run
the consumer's complete tests:
```sh
go mod tidy
go test ./...
```
Confirm that `go.mod` selects Promptkit `v0.9.0` and that no Go file imports
the former Scriptorium package:
```sh
test "$(
go list -m -f '{{.Path}}@{{.Version}}' \
gitea.maximumdirect.net/eric/promptkit
)" = 'gitea.maximumdirect.net/eric/promptkit@v0.9.0'
if git grep -n \
'gitea.maximumdirect.net/eric/scriptorium' \
-- '*.go'
then
printf '%s\n' 'a former Scriptorium Go import remains' >&2
exit 1
fi
```
## Compatibility And Additions
Promptkit preserves the established engine, request, result, profile,
source-option, model-client, artifact, validation-value, and public-error
shapes where practical. Exact declarations and current behavior belong to the
tagged [Promptkit consumer guide](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.9.0/docs/consumers/pkg-promptkit.md)
and Go source.
Promptkit also includes migration-relevant public contracts that were not in
Scriptorium `v0.11.1`:
- [`WithArtifactReader`](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.9.0/engine.go#L182)
and the
[`ArtifactReader` declaration](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.9.0/types.go#L304)
provide the artifact-reading extension described by the tagged
[extension-interface guide](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.9.0/docs/consumers/pkg-promptkit.md#extension-interfaces).
- [`ErrProfileRequired` and `ErrAPIKeyEnvMissing`](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.9.0/engine.go#L46-L61)
provide the specific identities described by the tagged
[error guide](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.9.0/docs/consumers/pkg-promptkit.md#errors).
Use those tagged owners for exact signatures, wrapping guarantees, and
extension behavior.
## Verify Consumer Behavior
Source compatibility is only the first check. Exercise the behavior the
consumer actually relies upon, especially:
- prompt, profile, and schema source selection;
- direct and environment-based credentials;
- caller, generation, and transport timeout layering;
- output validation and validation-failure handling;
- injected model-client and artifact-reader extensions; and
- every `errors.Is` branch used for recovery or classification.
Also verify any serialized values, redaction expectations, filesystem policy,
and provider integration behavior that crosses the consumer's own boundary.
Promptkit owns the in-process framework contract; Scriptorium owns only its
executable CLI and HTTP application interfaces.

View File

@@ -2,7 +2,7 @@
This is the contributor entry point for Scriptorium. Scriptorium is an
application that consumes the public
[Promptkit v0.1.0 package](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.1.0/docs/consumers/pkg-promptkit.md);
[Promptkit v0.9.0 package](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.9.0/docs/consumers/pkg-promptkit.md);
framework implementation work belongs in Promptkit.
## Initial Orientation
@@ -27,12 +27,12 @@ tests.
| HTTP routes, DTOs, limits, status mapping, or artifact policy | [HTTP API contract](api.md), [adapter internals](internal/adapters.md), and [source internals](internal/sources.md) |
| Application configuration or precedence | [Configuration contract](config.md), [adapter internals](internal/adapters.md), and [source internals](internal/sources.md) |
| Prepared-run presentation | [CLI contract](cli.md), [adapter internals](internal/adapters.md), and `internal/format` |
| Prompt, profile, schema, generation, or validation semantics | [Promptkit framework formats](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.1.0/docs/formats.md) and the [Promptkit consumer guide](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.1.0/docs/consumers/pkg-promptkit.md) |
| OpenAI-compatible outbound behavior or timeout layering | [Promptkit integration contract](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.1.0/docs/integrations/openai-compatible-chat.md) |
| Prompt, profile, schema, generation, or validation semantics | [Promptkit framework formats](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.9.0/docs/formats.md) and the [Promptkit consumer guide](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.9.0/docs/consumers/pkg-promptkit.md) |
| OpenAI-compatible outbound behavior or timeout layering | [Promptkit integration contract](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.9.0/docs/integrations/openai-compatible-chat.md) |
| Subprocess behavior | [Subprocess integration](integrations/subprocess.md) and [CLI contract](cli.md) |
| Runtime operation or recovery | [Operations](operations.md) |
| Release packaging | `.woodpecker/release.yml`, [operations](operations.md), and the architecture policy |
| Examples or copyable assets | The owning Scriptorium contract, the relevant [Promptkit format contract](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.1.0/docs/formats.md), and the related files under `examples/` |
| Release packaging or publication | The [release procedure](release.md), [hosted release workflow](../.woodpecker/release.yml), and [architecture policy](policy/architecture.md) |
| Examples or copyable assets | The owning Scriptorium contract, the relevant [Promptkit format contract](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.9.0/docs/formats.md), and the related files under `examples/` |
| Architecture decisions or future work | The [documentation policy](policy/documentation.md), relevant accepted ADRs, and relevant roadmap documents |
Cross-project changes land and release in Promptkit before Scriptorium adopts

View File

@@ -9,7 +9,7 @@ own IO and presentation mechanics, not framework decisions.
External contracts are canonical in the [CLI reference](../cli.md) and
[HTTP API reference](../api.md). Promptkit's public engine contract is
described by its tagged
[Go consumer guide](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.1.0/docs/consumers/pkg-promptkit.md).
[Go consumer guide](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.9.0/docs/consumers/pkg-promptkit.md).
## Components And Collaborators
@@ -29,15 +29,30 @@ described by its tagged
### CLI
`run` calls `promptkit.Engine.Run`; `render` calls
`promptkit.Engine.Prepare`. Both share request mapping for prompt/profile
selection, file inputs, variables, and presence-aware execution overrides.
Omitted framework settings remain zero values so Promptkit resolves its own
defaults.
`promptkit.Engine.Prepare`. Both share request mapping for prompt ID/version
and profile selection, optional file inputs, variables, and presence-aware
execution overrides. Omitted framework settings remain zero values so Promptkit
resolves its own defaults and definition-required inputs.
`inspect prompt` maps the selected ID and optional version to
`promptkit.Engine.InspectPrompt`, then formats Scriptorium-owned inspection
data. It constructs the same configuration-aware engine but does not perform
preparation or generation.
`inspect profile` maps an explicit profile ID to `Engine.InspectProfile` and
formats a safe application-owned effective profile view. It intentionally does
not require prompt, schema, or artifact sources.
`serve` constructs Scriptorium's restricted HTTP artifact reader, injects it
with `promptkit.WithArtifactReader`, passes the engine through the HTTP
adapter's consumer-owned `Runner` interface, and starts the server.
All three CLI paths assemble the engine from the same resolved prompt, profile,
and schema directories plus configured custom backends. Each backend is mapped
to Promptkit's public `Backend` value and registered during engine construction,
so one constructed server engine retains one immutable backend registry and its
associated capacity state.
### HTTP
The handler enforces transport limits and strict JSON decoding before mapping

View File

@@ -11,8 +11,8 @@ public behavior belongs in the linked contracts.
| `internal/adapter/http` | Owns routes, DTOs, strict decoding, limits, Promptkit request/result mapping, public error mapping, and restricted HTTP artifact reading. | [HTTP API](../api.md), [adapter internals](adapters.md), [source internals](sources.md) |
| `internal/config` | Discovers and strictly decodes application configuration and applies built-in and CLI precedence. | [configuration contract](../config.md), [adapter internals](adapters.md) |
| `internal/defaults` | Holds Scriptorium-owned application and HTTP defaults. | [configuration contract](../config.md) |
| `internal/format` | Formats Promptkit prepared-run values for CLI text or JSON output. | [CLI contract](../cli.md), [adapter internals](adapters.md) |
| `internal/format` | Formats Promptkit prepared-run and prompt/profile inspection values for CLI text or JSON output. | [CLI contract](../cli.md), [adapter internals](adapters.md) |
Framework implementation packages are provided by
[Promptkit v0.1.0](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.1.0/docs/consumers/pkg-promptkit.md)
[Promptkit v0.9.0](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.9.0/docs/consumers/pkg-promptkit.md)
and are not part of this repository.

View File

@@ -5,14 +5,16 @@
This document covers Scriptorium-owned source locations and the restricted HTTP
artifact reader. Prompt, profile, schema, and ordinary artifact semantics are
owned by the tagged
[Promptkit format reference](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.1.0/docs/formats.md).
[Promptkit format reference](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.9.0/docs/formats.md).
## Application Source Locations
`internal/config` resolves `prompt_dir`, `profile_dir`, and `schema_dir` from
Scriptorium defaults, configuration files, and CLI overrides.
`internal/adapter/cli` passes those paths into `promptkit.Config` when
constructing the engine.
Scriptorium defaults, configuration files, and CLI overrides. It also resolves
the application-owned `backends` mapping into sorted engine settings.
`internal/adapter/cli` passes the directories into `promptkit.Config` and maps
each configured backend to Promptkit's public engine registration when
constructing an engine shared by the command path.
Scriptorium does not search, parse, validate, or overlay framework source files
itself. Promptkit owns prompt selection, profile built-ins and overlays, schema

View File

@@ -9,7 +9,7 @@ fields, or HTTP wire behavior.
- [CLI reference](cli.md): commands, output destinations, and exit codes.
- [Configuration reference](config.md): application settings, source
locations, defaults, and credential mapping.
- [Promptkit framework formats](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.1.0/docs/formats.md):
- [Promptkit framework formats](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.9.0/docs/formats.md):
prompt, profile, schema, execution-setting, and framework credential
contracts.
- [HTTP API reference](api.md): route, request/response schema, status codes,
@@ -94,6 +94,14 @@ protect request bodies, HTTP file artifacts, and encoded responses; configure
them through the [configuration reference](config.md) and rely on the
[HTTP API reference](api.md) for their response effects.
Configured backend concurrency and queue capacity are enforced per constructed
Promptkit engine. A `serve` process constructs one engine for its handler, so
concurrent HTTP requests share that transient admission state. Scriptorium does
not retain workflow state: capacity is neither durable nor a queue of resumable
runs. When admission is exhausted, HTTP returns `503 capacity_exceeded` without
retry timing; callers choose any retry policy that is safe for another model
call.
Before increasing a limit:
1. measure representative input, generated-output, and optional raw-output

View File

@@ -5,16 +5,18 @@ durable development boundaries.
## System Shape
Scriptorium is an executable application with three entry paths: CLI `run`, CLI
`render`, and the HTTP service started by `serve`. It does not expose a reusable
root Go package.
Scriptorium is an executable application with four command entry paths: CLI
`run`, CLI `render`, CLI `inspect`, and the HTTP service started by `serve`.
The `inspect` command has prompt and profile modes. Scriptorium does not expose
a reusable root Go package.
The application consumes
[Promptkit v0.1.0](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.1.0/docs/consumers/pkg-promptkit.md)
[Promptkit v0.9.0](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.9.0/docs/consumers/pkg-promptkit.md)
through its supported root package. Promptkit owns prompt execution,
preparation, source formats, built-in profiles, model-client behavior, and
validation. Scriptorium owns application configuration, executable adapters,
prepared-run presentation, process behavior, and HTTP deployment policy.
prepared-run and definition-inspection presentation, process behavior, and HTTP
deployment policy.
The concrete package inventory is maintained in the
[internal overview](../internal/overview.md).
@@ -33,6 +35,7 @@ gitea.maximumdirect.net/eric/promptkit
- Retained application packages may import Promptkit's root package.
- They must not import Promptkit `internal` packages.
- They must not import Promptkit catalog modules directly.
- They must not import the removed Scriptorium root facade or recreate former
framework package families.
- Adapter-owned interfaces use Promptkit public values when a consumer-side
@@ -52,11 +55,12 @@ invariants.
- `internal/config` owns discovery and strict decoding of Scriptorium
application configuration.
- `internal/defaults` owns Scriptorium application and HTTP defaults only.
- `internal/format` owns deterministic prepared-run text and JSON presentation.
- `internal/format` owns deterministic prepared-run and definition-inspection
text and JSON presentation.
- Promptkit owns framework orchestration and contracts. Its
[format reference](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.1.0/docs/formats.md)
[format reference](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.9.0/docs/formats.md)
and
[outbound integration contract](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.1.0/docs/integrations/openai-compatible-chat.md)
[outbound integration contract](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.9.0/docs/integrations/openai-compatible-chat.md)
are canonical.
## HTTP Artifact Security Boundary

View File

@@ -71,6 +71,8 @@ secret values.
| CLI contract | `docs/cli.md` | Commands, arguments, flags, invocation semantics, and exit codes. | End-to-end operating procedures, configuration field definitions, runtime filesystem layout, module implementation details. |
| Configuration contract | `docs/config.md` | Application discovery and precedence, source locations, server fields, render default, HTTP limits, and credential mapping. | Promptkit framework formats and defaults, complete example files, CLI syntax, runtime lifecycle, and implementation detail. |
| Operations | `docs/operations.md` | Runtime workflows, physical filesystem and state layout, output, cache, and debug handling, resume, cleanup, permissions, recovery, and operational limits. | CLI flag syntax, configuration field definitions, logical output schemas, implementation mechanics. |
| Release procedure | `docs/release.md` | Candidate validation, version and tag operations, hosted-workflow observation, and published-artifact verification. | Runtime operations, version-specific announcements, and complete application-interface contracts. |
| Version-specific release notes | `docs/releases/` | Immutable release summaries, compatibility notices, and migration announcements for one published version. | Complete CLI, HTTP, configuration, operations, or dependency contracts. |
| Public HTTP contract | `docs/api.md` | Routes, authentication, media types, request and response schemas, status codes, pagination, caching, idempotency, rate limits, and HTTP retry semantics. | Client walkthroughs, upstream or downstream integration internals, implementation detail. |
| Consumer guidance | `docs/consumers/` | Choosing between Scriptorium's executable interfaces and understanding consumer responsibilities. | HTTP wire semantics, CLI syntax, Promptkit's Go package, and internal implementation detail. |
| External and durable integration contracts | `docs/integrations/` | Scriptorium-owned process and executable integration contracts. | Promptkit framework formats and outbound provider protocols, physical runtime placement, internal transformations, CLI syntax, and configuration defaults. |

377
docs/release.md Normal file
View File

@@ -0,0 +1,377 @@
# Release Procedure
## Release Model And Status
Scriptorium publishes annotated semantic tags and tag-triggered Linux binary
releases. The hosted
[release workflow](../.woodpecker/release.yml) builds `amd64` and `arm64`
executables, publishes their SHA-256 checksums, and uses the matching file
under `docs/releases/` as the hosted release body.
`v0.12.0` is the first published application-only release. For each later
release, select a new `vMAJOR.MINOR.PATCH` version according to the intended
compatibility change. A selected version remains an unreleased candidate until
its annotated tag is published, the hosted workflow succeeds, and every
published artifact is verified.
Run this procedure from the Scriptorium repository root. A release must not
depend on a Go workspace, module replacement, vendor tree, sibling checkout,
unpublished dependency, or unpushed source commit.
## Establish The Candidate
Start a POSIX shell, choose a semantic version that has not been published, and
export it as `RELEASE_VERSION`. For example, if `v0.12.1` is the intended next
version and remains unpublished, select:
```sh
export RELEASE_VERSION=v0.12.1
```
Use the version appropriate to the actual compatibility change rather than
assuming that the example is the next release. Then run the following guard in
that same shell:
```sh
set -eu
: "${RELEASE_VERSION:?export an unpublished vMAJOR.MINOR.PATCH version}"
if ! printf '%s\n' "$RELEASE_VERSION" |
grep -Eq '^v(0|[1-9][0-9]*)\.(0|[1-9][0-9]*)\.(0|[1-9][0-9]*)$'
then
printf '%s\n' "invalid release version: $RELEASE_VERSION" >&2
exit 1
fi
RELEASE_COMMIT=$(git rev-parse --verify 'HEAD^{commit}')
export RELEASE_COMMIT
check_release_candidate() {
test "$(git branch --show-current)" = main
test -z "$(git status --porcelain)"
gowork_value=$(go env GOWORK)
case "$gowork_value" in
''|off) ;;
*)
printf '%s\n' "active Go workspace: $gowork_value" >&2
return 1
;;
esac
test -z "$(git ls-files go.work go.work.sum)"
test ! -e vendor
if grep -Eq '^[[:space:]]*replace([[:space:]]|\()' go.mod
then
printf '%s\n' 'go.mod contains a replacement' >&2
return 1
fi
git fetch origin main --tags
test "$RELEASE_COMMIT" = \
"$(git rev-parse --verify 'refs/remotes/origin/main^{commit}')"
if git show-ref --verify --quiet "refs/tags/$RELEASE_VERSION"
then
printf '%s\n' "local tag already exists: $RELEASE_VERSION" >&2
return 1
fi
if test -n "$(
git ls-remote --tags origin \
"refs/tags/$RELEASE_VERSION" \
"refs/tags/$RELEASE_VERSION^{}"
)"
then
printf '%s\n' "remote tag already exists: $RELEASE_VERSION" >&2
return 1
fi
}
check_release_candidate
```
Do not continue unless this guard succeeds. It deliberately requires the
candidate to be the exact clean commit already published at `origin/main`.
## Verify Modules And Repository Boundaries
Confirm the module path and declared Go version:
```sh
test "$(
GOWORK=off go list -m -f '{{.Path}} {{.GoVersion}}'
)" = 'gitea.maximumdirect.net/eric/scriptorium 1.25.5'
```
Require Promptkit `v0.9.0` as both the direct module-graph edge and the selected
module version:
```sh
direct_promptkit=$(
GOWORK=off go mod graph |
awk '
$1 == "gitea.maximumdirect.net/eric/scriptorium" &&
$2 ~ /^gitea\.maximumdirect\.net\/eric\/promptkit@/ {
print $2
}
'
)
test "$direct_promptkit" = \
'gitea.maximumdirect.net/eric/promptkit@v0.9.0'
test "$(
GOWORK=off go list -m -f '{{.Path}}@{{.Version}}' \
gitea.maximumdirect.net/eric/promptkit
)" = 'gitea.maximumdirect.net/eric/promptkit@v0.9.0'
GOWORK=off go list -m all
```
Require tidy module metadata and recheck the repository exclusions:
```sh
GOWORK=off go mod tidy -diff
test -z "$(git ls-files go.work go.work.sum)"
test ! -e vendor
if grep -Eq '^[[:space:]]*replace([[:space:]]|\()' go.mod
then
printf '%s\n' 'go.mod contains a replacement' >&2
exit 1
fi
test -z "$(git status --porcelain)"
```
## Validate The Application
Run the complete application validation:
```sh
GOWORK=off go test ./...
GOWORK=off go test -race ./...
GOWORK=off go vet ./...
validation_build_dir=$(mktemp -d)
GOWORK=off go build \
-o "$validation_build_dir/scriptorium" \
./cmd/scriptorium
```
The ordinary test run includes the architecture guard that rejects a root Go
package, former framework package families, and imports of Promptkit internal
packages. Inspect the repository for generated binaries, credentials,
temporary output, sibling paths, and other files that do not belong in the
tracked release source.
Check every tracked Go file. This command must produce no output:
```sh
unformatted=$(
git ls-files '*.go' |
while IFS= read -r go_file
do
gofmt -l "$go_file"
done
)
test -z "$unformatted"
```
Run the maintained render and Promptkit v0.9 feature scripts, then smoke-test
both maintained configuration examples without a model call:
```sh
GOWORK=off ./examples/render-markdown-summary.sh
GOWORK=off ./examples/render-v0.9-features.sh
for config_file in examples/config.yml examples/config.full.yml
do
GOWORK=off go run ./cmd/scriptorium render \
--config "$config_file" \
--prompt generic.markdown_summary \
--input transcript=./examples/fixtures/transcript.md \
--input glossary=./examples/fixtures/glossary.yml \
--format json >/dev/null
done
```
Exercise usage output and offline rendering with the temporary native
executable:
```sh
usage_output="$validation_build_dir/usage.txt"
if "$validation_build_dir/scriptorium" >"$usage_output" 2>&1
then
printf '%s\n' 'expected an invocation without a command to fail' >&2
exit 1
fi
grep -F 'usage: scriptorium' "$usage_output"
"$validation_build_dir/scriptorium" render \
--config ./examples/config.yml \
--prompt generic.markdown_summary \
--input transcript=./examples/fixtures/transcript.md \
--input glossary=./examples/fixtures/glossary.yml \
--format text >/dev/null
```
Follow every maintained local, Promptkit-tagged, and other external Markdown
link. Confirm that all repository-relative link targets exist. Finish the
application checks with:
```sh
git diff --check
test -z "$(git status --porcelain)"
```
## Require Release Notes And Reproduce Packaging
The immutable release note must exist before tagging:
```sh
release_notes="docs/releases/$RELEASE_VERSION.md"
test -f "$release_notes"
test -s "$release_notes"
```
Validate every local and currently published link in the note. For links
pinned to the candidate Scriptorium tag, confirm that the corresponding
repository-relative path exists even though its tag URL is not live yet.
Reproduce the hosted build flags, targets, and filenames in a temporary
directory:
```sh
release_dist=$(mktemp -d)
release_package='gitea.maximumdirect.net/eric/scriptorium/cmd/scriptorium'
build_release_binary() {
target_os="$1"
target_arch="$2"
output="$release_dist/scriptorium-$RELEASE_VERSION-$target_os-$target_arch"
CGO_ENABLED=0 GOOS="$target_os" GOARCH="$target_arch" GOWORK=off \
go build -trimpath -ldflags '-s -w' \
-o "$output" "$release_package"
}
build_release_binary linux amd64
build_release_binary linux arm64
test -s "$release_dist/scriptorium-$RELEASE_VERSION-linux-amd64"
test -s "$release_dist/scriptorium-$RELEASE_VERSION-linux-arm64"
file "$release_dist/scriptorium-$RELEASE_VERSION-linux-amd64"
file "$release_dist/scriptorium-$RELEASE_VERSION-linux-arm64"
```
Require `file` to identify Linux executables for `x86-64` and `ARM aarch64`,
respectively. Inspect the
[hosted workflow](../.woodpecker/release.yml) and confirm that it uses the
same build flags and names, copies the selected release note to
`dist/RELEASE_NOTES.md`, publishes only `dist/scriptorium-*`, and keeps
checksum generation enabled.
## Create And Publish The Tag
Run the candidate guard again immediately before creating the tag:
```sh
check_release_candidate
test -f "$release_notes"
test -s "$release_notes"
```
Create an annotated tag explicitly bound to the validated commit, using the
version-specific release note as its message:
```sh
git tag --annotate "$RELEASE_VERSION" \
--file "$release_notes" \
"$RELEASE_COMMIT"
```
Inspect the tag and require it to resolve to the validated source:
```sh
test "$(git cat-file -t "refs/tags/$RELEASE_VERSION")" = tag
git show --no-patch --decorate "refs/tags/$RELEASE_VERSION"
test "$(
git rev-parse --verify "refs/tags/$RELEASE_VERSION^{commit}"
)" = "$RELEASE_COMMIT"
```
If inspection finds an error, delete only the unpublished local tag, correct
the candidate, and repeat the complete validation. Never move or recreate a
published tag.
Push only the selected tag ref:
```sh
git push origin \
"refs/tags/$RELEASE_VERSION:refs/tags/$RELEASE_VERSION"
```
## Observe And Verify Publication
Open the hosted workflow run for the selected tag. Require
`build-release-assets` to succeed before `publish-release`, then require the
publication step and hosted release to succeed. A queued, running, failed, or
partially published workflow is not a verified release.
Compare the local and remote annotated-tag objects and their source commits:
```sh
remote_tag=$(
git ls-remote --tags origin "refs/tags/$RELEASE_VERSION" |
awk 'NR == 1 { print $1 }'
)
remote_commit=$(
git ls-remote --tags origin "refs/tags/$RELEASE_VERSION^{}" |
awk 'NR == 1 { print $1 }'
)
test -n "$remote_tag"
test "$remote_tag" = \
"$(git rev-parse --verify "refs/tags/$RELEASE_VERSION")"
test "$remote_commit" = "$RELEASE_COMMIT"
```
Download the hosted binaries and checksum file into a temporary directory:
```sh
release_base="https://gitea.maximumdirect.net/eric/scriptorium/releases/download/$RELEASE_VERSION"
download_dir=$(mktemp -d)
(
cd "$download_dir"
for asset in \
"scriptorium-$RELEASE_VERSION-linux-amd64" \
"scriptorium-$RELEASE_VERSION-linux-arm64" \
SHA256SUMS
do
curl --fail --location --remote-name "$release_base/$asset"
done
test -s "scriptorium-$RELEASE_VERSION-linux-amd64"
test -s "scriptorium-$RELEASE_VERSION-linux-arm64"
test -s SHA256SUMS
sha256sum --check SHA256SUMS
test "$(wc -l < SHA256SUMS | tr -d ' ')" = 2
file "scriptorium-$RELEASE_VERSION-linux-amd64"
file "scriptorium-$RELEASE_VERSION-linux-arm64"
)
```
Require the same Linux architectures observed in the local packaging check and
confirm that the hosted release contains no unexpected asset. On a compatible
Linux host, make the matching downloaded binary executable and repeat the
usage-output and offline-render smoke checks against it.
Only after the tag, workflow, release body, binaries, architectures, and
checksums all pass verification is the candidate a verified published release.
## Handle Failures
Before tag publication, correct the release commit or note and restart the
complete procedure. After tag publication, never delete, move, overwrite, or
recreate the tag. A transient hosted failure may be retried only against the
same immutable tag and commit and only when doing so cannot overwrite or
silently retain partial assets. A source, packaging, note, or artifact defect
requires a new corrective semantic version from a new validated commit.
Record the selected version, validated commit, tag object, workflow result,
artifact names, checksum result, and smoke-check outcome in the release
checkpoint. Keep temporary builds and downloaded assets outside the repository,
and require a clean `main` synchronized with `origin/main` when verification
is complete.

36
docs/releases/v0.12.0.md Normal file
View File

@@ -0,0 +1,36 @@
# Scriptorium v0.12.0
## Breaking Project Boundary
Scriptorium is now an executable-only CLI and HTTP application. This is a
breaking change for Go consumers: the former root Go package is not included,
and no compatibility facade is provided.
Scriptorium `v0.11.1` was the final framework-bearing release. Former Go
consumers should follow the
[migration guide](https://gitea.maximumdirect.net/eric/scriptorium/src/tag/v0.12.0/docs/consumers/migrating-to-promptkit.md)
and adopt
[Promptkit `v0.1.0`](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.1.0/docs/consumers/pkg-promptkit.md)
for in-process prompt preparation and execution.
## Application Interfaces
The Scriptorium command-line and HTTP application interfaces remain. Their
canonical documentation defines the supported commands, configuration,
requests, responses, operational behavior, and deployment responsibilities:
- [CLI reference](https://gitea.maximumdirect.net/eric/scriptorium/src/tag/v0.12.0/docs/cli.md)
- [HTTP API reference](https://gitea.maximumdirect.net/eric/scriptorium/src/tag/v0.12.0/docs/api.md)
- [Configuration reference](https://gitea.maximumdirect.net/eric/scriptorium/src/tag/v0.12.0/docs/config.md)
- [Operations guide](https://gitea.maximumdirect.net/eric/scriptorium/src/tag/v0.12.0/docs/operations.md)
## Framework Dependency And Consumers
The released Scriptorium binaries use Promptkit `v0.1.0` as their framework
dependency. Promptkit owns the reusable engine, source formats, profiles,
generation boundary, and validation contracts. See the
[Promptkit Go consumer guide](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.1.0/docs/consumers/pkg-promptkit.md)
for that supported API.
All known downstream Go consumers were migrated to Promptkit before this
release.

59
docs/releases/v0.13.0.md Normal file
View File

@@ -0,0 +1,59 @@
# Scriptorium v0.13.0
## Promptkit v0.9 Adoption
Scriptorium now uses
[Promptkit `v0.9.0`](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.9.0/)
and supports the complete Promptkit v0.9 prompt and profile definition format
through its configured directory sources. This includes prompt versions,
optional inputs, all supported message roles, cache control, session templates,
output schemas and repair budgets, profile inheritance, backend selection,
provider controls, optional credential environment names, and JSON-compatible
extra parameters.
Promptkit remains the canonical owner of definition parsing and framework
validation. Scriptorium continues to own its CLI, HTTP, configuration,
presentation, and deployment contracts.
## Application Features
- `run` and `render` can select an explicit prompt version and can execute
prompts that do not declare or reference inputs.
- CLI and HTTP requests can provide a direct session ID. Reasoning controls can
inherit, replace, or explicitly clear the selected profile value.
- Application configuration can register custom OpenAI-compatible backends
with optional credential environment names, extra parameters, concurrency
limits, and queue policy.
- `inspect prompt` and `inspect profile` provide deterministic text or JSON
views without invoking a model. Profile inspection does not read credential
values or require a prompt definition.
- Prepared output, CLI summaries, and HTTP metadata report effective backend
identity when available.
- HTTP requests rejected by backend capacity policy return `503` with the
stable `capacity_exceeded` code. One server-scoped Promptkit engine enforces
those limits across concurrent requests.
See the versioned application contracts for exact behavior:
- [CLI reference](https://gitea.maximumdirect.net/eric/scriptorium/src/tag/v0.13.0/docs/cli.md)
- [HTTP API reference](https://gitea.maximumdirect.net/eric/scriptorium/src/tag/v0.13.0/docs/api.md)
- [Configuration reference](https://gitea.maximumdirect.net/eric/scriptorium/src/tag/v0.13.0/docs/config.md)
- [Operations guide](https://gitea.maximumdirect.net/eric/scriptorium/src/tag/v0.13.0/docs/operations.md)
- [Promptkit v0.9 format reference](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.9.0/docs/formats.md)
## Compatibility And Operations
There are no intentional removals from the v0.12 application interfaces. The
Promptkit upgrade does enforce its current definition-safety rules, including
supported message roles, absolute HTTP or HTTPS endpoints, contained content
and schema paths, bounded repair budgets, and JSON-compatible extra parameters.
Definitions rejected by these rules must be corrected before use.
Credential values remain outside configuration, definition files, CLI flags,
and HTTP payloads. Named environment sources are optional unless a selected
Promptkit profile explicitly requires a key. A positive repair budget permits
additional provider calls and can increase latency, token use, and cost.
The HTTP service retains only the shared operational state needed for backend
admission and in-flight request handling. Scriptorium does not retain durable
conversation, workflow, checkpoint, or resume state.

View File

@@ -1,64 +1,601 @@
# Step 7 Implementation Completion
# Promptkit v0.9.0 Adoption Implementation Plan
## Status
## Status And Authority
Complete as of 2026-07-28.
Complete. Implement the stages below in numeric order. The accepted feature
scope and target state are defined by the
[Promptkit v0.9.0 adoption roadmap](promptkit-v0.9.0-adoption.md), and the
durable product boundary is defined by
[ADR 0004](../adr/0004-definition-boundary.md). If this plan and either source
conflict, the ADR governs product boundaries and the feature roadmap governs
scope.
## Result
Follow the architecture, documentation, and testing policies under
`docs/policy/` throughout the work. In particular:
Scriptorium now consumes
`gitea.maximumdirect.net/eric/promptkit v0.1.0` as an ordinary direct module
dependency. The published tag resolves to Promptkit commit
`9e68a2bbf779545995270c47842048a3bc6c85dc`; no workspace, replacement,
vendored dependency, or sibling-repository state is required.
- use only Promptkit's public root package and its released v0.9.0 tag;
- do not add a workspace, module replacement, vendored Promptkit source,
direct catalog import, or Scriptorium reimplementation of Promptkit parsing
and validation;
- keep default tests deterministic, offline, and free of real credentials or
provider calls;
- update each current-state contract in the same stage that implements its
behavior; and
- prefer a small number of boundary and integration tests over duplicating
Promptkit's own framework test matrix.
The Scriptorium module root no longer exposes a Go package. The remaining
production boundary consists of:
At the start of every stage, inspect the working tree and preserve unrelated
changes. At the end of every stage, run the focused tests named in that stage,
`gofmt` changed Go files, and run `git diff --check`. Do not proceed while a
stage's completion conditions are unmet.
- `cmd/scriptorium`;
- `internal/adapter/cli`;
- `internal/adapter/http`;
- `internal/config`;
- `internal/defaults`; and
- `internal/format`.
## Stage 1: Adopt The Promptkit v0.9.0 Compatibility Baseline
The CLI constructs and invokes Promptkit engines, the HTTP adapter maps its
transport contract to Promptkit public values, and the HTTP-specific restricted
artifact reader is injected through Promptkit's public extension point.
Scriptorium retains application configuration, presentation, transport,
containment, process, packaging, and executable-example responsibilities.
**Completion: Complete.**
The former root facade, duplicated framework packages, built-in profile copy,
framework tests and fixtures, Go-library example, and framework-owned
documentation were removed. Permanent Scriptorium documentation now describes
the CLI and HTTP application and links to the tagged Promptkit documentation
for framework contracts.
Make the released dependency the build baseline before adding new application
surface area.
## Validation Evidence
### Implementation
Scriptorium passed, outside a Go workspace and without a replacement:
1. Update `go.mod` to require
`gitea.maximumdirect.net/eric/promptkit v0.9.0`, then run `go mod tidy` so
`go.mod` and `go.sum` contain the module graph selected by that tag. Accept
Promptkit's OpenRouter and Rakestrawhome catalog modules only as transitive
dependencies; do not import them from Scriptorium.
2. In the HTTP request DTO, change `model.reasoning_effort` from `string` to
`*string`, and pass the pointer directly to
`promptkit.ExecutionTargetOverride.ReasoningEffort`. This is required by the
v0.9.0 API and preserves omitted versus explicit-empty input. Keep the JSON
field itself a string when present.
3. Compile every Promptkit public struct literal against v0.9.0. Keep literals
keyed, do not restore the removed `RunRequest.Metadata` field, and do not
add compatibility wrappers around Promptkit.
4. Audit maintained prompt, profile, schema, configuration, and HTTP examples
against v0.9.0. Correct only actual incompatibilities: endpoints must be
absolute HTTP(S) URLs, roles must be `developer`, `system`, `user`, or
`assistant`, positive repair budgets must be at most three and paired with
`basic`, `json`, or `json_schema`, content and schema paths must remain
within their Promptkit source rules, and extra parameters must be
JSON-compatible and avoid reserved provider keys.
5. Replace v0.1.0 links and current-version statements in current-state
documentation with tagged v0.9.0 links. Do not rewrite historical version
statements in accepted ADRs. Update the owning documentation to state the
v0.9.0 credential and provider behavior: an optional missing environment
credential may result in an unauthenticated provider request, while a
profile that declares credentials required still fails as an invalid request
when no source is supplied and with `ErrAPIKeyEnvMissing` when its selected
environment source is empty; unset optional provider controls are omitted;
and a positive repair budget can add provider calls, latency, token use, and
cost.
6. Preserve the architecture guard for forbidden replacements, workspaces,
vendor trees, former facade packages, and Promptkit `internal` imports.
Extend it only if the dependency graph introduces a new realistic bypass
that the existing assertions do not cover.
- module identity, graph, tagged dependency, and `go mod tidy -diff` checks;
- ordinary and race-enabled tests, including retained HTTP decoding, limit,
restricted-file, error-mapping, and raw-output coverage;
- vet, formatting, whitespace, architecture, and tracked-tree hygiene checks;
- temporary native and release-equivalent Linux `amd64` and `arm64` builds;
- the maintained render script and both configuration examples without model
credentials; and
- all maintained local and tagged Promptkit documentation links.
### Verification
A fresh temporary module and build cache resolved Promptkit remotely, reported
the expected `v0.1.0` origin commit, and passed the Scriptorium test suite and
build without reading the sibling Promptkit checkout.
- Update HTTP mapping tests for the pointer-valued reasoning field, including
separate omitted and explicit-empty cases, and update `docs/api.md` with the
HTTP three-state behavior in this stage. Stage 4 adds the corresponding CLI
control and session-ID support.
- Run `go test ./...`, `go test -race ./...`, `go vet ./...`, and
`go build ./cmd/scriptorium` against the tagged module with no local
replacement.
- Run the existing maintained render example and any example-validation tests.
- Confirm that a search for `v0.1.0` finds only legitimate historical material,
and that searches for `replace`, `go.work`, and Promptkit `/internal/`
imports do not reveal a prohibited dependency path.
Promptkit was reconfirmed independently at the exact published tag. Its local
and remote tag targets agree, its working tree remains clean, and its
documented test, race, vet, build, example, formatting, module, link,
whitespace, and repository-hygiene checks pass.
### Completion Conditions
## Next Gate
The repository builds and all existing behavior tests pass on Promptkit v0.9.0;
current documentation identifies v0.9.0 as the framework contract; no local or
private Promptkit integration mechanism has been introduced.
[Step 8](migration.md#step-8-migrate-downstream-consumers-to-promptkit) may
now inventory and migrate downstream Go consumers. Step 7 did not migrate
external consumers, publish a breaking Scriptorium release, or complete the
release and documentation cutover assigned to Step 9.
## Stage 2: Remove Definition-Compatibility Restrictions In Run And Render
**Completion: Complete.**
Allow the executable adapters to address every valid directory-backed prompt
definition without imposing input requirements of their own.
### Implementation
1. Add `promptVersion string` to the shared run/render request configuration.
Register `--prompt-version <version>` for both `run` and `render`, include it
in usage text, and map it without local normalization to
`promptkit.RunRequest.PromptVersion`.
2. Make `--input` optional for `run` and `render`. Do not weaken
`parseMappings` for callers that actually supplied mappings; instead, skip
that parser when no input flag was supplied and pass a nil or empty input
map to Promptkit. Promptkit remains responsible for declared-required and
template-referenced input failures.
3. In `POST /v1/runs`, remove the adapter check that rejects a nil or empty
`inputs` object. Preserve strict JSON decoding and all validation of input
references that are present. Continue mapping the already-supported
`prompt_version` field exactly once into `RunRequest.PromptVersion`.
4. Keep extra supplied inputs legal and retain the CLI file-reader and HTTP
artifact containment and size policies.
5. Update `docs/cli.md` and `docs/api.md` in this stage: prompt ID remains
required, prompt version is optional, inputs are optional at the adapter
boundary, and Promptkit decides whether the selected definition requires
them. Update the shortest request examples only where necessary to avoid
implying that all prompts need inputs.
### Verification
- Add CLI behavior tests proving that an omitted version selects a sole prompt
definition, an explicit version selects the requested definition, and an
omitted version fails clearly when a prompt ID has multiple versions.
- Add CLI and HTTP adapter coverage for: a prompt with no declared inputs,
omitted optional inputs, a missing declared-required input, and a missing
template-referenced input. Use a capturing runner for pure request mapping
and one real Promptkit engine integration fixture for the definition-driven
failures; do not reproduce Promptkit's parser tests.
- Retain tests for malformed supplied mappings, invalid HTTP input references,
artifact containment, and request-size limits.
- Run `go test ./internal/adapter/cli ./internal/adapter/http` and the maintained
render workflow.
### Completion Conditions
Both executable interfaces can select a specific prompt version and can submit
no inputs; only Promptkit rejects missing definition-required data.
## Stage 3: Add Engine-Scoped Custom Backend Configuration
**Completion: Complete.**
Add the application-owned configuration needed for Promptkit profiles to select
custom backend IDs and capacity policies.
### Configuration Contract
Use this exact strict-YAML shape:
```yaml
backends:
local-gpu:
endpoint: http://localhost:11434/v1
api_key_env: LOCAL_GPU_API_KEY
extra_params:
provider_option: enabled
concurrency_limit: 2
queue_capacity: 0
```
The mapping key is the backend ID. `endpoint` is required by the effective
contract. `api_key_env`, `extra_params`, `concurrency_limit`, and
`queue_capacity` are optional. `concurrency_limit` uses zero as Promptkit's
unlimited value. Represent `queue_capacity` as `*int` so omission uses
Promptkit's default queue capacity and explicit zero disables queuing. There
are no backend CLI overrides and no raw API-key field.
### Implementation
1. Add `Backends map[string]BackendConfig` to the on-disk `config.Config` and a
`BackendConfig` with exactly the five YAML fields above. Add a resolved
`BackendSettings` value containing `ID` plus the corresponding fields, and
carry `[]BackendSettings` on `AppSettings`.
2. During config application, sort the YAML map keys and build the resolved
slice in that order. Preserve the `queue_capacity` pointer and the complete
`extra_params` value. The deterministic order makes multi-error diagnostics
and tests stable; it is not backend precedence.
3. Do not duplicate Promptkit's backend validators in `internal/config`.
Strict YAML decoding owns unknown-field and type errors. Promptkit
`NewEngine` owns blank/reserved/duplicate IDs, endpoint and environment-name
rules, reserved or invalid extra parameters, and capacity relationships.
4. Refactor CLI engine construction around a small shared engine-settings value
containing prompt, profile, and schema directories plus resolved backends.
Convert every `BackendSettings` to `promptkit.Backend`, register it with
`promptkit.WithBackend`, and use the same constructor for `run`, `render`,
and `serve`. Wrap construction failures with application-configuration and
engine-initialization context while preserving `errors.Is` identities.
5. Retain one immutable backend registry per constructed engine. Do not create
registries per HTTP request, and do not add direct imports of Promptkit's
catalog modules; Promptkit's built-ins remain available automatically.
6. Document `backends` in `docs/config.md`, including key identity,
queue-capacity presence, validation ownership, optional environment
credentials, and the raw-secret prohibition. Update
`docs/internal/adapters.md` and `docs/internal/sources.md` to describe the
implemented mapping and engine assembly. Add the exact shape to
`examples/config.full.yml`; keep the minimal config minimal.
### Verification
- In `internal/config`, test strict decoding of all fields, unknown and raw
secret-field rejection, deterministic resolved ordering, JSON-compatible
nested `extra_params`, and omitted versus explicit-zero `queue_capacity`.
- At the CLI/engine boundary, test a valid custom backend selected by a custom
profile through `render`, and representative Promptkit validation failures:
reserved backend ID, invalid endpoint, invalid capacity relationship, and a
reserved `extra_params` key. Assert the stable error identity or a small
diagnostic fragment, not Promptkit's full text.
- Test that endpoint-only profiles and built-in profiles still work without a
`backends` section.
- Run `go test ./internal/config ./internal/adapter/cli` and render with the
complete example without contacting a provider.
### Completion Conditions
Every command constructs its engine from the same resolved custom-backend
configuration, Promptkit remains the validator, and profiles can select custom,
built-in, or endpoint-only execution targets.
## Stage 4: Expose Session And Presence-Aware Reasoning Controls
**Completion: Complete.**
Complete the request mapping for direct session identifiers and the v0.9.0
reasoning override semantics.
### Implementation
1. Add `--session-id <id>` and `--reasoning-effort <value>` to both `run` and
`render`, with no aliases. Map the session value to
`RunRequest.SessionID`.
2. Track flag presence for `--reasoning-effort`. Omission leaves
`ExecutionTargetOverride.ReasoningEffort` nil, a nonblank value supplies a
pointer to that value, and `--reasoning-effort=` supplies a pointer to the
empty string. Include reasoning presence when deciding whether to allocate
the enclosing execution override.
3. Add optional `session_id` to the strict HTTP run-request DTO and map it to
`RunRequest.SessionID` without application-level normalization. Keep
`model.reasoning_effort` pointer-valued so omitted, nonblank, and empty JSON
strings remain distinct.
4. Add optional `session_id` to successful HTTP metadata and populate it from
the effective `RunResult.SessionID`, not directly from the request. Use
`omitempty`; absence means no effective session ID.
5. Continue to reject unknown HTTP fields and do not add `api_key`, appended
messages, or request-level output-contract fields.
6. Update `docs/cli.md` and `docs/api.md` with the exact flag/field names,
inheritance/replacement/clearing semantics, the non-secret nature and
Promptkit length rules of session IDs, and effective-session response
behavior.
### Verification
- CLI tests must distinguish an omitted reasoning flag, a nonblank replacement,
and an explicit empty clear by observing the resulting prepared run through
`render`. Add a mapping assertion that an explicit empty value still creates
an execution override.
- HTTP tests must capture and distinguish the same three reasoning states and
continue to reject non-string values under the strict DTO contract. Treat
JSON `null` like omission, matching ordinary pointer-field decoding; the
documented explicit clear remains the empty JSON string.
- CLI and HTTP tests must cover a direct session ID, a definition-rendered
session ID when no direct value is supplied, no effective session ID, and an
invalid overlong direct ID. Assert that successful HTTP metadata reports the
engine's effective result.
- Run `go test ./internal/adapter/cli ./internal/adapter/http ./internal/format`.
### Completion Conditions
CLI and HTTP callers can inherit, replace, or clear reasoning effort and can
supply a direct session ID, with Promptkit retaining normalization and
validation ownership.
## Stage 5: Present Backend Identity And Map Capacity Outcomes
**Completion: Complete.**
Expose Promptkit's selected routing identity and make overload behavior a
stable application contract.
### Implementation
1. In prepared-run text output, add `selected_backend_id` immediately after
`selected_profile_id` when it is nonempty. Preserve direct JSON marshaling of
`promptkit.PreparedRun`; v0.9.0 already supplies `selected_backend_id` and
`effective_model_params.backend_id` JSON fields.
2. In the CLI run summary, append `backend=<id>` when
`RunResult.SelectedBackendID` is nonempty. Omit the token for endpoint-only
profiles rather than deriving an ID from the endpoint.
3. Add `selected_backend_id,omitempty` to HTTP metadata and
`backend_id,omitempty` to `metadata.model_params`, mapping them from
`RunResult.SelectedBackendID` and `ExecutionTarget.BackendID` respectively.
Keep the two values independently mapped so an upstream inconsistency is not
hidden by Scriptorium.
4. In HTTP error mapping, classify `promptkit.ErrCapacityExceeded` before the
general generation failure and return HTTP 503 with code
`capacity_exceeded` and message `model backend capacity is exhausted`. Do
not emit `Retry-After` and do not expose `CapacityError` details.
5. In CLI `run`, classify the same identity as the existing runtime-error exit
status 1 and print the stable safe diagnostic
`run error: model backend capacity is exhausted`. Keep all other runtime
error handling unchanged.
6. Preserve `serveCommand`'s one-engine/one-handler construction outside the
request path. Do not add per-request engine construction; that would split
capacity state and violate ADR 0004.
7. Update `docs/cli.md`, `docs/api.md`, and `docs/operations.md` with backend
identity, endpoint-only omission, capacity status, lack of retry timing,
shared per-engine admission, and the distinction between transient
in-flight state and durable workflow state.
### Verification
- Formatter, CLI, and HTTP tests must cover a registered backend and an
endpoint-only profile. Assert supplied backend IDs and empty/omitted identity
rather than endpoint-derived guesses.
- Add HTTP error-table coverage proving capacity is 503
`capacity_exceeded`, provider failure remains 502 `llm_failed`, and neither
response leaks wrapped provider or capacity diagnostics.
- Add one offline concurrent HTTP integration test using a single real
Promptkit engine, a custom backend with `concurrency_limit: 1` and explicit
`queue_capacity: 0`, and a blocking fake LLM client. Hold the first request
in generation, issue a second request, assert the second receives the 503
contract, release the first, assert it succeeds, and assert observed peak
generation concurrency is one. This test owns the Scriptorium shared-engine
interaction; do not replicate Promptkit's FIFO or broader scheduler suite.
- Run `go test -race ./internal/adapter/http ./internal/adapter/cli ./internal/format`.
### Completion Conditions
All presentations report actual backend identity when present, endpoint-only
profiles remain identity-free, and one server instance enforces one engine's
backend capacity across concurrent requests.
## Stage 6: Add Stable Prompt Inspection Output And CLI
**Completion: Complete.**
Introduce reusable application-owned inspection presentation, then expose
prompt inspection without model execution.
### Presentation Contract
1. Generalize the internal prepared-run format selector to an internal
`OutputFormat` with `text` and `json` values, a text default, and one parser.
Update config and prepared-run call sites to use it without changing
existing prepared-run bytes or `defaults.render_format` behavior.
2. Define a Scriptorium-owned prompt-inspection DTO; never marshal
`promptkit.PromptInspection` directly. JSON contains these fields in this
shape, including empty strings and an empty array where applicable:
```json
{
"prompt_id": "example",
"prompt_version": "1",
"prompt_hash": "opaque",
"default_profile_id": "",
"inputs": [
{
"name": "source",
"required": true,
"content_type": "text/plain",
"description": "Input text"
}
],
"output_contract": {
"format": "text",
"validation_mode": "none",
"schema_path": "",
"repair_attempts": 0
}
}
```
3. Preserve Promptkit's declared input order. Emit JSON with deterministic
indentation and one trailing newline. Text output uses the same field order,
renders `inputs: []` when empty, lists input records in declaration order,
quotes free-form descriptions safely, includes empty declared values, and
ends in one newline. The prompt hash is opaque and must not be interpreted.
### CLI Contract And Implementation
1. Add `inspect` to top-level dispatch and usage, with implementation in a new
focused CLI file rather than further expanding `run.go`.
2. Implement:
```text
scriptorium inspect prompt --prompt ID [--prompt-version VERSION]
[--config PATH] [--prompt-dir DIR] [--format text|json] [--out PATH]
```
`--prompt` is required. All other flags are optional. Positional arguments
and unknown inspection modes are errors. There are no deprecated aliases.
3. Apply ordinary config discovery and CLI-over-file precedence. Require an
effective prompt directory, pass the optional version directly to
`Engine.InspectPrompt`, and construct the engine through the shared settings
path so configured backends are validated consistently. Do not require a
profile or schema directory for this operation.
4. Default inspection format to text independently of
`defaults.render_format`; `--format` is the only inspection format override.
Reuse ordinary output-file handling. Write output only after successful
inspection and formatting. Use exit 0 for success and exit 1 for parsing,
configuration, load, formatting, or write failure.
5. Update `docs/cli.md`, `docs/internal/adapters.md`, and the internal format
component description. Make clear that inspection loads and normalizes a
definition but does not resolve a profile, load a schema, render templates,
reserve backend capacity, or call a model.
### Verification
- Formatter tests cover deterministic text and JSON, declared order, empty
inputs/default profile/schema path, all nonempty fields, and nil inspection
handling. Prefer structural JSON assertions plus a few stable text fragments
over large snapshots.
- CLI tests cover required flags, rejected positional arguments, config source
selection, `--prompt-dir` precedence, sole and explicit multi-version
selection, default and explicit formats, stdout versus `--out`, no partial
output on failure, and Promptkit prompt-not-found/load identities.
- Use a fake or deliberately failing model client to prove inspection performs
no generation; do not assert Promptkit's internal call sequence.
- Run `go test ./internal/format ./internal/adapter/cli`.
### Completion Conditions
`inspect prompt` produces stable Scriptorium text or JSON for every valid
directory-backed v0.9.0 prompt definition without inputs or model access, and
prepared-run presentation remains backward compatible.
## Stage 7: Add Stable Profile Inspection Output And CLI
**Completion: Complete.**
Complete the inspection command family with inherited, built-in, custom, and
endpoint-only profile support.
### Presentation Contract
1. Define a Scriptorium-owned profile-inspection DTO; never marshal
`promptkit.ProfileInspection` directly. JSON has the following complete
shape. Keep empty strings and zero numeric values because Promptkit reports
unresolved optional controls as zero, and normalize absent `extra_params`
to an empty object:
```json
{
"profile_id": "example",
"effective_model_params": {
"backend_id": "",
"endpoint": "https://example.test/v1",
"model": "model-name",
"temperature": 0,
"max_tokens": 0,
"top_p": 0,
"timeout_seconds": 0,
"service_tier": "",
"reasoning_effort": "",
"api_key_env": "",
"extra_params": {}
},
"api_key_required": false
}
```
2. Text output follows the same order, nests effective model parameters,
encodes extra-parameter values as deterministic compact JSON with sorted
keys, includes an empty `backend_id` for endpoint-only profiles, and ends in
one newline. Never read or print an environment-variable value or direct
API key.
### CLI Contract And Implementation
1. Implement the second mode:
```text
scriptorium inspect profile --profile ID
[--config PATH] [--profile-dir DIR] [--format text|json] [--out PATH]
```
`--profile` is required. The common inspection parse, format, output, and
exit rules from Stage 6 apply. No prompt, prompt directory, schema directory,
or input is required, so built-in profiles are inspectable with only their
ID.
2. Resolve config and `--profile-dir` with normal precedence, construct the
engine through the shared settings path so custom backends participate, and
call `Engine.InspectProfile` exactly once.
3. Update `docs/cli.md`, `docs/config.md`, and internal adapter/source docs.
Explain that profile inspection resolves inheritance and backend defaults
but does not read credentials, load a prompt, reserve capacity, or call a
model. Document that zero provider-control values may mean unspecified,
consistent with Promptkit v0.9.0.
### Verification
- Formatter tests cover deterministic text and JSON, registered and empty
backend identity, optional environment-variable names, `api_key_required`,
nested extra parameters, and the absence of credential values.
- CLI tests cover built-in, inherited custom, custom-backend, and endpoint-only
profiles; config and `--profile-dir` precedence; default and explicit format;
stdout and `--out`; missing and invalid profiles; no partial output; and no
requirement for `prompt_dir`.
- Include a sentinel environment secret in a test and assert it appears in no
inspection output or diagnostic while its variable name may appear.
- Run `go test ./internal/format ./internal/adapter/cli ./internal/config`.
### Completion Conditions
`inspect profile` reports the complete safe effective profile view for every
valid directory-backed v0.9.0 profile and configured backend combination
without requiring a prompt or contacting a model.
## Stage 8: Complete Cross-Feature Conformance And Release Validation
**Completion: Complete.**
Close the roadmap with representative definition-level integration coverage,
copyable examples, documentation alignment, and full validation. Do not defer
contract documentation from earlier stages to this stage; this is a final
cross-check.
### Implementation And Documentation
1. Maintain one compact shared fixture set under
`testdata/promptkit-v0.9/` with `prompts`, `profiles`, `schemas`, and any
message content files needed by CLI and HTTP integration tests. Through a
small set of composable definitions, cover:
- inline and file-backed messages;
- `developer`, `system`, `user`, and `assistant` roles and message cache
control;
- required and optional inputs plus template-referenced values;
- prompt versions, default profiles, and session-ID templates;
- text and JSON output plus `none`, `basic`, `json`, and `json_schema`
validation, a schema path, and a valid positive repair budget;
- standalone and inherited profiles;
- built-in, configured custom, and endpoint-only backends;
- model controls, service tier, reasoning effort, timeout, optional
credential environment names, and nested JSON-compatible extra
parameters.
Exercise these fixtures through Scriptorium's `render` and inspection
boundaries. Assert representative effective outcomes; do not independently
decode YAML or exhaustively retest every Promptkit field combination.
2. Add a copyable `examples/render-v0.9-features.sh` workflow that uses only
`render` and `inspect` operations to demonstrate prompt-version selection,
a direct session ID, reasoning replacement or clearing, a configured custom
backend, and both inspection modes without provider access. Keep
`examples/config.full.yml` complete and keep raw credentials out of every
example.
3. Review every canonical current-state owner named by the feature roadmap.
Remove stale claims that inputs are always required, that every command
needs `prompt_dir`, or that v0.1.0 semantics still apply. Ensure CLI, API,
configuration, operations, internal adapter/source, development, and
overview documents link instead of duplicating one another's contracts.
Change architecture policy or internal component boundaries only if the
implementation actually changed those durable responsibilities.
4. Verify that all explicit non-goals remain absent: appended messages,
retained prepared handles, HTTP inspection routes, alternative Promptkit
source kinds, request output-contract replacement, public provider details,
direct catalog imports, raw API keys, and durable workflow state.
5. After every completion criterion below passes, change the feature roadmap's
Status from `Planned` to `Complete`. Do not mark it complete for a partial
implementation.
### Final Validation
Run all of the following from the repository root:
```bash
gofmt -w $(git ls-files '*.go')
go test ./...
go test -race ./...
go vet ./...
go build ./cmd/scriptorium
go run ./cmd/scriptorium render --config ./examples/config.yml --prompt generic.markdown_summary --input transcript=./examples/fixtures/transcript.md --input glossary=./examples/fixtures/glossary.yml
bash ./examples/render-v0.9-features.sh
git diff --check
```
Also verify the architecture guard, strict example decoding, documentation
links, executable permissions on shell examples, and absence of secrets. Review
the suite for redundant upstream-semantic tests and retain only tests that
protect Scriptorium mappings, presentation, configuration, concurrency
integration, or complete workflows.
### Completion Conditions
Every roadmap validation criterion is satisfied on the tagged dependency; all
current-state documentation and examples describe implemented behavior; the
full application supports every Promptkit v0.9.0 feature expressible in its
directory-backed prompt and profile definitions; and the ADR 0004 non-goals
remain intact.
## Open Questions
None. The external syntax, application mappings, validation ownership,
inspection wire shapes, concurrency behavior, testing boundaries, and stage
order required for implementation are fixed above.

View File

@@ -1,335 +0,0 @@
# Promptkit Migration Roadmap
## Status
Accepted plan. Steps 1 through 7 are complete. Steps 8 and 9 remain proposed
and are not yet implemented.
## Objective
Split the current repository into two projects:
- **Promptkit**: the reusable Go framework, public Go facade, execution engine,
source and validation support, OpenAI-compatible client, extension
interfaces, and built-in execution-profile registry.
- **Scriptorium**: a slim runnable application that imports Promptkit and
provides the CLI and HTTP interfaces.
Scriptorium will become another downstream Promptkit consumer rather than the
owner of the framework.
## Compatibility And Migration Policy
This is an intentionally breaking change.
- New and migrated Go consumers must import Promptkit instead of Scriptorium.
- Scriptorium will not retain type aliases, forwarding packages, deprecated
facade APIs, or other source-compatibility shims.
- Existing consumers may continue using a previously tagged Scriptorium module
version until they are migrated.
- The migration does not need to preserve compatibility between intermediate
development states. Each completed phase must instead leave the affected
repository internally consistent and tested.
- Promptkit should initially preserve the useful shape and behavior of the
current public Go facade where doing so reduces extraction risk. Broader API
redesign should follow the split unless required to establish the new
boundary.
## Target Ownership
Promptkit should own application-neutral framework behavior:
- public engine, request, result, option, extension, and error APIs;
- prompt-definition loading and rendering;
- execution profiles, overlays, and the built-in profile registry;
- artifact-loading interfaces and general-purpose `file` and `inline` support;
- schema loading and output validation;
- LLM client boundaries and the OpenAI-compatible implementation;
- preparation and execution orchestration;
- framework and execution defaults.
Scriptorium should own executable and transport concerns:
- the `scriptorium` command and its `run`, `render`, and `serve` interfaces;
- CLI parsing, output formatting, exit codes, and process behavior;
- application-config discovery and CLI precedence;
- HTTP routing, request and response DTOs, limits, and error/status mapping;
- HTTP artifact-root and deployment security policy;
- server and adapter defaults;
- executable examples, operations guidance, and transport documentation.
The intended dependency direction is:
```text
Scriptorium CLI and HTTP adapters
|
v
Promptkit
|
v
consumer-supplied sources and clients
```
Scriptorium must use Promptkit's public API. It must not depend on Promptkit
implementation packages or reproduce framework orchestration.
## Migration Steps
### Step 1: Refresh And Synchronize Documentation
Perform a repository-wide documentation refresh before migration development.
At minimum:
- reconcile all current-behavior documentation with the code, tests, examples,
defaults, and current public contracts;
- introduce the planned documentation-policy updates;
- establish an architecture decision record policy and canonical ADR location;
- resolve stale, duplicated, or misplaced material;
- validate documentation links and maintained examples;
- leave future migration behavior in `docs/roadmap/` until implemented.
**Gate:** Do not begin architectural migration work until the documentation
refresh and policy updates are merged and the repository has an agreed,
accurate baseline.
**Gate status:** Complete as of 2026-07-26. The completed documentation
refresh, follow-up verification, and layered-timeout correction remain recorded
in repository history. Step 1 remains complete after that validation.
### Step 2: Record The Architectural Decision And Detailed Boundary
Create an ADR, under the policy established in Step 1, that records:
- the decision to split Promptkit from Scriptorium;
- the target ownership and dependency direction;
- the selected Promptkit repository and Go module paths;
- the breaking-change and versioning policy;
- ownership of configuration fields and defaults;
- artifact-reader and HTTP containment responsibilities;
- local multi-repository development and release coordination;
- documentation ownership after the split.
Use the ADR to resolve any remaining public-boundary decisions before code is
moved.
**Gate:** The ADR is accepted, and every existing package, public contract,
configuration category, and maintained asset has a target owner.
**Gate status:** Complete as of 2026-07-26.
[ADR 0002: Split Promptkit From Scriptorium](../adr/0002-split-promptkit-from-scriptorium.md)
is accepted and records the required ownership and coordination decisions.
### Step 3: Characterize Existing Framework Behavior
Strengthen or add contract-focused tests where needed so extraction can be
verified without relying on package placement.
The completed Step 3 gate records the accepted implementation scope and
intended completion state.
Preserve coverage of:
- `Prepare` and `Run` behavior;
- prompt, profile, execution-default, and request-override precedence;
- presence-aware numeric overrides;
- built-in profile fallback and custom-profile overlays;
- strict YAML and JSON decoding;
- prompt, profile, schema, and artifact source behavior;
- structured-output requests and output validation;
- validation failures versus validation runtime errors;
- secret handling and redaction;
- public error classification;
- HTTP artifact restrictions and transport mappings.
**Gate:** Current framework and adapter contracts are represented by passing
tests sufficient to detect behavioral regressions during the split.
**Gate status:** Complete as of 2026-07-27. The framework contract corpus,
public `Engine` characterization, ownership audit, full test and vet suites,
temporary executable build, and maintained offline examples passed.
### Step 4: Make Scriptorium Adapters Consume The Public Facade
Within the current repository, refactor the CLI and HTTP adapters to use the
public framework facade rather than constructing or importing internal runner
components directly.
Add only the minimum public capabilities needed to support this boundary. These
may include:
- a small `Run`/`Prepare` consumer interface;
- injectable artifact-reading behavior for Scriptorium's restricted HTTP
policy;
- source options currently available only through internal constructors;
- prepared-run formatting based on public types;
- stable public error classification required by CLI and HTTP mappings.
Do not broadly export internal repositories, domain types, or use-case
implementations.
**Gate:** The CLI and HTTP adapters use only the public framework API for
framework behavior, and all tests and documented smoke commands pass.
**Gate status:** Complete as of 2026-07-28. CLI `run`, `render`, and `serve`,
the HTTP handler, and prepared-run formatting use the public facade; the
restricted HTTP reader is injected through the public extension point. The
post-implementation public-error, deterministic MIME, and recursive
dependency-guard corrections passed full tests, vet, build, race checks,
maintained examples, and configuration smoke checks.
### Step 5: Create The Promptkit Repository
Promptkit was established as an independent repository and Go module through
the completed out-of-band workflow recorded in repository history. Its
foundation includes:
- confirmed repository access, governance, origin, and default-branch tracking;
- module `gitea.maximumdirect.net/eric/promptkit` at Go `1.25.5`;
- a minimal root `promptkit` public package boundary with no placeholder
framework packages;
- library-specific development, architecture, documentation, testing, and
release policies;
- documented maintainer-run test, vet, build, formatting, documentation-link,
and repository-hygiene validation;
- source-commit and semantic Go module tag releases without runnable binaries
or binary packaging; and
- temporary workspace and uncommitted replacement workflows for coordinated
development without committed repository coupling.
[ADR 0003](../adr/0003-use-maintainer-run-validation-and-tag-only-releases-for-promptkit.md)
records the controlling Promptkit validation and release decision.
**Gate status:** Complete as of 2026-07-28. Promptkit passed its documented
validation independently, all maintained links and repository-hygiene checks
passed, and no workspace, replacement, CI configuration, binary, tag, command,
or placeholder package was added. The completed repository foundation
supported the Step 6 extraction.
### Step 6: Extract And Stabilize Promptkit
Move the application-neutral framework and built-in profile assets into
Promptkit. Preserve implementation packages as internal where practical.
The initial public API should remain focused on the established engine workflow
and the source and client extension points required by real consumers. Avoid
combining the extraction with unrelated API redesign.
Move or recreate the relevant:
- framework implementation;
- public package tests and framework contract tests;
- built-in profile assets and registry tests;
- Go consumer examples;
- framework, consumer, configuration-format, and integration documentation.
Verify that Promptkit can be built, tested, and consumed independently of the
Scriptorium repository.
**Gate:** Promptkit independently provides the agreed framework contract,
passes its documented validation, and has published its first versioned tag
before Scriptorium or another consumer adopts it, as required by
[ADR 0003](../adr/0003-use-maintainer-run-validation-and-tag-only-releases-for-promptkit.md).
**Gate status:** Complete as of 2026-07-28. Repository history records source
Scriptorium commit `c7263ab2a8e58f7fb97280082d327a820c7cece7`,
accepted Promptkit commit
`9e68a2bbf779545995270c47842048a3bc6c85dc`, independently passing acceptance,
published annotated tag `v0.1.0`, and successful remote-consumer validation.
Scriptorium remains unchanged at its pre-cutover boundary. Step 7 adoption of
the tagged module and removal of the duplicated framework is the next gate.
### Step 7: Slim Scriptorium And Adopt Promptkit
Update Scriptorium to import the tagged Promptkit module and remove the
framework implementation and public Go facade that Promptkit replaces.
Retain only Scriptorium-owned executable and transport behavior. In particular:
- wire CLI and HTTP requests through Promptkit's public API;
- keep application config and transport defaults in Scriptorium;
- keep restricted HTTP artifact policy in Scriptorium while injecting it
through Promptkit's supported boundary;
- remove obsolete framework packages, tests, and documentation;
- update Scriptorium examples and docs to describe the CLI and HTTP application;
- direct Go framework consumers to Promptkit without providing compatibility
aliases or forwarding APIs.
**Gate:** Scriptorium builds and passes all tests using a tagged Promptkit
dependency, contains no duplicate framework implementation, and its current
documentation describes only the slimmed application.
**Gate status:** Complete as of 2026-07-28. Scriptorium directly resolves
Promptkit `v0.1.0`, no longer contains the framework copy or public Go facade,
and retains only its application, adapter, configuration, presentation,
transport, packaging, and executable-example responsibilities. Release-grade
Scriptorium validation passed with a fresh remote dependency cache, and the
published Promptkit tag passed its documented validation independently. The
application is ready for the downstream-consumer migrations in Step 8.
### Step 8: Migrate Downstream Consumers To Promptkit
Inventory downstream Go consumers and migrate each from the Scriptorium package
to Promptkit. This work may occur in external repositories and must be tracked
explicitly.
For each consumer:
- update module imports and dependencies;
- adapt to any intentionally changed public API;
- run its tests and relevant integration or smoke checks;
- confirm configuration, source, validation, and error behavior;
- release or deploy the migrated consumer through its normal process.
Consumers that cannot migrate immediately may remain pinned to the last
framework-bearing Scriptorium tag. No compatibility work is required in the new
Scriptorium project for those consumers.
**Gate:** All in-scope downstream consumers are either migrated and verified or
explicitly recorded as remaining on the previous Scriptorium version with an
owner and follow-up plan. Do not declare the ecosystem migration complete until
the required out-of-band consumer changes are confirmed.
### Step 9: Complete Release And Documentation Cutover
Complete the coordinated project transition:
- publish Promptkit before dependent Scriptorium releases;
- release the breaking Scriptorium version against the tagged Promptkit
dependency;
- publish migration guidance that maps the former Scriptorium Go API to
Promptkit;
- update cross-project links, examples, package documentation, and release
notes;
- verify that no release artifact depends on local workspaces or replacements;
- archive completed roadmap material according to the documentation policy in
effect at that time.
**Gate:** Promptkit and Scriptorium are independently releasable, their
documentation has distinct and accurate ownership, and the migration status of
all identified downstream consumers is recorded.
## Cross-Cutting Constraints
- Preserve the invariant that execution orchestration remains narrow and
application-neutral.
- Keep adapter-specific decisions out of Promptkit.
- Keep Scriptorium dependent only on Promptkit's supported public API.
- Preserve strict external decoding, error classification, validation
semantics, and secret redaction throughout the migration.
- Keep each repository buildable and testable at merged phase boundaries.
- Coordinate cross-repository changes through tagged dependencies and explicit
gates rather than assuming atomic commits across repositories.
- Document only implemented behavior outside roadmap files.
## Completion Criteria
The migration is complete when:
- Promptkit is the independent owner of the reusable framework and built-in
profiles;
- Scriptorium is a slim CLI and HTTP consumer of Promptkit;
- Scriptorium no longer exposes or maintains the former public Go framework;
- all required downstream migrations and external repository work have been
completed or explicitly dispositioned;
- both repositories build, test, document, version, and release independently.

View File

@@ -0,0 +1,351 @@
# Promptkit v0.9.0 Adoption Roadmap
## Status
Complete. This document records the delivered feature scope and targeted end
state. The completed delivery plan is maintained in
[implementation.md](implementation.md); current behavior remains defined by
the canonical application contracts linked below.
The durable product and integration boundary for this work is established by
[ADR 0004](../adr/0004-definition-boundary.md). This roadmap applies that
decision to Promptkit v0.9.0 and owns implementation scope and completion
status.
## Objective
Bring Scriptorium from Promptkit v0.1.0 to v0.9.0 and expose the newer
framework capabilities that fit Scriptorium's existing CLI, HTTP,
configuration, presentation, and process boundaries.
Scriptorium must accept the complete Promptkit v0.9.0 feature set expressed by
directory-backed prompt, profile, and schema definitions. Application adapters
must not narrow that format contract by requiring request values that the
selected definition does not require or by omitting selection values needed to
address a valid definition. Promptkit's tagged
[framework format reference](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.9.0/docs/formats.md)
remains canonical; Scriptorium must delegate parsing and framework validation
rather than reproduce those rules.
The completed application will retain Promptkit as the owner of reusable
prompt execution, source formats, profiles, backend definitions, capacity,
inspection, generation, and validation. Scriptorium will continue to own
application configuration, command and HTTP contracts, error and exit mapping,
output presentation, and deployment policy.
## Targeted End State
When this roadmap is complete:
- Scriptorium pins the released Promptkit v0.9.0 module without a workspace,
replacement, vendored source, direct catalog import, or Promptkit internal
import.
- Existing valid CLI, HTTP, configuration, prompt, profile, schema, and render
workflows remain supported subject to Promptkit's documented v0.2.0 through
v0.9.0 compatibility corrections.
- Every Promptkit v0.9.0 prompt and profile feature expressible in definitions
under Scriptorium's configured directory sources is usable through the
executable application, including prompts with no inputs and explicitly
selected prompt versions.
- CLI and HTTP requests can supply a direct non-secret session ID.
- HTTP requests and a presence-aware CLI flag can inherit, replace, or
explicitly clear reasoning effort.
- Prepared output, CLI run summaries, and HTTP result metadata expose effective
backend identity when Promptkit supplies one.
- HTTP capacity rejection has a stable Scriptorium-owned overload response.
- Application configuration can register reusable engine-scoped custom
backends, including bounded concurrency and queue policy, for selection by
Promptkit profiles.
- CLI users can inspect prompt and profile definitions without supplying
placeholder inputs or invoking a model.
- Scriptorium's contracts and examples point to Promptkit v0.9.0 and accurately
describe its current credential, validation-repair, provider-control,
message-role, backend, and source-validation semantics.
## Feature Scope
### Complete Prompt And Profile Definition Compatibility
Scriptorium will treat the tagged Promptkit v0.9.0 format contract as one
indivisible downstream compatibility boundary. This includes prompt identity
and version selection, optional and required input declarations, message and
content-file templates, supported roles, cache control, session templates,
default profiles, output contracts, schemas, and repair budgets. It also
includes standalone and inherited profiles, built-in and custom backend
selection, endpoint overrides, execution controls, optional credential
environment sources, and JSON-compatible extra parameters.
Most of this support is provided by passing configured prompt, profile, and
schema directories directly to Promptkit. Scriptorium will not independently
decode these framework definitions or maintain a field-level subset. Two
current adapter restrictions must be removed so the executable interfaces do
not reject definitions that Promptkit accepts:
- `run` and `render` will accept an optional prompt-version flag and map it to
`promptkit.RunRequest.PromptVersion`; and
- `run`, `render`, and `POST /v1/runs` will permit omitted or empty input maps.
Promptkit will decide whether an input is required by the selected
definition or referenced template.
The HTTP API already carries `prompt_version`; its mapping must remain covered
by adapter tests. CLI and HTTP callers may continue to supply extra inputs,
subject to their existing file, containment, and size policies. Removing the
application-level nonempty-input requirement does not weaken Promptkit's
declared-required-input or template-reference validation.
Profiles that name built-in backend IDs require no application registration.
Profiles that name other backend IDs become fully usable through the custom
backend configuration in this roadmap. Profile inheritance and all other
profile-field resolution remain Promptkit responsibilities.
This compatibility requirement concerns fields and behavior expressed in
Promptkit YAML definitions. Go-consumer construction alternatives such as
single-file sources, injected `fs.FS` sources, in-memory profiles, fallback
profile filesystems, and prepared-execution handles are library-integration
features and are not required merely to support the complete definition
format.
### Promptkit v0.9.0 Compatibility Baseline
Required dependency and source changes:
- update `go.mod` and `go.sum` to select Promptkit v0.9.0 and its selected
OpenRouter and Rakestrawhome catalog dependencies;
- change Scriptorium's inbound HTTP reasoning-effort representation to preserve
Promptkit's pointer semantics;
- retain keyed Promptkit public struct literals and confirm no removed
`RunRequest.Metadata` use exists;
- update tagged Promptkit links and version references throughout Scriptorium;
and
- preserve Scriptorium's architecture guard against replacements, workspaces,
vendored Promptkit code, former facade packages, and Promptkit internal
imports.
Required compatibility review:
- verify application and maintained example endpoints satisfy Promptkit's
absolute HTTP/HTTPS endpoint rules;
- verify prompt content paths, file artifacts, identities, JSON documents, and
JSON-compatible extra parameters satisfy the v0.6.0 safety boundaries;
- verify maintained prompt roles are limited to `developer`, `system`, `user`,
and `assistant`;
- verify every positive repair budget is no greater than three and is paired
with `basic`, `json`, or `json_schema` validation;
- document that positive repair budgets authorize additional provider calls,
latency, token use, and cost;
- document that unset optional provider controls are omitted and that a zero
effective value can represent an unspecified provider control unless an
explicit request override supplied it; and
- update credential guidance for Promptkit's optional environment lookup
behavior, profile `api_key_env` semantics, and Scriptorium's prohibition on
raw API-key inputs.
### Session And Reasoning Request Controls
The CLI `run` and `render` commands will accept a direct session-ID flag and map
it to `promptkit.RunRequest.SessionID`. The value remains non-secret
application correlation metadata and is subject to Promptkit's normalization
and length rules.
The HTTP run request will accept `session_id` and map it through the existing
strict DTO boundary. Successful HTTP metadata will report the effective
session ID when present.
The CLI `run` and `render` commands will also accept a presence-aware reasoning
effort flag. Omission inherits the selected profile, a nonblank value replaces
it, and an explicitly supplied empty value clears it. The existing HTTP
`model.reasoning_effort` field will gain the same three-state behavior while
remaining a JSON string when present.
These additions must not introduce raw direct API-key flags or fields.
### Backend Identity And Capacity Outcomes
Scriptorium presentation will expose `SelectedBackendID` and the effective
target's `BackendID` without deriving identity from endpoint text. Backend
identity will be included where applicable in:
- prepared-run text and JSON output;
- the CLI run summary; and
- HTTP response metadata and effective model parameters.
Endpoint-only profiles continue to have no backend ID, and empty identities
must remain distinguishable from registered built-in or custom backends.
The HTTP adapter will classify `promptkit.ErrCapacityExceeded` separately from
provider generation failures. The target public outcome is HTTP `503` with a
stable `capacity_exceeded` code and a generic message. Scriptorium will not
invent retry timing or expose an untrusted diagnostic. CLI capacity rejection
continues to be a runtime failure, with a clear safe diagnostic and the
existing runtime-error exit status.
The HTTP server will use one appropriately scoped Promptkit engine across its
requests so backend admission, active-generation limits, and queue capacity
apply across concurrent in-flight work. The server may retain the operational
state needed for those limits and for request cancellation and lifecycle
management. This does not create durable per-request, conversation, or resume
state.
Scriptorium will continue to return generic public model-generation failures.
Promptkit `GenerationError` provider code, type, and message values will not be
added to the public HTTP response because they are untrusted and potentially
sensitive.
### Custom Backend Configuration
The strict Scriptorium application configuration will gain an optional custom
backend collection. Each entry will support the application-owned mapping
needed to construct a public `promptkit.Backend`:
- backend ID;
- OpenAI-compatible endpoint;
- optional API-key environment-variable name;
- optional JSON-compatible request-wide extra parameters;
- non-negative concurrency limit; and
- queue capacity with presence preserved so omission and explicit zero remain
different.
Raw API-key values will remain invalid. Unknown fields will remain errors.
Configured backends will be registered through `promptkit.WithBackend` during
engine construction and will be immutable and engine-scoped. Promptkit will
remain the canonical validator for backend IDs, endpoints, reserved request
parameters, capacity bounds, duplicate or reserved IDs, environment-variable
names, and JSON-compatible values. Scriptorium will add context appropriate to
its configuration error boundary without copying those framework rules.
The collection will be configuration-file-owned. This roadmap does not add
per-backend CLI flags. Existing endpoint-only profiles remain valid, while
profiles may select configured custom backend IDs through Promptkit's existing
`backend` field. Promptkit's built-in OpenRouter and Rakestrawhome backends and
profiles remain available without Scriptorium registration.
The application configuration shape is a `backends` mapping whose keys are the
backend IDs registered with Promptkit:
```yaml
backends:
local-gpu:
endpoint: http://localhost:11434/v1
api_key_env: LOCAL_GPU_API_KEY
extra_params:
provider_option: enabled
concurrency_limit: 2
queue_capacity: 0
```
`endpoint` is required for each entry. The other fields are optional. Omitted
`queue_capacity` uses Promptkit's default queue policy when concurrency is
bounded, while an explicit zero disables queuing. The configuration contract
and complete maintained configuration example will document this shape when
the feature is implemented.
### Prompt And Profile Inspection CLI
Scriptorium will add a CLI inspection command with prompt and profile modes.
The intended command family is:
- `scriptorium inspect prompt`, backed by `Engine.InspectPrompt`; and
- `scriptorium inspect profile`, backed by `Engine.InspectProfile`.
Prompt inspection will select an ID and optional version and report the
normalized prompt identity, opaque prompt hash, declared default profile,
declared inputs, and normalized output contract. It will not require input
artifacts, resolve a profile, load a schema, render templates, reserve backend
capacity, or contact a model.
Profile inspection will select an explicit profile ID and report its resolved
effective target, backend identity, optional credential source name, and
whether a later request must provide credential configuration. It will not
read credential values, load a prompt, reserve capacity, or contact a model.
Inspection will support deterministic Scriptorium-owned text and JSON output
and normal output-file handling. Scriptorium will define explicit output DTOs
rather than treating Promptkit inspection structs as stable wire formats.
Inspection failures will use existing public Promptkit error identities and
Scriptorium-owned CLI diagnostics and exit behavior.
The prompt form accepts `--prompt`, optional `--prompt-version`, `--config`,
`--prompt-dir`, `--format text|json`, and `--out`. The profile form accepts
`--profile`, `--config`, `--profile-dir`, `--format text|json`, and `--out`.
Both forms reject positional arguments and default to text output independently
of `defaults.render_format`. Configuration discovery and CLI-over-file source
precedence match the existing command family. Prompt inspection requires an
effective prompt directory; profile inspection does not, because Promptkit's
built-in profiles remain inspectable without one.
## Documentation And Example Changes
Implementation of this roadmap requires coordinated updates to the canonical
owners of affected behavior:
- `docs/cli.md` for session and reasoning flags, inspection commands, output,
prompt-version selection, optional inputs, and exit behavior;
- `docs/api.md` for `session_id`, effective backend/session metadata, reasoning
presence semantics, optional inputs, and the capacity response;
- `docs/config.md` for custom backend configuration and current Promptkit
credential and provider-default semantics;
- `docs/operations.md` for capacity and custom-backend operational guidance
where deployment handling is affected;
- `docs/internal/adapters.md` and `docs/internal/sources.md` for implemented
mapping and engine-assembly behavior;
- `docs/policy/architecture.md` and `docs/internal/overview.md` only if concrete
component responsibilities or durable boundaries change; and
- maintained examples for valid v0.9.0 prompt/profile formats and custom
backend configuration.
Current-state documents must not describe these features as implemented until
the corresponding code lands.
## Validation And Completion Criteria
The roadmap is complete only when:
- ordinary and race-enabled tests pass against the tagged Promptkit v0.9.0
dependency;
- `go vet ./...` and `go build ./cmd/scriptorium` pass;
- Go formatting and `git diff --check` pass;
- the architecture guard passes with no workspace, replacement, vendor tree,
former facade, or Promptkit internal import;
- maintained render, custom-backend, session, reasoning, and inspection
examples execute successfully without real credentials or provider calls;
- CLI tests cover explicit and omitted prompt versions for prompt IDs with one
or multiple definitions;
- CLI and HTTP tests cover prompts with no declared inputs, omitted optional
inputs, missing required inputs, and template-referenced inputs;
- HTTP tests cover omitted, replacement, and clearing reasoning states;
- CLI and HTTP tests cover session propagation and invalid session handling;
- presentation tests cover registered and endpoint-only backend identity;
- HTTP tests cover capacity rejection independently from provider failure;
- concurrent HTTP tests demonstrate that one server's requests share backend
admission and cannot bypass capacity through per-request engine creation;
- configuration tests cover strict decoding, precedence, queue-capacity
presence, invalid backends, and secret-field rejection;
- inspection tests cover deterministic text and JSON, source selection,
output-file handling, safe credential presentation, and public error
mapping;
- all maintained prompt roles and repair budgets are valid under Promptkit
v0.9.0; and
- representative definition fixtures cover inline and file-backed messages,
cache control, session templates, every output validation mode, schemas,
profile inheritance, built-in and custom backends, execution controls,
credential environment names, and extra parameters without Scriptorium
independently parsing those fields; and
- affected documentation links and copyable examples validate successfully.
## Explicit Non-Goals
This roadmap does not include:
- appended request messages in either executable interface;
- retained or durable prepared-execution handles;
- HTTP prompt or profile inspection routes;
- embedded application fallback profiles;
- single-file, injected `fs.FS`, or in-memory prompt and profile source
configuration;
- request-level output-contract replacement through the executable interfaces;
- public disclosure of structured provider error details;
- direct imports or configuration of Promptkit's external catalog modules;
- raw API-key configuration, CLI flags, or HTTP fields;
- durable run state, conversations, retries, archives, checkpoints, or resume;
or
- changes to Promptkit itself.

View File

@@ -1,64 +0,0 @@
# Migration Step 7: Slim Scriptorium And Adopt Promptkit
## Status
Complete as of 2026-07-28.
## Result
Scriptorium is now an application-only consumer of the published
`gitea.maximumdirect.net/eric/promptkit v0.1.0` module. The command, CLI and
HTTP adapters, restricted artifact reader, and prepared-run formatter use
Promptkit's supported public API. The dependency resolves normally to
Promptkit commit `9e68a2bbf779545995270c47842048a3bc6c85dc` without a workspace,
replacement, vendored copy, or unpublished revision.
The Scriptorium module root no longer provides an importable Go package or
compatibility facade. The reusable framework packages, embedded built-in
profiles, framework contract tests and fixtures, and Go-library example were
removed. Architecture tests prevent production imports of the former root
facade, removed framework package families, and Promptkit internals.
## Retained Application Boundary
Scriptorium owns only its runnable process and application concerns:
- command and CLI parsing, assembly, streams, summaries, and exit behavior;
- HTTP routes, DTOs, strict decoding, limits, public error mapping, and
restricted artifact access;
- application configuration discovery, validation, precedence, and transport
defaults;
- prepared-run text and JSON presentation;
- executable examples, operations guidance, and binary packaging.
Promptkit owns prompt execution, framework sources and formats, validation,
model integration, built-in profiles, and ordinary artifact loading.
Scriptorium's permanent documentation reflects that boundary and links to the
tagged Promptkit contracts rather than duplicating them.
## Acceptance
The completed cutover passed:
- module graph and tidiness checks selecting exactly Promptkit `v0.1.0`;
- ordinary and race-enabled Scriptorium tests, vet, formatting, whitespace,
architecture, and repository-hygiene checks;
- temporary native and release-equivalent cross-builds;
- maintained offline render workflows with both application configurations;
- maintained local and tagged-documentation link checks;
- fresh-cache remote dependency resolution, tests, and build with the expected
Promptkit origin commit; and
- Promptkit's complete documented validation independently at its clean local
and remote `v0.1.0` tag.
These checks cover the retained CLI, HTTP, configuration, containment,
formatting, error-mapping, security, and representative executable behavior.
The working trees contain no generated binaries, temporary workspaces,
replacement directives, credentials, or migration residue.
## Handoff
The Step 7 gate is satisfied. [Step 8](migration.md#step-8-migrate-downstream-consumers-to-promptkit)
owns downstream consumer inventory and migration. Step 9 remains responsible
for the breaking Scriptorium release and final documentation cutover; neither
later gate is complete.

View File

@@ -2,6 +2,15 @@ prompt_dir: ./examples/prompts
profile_dir: ./examples/profiles
schema_dir: ./examples/schemas
backends:
local-gpu:
endpoint: http://localhost:11434/v1
api_key_env: LOCAL_GPU_API_KEY
extra_params:
provider_option: enabled
concurrency_limit: 2
queue_capacity: 0
server:
addr: 127.0.0.1:8080
artifact_root: .

View File

@@ -0,0 +1,4 @@
id: local-gpu
backend: local-gpu
model: local-model
reasoning_effort: low

View File

@@ -0,0 +1,25 @@
#!/usr/bin/env bash
set -euo pipefail
repo_root="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
cd "$repo_root"
go run ./cmd/scriptorium render \
--config ./examples/config.full.yml \
--prompt generic.markdown_summary \
--prompt-version 1.0.0 \
--profile local-gpu \
--session-id example-session \
--reasoning-effort= \
--input transcript=./examples/fixtures/transcript.md \
--input glossary=./examples/fixtures/glossary.yml
go run ./cmd/scriptorium inspect prompt \
--config ./examples/config.full.yml \
--prompt generic.markdown_summary \
--format json
go run ./cmd/scriptorium inspect profile \
--config ./examples/config.full.yml \
--profile local-gpu \
--format json

4
go.mod
View File

@@ -3,11 +3,13 @@ module gitea.maximumdirect.net/eric/scriptorium
go 1.25.5
require (
gitea.maximumdirect.net/eric/promptkit v0.1.0
gitea.maximumdirect.net/eric/promptkit v0.9.0
gopkg.in/yaml.v3 v3.0.1
)
require (
gitea.maximumdirect.net/eric/promptkit-backend-openrouter v1.0.0 // indirect
gitea.maximumdirect.net/eric/promptkit-backend-rakestrawhome v1.0.0 // indirect
github.com/santhosh-tekuri/jsonschema/v6 v6.0.2 // indirect
golang.org/x/text v0.14.0 // indirect
)

8
go.sum
View File

@@ -1,5 +1,9 @@
gitea.maximumdirect.net/eric/promptkit v0.1.0 h1:vuKeBxkiY8E54LRFbLQFjlJJCiOfMvB1++DYBCrD/ug=
gitea.maximumdirect.net/eric/promptkit v0.1.0/go.mod h1:R95NM6fbMDGDC0/UomgnSBP6ui2ns+8SZb8bESNvrDQ=
gitea.maximumdirect.net/eric/promptkit v0.9.0 h1:IpvDRC8L6xRxQ9hpuyKOmMc5b6MeLTKYyx+h1YAjy08=
gitea.maximumdirect.net/eric/promptkit v0.9.0/go.mod h1:oMJ/WUJImUtwJ5e+6MAGECPYAErAkOaKel0G+3T/b4E=
gitea.maximumdirect.net/eric/promptkit-backend-openrouter v1.0.0 h1:lc062euk2qseO//D762i3JaFyulDNML3eQQX7DkYTho=
gitea.maximumdirect.net/eric/promptkit-backend-openrouter v1.0.0/go.mod h1:AIa7kAu2mfrRQgcspe4L+DW51WqgnALQT60lqkEywJI=
gitea.maximumdirect.net/eric/promptkit-backend-rakestrawhome v1.0.0 h1:j9YY7wsTVjzke2kHH4YAzpU0oUpM+x+nXwl1IeS+2eg=
gitea.maximumdirect.net/eric/promptkit-backend-rakestrawhome v1.0.0/go.mod h1:4RNS+LILDg4JbS4Ts9Lwy1C92wauXJIbeQaalps4Koo=
github.com/dlclark/regexp2 v1.11.0 h1:G/nrcoOa7ZXlpoa/91N3X7mM3r8eIlMBBJZvsz/mxKI=
github.com/dlclark/regexp2 v1.11.0/go.mod h1:DHkYz0B9wPfa6wondMfaivmHpzrQ3v9q8cnmRbL6yW8=
github.com/santhosh-tekuri/jsonschema/v6 v6.0.2 h1:KRzFb2m7YtdldCEkzs6KqmJw4nqEVZGK7IN2kJkjTuQ=

View File

@@ -0,0 +1,200 @@
package cli
import (
"context"
"errors"
"flag"
"fmt"
"io"
"io/fs"
"path/filepath"
"strings"
"gitea.maximumdirect.net/eric/promptkit"
appconfig "gitea.maximumdirect.net/eric/scriptorium/internal/config"
appformat "gitea.maximumdirect.net/eric/scriptorium/internal/format"
)
type promptInspectionConfig struct {
configPath, promptDir, promptID, promptVersion, outputPath string
configExplicit bool
outputFormat appformat.OutputFormat
}
type profileInspectionConfig struct {
configPath, profileDir, profileID, outputPath string
configExplicit bool
outputFormat appformat.OutputFormat
}
// emptyPromptDefinitionFS satisfies Promptkit's engine-level prompt-source
// requirement without exposing the caller's working directory. Profile
// inspection never reads this source.
type emptyPromptDefinitionFS struct{}
func (emptyPromptDefinitionFS) Open(name string) (fs.File, error) {
return nil, &fs.PathError{Op: "open", Path: name, Err: fs.ErrNotExist}
}
func inspectCommand(args []string, stdout, stderr io.Writer) int {
if len(args) == 0 {
fmt.Fprintln(stderr, "inspect parse error: inspection mode is required")
return ExitRuntimeError
}
if args[0] == "profile" {
return inspectProfileCommand(args[1:], stdout, stderr)
}
if args[0] != "prompt" {
fmt.Fprintln(stderr, "inspect parse error: unknown inspection mode")
return ExitRuntimeError
}
cfg, err := parsePromptInspectionArgs(args[1:])
if err != nil {
fmt.Fprintf(stderr, "inspect parse error: %v\n", err)
return ExitRuntimeError
}
settings, err := resolveAppSettingsForPromptInspection(cfg)
if err != nil {
fmt.Fprintf(stderr, "inspect error: %v\n", err)
return ExitRuntimeError
}
engine, err := newEngine(settings)
if err != nil {
fmt.Fprintf(stderr, "engine error: %v\n", err)
return ExitRuntimeError
}
inspection, err := engine.InspectPrompt(context.Background(), cfg.promptID, cfg.promptVersion)
if err != nil {
fmt.Fprintf(stderr, "inspect error: %v\n", err)
return ExitRuntimeError
}
data, err := appformat.FormatPromptInspection(inspection, cfg.outputFormat)
if err != nil {
fmt.Fprintf(stderr, "inspect error: %v\n", err)
return ExitRuntimeError
}
if err := writeOutput(stdout, cfg.outputPath, data); err != nil {
fmt.Fprintf(stderr, "output write error: %v\n", err)
return ExitRuntimeError
}
return ExitOK
}
func inspectProfileCommand(args []string, stdout, stderr io.Writer) int {
cfg, err := parseProfileInspectionArgs(args)
if err != nil {
fmt.Fprintf(stderr, "inspect parse error: %v\n", err)
return ExitRuntimeError
}
settings, err := resolveAppSettingsForProfileInspection(cfg)
if err != nil {
fmt.Fprintf(stderr, "inspect error: %v\n", err)
return ExitRuntimeError
}
engine, err := newEngine(settings, promptkit.WithPromptFS(emptyPromptDefinitionFS{}, "."))
if err != nil {
fmt.Fprintf(stderr, "engine error: %v\n", err)
return ExitRuntimeError
}
inspection, err := engine.InspectProfile(context.Background(), cfg.profileID)
if err != nil {
fmt.Fprintf(stderr, "inspect error: %v\n", err)
return ExitRuntimeError
}
data, err := appformat.FormatProfileInspection(inspection, cfg.outputFormat)
if err != nil {
fmt.Fprintf(stderr, "inspect error: %v\n", err)
return ExitRuntimeError
}
if err := writeOutput(stdout, cfg.outputPath, data); err != nil {
fmt.Fprintf(stderr, "output write error: %v\n", err)
return ExitRuntimeError
}
return ExitOK
}
func parseProfileInspectionArgs(args []string) (*profileInspectionConfig, error) {
cfg := &profileInspectionConfig{outputFormat: appformat.DefaultOutputFormat}
fs := flag.NewFlagSet("inspect profile", flag.ContinueOnError)
fs.SetOutput(io.Discard)
registerConfigPathFlag(fs, &cfg.configPath)
fs.StringVar(&cfg.profileDir, "profile-dir", "", "directory containing execution profiles")
fs.StringVar(&cfg.profileID, "profile", "", "profile ID to inspect")
rawFormat := ""
fs.StringVar(&rawFormat, "format", "", "output format: text or json")
fs.StringVar(&cfg.outputPath, "out", "", "optional output file path")
if err := fs.Parse(args); err != nil {
return nil, err
}
cfg.configExplicit = flagWasSet(fs, "config")
if fs.NArg() > 0 {
return nil, fmt.Errorf("unexpected positional args: %v", fs.Args())
}
if strings.TrimSpace(cfg.profileID) == "" {
return nil, errors.New("--profile is required")
}
format, err := appformat.ParseOutputFormat(rawFormat)
if err != nil {
return nil, err
}
cfg.outputFormat = format
if cfg.outputPath != "" {
cfg.outputPath = filepath.Clean(cfg.outputPath)
}
return cfg, nil
}
func resolveAppSettingsForProfileInspection(cfg *profileInspectionConfig) (engineSettings, error) {
settings, err := resolveAppSettingsWithConfigPresence(cfg.configPath, cfg.configExplicit, appconfig.CLIOverrides{
ProfileDir: cfg.profileDir,
})
if err != nil {
return engineSettings{}, err
}
return engineSettings{profileDir: settings.ProfileDir, backends: settings.Backends}, nil
}
func parsePromptInspectionArgs(args []string) (*promptInspectionConfig, error) {
cfg := &promptInspectionConfig{outputFormat: appformat.DefaultOutputFormat}
fs := flag.NewFlagSet("inspect prompt", flag.ContinueOnError)
fs.SetOutput(io.Discard)
registerConfigPathFlag(fs, &cfg.configPath)
fs.StringVar(&cfg.promptDir, "prompt-dir", "", "directory containing prompt definitions")
fs.StringVar(&cfg.promptID, "prompt", "", "prompt ID to inspect")
fs.StringVar(&cfg.promptVersion, "prompt-version", "", "optional prompt version")
rawFormat := ""
fs.StringVar(&rawFormat, "format", "", "output format: text or json")
fs.StringVar(&cfg.outputPath, "out", "", "optional output file path")
if err := fs.Parse(args); err != nil {
return nil, err
}
cfg.configExplicit = flagWasSet(fs, "config")
if fs.NArg() > 0 {
return nil, fmt.Errorf("unexpected positional args: %v", fs.Args())
}
if strings.TrimSpace(cfg.promptID) == "" {
return nil, errors.New("--prompt is required")
}
format, err := appformat.ParseOutputFormat(rawFormat)
if err != nil {
return nil, err
}
cfg.outputFormat = format
if cfg.outputPath != "" {
cfg.outputPath = filepath.Clean(cfg.outputPath)
}
return cfg, nil
}
func resolveAppSettingsForPromptInspection(cfg *promptInspectionConfig) (engineSettings, error) {
settings, err := resolveAppSettingsWithConfigPresence(cfg.configPath, cfg.configExplicit, appconfig.CLIOverrides{
PromptDir: cfg.promptDir,
})
if err != nil {
return engineSettings{}, err
}
if strings.TrimSpace(settings.PromptDir) == "" {
return engineSettings{}, errors.New(errPromptDirRequired)
}
return engineSettings{promptDir: settings.PromptDir, backends: settings.Backends}, nil
}

View File

@@ -32,31 +32,36 @@ const (
type runConfig struct {
configPath string
promptDir string
profileDir string
promptID string
profileID string
inputRaw listFlag
varRaw listFlag
outputPath string
llmBaseURL string
apiKeyEnv string
model string
temperature float64
maxTokens int
topP float64
schemaDir string
timeout time.Duration
promptDir string
profileDir string
promptID string
promptVersion string
profileID string
sessionID string
inputRaw listFlag
varRaw listFlag
outputPath string
llmBaseURL string
apiKeyEnv string
model string
temperature float64
maxTokens int
topP float64
reasoningEffort string
schemaDir string
backends []appconfig.BackendSettings
timeout time.Duration
defaultRenderFormat renderformat.PreparedRunOutputFormat
llmBaseURLSet bool
apiKeyEnvSet bool
modelSet bool
temperatureSet bool
maxTokensSet bool
topPSet bool
timeoutSet bool
llmBaseURLSet bool
apiKeyEnvSet bool
modelSet bool
temperatureSet bool
maxTokensSet bool
topPSet bool
reasoningEffortSet bool
timeoutSet bool
}
type renderConfig struct {
@@ -75,6 +80,7 @@ type serveConfig struct {
maxRequestBytes int64
maxArtifactBytes int64
maxResponseBytes int64
backends []appconfig.BackendSettings
}
type commonCommandSettings struct {
@@ -87,6 +93,7 @@ type commonCommandSettings struct {
maxArtifactBytes int64
maxResponseBytes int64
defaultRenderFormat renderformat.PreparedRunOutputFormat
backends []appconfig.BackendSettings
}
type listFlag []string
@@ -113,6 +120,8 @@ func Run(args []string, stdout, stderr io.Writer) int {
return renderCommand(args[1:], stdout, stderr)
case "serve":
return serveCommand(args[1:], stderr)
case "inspect":
return inspectCommand(args[1:], stdout, stderr)
default:
fmt.Fprintf(stderr, "unknown command %q\n", args[0])
printUsage(stderr)
@@ -133,7 +142,7 @@ func runCommand(args []string, stdout, stderr io.Writer) int {
return ExitRuntimeError
}
engine, err := newEngine(cfg)
engine, err := newEngine(cfg.engineSettings())
if err != nil {
fmt.Fprintf(stderr, "engine error: %v\n", err)
return ExitRuntimeError
@@ -141,7 +150,7 @@ func runCommand(args []string, stdout, stderr io.Writer) int {
res, runErr := engine.Run(context.Background(), req)
if runErr != nil {
fmt.Fprintf(stderr, "run error: %v\n", runErr)
fmt.Fprintln(stderr, runErrorMessage(runErr))
return ExitRuntimeError
}
@@ -167,7 +176,7 @@ func renderCommand(args []string, stdout, stderr io.Writer) int {
return ExitRuntimeError
}
engine, err := newEngine(&cfg.runConfig)
engine, err := newEngine(cfg.runConfig.engineSettings())
if err != nil {
fmt.Fprintf(stderr, "engine error: %v\n", err)
return ExitRuntimeError
@@ -205,11 +214,7 @@ func serveCommand(args []string, stderr io.Writer) int {
return ExitRuntimeError
}
engine, err := newEngine(&runConfig{
promptDir: cfg.promptDir,
profileDir: cfg.profileDir,
schemaDir: cfg.schemaDir,
}, promptkit.WithArtifactReader(artifactReader))
engine, err := newEngine(cfg.engineSettings(), promptkit.WithArtifactReader(artifactReader))
if err != nil {
fmt.Fprintf(stderr, "engine error: %v\n", err)
return ExitRuntimeError
@@ -329,6 +334,7 @@ func parseServeArgs(args []string) (*serveConfig, error) {
cfg.maxRequestBytes = settings.maxRequestBytes
cfg.maxArtifactBytes = settings.maxArtifactBytes
cfg.maxResponseBytes = settings.maxResponseBytes
cfg.backends = settings.backends
if err := validateRequiredLibraryDirs(cfg.promptDir); err != nil {
return nil, err
@@ -349,7 +355,9 @@ func registerExecutionRequestFlags(fs *flag.FlagSet, cfg *runConfig) {
fs.StringVar(&cfg.promptDir, "prompt-dir", "", "directory containing prompt definition YAML files")
fs.StringVar(&cfg.profileDir, "profile-dir", "", "directory containing execution profile YAML files")
fs.StringVar(&cfg.promptID, "prompt", "", "prompt ID to run")
fs.StringVar(&cfg.promptVersion, "prompt-version", "", "optional prompt definition version")
fs.StringVar(&cfg.profileID, "profile", "", "optional execution profile ID; if omitted, prompt default_profile is used")
fs.StringVar(&cfg.sessionID, "session-id", "", "optional session ID")
fs.Var(&cfg.inputRaw, "input", "input mapping(s): name=path (repeatable, comma-separated)")
fs.Var(&cfg.varRaw, "var", "variable mapping(s): name=value (repeatable, comma-separated)")
fs.StringVar(&cfg.outputPath, "out", "", "optional output file path")
@@ -359,6 +367,7 @@ func registerExecutionRequestFlags(fs *flag.FlagSet, cfg *runConfig) {
fs.Float64Var(&cfg.temperature, "temperature", 0, "optional temperature override")
fs.IntVar(&cfg.maxTokens, "max-tokens", 0, "optional max tokens override")
fs.Float64Var(&cfg.topP, "top-p", 0, "optional top_p override")
fs.StringVar(&cfg.reasoningEffort, "reasoning-effort", "", "optional reasoning effort override")
fs.DurationVar(&cfg.timeout, "timeout", 0, "LLM request timeout")
fs.StringVar(&cfg.promptID, "prompt-id", "", "deprecated alias for --prompt")
fs.StringVar(&cfg.profileID, "profile-id", "", "deprecated alias for --profile")
@@ -382,6 +391,7 @@ func finalizeExecutionRequestConfig(fs *flag.FlagSet, cfg *runConfig) error {
cfg.profileDir = settings.profileDir
cfg.schemaDir = settings.schemaDir
cfg.defaultRenderFormat = settings.defaultRenderFormat
cfg.backends = settings.backends
if err := validateRequiredLibraryDirs(cfg.promptDir); err != nil {
return err
@@ -389,9 +399,6 @@ func finalizeExecutionRequestConfig(fs *flag.FlagSet, cfg *runConfig) error {
if strings.TrimSpace(cfg.promptID) == "" {
return errors.New("--prompt is required")
}
if len(cfg.inputRaw) == 0 {
return errors.New("at least one --input is required")
}
cfg.promptDir = filepath.Clean(cfg.promptDir)
if strings.TrimSpace(cfg.profileDir) != "" {
cfg.profileDir = filepath.Clean(cfg.profileDir)
@@ -405,6 +412,7 @@ func finalizeExecutionRequestConfig(fs *flag.FlagSet, cfg *runConfig) error {
cfg.temperatureSet = flagWasSet(fs, "temperature")
cfg.maxTokensSet = flagWasSet(fs, "max-tokens")
cfg.topPSet = flagWasSet(fs, "top-p")
cfg.reasoningEffortSet = flagWasSet(fs, "reasoning-effort")
cfg.timeoutSet = flagWasSet(fs, "timeout")
return nil
}
@@ -500,7 +508,11 @@ func registerConfigPathFlag(fs *flag.FlagSet, target *string) {
}
func resolveAppSettings(fs *flag.FlagSet, configPath string, overrides appconfig.CLIOverrides) (appconfig.AppSettings, error) {
settings, err := appconfig.LoadConfig(configPath, flagWasSet(fs, "config"))
return resolveAppSettingsWithConfigPresence(configPath, flagWasSet(fs, "config"), overrides)
}
func resolveAppSettingsWithConfigPresence(configPath string, configExplicit bool, overrides appconfig.CLIOverrides) (appconfig.AppSettings, error) {
settings, err := appconfig.LoadConfig(configPath, configExplicit)
if err != nil {
return appconfig.AppSettings{}, fmt.Errorf("application config error: %w", err)
}
@@ -528,6 +540,7 @@ func resolveCommonSettings(fs *flag.FlagSet, configPath string, overrides appcon
maxArtifactBytes: settings.MaxArtifactBytes,
maxResponseBytes: settings.MaxResponseBytes,
defaultRenderFormat: settings.DefaultRenderFormat,
backends: settings.Backends,
}, nil
}
@@ -538,18 +551,53 @@ func validateRequiredLibraryDirs(promptDir string) error {
return nil
}
func newEngine(cfg *runConfig, options ...promptkit.Option) (*promptkit.Engine, error) {
return promptkit.NewEngine(promptkit.Config{
PromptDir: cfg.promptDir,
ProfileDir: cfg.profileDir,
SchemaDir: cfg.schemaDir,
}, options...)
type engineSettings struct {
promptDir string
profileDir string
schemaDir string
backends []appconfig.BackendSettings
}
func (c runConfig) engineSettings() engineSettings {
return engineSettings{promptDir: c.promptDir, profileDir: c.profileDir, schemaDir: c.schemaDir, backends: c.backends}
}
func (c serveConfig) engineSettings() engineSettings {
return engineSettings{promptDir: c.promptDir, profileDir: c.profileDir, schemaDir: c.schemaDir, backends: c.backends}
}
func newEngine(settings engineSettings, options ...promptkit.Option) (*promptkit.Engine, error) {
engineOptions := make([]promptkit.Option, 0, len(settings.backends)+len(options))
for _, configured := range settings.backends {
engineOptions = append(engineOptions, promptkit.WithBackend(promptkit.Backend{
ID: configured.ID,
Endpoint: configured.Endpoint,
APIKeyEnv: configured.APIKeyEnv,
ExtraParams: configured.ExtraParams,
ConcurrencyLimit: configured.ConcurrencyLimit,
QueueCapacity: configured.QueueCapacity,
}))
}
engineOptions = append(engineOptions, options...)
engine, err := promptkit.NewEngine(promptkit.Config{
PromptDir: settings.promptDir,
ProfileDir: settings.profileDir,
SchemaDir: settings.schemaDir,
}, engineOptions...)
if err != nil {
return nil, fmt.Errorf("engine initialization from application configuration: %w", err)
}
return engine, nil
}
func buildRunRequestFromConfig(cfg *runConfig) (promptkit.RunRequest, error) {
inputMappings, err := parseMappings(cfg.inputRaw, false)
if err != nil {
return promptkit.RunRequest{}, fmt.Errorf("input parse error: %w", err)
var inputMappings map[string]string
var err error
if len(cfg.inputRaw) > 0 {
inputMappings, err = parseMappings(cfg.inputRaw, false)
if err != nil {
return promptkit.RunRequest{}, fmt.Errorf("input parse error: %w", err)
}
}
varMappings := map[string]string{}
@@ -560,13 +608,16 @@ func buildRunRequestFromConfig(cfg *runConfig) (promptkit.RunRequest, error) {
}
}
inputs := make(map[string]promptkit.ArtifactRef, len(inputMappings))
for name, path := range inputMappings {
inputs[name] = promptkit.File(path)
var inputs map[string]promptkit.ArtifactRef
if len(inputMappings) > 0 {
inputs = make(map[string]promptkit.ArtifactRef, len(inputMappings))
for name, path := range inputMappings {
inputs[name] = promptkit.File(path)
}
}
var modelOverride *promptkit.ExecutionTargetOverride
if cfg.llmBaseURLSet || cfg.modelSet || cfg.temperatureSet || cfg.maxTokensSet || cfg.topPSet || cfg.apiKeyEnvSet || cfg.timeoutSet {
if cfg.llmBaseURLSet || cfg.modelSet || cfg.temperatureSet || cfg.maxTokensSet || cfg.topPSet || cfg.reasoningEffortSet || cfg.apiKeyEnvSet || cfg.timeoutSet {
modelOverride = &promptkit.ExecutionTargetOverride{
Endpoint: cfg.llmBaseURL,
Model: cfg.model,
@@ -581,6 +632,9 @@ func buildRunRequestFromConfig(cfg *runConfig) (promptkit.RunRequest, error) {
if cfg.topPSet {
modelOverride.TopP = &cfg.topP
}
if cfg.reasoningEffortSet {
modelOverride.ReasoningEffort = &cfg.reasoningEffort
}
if cfg.timeoutSet {
timeoutSeconds := int(cfg.timeout.Seconds())
modelOverride.TimeoutSeconds = &timeoutSeconds
@@ -588,11 +642,13 @@ func buildRunRequestFromConfig(cfg *runConfig) (promptkit.RunRequest, error) {
}
return promptkit.RunRequest{
PromptID: cfg.promptID,
ProfileID: cfg.profileID,
Inputs: inputs,
Vars: varMappings,
Execution: modelOverride,
PromptID: cfg.promptID,
PromptVersion: cfg.promptVersion,
ProfileID: cfg.profileID,
SessionID: cfg.sessionID,
Inputs: inputs,
Vars: varMappings,
Execution: modelOverride,
}, nil
}
@@ -682,12 +738,24 @@ func printSummary(stderr io.Writer, res *promptkit.RunResult) {
if res.Usage.CachedTokens != 0 || res.Usage.CacheWriteTokens != 0 {
fmt.Fprintf(stderr, " cached_tokens=%d cache_write_tokens=%d", res.Usage.CachedTokens, res.Usage.CacheWriteTokens)
}
if res.SelectedBackendID != "" {
fmt.Fprintf(stderr, " backend=%s", res.SelectedBackendID)
}
fmt.Fprintln(stderr)
}
func printUsage(w io.Writer) {
fmt.Fprintln(w, "usage: scriptorium <run|render|serve> ...")
fmt.Fprintln(w, " run: scriptorium run [--config PATH] [--prompt-dir DIR] [--profile-dir DIR] --prompt ID --input name=path [--input ...] [--profile ID] [--llm-base-url URL] [--model NAME] [--api-key-env ENV] [--temperature N] [--max-tokens N] [--top-p N] [--var k=v] [--out path] [--timeout 10m]")
fmt.Fprintln(w, " render: scriptorium render [--config PATH] [--prompt-dir DIR] [--profile-dir DIR] --prompt ID --input name=path [--input ...] [--profile ID] [--llm-base-url URL] [--model NAME] [--api-key-env ENV] [--temperature N] [--max-tokens N] [--top-p N] [--var k=v] [--format text|json] [--out path] [--timeout 10m]")
fmt.Fprintf(w, " serve: scriptorium serve [--config PATH] [--addr %s] [--prompt-dir DIR] [--profile-dir DIR] [--schema-dir DIR] [--artifact-root DIR] [--max-request-bytes N] [--max-artifact-bytes N] [--max-response-bytes N]\n", defaults.HTTPAddrDefault)
func runErrorMessage(err error) string {
if errors.Is(err, promptkit.ErrCapacityExceeded) {
return "run error: model backend capacity is exhausted"
}
return fmt.Sprintf("run error: %v", err)
}
func printUsage(w io.Writer) {
fmt.Fprintln(w, "usage: scriptorium <run|render|serve|inspect> ...")
fmt.Fprintln(w, " run: scriptorium run [--config PATH] [--prompt-dir DIR] [--profile-dir DIR] --prompt ID [--prompt-version VERSION] [--input name=path] [--profile ID] [--session-id ID] [--llm-base-url URL] [--model NAME] [--api-key-env ENV] [--temperature N] [--max-tokens N] [--top-p N] [--reasoning-effort VALUE] [--var k=v] [--out path] [--timeout 10m]")
fmt.Fprintln(w, " render: scriptorium render [--config PATH] [--prompt-dir DIR] [--profile-dir DIR] --prompt ID [--prompt-version VERSION] [--input name=path] [--profile ID] [--session-id ID] [--llm-base-url URL] [--model NAME] [--api-key-env ENV] [--temperature N] [--max-tokens N] [--top-p N] [--reasoning-effort VALUE] [--var k=v] [--format text|json] [--out path] [--timeout 10m]")
fmt.Fprintf(w, " serve: scriptorium serve [--config PATH] [--addr %s] [--prompt-dir DIR] [--profile-dir DIR] [--schema-dir DIR] [--artifact-root DIR] [--max-request-bytes N] [--max-artifact-bytes N] [--max-response-bytes N]\n", defaults.HTTPAddrDefault)
fmt.Fprintln(w, " inspect prompt: scriptorium inspect prompt --prompt ID [--prompt-version VERSION] [--config PATH] [--prompt-dir DIR] [--format text|json] [--out path]")
fmt.Fprintln(w, " inspect profile: scriptorium inspect profile --profile ID [--config PATH] [--profile-dir DIR] [--format text|json] [--out path]")
}

View File

@@ -87,9 +87,12 @@ func TestParseRunArgsRequiredFlags(t *testing.T) {
t.Fatal("expected missing --prompt error")
}
_, err = parseRunArgs([]string{"--config", configPath, "--prompt-dir", "./prompts", "--profile-dir", "./profiles", "--prompt", "p"})
if err == nil {
t.Fatal("expected missing --input error")
cfg, err = parseRunArgs([]string{"--config", configPath, "--prompt-dir", "./prompts", "--profile-dir", "./profiles", "--prompt", "p"})
if err != nil {
t.Fatalf("expected omitted --input to be accepted, got %v", err)
}
if len(cfg.inputRaw) != 0 {
t.Fatalf("expected no input mappings, got %#v", cfg.inputRaw)
}
}
@@ -98,13 +101,16 @@ func TestParseRunArgsFlagMapping(t *testing.T) {
"--prompt-dir", "./prompts",
"--profile-dir", "./profiles",
"--prompt", "prompt.a",
"--prompt-version", "2",
"--profile", "profile.a",
"--session-id", "session-1",
"--input", "a=b",
"--llm-base-url", "http://x/v1",
"--model", "m",
"--temperature", "0.7",
"--max-tokens", "111",
"--top-p", "0.8",
"--reasoning-effort", "medium",
"--timeout", "30s",
"--api-key-env", "SCRIPTORIUM_API_KEY",
})
@@ -114,8 +120,11 @@ func TestParseRunArgsFlagMapping(t *testing.T) {
if cfg.promptDir != filepath.Clean("./prompts") || cfg.profileDir != filepath.Clean("./profiles") {
t.Fatalf("unexpected dirs: prompt=%q profile=%q", cfg.promptDir, cfg.profileDir)
}
if cfg.promptID != "prompt.a" || cfg.profileID != "profile.a" {
t.Fatalf("unexpected prompt/profile ids: %q %q", cfg.promptID, cfg.profileID)
if cfg.promptID != "prompt.a" || cfg.promptVersion != "2" || cfg.profileID != "profile.a" {
t.Fatalf("unexpected prompt/version/profile ids: %q %q %q", cfg.promptID, cfg.promptVersion, cfg.profileID)
}
if cfg.sessionID != "session-1" || cfg.reasoningEffort != "medium" || !cfg.reasoningEffortSet {
t.Fatalf("unexpected session or reasoning configuration: %+v", cfg)
}
if !cfg.llmBaseURLSet || !cfg.modelSet || !cfg.temperatureSet || !cfg.maxTokensSet || !cfg.topPSet || !cfg.timeoutSet || !cfg.apiKeyEnvSet {
t.Fatalf("expected override flags set, got %+v", cfg)
@@ -192,7 +201,7 @@ func TestParseServeArgsRejectsRuntimeOverrideFlags(t *testing.T) {
}
}
func TestUsageIncludesServeFileAndSizeLimitFlags(t *testing.T) {
func TestUsageIncludesExecutionAndServeFlags(t *testing.T) {
var stderr bytes.Buffer
code := Run(nil, io.Discard, &stderr)
if code != ExitRuntimeError {
@@ -201,10 +210,15 @@ func TestUsageIncludesServeFileAndSizeLimitFlags(t *testing.T) {
usage := stderr.String()
for _, want := range []string{
"--prompt-version VERSION",
"--session-id ID",
"--reasoning-effort VALUE",
"--artifact-root",
"--max-request-bytes",
"--max-artifact-bytes",
"--max-response-bytes",
"inspect prompt",
"inspect profile",
} {
if !strings.Contains(usage, want) {
t.Fatalf("expected usage to include %q, got %q", want, usage)
@@ -689,6 +703,55 @@ func TestBuildRunRequestPreservesNumericOverridePresence(t *testing.T) {
}
}
func TestBuildRunRequestPreservesReasoningEffortPresenceAndSessionID(t *testing.T) {
omitted, err := buildRunRequestFromConfig(&runConfig{promptID: "prompt-1"})
if err != nil {
t.Fatalf("expected omitted request to build, got %v", err)
}
if omitted.Execution != nil {
t.Fatalf("expected omitted reasoning flag to leave execution nil, got %#v", omitted.Execution)
}
for _, tc := range []struct {
name string
value string
}{
{name: "replacement", value: "high"},
{name: "clear", value: ""},
} {
t.Run(tc.name, func(t *testing.T) {
req, err := buildRunRequestFromConfig(&runConfig{
promptID: "prompt-1",
sessionID: "session-1",
reasoningEffort: tc.value,
reasoningEffortSet: true,
})
if err != nil {
t.Fatalf("expected request to build, got %v", err)
}
if req.SessionID != "session-1" || req.Execution == nil || req.Execution.ReasoningEffort == nil || *req.Execution.ReasoningEffort != tc.value {
t.Fatalf("unexpected mapped request: %#v", req)
}
})
}
}
func TestBuildRunRequestAllowsOmittedInputsAndMapsPromptVersion(t *testing.T) {
req, err := buildRunRequestFromConfig(&runConfig{
promptID: "prompt-1",
promptVersion: "2",
})
if err != nil {
t.Fatalf("expected request without inputs to build, got %v", err)
}
if req.PromptVersion != "2" {
t.Fatalf("expected prompt version to be mapped, got %q", req.PromptVersion)
}
if req.Inputs != nil {
t.Fatalf("expected omitted inputs to remain nil, got %#v", req.Inputs)
}
}
func TestParseRunArgsFailsClearlyWhenNoEffectivePromptDir(t *testing.T) {
configPath := writeAppConfigFile(t, `
profile_dir: ./profiles
@@ -942,6 +1005,452 @@ profile_dir: %s
}
}
func TestRenderCommandUsesConfiguredCustomBackend(t *testing.T) {
lib := newCLITestLibrary(t)
writePromptDefinition(t, lib.promptDir, "custom.yaml", `id: custom
version: "1"
default_profile: local-gpu
messages:
- role: user
content: "hello"
output:
format: text
validation_mode: none
`)
if err := os.WriteFile(filepath.Join(lib.profileDir, "local-gpu.yaml"), []byte(`id: local-gpu
backend: local-gpu
model: local-model
`), 0o644); err != nil {
t.Fatalf("write profile fixture: %v", err)
}
configPath := writeAppConfigFile(t, fmt.Sprintf(`
prompt_dir: %s
profile_dir: %s
backends:
local-gpu:
endpoint: http://localhost:11434/v1
extra_params:
provider_option: enabled
concurrency_limit: 2
queue_capacity: 0
`, lib.promptDir, lib.profileDir))
code, stdout, stderr := runCLICommand(t, renderCommand, []string{
"--config", configPath,
"--prompt", "custom",
})
if code != ExitOK {
t.Fatalf("expected ExitOK, got %d stderr=%q", code, stderr)
}
if !strings.Contains(stdout, "selected_backend_id: local-gpu") {
t.Fatalf("expected configured backend in prepared output, got:\n%s", stdout)
}
}
func TestRenderCommandMapsReasoningEffortAndSessionID(t *testing.T) {
lib := newCLITestLibrary(t)
writePromptDefinition(t, lib.promptDir, "session.yaml", `id: session
version: "1"
default_profile: local
session_id: definition-session
messages:
- role: user
content: "hello"
output:
format: text
validation_mode: none
`)
if err := os.WriteFile(filepath.Join(lib.profileDir, "local.yaml"), []byte(`id: local
endpoint: http://127.0.0.1:1/v1
model: local-model
reasoning_effort: low
`), 0o644); err != nil {
t.Fatalf("write profile fixture: %v", err)
}
for _, tc := range []struct {
name string
args []string
wantReasoning string
wantSessionID string
absentReasoning bool
}{
{name: "omitted reasoning inherits profile", wantReasoning: "low", wantSessionID: "definition-session"},
{name: "nonblank reasoning replaces profile", args: []string{"--reasoning-effort", "high"}, wantReasoning: "high", wantSessionID: "definition-session"},
{name: "empty reasoning clears profile", args: []string{"--reasoning-effort="}, wantSessionID: "definition-session", absentReasoning: true},
{name: "direct session replaces definition", args: []string{"--session-id", "direct-session"}, wantReasoning: "low", wantSessionID: "direct-session"},
} {
t.Run(tc.name, func(t *testing.T) {
args := []string{"--prompt-dir", lib.promptDir, "--profile-dir", lib.profileDir, "--prompt", "session"}
args = append(args, tc.args...)
code, stdout, stderr := runCLICommand(t, renderCommand, args)
if code != ExitOK {
t.Fatalf("expected ExitOK, got %d stderr=%q", code, stderr)
}
if !strings.Contains(stdout, "session_id: "+tc.wantSessionID) {
t.Fatalf("expected session ID %q, got:\n%s", tc.wantSessionID, stdout)
}
hasReasoning := strings.Contains(stdout, "reasoning_effort:")
if tc.absentReasoning {
if hasReasoning {
t.Fatalf("expected cleared reasoning to be omitted, got:\n%s", stdout)
}
return
}
if !strings.Contains(stdout, "reasoning_effort: "+tc.wantReasoning) {
t.Fatalf("expected reasoning effort %q, got:\n%s", tc.wantReasoning, stdout)
}
})
}
}
func TestRenderCommandOmitsEmptyEffectiveSessionID(t *testing.T) {
lib := newCLITestLibrary(t)
writePromptDefinition(t, lib.promptDir, "plain.yaml", `id: plain
version: "1"
default_profile: local
messages:
- role: user
content: "hello"
output:
format: text
validation_mode: none
`)
writeProfileFile(t, lib.profileDir, "local", "http://127.0.0.1:1/v1", "local-model")
code, stdout, stderr := runCLICommand(t, renderCommand, []string{
"--prompt-dir", lib.promptDir,
"--profile-dir", lib.profileDir,
"--prompt", "plain",
})
if code != ExitOK {
t.Fatalf("expected ExitOK, got %d stderr=%q", code, stderr)
}
if strings.Contains(stdout, "session_id:") {
t.Fatalf("expected no effective session ID, got:\n%s", stdout)
}
}
func TestRenderCommandRejectsOverlongSessionID(t *testing.T) {
lib := newCLITestLibrary(t)
writePromptDefinition(t, lib.promptDir, "session.yaml", `id: session
version: "1"
default_profile: local
messages:
- role: user
content: "hello"
output:
format: text
validation_mode: none
`)
writeProfileFile(t, lib.profileDir, "local", "http://127.0.0.1:1/v1", "local-model")
code, _, stderr := runCLICommand(t, renderCommand, []string{
"--prompt-dir", lib.promptDir,
"--profile-dir", lib.profileDir,
"--prompt", "session",
"--session-id", strings.Repeat("x", 257),
})
if code != ExitRuntimeError {
t.Fatalf("expected runtime error, got %d stderr=%q", code, stderr)
}
if !strings.Contains(stderr, "invalid run request") {
t.Fatalf("expected invalid-request context, got %q", stderr)
}
}
func TestInspectPromptCommandFormatsDefinitionWithoutProfileOrGeneration(t *testing.T) {
lib := newCLITestLibrary(t)
writePromptDefinition(t, lib.promptDir, "inspect.yaml", `id: inspect
version: "1"
messages:
- role: user
content: "hello"
output:
format: text
validation_mode: none
`)
code, stdout, stderr := runCLICommand(t, inspectCommand, []string{"prompt", "--prompt-dir", lib.promptDir, "--prompt", "inspect", "--format", "json"})
if code != ExitOK {
t.Fatalf("expected ExitOK, got %d stderr=%q", code, stderr)
}
if !strings.Contains(stdout, `"prompt_id": "inspect"`) || !strings.Contains(stdout, `"inputs": []`) {
t.Fatalf("unexpected inspection output: %s", stdout)
}
code, stdout, stderr = runCLICommand(t, inspectCommand, []string{"prompt", "--prompt-dir", lib.promptDir, "--prompt", "missing"})
if code != ExitRuntimeError || stdout != "" || !strings.Contains(stderr, "inspect error") {
t.Fatalf("expected failed inspection without output, got code=%d stdout=%q stderr=%q", code, stdout, stderr)
}
}
func TestInspectionParsersPreserveExplicitEmptyConfigPath(t *testing.T) {
promptConfig, err := parsePromptInspectionArgs([]string{"--config=", "--prompt", "fixture"})
if err != nil {
t.Fatalf("parse prompt inspection: %v", err)
}
if promptConfig.configPath != "" || !promptConfig.configExplicit {
t.Fatalf("expected explicit empty prompt config path, got %+v", promptConfig)
}
profileConfig, err := parseProfileInspectionArgs([]string{"--config=", "--profile", "fixture"})
if err != nil {
t.Fatalf("parse profile inspection: %v", err)
}
if profileConfig.configPath != "" || !profileConfig.configExplicit {
t.Fatalf("expected explicit empty profile config path, got %+v", profileConfig)
}
}
func TestPromptkitV09DefinitionsRenderThroughCLI(t *testing.T) {
fixtureRoot := promptkitV09FixtureRoot(t)
configPath := writePromptkitV09Config(t, fixtureRoot, true)
inputPath := filepath.Join(fixtureRoot, "inputs", "source.md")
code, stdout, stderr := runCLICommand(t, renderCommand, []string{
"--config", configPath,
"--prompt", "compat.complete",
"--prompt-version", "1.0.0",
"--input", "source=" + inputPath,
"--var", "topic=testing",
"--format", "json",
})
if code != ExitOK {
t.Fatalf("expected ExitOK, got %d stderr=%q", code, stderr)
}
var payload map[string]any
if err := json.Unmarshal([]byte(stdout), &payload); err != nil {
t.Fatalf("decode rendered fixture: %v\nbody=%s", err, stdout)
}
if payload["prompt_version"] != "1.0.0" || payload["selected_profile_id"] != "custom-derived" || payload["selected_backend_id"] != "fixture-custom" {
t.Fatalf("unexpected selected definition and target: %#v", payload)
}
if payload["session_id"] != "fixture-testing" {
t.Fatalf("expected rendered session template, got %#v", payload["session_id"])
}
outputContract := payload["output_contract"].(map[string]any)
if outputContract["format"] != "json" || outputContract["validation_mode"] != "json_schema" || outputContract["repair_attempts"] != float64(2) {
t.Fatalf("unexpected output contract: %#v", outputContract)
}
if _, ok := payload["structured_output"].(map[string]any)["json_schema"]; !ok {
t.Fatalf("expected loaded JSON schema metadata, got %#v", payload["structured_output"])
}
messages := payload["messages"].([]any)
wantRoles := []string{"developer", "system", "user", "assistant"}
for i, wantRole := range wantRoles {
message := messages[i].(map[string]any)
if message["role"] != wantRole {
t.Fatalf("message %d: expected role %q, got %#v", i, wantRole, message["role"])
}
}
cacheControl := messages[0].(map[string]any)["cache_control"].(map[string]any)
if cacheControl["type"] != "ephemeral" || cacheControl["ttl"] != "1h" {
t.Fatalf("unexpected cache control: %#v", cacheControl)
}
if !strings.Contains(messages[2].(map[string]any)["content"].(string), "stable, synthetic material") {
t.Fatalf("file-backed input template was not rendered: %#v", messages[2])
}
inputHashes := payload["input_hashes"].(map[string]any)
if len(inputHashes) != 1 || inputHashes["source"] == "" {
t.Fatalf("required and omitted optional inputs were not preserved: %#v", inputHashes)
}
}
func TestPromptkitV09PromptInspectionSelectsVersionsAndContracts(t *testing.T) {
fixtureRoot := promptkitV09FixtureRoot(t)
configPath := writePromptkitV09Config(t, fixtureRoot, true)
tests := []struct {
promptID string
version string
format string
mode string
}{
{promptID: "compat.complete", version: "1.0.0", format: "json", mode: "json_schema"},
{promptID: "compat.complete", version: "2.0.0", format: "text", mode: "basic"},
{promptID: "compat.json", version: "1.0.0", format: "json", mode: "json"},
{promptID: "compat.none", version: "1.0.0", format: "text", mode: "none"},
}
for _, tc := range tests {
t.Run(tc.promptID+"@"+tc.version, func(t *testing.T) {
code, stdout, stderr := runCLICommand(t, inspectCommand, []string{
"prompt",
"--config", configPath,
"--prompt", tc.promptID,
"--prompt-version", tc.version,
"--format", "json",
})
if code != ExitOK {
t.Fatalf("expected ExitOK, got %d stderr=%q", code, stderr)
}
var inspection renderformat.PromptInspection
if err := json.Unmarshal([]byte(stdout), &inspection); err != nil {
t.Fatalf("decode prompt inspection: %v\nbody=%s", err, stdout)
}
if inspection.PromptVersion != tc.version || inspection.OutputContract.Format != tc.format || inspection.OutputContract.ValidationMode != tc.mode {
t.Fatalf("unexpected prompt inspection: %+v", inspection)
}
})
}
}
func TestPromptkitV09ProfileInspectionResolvesSupportedTargets(t *testing.T) {
const secret = "sentinel-profile-secret"
t.Setenv("FIXTURE_PROFILE_API_KEY", secret)
fixtureRoot := promptkitV09FixtureRoot(t)
configPath := writePromptkitV09Config(t, fixtureRoot, true)
tests := []struct {
name string
profileID string
wantBackend string
wantModel string
wantAPIKeyEnv string
}{
{name: "inherited custom backend", profileID: "custom-derived", wantBackend: "fixture-custom", wantModel: "fixture-derived-model", wantAPIKeyEnv: "FIXTURE_PROFILE_API_KEY"},
{name: "endpoint only", profileID: "endpoint-only", wantModel: "fixture-endpoint-model"},
{name: "built in", profileID: "deepseek-4-flash", wantBackend: "openrouter", wantModel: "deepseek/deepseek-v4-flash", wantAPIKeyEnv: "OPENROUTER_API_KEY"},
}
for _, tc := range tests {
t.Run(tc.name, func(t *testing.T) {
args := []string{"profile", "--config", configPath, "--profile", tc.profileID, "--format", "json"}
code, stdout, stderr := runCLICommand(t, inspectCommand, args)
if code != ExitOK {
t.Fatalf("expected ExitOK, got %d stderr=%q", code, stderr)
}
if strings.Contains(stdout, secret) || strings.Contains(stderr, secret) {
t.Fatalf("inspection exposed environment secret: stdout=%q stderr=%q", stdout, stderr)
}
var inspection renderformat.ProfileInspection
if err := json.Unmarshal([]byte(stdout), &inspection); err != nil {
t.Fatalf("decode profile inspection: %v\nbody=%s", err, stdout)
}
params := inspection.EffectiveModelParams
if inspection.ProfileID != tc.profileID || params.BackendID != tc.wantBackend || params.Model != tc.wantModel || params.APIKeyEnv != tc.wantAPIKeyEnv {
t.Fatalf("unexpected effective profile: %+v", inspection)
}
if tc.profileID == "custom-derived" {
if params.ServiceTier != "flex" || params.ReasoningEffort != "high" || params.TimeoutSeconds != 45 || len(params.ExtraParams) == 0 {
t.Fatalf("inherited profile controls were not resolved: %+v", params)
}
}
})
}
configWithoutPromptDir := writePromptkitV09Config(t, fixtureRoot, false)
code, _, stderr := runCLICommand(t, inspectCommand, []string{
"profile",
"--config", configWithoutPromptDir,
"--profile-dir", filepath.Join(fixtureRoot, "profiles"),
"--profile", "custom-derived",
})
if code != ExitOK {
t.Fatalf("profile inspection unexpectedly required a prompt directory: %q", stderr)
}
}
func TestProfileInspectionHonorsDirectoryPrecedenceOutputAndFailures(t *testing.T) {
fixtureRoot := promptkitV09FixtureRoot(t)
configPath := writePromptkitV09Config(t, fixtureRoot, true)
overrideDir := t.TempDir()
writeProfileFile(t, overrideDir, "custom-derived", "http://127.0.0.1:9000/v1", "override-model")
outPath := filepath.Join(t.TempDir(), "inspection.json")
code, stdout, stderr := runCLICommand(t, inspectCommand, []string{
"profile",
"--config", configPath,
"--profile-dir", overrideDir,
"--profile", "custom-derived",
"--format", "json",
"--out", outPath,
})
if code != ExitOK || stdout != "" {
t.Fatalf("expected file output, got code=%d stdout=%q stderr=%q", code, stdout, stderr)
}
output, err := os.ReadFile(outPath)
if err != nil {
t.Fatalf("read profile inspection output: %v", err)
}
if !strings.Contains(string(output), `"model": "override-model"`) || !strings.Contains(string(output), `"backend_id": ""`) {
t.Fatalf("profile directory override was not used: %s", output)
}
for _, profileID := range []string{"missing", "invalid"} {
t.Run(profileID, func(t *testing.T) {
profileDir := filepath.Join(fixtureRoot, "profiles")
if profileID == "invalid" {
profileDir = t.TempDir()
writePromptDefinition(t, profileDir, "invalid.yaml", "id: invalid\nendpoint: not-a-url\nmodel: fixture\n")
}
code, stdout, stderr := runCLICommand(t, inspectCommand, []string{
"profile",
"--config", configPath,
"--profile-dir", profileDir,
"--profile", profileID,
})
if code != ExitRuntimeError || stdout != "" || !strings.Contains(stderr, "inspect error") {
t.Fatalf("expected inspection failure without partial output, got code=%d stdout=%q stderr=%q", code, stdout, stderr)
}
})
}
}
func TestConfiguredBackendValidationComesFromPromptkit(t *testing.T) {
lib := newCLITestLibrary(t)
for _, tc := range []struct {
name string
backend string
}{
{
name: "reserved ID",
backend: `openrouter:
endpoint: http://localhost:11434/v1`,
},
{
name: "invalid endpoint",
backend: `local-gpu:
endpoint: not-a-url`,
},
{
name: "invalid capacity relationship",
backend: `local-gpu:
endpoint: http://localhost:11434/v1
queue_capacity: 0`,
},
{
name: "reserved extra parameter",
backend: `local-gpu:
endpoint: http://localhost:11434/v1
extra_params:
model: forbidden`,
},
} {
t.Run(tc.name, func(t *testing.T) {
configPath := writeAppConfigFile(t, fmt.Sprintf(`
prompt_dir: %s
profile_dir: %s
backends:
%s
`, lib.promptDir, lib.profileDir, tc.backend))
cfg, err := parseRenderArgs([]string{"--config", configPath, "--prompt", "custom"})
if err != nil {
t.Fatalf("expected config decoding to succeed, got %v", err)
}
_, err = newEngine(cfg.runConfig.engineSettings())
if !errors.Is(err, promptkit.ErrInvalidConfig) {
t.Fatalf("expected Promptkit ErrInvalidConfig, got %v", err)
}
if !strings.Contains(err.Error(), "engine initialization from application configuration") {
t.Fatalf("expected application context, got %v", err)
}
})
}
}
func TestRenderCommandExplicitTextFormatWorks(t *testing.T) {
lib := newCLITestLibrary(t)
inputPath := lib.writeInputFile(t, "transcript.md", "hello transcript")
@@ -1122,6 +1631,104 @@ func TestRenderCommandExplicitProfileOverridesPromptDefault(t *testing.T) {
}
}
func TestRenderCommandUsesDefinitionInputRulesAndPromptVersions(t *testing.T) {
lib := newCLITestLibrary(t)
writeProfileFile(t, lib.profileDir, "local", "http://127.0.0.1:1/v1", "model")
writePromptDefinition(t, lib.promptDir, "sole.yaml", `id: sole
version: "1"
default_profile: local
messages:
- role: user
content: "hello"
output:
format: text
validation_mode: none
`)
writePromptDefinition(t, lib.promptDir, "versioned-one.yaml", `id: versioned
version: "1"
default_profile: local
messages:
- role: user
content: "one"
output:
format: text
validation_mode: none
`)
writePromptDefinition(t, lib.promptDir, "versioned-two.yaml", `id: versioned
version: "2"
default_profile: local
messages:
- role: user
content: "two"
output:
format: text
validation_mode: none
`)
writePromptDefinition(t, lib.promptDir, "optional.yaml", `id: optional
version: "1"
default_profile: local
inputs:
- name: note
required: false
messages:
- role: user
content: "hello"
output:
format: text
validation_mode: none
`)
writePromptDefinition(t, lib.promptDir, "required.yaml", `id: required
version: "1"
default_profile: local
inputs:
- name: note
required: true
messages:
- role: user
content: "hello"
output:
format: text
validation_mode: none
`)
writePromptDefinition(t, lib.promptDir, "template.yaml", `id: template
version: "1"
default_profile: local
messages:
- role: user
content: '{{input "note"}}'
output:
format: text
validation_mode: none
`)
baseArgs := []string{"--prompt-dir", lib.promptDir, "--profile-dir", lib.profileDir}
for _, tc := range []struct {
name string
args []string
wantCode int
wantText string
}{
{name: "sole version selected when omitted", args: []string{"--prompt", "sole"}, wantCode: ExitOK, wantText: "prompt_version: 1"},
{name: "explicit version selected", args: []string{"--prompt", "versioned", "--prompt-version", "2"}, wantCode: ExitOK, wantText: "prompt_version: 2"},
{name: "multiple versions require selection", args: []string{"--prompt", "versioned"}, wantCode: ExitRuntimeError, wantText: "duplicate prompt definition id"},
{name: "no declared inputs", args: []string{"--prompt", "sole"}, wantCode: ExitOK, wantText: "prompt: sole"},
{name: "optional input omitted", args: []string{"--prompt", "optional"}, wantCode: ExitOK, wantText: "prompt: optional"},
{name: "required input omitted", args: []string{"--prompt", "required"}, wantCode: ExitRuntimeError, wantText: "required"},
{name: "template input omitted", args: []string{"--prompt", "template"}, wantCode: ExitRuntimeError, wantText: "note"},
} {
t.Run(tc.name, func(t *testing.T) {
code, stdout, stderr := runCLICommand(t, renderCommand, append(append([]string{}, baseArgs...), tc.args...))
if code != tc.wantCode {
t.Fatalf("expected exit %d, got %d stderr=%q", tc.wantCode, code, stderr)
}
if !strings.Contains(stdout+stderr, tc.wantText) {
t.Fatalf("expected output to contain %q, stdout=%q stderr=%q", tc.wantText, stdout, stderr)
}
})
}
}
func TestRunCommandPromptDefaultProfileWorksThroughCLIPath(t *testing.T) {
lib := newCLITestLibrary(t)
inputPath := lib.writeInputFile(t, "transcript.md", "hello")
@@ -1261,6 +1868,9 @@ func TestWriteOutputAndSummaryUseSeparateWriters(t *testing.T) {
if strings.Contains(stderr.String(), "cached_tokens=") || strings.Contains(stderr.String(), "cache_write_tokens=") {
t.Fatalf("expected zero cache usage to be omitted from summary, got %q", stderr.String())
}
if strings.Contains(stderr.String(), "backend=") {
t.Fatalf("expected endpoint-only backend to be omitted from summary, got %q", stderr.String())
}
}
func TestPrintSummaryIncludesCacheUsageWhenPresent(t *testing.T) {
@@ -1290,6 +1900,32 @@ func TestPrintSummaryIncludesCacheUsageWhenPresent(t *testing.T) {
if !strings.Contains(summary, "cached_tokens=0 cache_write_tokens=3") {
t.Fatalf("expected cache usage in summary, got %q", summary)
}
if strings.Contains(summary, "backend=") {
t.Fatalf("expected endpoint-only backend to be omitted from summary, got %q", summary)
}
}
func TestPrintSummaryIncludesBackendWhenPresent(t *testing.T) {
var stderr bytes.Buffer
printSummary(&stderr, &promptkit.RunResult{
PromptID: "p",
PromptVersion: "1",
SelectedProfileID: "exec",
SelectedBackendID: "local",
ModelName: "m",
Validation: promptkit.ValidationResult{Status: promptkit.ValidationPassed, Mode: promptkit.ValidationBasic},
RenderedPromptHash: "h",
})
if !strings.Contains(stderr.String(), "backend=local") {
t.Fatalf("expected backend in summary, got %q", stderr.String())
}
}
func TestRunErrorMessageDoesNotExposeCapacityDetails(t *testing.T) {
got := runErrorMessage(&promptkit.CapacityError{BackendID: "private-backend"})
if got != "run error: model backend capacity is exhausted" {
t.Fatalf("unexpected capacity diagnostic: %q", got)
}
}
type cliTestLibrary struct {
@@ -1335,11 +1971,48 @@ func runCLICommand(t *testing.T, command func([]string, io.Writer, io.Writer) in
return code, stdout.String(), stderr.String()
}
func promptkitV09FixtureRoot(t *testing.T) string {
t.Helper()
root, err := filepath.Abs(filepath.Join("..", "..", "..", "testdata", "promptkit-v0.9"))
if err != nil {
t.Fatalf("resolve Promptkit v0.9 fixture root: %v", err)
}
if _, err := os.Stat(root); err != nil {
t.Fatalf("stat Promptkit v0.9 fixture root: %v", err)
}
return root
}
func writePromptkitV09Config(t *testing.T, fixtureRoot string, includeSources bool) string {
t.Helper()
sources := ""
if includeSources {
sources = fmt.Sprintf("prompt_dir: %q\nprofile_dir: %q\nschema_dir: %q\n", filepath.Join(fixtureRoot, "prompts"), filepath.Join(fixtureRoot, "profiles"), filepath.Join(fixtureRoot, "schemas"))
}
return writeAppConfigFile(t, sources+`backends:
fixture-custom:
endpoint: http://127.0.0.1:11434/v1
api_key_env: FIXTURE_BACKEND_API_KEY
extra_params:
backend_option:
enabled: true
concurrency_limit: 2
queue_capacity: 0
`)
}
func writePromptFile(t *testing.T, dir, id, defaultProfile string) {
t.Helper()
writePromptFileWithTemplate(t, dir, id, defaultProfile, "Summarize: {{input \"transcript\"}}")
}
func writePromptDefinition(t *testing.T, dir, name, definition string) {
t.Helper()
if err := os.WriteFile(filepath.Join(dir, name), []byte(definition), 0o644); err != nil {
t.Fatalf("write prompt definition: %v", err)
}
}
func writePromptFileWithTemplate(t *testing.T, dir, id, defaultProfile, templateContent string) {
t.Helper()
data := fmt.Sprintf(`id: %s

View File

@@ -14,8 +14,10 @@ import (
)
const (
scriptoriumModulePath = "gitea.maximumdirect.net/eric/scriptorium"
promptkitInternalPath = "gitea.maximumdirect.net/eric/promptkit/internal"
scriptoriumModulePath = "gitea.maximumdirect.net/eric/scriptorium"
promptkitInternalPath = "gitea.maximumdirect.net/eric/promptkit/internal"
promptkitOpenRouterCatalogPath = "gitea.maximumdirect.net/eric/promptkit-backend-openrouter"
promptkitRakestrawhomeCatalogPath = "gitea.maximumdirect.net/eric/promptkit-backend-rakestrawhome"
)
var (
@@ -120,6 +122,24 @@ func TestForbiddenImportScannerDetectsPromptkitInternalPackages(t *testing.T) {
}
}
func TestForbiddenImportScannerDetectsPromptkitCatalogPackages(t *testing.T) {
for _, importPath := range []string{
promptkitOpenRouterCatalogPath,
promptkitRakestrawhomeCatalogPath,
} {
t.Run(importPath, func(t *testing.T) {
root := t.TempDir()
sourcePath := writeGoSource(t, root, "nested/consumer/catalog.go", importPath)
violations, err := findForbiddenProductionImports(root)
if err != nil {
t.Fatalf("scan source fixture: %v", err)
}
assertSingleViolation(t, violations, sourcePath, importPath)
})
}
}
func TestForbiddenImportScannerAllowsRetainedApplicationPackages(t *testing.T) {
root := t.TempDir()
sourcePath := filepath.Join(root, "nested/consumer/application.go")
@@ -201,6 +221,12 @@ func isForbiddenProductionImport(importPath string) bool {
if importPath == promptkitInternalPath || strings.HasPrefix(importPath, promptkitInternalPath+"/") {
return true
}
if importPath == promptkitOpenRouterCatalogPath || strings.HasPrefix(importPath, promptkitOpenRouterCatalogPath+"/") {
return true
}
if importPath == promptkitRakestrawhomeCatalogPath || strings.HasPrefix(importPath, promptkitRakestrawhomeCatalogPath+"/") {
return true
}
for _, root := range removedFrameworkPackageRoots {
if importPath == root || strings.HasPrefix(importPath, root+"/") {
return true

View File

@@ -8,6 +8,7 @@ type runRequestDTO struct {
PromptID string `json:"prompt_id"`
PromptVersion string `json:"prompt_version,omitempty"`
ProfileID string `json:"profile_id,omitempty"`
SessionID string `json:"session_id,omitempty"`
Inputs map[string]inputRefDTO `json:"inputs"`
Vars map[string]string `json:"vars,omitempty"`
Model *modelOverrideRequestDTO `json:"model,omitempty"`
@@ -28,7 +29,7 @@ type modelOverrideRequestDTO struct {
TopP *float64 `json:"top_p,omitempty"`
TimeoutSeconds *int `json:"timeout_seconds,omitempty"`
ServiceTier string `json:"service_tier,omitempty"`
ReasoningEffort string `json:"reasoning_effort,omitempty"`
ReasoningEffort *string `json:"reasoning_effort,omitempty"`
APIKeyEnv string `json:"api_key_env,omitempty"`
ExtraParams map[string]any `json:"extra_params,omitempty"`
}
@@ -56,6 +57,8 @@ type metadataDTO struct {
PromptHash string `json:"prompt_hash"`
RenderedPromptHash string `json:"rendered_prompt_hash"`
SelectedProfileID string `json:"selected_profile_id"`
SelectedBackendID string `json:"selected_backend_id,omitempty"`
SessionID string `json:"session_id,omitempty"`
ModelName string `json:"model_name"`
Endpoint string `json:"endpoint"`
ModelParams modelParamsDTO `json:"model_params"`
@@ -71,6 +74,7 @@ type metadataDTO struct {
type modelParamsDTO struct {
Endpoint string `json:"endpoint"`
BackendID string `json:"backend_id,omitempty"`
Model string `json:"model"`
Temperature float64 `json:"temperature"`
MaxTokens int `json:"max_tokens"`

View File

@@ -76,17 +76,15 @@ func (h *Handler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
writeError(w, http.StatusBadRequest, "invalid_request", "prompt_id is required")
return
}
if len(req.Inputs) == 0 {
writeError(w, http.StatusBadRequest, "invalid_request", "inputs is required")
return
}
mappedInputs := make(map[string]promptkit.ArtifactRef, len(req.Inputs))
for name, in := range req.Inputs {
mappedInputs[name] = promptkit.ArtifactRef{
Type: promptkit.ArtifactRefType(in.Type),
URI: in.URI,
Body: in.Body,
var mappedInputs map[string]promptkit.ArtifactRef
if len(req.Inputs) > 0 {
mappedInputs = make(map[string]promptkit.ArtifactRef, len(req.Inputs))
for name, in := range req.Inputs {
mappedInputs[name] = promptkit.ArtifactRef{
Type: promptkit.ArtifactRefType(in.Type),
URI: in.URI,
Body: in.Body,
}
}
}
@@ -99,6 +97,7 @@ func (h *Handler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
PromptID: req.PromptID,
PromptVersion: req.PromptVersion,
ProfileID: req.ProfileID,
SessionID: req.SessionID,
Inputs: mappedInputs,
Vars: req.Vars,
Execution: model,
@@ -126,6 +125,8 @@ func (h *Handler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
PromptHash: res.PromptHash,
RenderedPromptHash: res.RenderedPromptHash,
SelectedProfileID: res.SelectedProfileID,
SelectedBackendID: res.SelectedBackendID,
SessionID: res.SessionID,
ModelName: res.ModelName,
Endpoint: res.Endpoint,
ModelParams: modelParamsDTOFromExecutionTarget(res.EffectiveModelParams),
@@ -173,6 +174,7 @@ func executionTargetOverrideFromModelOverrideDTO(dto *modelOverrideRequestDTO) *
func modelParamsDTOFromExecutionTarget(target promptkit.ExecutionTarget) modelParamsDTO {
return modelParamsDTO{
Endpoint: target.Endpoint,
BackendID: target.BackendID,
Model: target.Model,
Temperature: target.Temperature,
MaxTokens: target.MaxTokens,
@@ -220,6 +222,8 @@ func mapRunError(err error) (int, string, string) {
return http.StatusBadRequest, "artifact_read_failed", "failed to read input artifact"
case errors.Is(err, promptkit.ErrPromptRender):
return http.StatusBadRequest, "prompt_render_failed", "failed to render prompt"
case errors.Is(err, promptkit.ErrCapacityExceeded):
return http.StatusServiceUnavailable, "capacity_exceeded", "model backend capacity is exhausted"
case errors.Is(err, promptkit.ErrLLMGenerate):
return http.StatusBadGateway, "llm_failed", "model generation request failed"
case errors.Is(err, promptkit.ErrValidation):

View File

@@ -11,6 +11,7 @@ import (
"path/filepath"
"reflect"
"strings"
"sync/atomic"
"testing"
"time"
@@ -69,6 +70,34 @@ func (handlerLLMClient) Generate(ctx context.Context, req promptkit.GenerateRequ
return &promptkit.GenerateResponse{Content: "ok"}, nil
}
type blockingLLMClient struct {
started chan struct{}
release chan struct{}
current int32
peak int32
}
func (c *blockingLLMClient) Generate(ctx context.Context, req promptkit.GenerateRequest) (*promptkit.GenerateResponse, error) {
current := atomic.AddInt32(&c.current, 1)
defer atomic.AddInt32(&c.current, -1)
for {
peak := atomic.LoadInt32(&c.peak)
if current <= peak || atomic.CompareAndSwapInt32(&c.peak, peak, current) {
break
}
}
select {
case c.started <- struct{}{}:
default:
}
select {
case <-c.release:
return &promptkit.GenerateResponse{Content: "ok"}, nil
case <-ctx.Done():
return nil, ctx.Err()
}
}
func TestHandlerPostRunsSuccessWithExplicitProfileID(t *testing.T) {
start := time.Now().UTC()
end := start.Add(2 * time.Second)
@@ -90,9 +119,11 @@ func TestHandlerPostRunsSuccessWithExplicitProfileID(t *testing.T) {
PromptHash: "phash",
RenderedPromptHash: "rhash",
SelectedProfileID: "exec-default",
SelectedBackendID: "local",
ModelName: "m1",
Endpoint: "http://llm/v1",
EffectiveModelParams: promptkit.ExecutionTarget{
BackendID: "local",
Endpoint: "http://llm/v1",
Model: "m1",
Temperature: 0.2,
@@ -151,6 +182,9 @@ func TestHandlerPostRunsSuccessWithExplicitProfileID(t *testing.T) {
if metadata["selected_profile_id"] != "exec-default" {
t.Fatalf("unexpected metadata.selected_profile_id: %#v", metadata["selected_profile_id"])
}
if metadata["selected_backend_id"] != "local" {
t.Fatalf("unexpected metadata.selected_backend_id: %#v", metadata["selected_backend_id"])
}
if metadata["model_name"] != "m1" || metadata["endpoint"] != "http://llm/v1" {
t.Fatalf("unexpected model metadata: name=%#v endpoint=%#v", metadata["model_name"], metadata["endpoint"])
}
@@ -162,6 +196,9 @@ func TestHandlerPostRunsSuccessWithExplicitProfileID(t *testing.T) {
t.Fatalf("unexpected cache usage metadata: %#v", usage)
}
modelParams := metadata["model_params"].(map[string]any)
if modelParams["backend_id"] != "local" {
t.Fatalf("expected model_params.backend_id=local, got %#v", modelParams["backend_id"])
}
if modelParams["api_key_env"] != envName {
t.Fatalf("expected model_params.api_key_env=%q, got %#v", envName, modelParams["api_key_env"])
}
@@ -192,6 +229,124 @@ func TestHandlerPostRunsSuccessWithExplicitProfileID(t *testing.T) {
}
}
func TestHandlerAllowsOmittedInputsAndMapsPromptVersion(t *testing.T) {
r := &fakeRunner{result: &promptkit.RunResult{
Artifact: promptkit.Artifact{Body: []byte("ok")},
Validation: promptkit.ValidationResult{Status: promptkit.ValidationPassed, Mode: promptkit.ValidationNone, IsValid: true},
EffectiveModelParams: promptkit.ExecutionTarget{Endpoint: "http://llm/v1", Model: "m1"},
}}
h := NewHandler(r)
for _, body := range []string{
`{"prompt_id":"prompt-1","prompt_version":"2"}`,
`{"prompt_id":"prompt-1","prompt_version":"2","inputs":{}}`,
} {
req := httptest.NewRequest(http.MethodPost, "/v1/runs", bytes.NewBufferString(body))
w := httptest.NewRecorder()
h.ServeHTTP(w, req)
if w.Code != http.StatusOK {
t.Fatalf("expected 200, got %d body=%s", w.Code, w.Body.String())
}
if r.last.PromptVersion != "2" {
t.Fatalf("expected prompt version to be mapped, got %q", r.last.PromptVersion)
}
if r.last.Inputs != nil {
t.Fatalf("expected omitted or empty inputs to remain nil, got %#v", r.last.Inputs)
}
}
}
func TestHandlerDelegatesDefinitionInputRequirements(t *testing.T) {
tests := []struct {
name string
definition string
wantStatus int
wantCode string
}{
{
name: "no declared inputs",
definition: `id: p
version: "1"
default_profile: exec
messages:
- role: user
content: "hello"
output:
format: text
validation_mode: none
`,
wantStatus: http.StatusOK,
},
{
name: "optional input omitted",
definition: `id: p
version: "1"
default_profile: exec
inputs:
- name: note
required: false
messages:
- role: user
content: "hello"
output:
format: text
validation_mode: none
`,
wantStatus: http.StatusOK,
},
{
name: "required input omitted",
definition: `id: p
version: "1"
default_profile: exec
inputs:
- name: note
required: true
messages:
- role: user
content: "hello"
output:
format: text
validation_mode: none
`,
wantStatus: http.StatusBadRequest,
wantCode: "prompt_render_failed",
},
{
name: "template input omitted",
definition: `id: p
version: "1"
default_profile: exec
messages:
- role: user
content: '{{input "note"}}'
output:
format: text
validation_mode: none
`,
wantStatus: http.StatusBadRequest,
wantCode: "prompt_render_failed",
},
}
for _, tc := range tests {
t.Run(tc.name, func(t *testing.T) {
h := newDefinitionHandler(t, tc.definition)
req := httptest.NewRequest(http.MethodPost, "/v1/runs", bytes.NewBufferString(`{"prompt_id":"p"}`))
w := httptest.NewRecorder()
h.ServeHTTP(w, req)
if w.Code != tc.wantStatus {
t.Fatalf("expected %d, got %d body=%s", tc.wantStatus, w.Code, w.Body.String())
}
if tc.wantCode != "" {
assertHTTPErrorCode(t, w, tc.wantStatus, tc.wantCode)
}
})
}
}
func TestHandlerInlineRefsWorkWithoutArtifactRoot(t *testing.T) {
h := newArtifactRootHandler(t, "")
@@ -365,10 +520,12 @@ func TestHandlerModelOverrideMapsAllSupportedExecutionFields(t *testing.T) {
if got.Endpoint != "http://override/v1" ||
got.Model != "override-model" ||
got.ServiceTier != "flex" ||
got.ReasoningEffort != "medium" ||
got.APIKeyEnv != "SCRIPTORIUM_API_KEY" {
t.Fatalf("unexpected mapped execution target: %+v", got)
}
if got.ReasoningEffort == nil || *got.ReasoningEffort != "medium" {
t.Fatalf("unexpected mapped reasoning_effort: %#v", got.ReasoningEffort)
}
if got.Temperature == nil || *got.Temperature != 0.6 {
t.Fatalf("unexpected mapped temperature: %#v", got.Temperature)
}
@@ -386,6 +543,251 @@ func TestHandlerModelOverrideMapsAllSupportedExecutionFields(t *testing.T) {
}
}
func TestHandlerModelOverridePreservesReasoningEffortPresence(t *testing.T) {
tests := []struct {
name string
model string
wantPresent bool
wantValue string
}{
{name: "omitted", model: `{}`},
{name: "nonblank", model: `{"reasoning_effort":"high"}`, wantPresent: true, wantValue: "high"},
{name: "explicit empty", model: `{"reasoning_effort":""}`, wantPresent: true},
{name: "null", model: `{"reasoning_effort":null}`},
}
for _, tc := range tests {
t.Run(tc.name, func(t *testing.T) {
r := &fakeRunner{result: &promptkit.RunResult{
Artifact: promptkit.Artifact{Body: []byte("ok")},
Validation: promptkit.ValidationResult{Status: promptkit.ValidationPassed, Mode: promptkit.ValidationBasic, IsValid: true},
EffectiveModelParams: promptkit.ExecutionTarget{Endpoint: "http://llm/v1", Model: "m1"},
}}
h := NewHandler(r)
req := httptest.NewRequest(http.MethodPost, "/v1/runs", bytes.NewBufferString(`{"prompt_id":"prompt-1","inputs":{"transcript":{"type":"file","uri":"./t.md"}},"model":`+tc.model+`}`))
w := httptest.NewRecorder()
h.ServeHTTP(w, req)
if w.Code != http.StatusOK {
t.Fatalf("expected 200, got %d body=%s", w.Code, w.Body.String())
}
if r.last.Execution == nil {
t.Fatal("expected execution override")
}
if (r.last.Execution.ReasoningEffort != nil) != tc.wantPresent {
t.Fatalf("unexpected reasoning_effort presence: %#v", r.last.Execution.ReasoningEffort)
}
if tc.wantPresent && *r.last.Execution.ReasoningEffort != tc.wantValue {
t.Fatalf("unexpected reasoning_effort: got %q want %q", *r.last.Execution.ReasoningEffort, tc.wantValue)
}
})
}
}
func TestHandlerMapsSessionIDAndReportsEffectiveResultSessionID(t *testing.T) {
r := &fakeRunner{result: &promptkit.RunResult{
Artifact: promptkit.Artifact{Body: []byte("ok")},
Validation: promptkit.ValidationResult{Status: promptkit.ValidationPassed, Mode: promptkit.ValidationBasic, IsValid: true},
EffectiveModelParams: promptkit.ExecutionTarget{Endpoint: "http://llm/v1", Model: "m1"},
SessionID: "effective-session",
}}
h := NewHandler(r)
req := httptest.NewRequest(http.MethodPost, "/v1/runs", bytes.NewBufferString(`{"prompt_id":"prompt-1","session_id":"request-session"}`))
w := httptest.NewRecorder()
h.ServeHTTP(w, req)
if w.Code != http.StatusOK {
t.Fatalf("expected 200, got %d body=%s", w.Code, w.Body.String())
}
if r.last.SessionID != "request-session" {
t.Fatalf("expected session ID in run request, got %q", r.last.SessionID)
}
var resp map[string]any
if err := json.Unmarshal(w.Body.Bytes(), &resp); err != nil {
t.Fatalf("invalid JSON response: %v", err)
}
metadata := resp["metadata"].(map[string]any)
if metadata["session_id"] != "effective-session" {
t.Fatalf("expected effective session ID in response, got %#v", metadata["session_id"])
}
}
func TestHandlerOmitsEmptyEffectiveSessionID(t *testing.T) {
r := &fakeRunner{result: &promptkit.RunResult{
Artifact: promptkit.Artifact{Body: []byte("ok")},
Validation: promptkit.ValidationResult{Status: promptkit.ValidationPassed, Mode: promptkit.ValidationBasic, IsValid: true},
EffectiveModelParams: promptkit.ExecutionTarget{Endpoint: "http://llm/v1", Model: "m1"},
}}
h := NewHandler(r)
req := httptest.NewRequest(http.MethodPost, "/v1/runs", bytes.NewBufferString(`{"prompt_id":"prompt-1"}`))
w := httptest.NewRecorder()
h.ServeHTTP(w, req)
if w.Code != http.StatusOK {
t.Fatalf("expected 200, got %d body=%s", w.Code, w.Body.String())
}
var resp map[string]any
if err := json.Unmarshal(w.Body.Bytes(), &resp); err != nil {
t.Fatalf("invalid JSON response: %v", err)
}
metadata := resp["metadata"].(map[string]any)
if _, ok := metadata["session_id"]; ok {
t.Fatalf("expected empty effective session ID to be omitted, got %#v", metadata)
}
}
func TestHandlerSessionIDUsesPromptkitResolution(t *testing.T) {
for _, tc := range []struct {
name string
definitionSession string
requestSession string
wantSession string
}{
{name: "definition session", definitionSession: "definition-session", wantSession: "definition-session"},
{name: "direct session", definitionSession: "definition-session", requestSession: "direct-session", wantSession: "direct-session"},
{name: "no effective session"},
} {
t.Run(tc.name, func(t *testing.T) {
promptDir := t.TempDir()
profileDir := t.TempDir()
sessionLine := ""
if tc.definitionSession != "" {
sessionLine = "session_id: " + tc.definitionSession + "\n"
}
if err := os.WriteFile(filepath.Join(promptDir, "prompt.yaml"), []byte("id: p\nversion: \"1\"\ndefault_profile: exec\n"+sessionLine+`messages:
- role: user
content: "hi"
output:
format: text
validation_mode: none
`), 0o644); err != nil {
t.Fatalf("write prompt fixture: %v", err)
}
if err := os.WriteFile(filepath.Join(profileDir, "profile.yaml"), []byte("id: exec\nendpoint: http://127.0.0.1:1/v1\nmodel: test\n"), 0o644); err != nil {
t.Fatalf("write profile fixture: %v", err)
}
engine, err := promptkit.NewEngine(promptkit.Config{PromptDir: promptDir, ProfileDir: profileDir}, promptkit.WithLLMClient(handlerLLMClient{}))
if err != nil {
t.Fatalf("new engine: %v", err)
}
body := `{"prompt_id":"p"}`
if tc.requestSession != "" {
body = `{"prompt_id":"p","session_id":"` + tc.requestSession + `"}`
}
w := httptest.NewRecorder()
NewHandler(engine).ServeHTTP(w, httptest.NewRequest(http.MethodPost, "/v1/runs", bytes.NewBufferString(body)))
if w.Code != http.StatusOK {
t.Fatalf("expected 200, got %d body=%s", w.Code, w.Body.String())
}
var resp map[string]any
if err := json.Unmarshal(w.Body.Bytes(), &resp); err != nil {
t.Fatalf("invalid JSON response: %v", err)
}
metadata := resp["metadata"].(map[string]any)
if tc.wantSession == "" {
if _, ok := metadata["session_id"]; ok {
t.Fatalf("expected session_id to be omitted, got %#v", metadata)
}
return
}
if metadata["session_id"] != tc.wantSession {
t.Fatalf("expected effective session ID %q, got %#v", tc.wantSession, metadata["session_id"])
}
})
}
}
func TestHandlerSharesBackendCapacityAcrossConcurrentRequests(t *testing.T) {
promptDir := t.TempDir()
profileDir := t.TempDir()
if err := os.WriteFile(filepath.Join(promptDir, "prompt.yaml"), []byte(`id: p
version: "1"
default_profile: limited
messages:
- role: user
content: "hi"
output:
format: text
validation_mode: none
`), 0o644); err != nil {
t.Fatalf("write prompt fixture: %v", err)
}
if err := os.WriteFile(filepath.Join(profileDir, "profile.yaml"), []byte(`id: limited
backend: limited
model: test
`), 0o644); err != nil {
t.Fatalf("write profile fixture: %v", err)
}
queueCapacity := 0
client := &blockingLLMClient{started: make(chan struct{}, 1), release: make(chan struct{})}
engine, err := promptkit.NewEngine(
promptkit.Config{PromptDir: promptDir, ProfileDir: profileDir},
promptkit.WithBackend(promptkit.Backend{
ID: "limited",
Endpoint: "http://127.0.0.1:1/v1",
ConcurrencyLimit: 1,
QueueCapacity: &queueCapacity,
}),
promptkit.WithLLMClient(client),
)
if err != nil {
t.Fatalf("new engine: %v", err)
}
h := NewHandler(engine)
first := httptest.NewRecorder()
firstDone := make(chan struct{})
go func() {
defer close(firstDone)
h.ServeHTTP(first, httptest.NewRequest(http.MethodPost, "/v1/runs", bytes.NewBufferString(`{"prompt_id":"p"}`)))
}()
select {
case <-client.started:
case <-time.After(time.Second):
t.Fatal("first request did not reach generation")
}
second := httptest.NewRecorder()
h.ServeHTTP(second, httptest.NewRequest(http.MethodPost, "/v1/runs", bytes.NewBufferString(`{"prompt_id":"p"}`)))
assertHTTPErrorCode(t, second, http.StatusServiceUnavailable, "capacity_exceeded")
if second.Header().Get("Retry-After") != "" {
t.Fatalf("expected no Retry-After header, got %q", second.Header().Get("Retry-After"))
}
if strings.Contains(second.Body.String(), "limited") {
t.Fatalf("capacity response leaked backend details: %s", second.Body.String())
}
close(client.release)
select {
case <-firstDone:
case <-time.After(time.Second):
t.Fatal("first request did not complete")
}
if first.Code != http.StatusOK {
t.Fatalf("expected first request to succeed, got %d body=%s", first.Code, first.Body.String())
}
if atomic.LoadInt32(&client.peak) != 1 {
t.Fatalf("expected peak generation concurrency of one, got %d", atomic.LoadInt32(&client.peak))
}
}
func TestHandlerRejectsOverlongSessionIDAndNonStringReasoningEffort(t *testing.T) {
engine := newHandlerEngine(t)
for _, body := range []string{
`{"prompt_id":"p","session_id":"` + strings.Repeat("x", 257) + `"}`,
`{"prompt_id":"p","model":{"reasoning_effort":1}}`,
} {
w := httptest.NewRecorder()
NewHandler(engine).ServeHTTP(w, httptest.NewRequest(http.MethodPost, "/v1/runs", bytes.NewBufferString(body)))
if w.Code != http.StatusBadRequest {
t.Fatalf("expected 400, got %d body=%s", w.Code, w.Body.String())
}
}
}
func TestHandlerModelOverrideAcceptsJSONCompatibleExtraParams(t *testing.T) {
r := &fakeRunner{result: &promptkit.RunResult{
Artifact: promptkit.Artifact{Body: []byte("ok")},
@@ -722,6 +1124,7 @@ func TestHandlerPublicErrorMapping(t *testing.T) {
{name: "file too large", err: ErrFileTooLarge, status: http.StatusRequestEntityTooLarge, code: "artifact_too_large", message: "file input artifact is too large"},
{name: "artifact", err: wrap(promptkit.ErrArtifactLoad, fmt.Errorf("read failed")), status: http.StatusBadRequest, code: "artifact_read_failed", message: "failed to read input artifact", avoidCause: "read failed"},
{name: "prompt render", err: wrap(promptkit.ErrPromptRender, fmt.Errorf("render failed")), status: http.StatusBadRequest, code: "prompt_render_failed", message: "failed to render prompt", avoidCause: "render failed"},
{name: "capacity", err: &promptkit.CapacityError{BackendID: "private-backend"}, status: http.StatusServiceUnavailable, code: "capacity_exceeded", message: "model backend capacity is exhausted", avoidCause: "private-backend"},
{name: "llm", err: wrap(promptkit.ErrLLMGenerate, fmt.Errorf("llm failed")), status: http.StatusBadGateway, code: "llm_failed", message: "model generation request failed", avoidCause: "llm failed"},
{name: "validation runtime", err: wrap(promptkit.ErrValidation, fmt.Errorf("validator broke")), status: http.StatusInternalServerError, code: "validation_runtime_failed", message: "validation runtime failed", avoidCause: "validator broke"},
}
@@ -889,6 +1292,31 @@ model: model
return engine
}
func newDefinitionHandler(t *testing.T, definition string) *Handler {
t.Helper()
promptDir := t.TempDir()
profileDir := t.TempDir()
if err := os.WriteFile(filepath.Join(promptDir, "prompt.yaml"), []byte(definition), 0o644); err != nil {
t.Fatalf("write prompt fixture: %v", err)
}
if err := os.WriteFile(filepath.Join(profileDir, "profile.yaml"), []byte(`id: exec
endpoint: http://example.invalid/v1
model: model
`), 0o644); err != nil {
t.Fatalf("write profile fixture: %v", err)
}
engine, err := promptkit.NewEngine(promptkit.Config{
PromptDir: promptDir,
ProfileDir: profileDir,
}, promptkit.WithLLMClient(handlerLLMClient{}))
if err != nil {
t.Fatalf("construct public engine: %v", err)
}
return NewHandler(engine)
}
func assertHTTPErrorCode(t *testing.T, w *httptest.ResponseRecorder, status int, code string) {
t.Helper()

View File

@@ -7,6 +7,7 @@ import (
"io"
"os"
"path/filepath"
"sort"
"strings"
"gitea.maximumdirect.net/eric/scriptorium/internal/defaults"
@@ -32,11 +33,29 @@ var (
// Config is the on-disk YAML shape for application-level settings.
type Config struct {
PromptDir string `yaml:"prompt_dir"`
ProfileDir string `yaml:"profile_dir"`
SchemaDir string `yaml:"schema_dir"`
Server ServerConfig `yaml:"server"`
Defaults DefaultsConfig `yaml:"defaults"`
PromptDir string `yaml:"prompt_dir"`
ProfileDir string `yaml:"profile_dir"`
SchemaDir string `yaml:"schema_dir"`
Server ServerConfig `yaml:"server"`
Defaults DefaultsConfig `yaml:"defaults"`
Backends map[string]BackendConfig `yaml:"backends"`
}
type BackendConfig struct {
Endpoint string `yaml:"endpoint"`
APIKeyEnv string `yaml:"api_key_env"`
ExtraParams map[string]any `yaml:"extra_params"`
ConcurrencyLimit int `yaml:"concurrency_limit"`
QueueCapacity *int `yaml:"queue_capacity"`
}
type BackendSettings struct {
ID string
Endpoint string
APIKeyEnv string
ExtraParams map[string]any
ConcurrencyLimit int
QueueCapacity *int
}
type ServerConfig struct {
@@ -62,6 +81,7 @@ type AppSettings struct {
MaxArtifactBytes int64
MaxResponseBytes int64
DefaultRenderFormat renderformat.PreparedRunOutputFormat
Backends []BackendSettings
}
// CLIOverrides can be applied after config load to enforce precedence.
@@ -245,6 +265,25 @@ func applyConfig(base AppSettings, cfg Config) (AppSettings, error) {
}
out.DefaultRenderFormat = parsed
}
if len(cfg.Backends) > 0 {
ids := make([]string, 0, len(cfg.Backends))
for id := range cfg.Backends {
ids = append(ids, id)
}
sort.Strings(ids)
out.Backends = make([]BackendSettings, 0, len(ids))
for _, id := range ids {
backend := cfg.Backends[id]
out.Backends = append(out.Backends, BackendSettings{
ID: id,
Endpoint: backend.Endpoint,
APIKeyEnv: backend.APIKeyEnv,
ExtraParams: backend.ExtraParams,
ConcurrencyLimit: backend.ConcurrencyLimit,
QueueCapacity: backend.QueueCapacity,
})
}
}
return out, nil
}

View File

@@ -4,6 +4,7 @@ import (
"errors"
"os"
"path/filepath"
"reflect"
"testing"
"gitea.maximumdirect.net/eric/scriptorium/internal/defaults"
@@ -20,7 +21,7 @@ func TestLoadConfigMissingImplicitPathUsesBuiltInDefaults(t *testing.T) {
}
want := BuiltInDefaults()
if got != want {
if !reflect.DeepEqual(got, want) {
t.Fatalf("unexpected settings: got=%+v want=%+v", got, want)
}
}
@@ -100,6 +101,78 @@ func TestLoadConfigAPIKeyFieldIsRejectedAsUnknown(t *testing.T) {
}
}
func TestLoadConfigBackendsRetainsResolvedSettingsInSortedOrder(t *testing.T) {
path := writeConfigFile(t, "config.yml", `
backends:
zebra:
endpoint: https://zebra.example/v1
api_key_env: ZEBRA_API_KEY
extra_params:
provider_option: enabled
nested:
enabled: true
attempts: 2
concurrency_limit: 2
alpha:
endpoint: http://alpha.example/v1
queue_capacity: 0
`)
got, err := LoadConfig(path, true)
if err != nil {
t.Fatalf("expected no error, got %v", err)
}
if len(got.Backends) != 2 {
t.Fatalf("expected two backends, got %#v", got.Backends)
}
if got.Backends[0].ID != "alpha" || got.Backends[1].ID != "zebra" {
t.Fatalf("expected sorted backend IDs, got %#v", got.Backends)
}
if got.Backends[0].QueueCapacity == nil || *got.Backends[0].QueueCapacity != 0 {
t.Fatalf("expected explicit zero queue capacity, got %#v", got.Backends[0].QueueCapacity)
}
if got.Backends[1].QueueCapacity != nil {
t.Fatalf("expected omitted queue capacity to remain nil, got %#v", got.Backends[1].QueueCapacity)
}
wantParams := map[string]any{
"provider_option": "enabled",
"nested": map[string]any{
"enabled": true,
"attempts": 2,
},
}
if !reflect.DeepEqual(got.Backends[1].ExtraParams, wantParams) {
t.Fatalf("unexpected extra params: got=%#v want=%#v", got.Backends[1].ExtraParams, wantParams)
}
if got.Backends[1].Endpoint != "https://zebra.example/v1" || got.Backends[1].APIKeyEnv != "ZEBRA_API_KEY" || got.Backends[1].ConcurrencyLimit != 2 {
t.Fatalf("unexpected zebra backend: %#v", got.Backends[1])
}
}
func TestLoadConfigRejectsUnknownOrSecretBackendFields(t *testing.T) {
for name, body := range map[string]string{
"unknown": "backends:\n local:\n endpoint: http://localhost:11434/v1\n unexpected: value\n",
"secret": "backends:\n local:\n endpoint: http://localhost:11434/v1\n api_key: secret\n",
} {
t.Run(name, func(t *testing.T) {
path := writeConfigFile(t, "config.yml", body)
_, err := LoadConfig(path, true)
if !errors.Is(err, ErrInvalidConfigYAML) {
t.Fatalf("expected ErrInvalidConfigYAML, got %v", err)
}
})
}
}
func TestLoadConfigRejectsInvalidBackendFieldTypes(t *testing.T) {
path := writeConfigFile(t, "config.yml", "backends:\n local:\n endpoint: http://localhost:11434/v1\n queue_capacity: not-a-number\n")
_, err := LoadConfig(path, true)
if !errors.Is(err, ErrInvalidConfigYAML) {
t.Fatalf("expected ErrInvalidConfigYAML, got %v", err)
}
}
func TestLoadConfigValidConfigSetsDirectoriesAndServerAddr(t *testing.T) {
path := writeConfigFile(t, "config.yml", `
prompt_dir: ./prompts
@@ -196,7 +269,7 @@ func TestLoadConfigEmptyFileResolvesToBuiltInDefaults(t *testing.T) {
}
want := BuiltInDefaults()
if got != want {
if !reflect.DeepEqual(got, want) {
t.Fatalf("unexpected settings: got=%+v want=%+v", got, want)
}
}

View File

@@ -0,0 +1,122 @@
package format
import (
"bytes"
"encoding/json"
"errors"
"fmt"
"strconv"
"gitea.maximumdirect.net/eric/promptkit"
)
type PromptInspection struct {
PromptID string `json:"prompt_id"`
PromptVersion string `json:"prompt_version"`
PromptHash string `json:"prompt_hash"`
DefaultProfileID string `json:"default_profile_id"`
Inputs []PromptInspectionInput `json:"inputs"`
OutputContract OutputContract `json:"output_contract"`
}
type PromptInspectionInput struct {
Name string `json:"name"`
Required bool `json:"required"`
ContentType string `json:"content_type"`
Description string `json:"description"`
}
type OutputContract struct {
Format string `json:"format"`
ValidationMode string `json:"validation_mode"`
SchemaPath string `json:"schema_path"`
RepairAttempts int `json:"repair_attempts"`
}
type ProfileInspection struct {
ProfileID string `json:"profile_id"`
EffectiveModelParams ProfileModelParams `json:"effective_model_params"`
APIKeyRequired bool `json:"api_key_required"`
}
type ProfileModelParams struct {
BackendID string `json:"backend_id"`
Endpoint string `json:"endpoint"`
Model string `json:"model"`
Temperature float64 `json:"temperature"`
MaxTokens int `json:"max_tokens"`
TopP float64 `json:"top_p"`
TimeoutSeconds int `json:"timeout_seconds"`
ServiceTier string `json:"service_tier"`
ReasoningEffort string `json:"reasoning_effort"`
APIKeyEnv string `json:"api_key_env"`
ExtraParams map[string]any `json:"extra_params"`
}
func FormatPromptInspection(value *promptkit.PromptInspection, outputFormat OutputFormat) ([]byte, error) {
if value == nil {
return nil, errors.New("prompt inspection is nil")
}
dto := PromptInspection{
PromptID: value.PromptID, PromptVersion: value.PromptVersion, PromptHash: value.PromptHash, DefaultProfileID: value.DefaultProfileID,
Inputs: make([]PromptInspectionInput, len(value.Inputs)),
OutputContract: OutputContract{Format: string(value.OutputContract.Format), ValidationMode: string(value.OutputContract.ValidationMode), SchemaPath: value.OutputContract.SchemaPath, RepairAttempts: value.OutputContract.RepairAttempts},
}
for i, input := range value.Inputs {
dto.Inputs[i] = PromptInspectionInput{Name: input.Name, Required: input.Required, ContentType: input.ContentType, Description: input.Description}
}
switch outputFormat {
case OutputFormatJSON:
data, err := json.MarshalIndent(dto, "", " ")
if err != nil {
return nil, err
}
return append(data, '\n'), nil
case OutputFormatText:
var b bytes.Buffer
fmt.Fprintf(&b, "prompt_id: %s\nprompt_version: %s\nprompt_hash: %s\ndefault_profile_id: %s\ninputs:", dto.PromptID, dto.PromptVersion, dto.PromptHash, dto.DefaultProfileID)
if len(dto.Inputs) == 0 {
fmt.Fprintln(&b, " []")
} else {
fmt.Fprintln(&b)
for _, input := range dto.Inputs {
fmt.Fprintf(&b, " - name: %s\n required: %t\n content_type: %s\n description: %s\n", input.Name, input.Required, input.ContentType, strconv.Quote(input.Description))
}
}
fmt.Fprintf(&b, "output_contract:\n format: %s\n validation_mode: %s\n schema_path: %s\n repair_attempts: %d\n", dto.OutputContract.Format, dto.OutputContract.ValidationMode, dto.OutputContract.SchemaPath, dto.OutputContract.RepairAttempts)
return b.Bytes(), nil
default:
return nil, fmt.Errorf("%w: %q", ErrUnknownPreparedRunFormat, outputFormat)
}
}
func FormatProfileInspection(value *promptkit.ProfileInspection, outputFormat OutputFormat) ([]byte, error) {
if value == nil {
return nil, errors.New("profile inspection is nil")
}
target := value.EffectiveModelParams
params := map[string]any{}
for key, item := range target.ExtraParams {
params[key] = item
}
dto := ProfileInspection{ProfileID: value.ProfileID, APIKeyRequired: value.APIKeyRequired, EffectiveModelParams: ProfileModelParams{BackendID: target.BackendID, Endpoint: target.Endpoint, Model: target.Model, Temperature: target.Temperature, MaxTokens: target.MaxTokens, TopP: target.TopP, TimeoutSeconds: target.TimeoutSeconds, ServiceTier: target.ServiceTier, ReasoningEffort: target.ReasoningEffort, APIKeyEnv: target.APIKeyEnv, ExtraParams: params}}
switch outputFormat {
case OutputFormatJSON:
data, err := json.MarshalIndent(dto, "", " ")
if err != nil {
return nil, err
}
return append(data, '\n'), nil
case OutputFormatText:
var b bytes.Buffer
fmt.Fprintf(&b, "profile_id: %s\neffective_model_params:\n backend_id: %s\n endpoint: %s\n model: %s\n temperature: %g\n max_tokens: %d\n top_p: %g\n timeout_seconds: %d\n service_tier: %s\n reasoning_effort: %s\n api_key_env: %s\n extra_params: ", dto.ProfileID, dto.EffectiveModelParams.BackendID, dto.EffectiveModelParams.Endpoint, dto.EffectiveModelParams.Model, dto.EffectiveModelParams.Temperature, dto.EffectiveModelParams.MaxTokens, dto.EffectiveModelParams.TopP, dto.EffectiveModelParams.TimeoutSeconds, dto.EffectiveModelParams.ServiceTier, dto.EffectiveModelParams.ReasoningEffort, dto.EffectiveModelParams.APIKeyEnv)
paramsJSON, err := json.Marshal(dto.EffectiveModelParams.ExtraParams)
if err != nil {
return nil, err
}
fmt.Fprintf(&b, "%s\napi_key_required: %t\n", paramsJSON, dto.APIKeyRequired)
return b.Bytes(), nil
default:
return nil, fmt.Errorf("%w: %q", ErrUnknownPreparedRunFormat, outputFormat)
}
}

View File

@@ -0,0 +1,124 @@
package format
import (
"encoding/json"
"strings"
"testing"
"gitea.maximumdirect.net/eric/promptkit"
)
func TestFormatPromptInspectionPreservesDeclaredOrderAndEmptyValues(t *testing.T) {
value := &promptkit.PromptInspection{PromptID: "p", PromptVersion: "1", PromptHash: "hash", Inputs: []promptkit.PromptInputDefinition{{Name: "first", Required: true, ContentType: "text/plain", Description: "first input"}, {Name: "second"}}, OutputContract: promptkit.OutputContract{Format: "text", ValidationMode: promptkit.ValidationNone}}
text, err := FormatPromptInspection(value, OutputFormatText)
if err != nil {
t.Fatal(err)
}
if !strings.Contains(string(text), "default_profile_id: \ninputs:\n - name: first") || strings.Index(string(text), "name: first") > strings.Index(string(text), "name: second") {
t.Fatalf("unexpected text inspection: %s", text)
}
jsonOutput, err := FormatPromptInspection(value, OutputFormatJSON)
if err != nil {
t.Fatal(err)
}
if !strings.HasSuffix(string(jsonOutput), "\n") || !strings.Contains(string(jsonOutput), `"default_profile_id": ""`) {
t.Fatalf("unexpected json inspection: %s", jsonOutput)
}
}
func TestFormatPromptInspectionRejectsNil(t *testing.T) {
if _, err := FormatPromptInspection(nil, OutputFormatText); err == nil {
t.Fatal("expected nil inspection error")
}
}
func TestFormatProfileInspectionPreservesSafeEffectiveValues(t *testing.T) {
const secret = "sentinel-secret-must-not-appear"
t.Setenv("FIXTURE_PROFILE_API_KEY", secret)
value := &promptkit.ProfileInspection{
ProfileID: "custom-derived",
EffectiveModelParams: promptkit.ExecutionTarget{
BackendID: "fixture-custom",
Endpoint: "http://127.0.0.1:11434/v1",
Model: "fixture-model",
Temperature: 0.25,
MaxTokens: 640,
TopP: 0.9,
TimeoutSeconds: 45,
ServiceTier: "flex",
ReasoningEffort: "high",
APIKeyEnv: "FIXTURE_PROFILE_API_KEY",
ExtraParams: map[string]any{
"zeta": true,
"alpha": map[string]any{"nested": []any{"first", 2.0}},
},
},
APIKeyRequired: true,
}
jsonOutput, err := FormatProfileInspection(value, OutputFormatJSON)
if err != nil {
t.Fatal(err)
}
secondJSONOutput, err := FormatProfileInspection(value, OutputFormatJSON)
if err != nil {
t.Fatal(err)
}
if string(jsonOutput) != string(secondJSONOutput) || !strings.HasSuffix(string(jsonOutput), "\n") {
t.Fatalf("expected deterministic newline-terminated JSON, got %q", jsonOutput)
}
if strings.Contains(string(jsonOutput), secret) {
t.Fatalf("inspection exposed an environment secret: %s", jsonOutput)
}
var decoded ProfileInspection
if err := json.Unmarshal(jsonOutput, &decoded); err != nil {
t.Fatalf("decode profile inspection: %v", err)
}
if decoded.ProfileID != "custom-derived" || decoded.EffectiveModelParams.BackendID != "fixture-custom" {
t.Fatalf("unexpected profile identity: %+v", decoded)
}
if decoded.EffectiveModelParams.APIKeyEnv != "FIXTURE_PROFILE_API_KEY" || !decoded.APIKeyRequired {
t.Fatalf("unexpected credential metadata: %+v", decoded)
}
alpha, ok := decoded.EffectiveModelParams.ExtraParams["alpha"].(map[string]any)
if !ok || len(alpha["nested"].([]any)) != 2 {
t.Fatalf("nested extra parameters were not preserved: %#v", decoded.EffectiveModelParams.ExtraParams)
}
textOutput, err := FormatProfileInspection(value, OutputFormatText)
if err != nil {
t.Fatal(err)
}
textValue := string(textOutput)
if !strings.Contains(textValue, "backend_id: fixture-custom") ||
!strings.Contains(textValue, "api_key_env: FIXTURE_PROFILE_API_KEY") ||
!strings.Contains(textValue, `extra_params: {"alpha":{"nested":["first",2]},"zeta":true}`) ||
!strings.Contains(textValue, "api_key_required: true") {
t.Fatalf("unexpected text inspection: %s", textValue)
}
if strings.Contains(textValue, secret) {
t.Fatalf("text inspection exposed an environment secret: %s", textValue)
}
}
func TestFormatProfileInspectionPreservesEmptyBackendAndRejectsNil(t *testing.T) {
output, err := FormatProfileInspection(&promptkit.ProfileInspection{
ProfileID: "endpoint-only",
EffectiveModelParams: promptkit.ExecutionTarget{
Endpoint: "http://127.0.0.1:8000/v1",
Model: "fixture-model",
},
}, OutputFormatJSON)
if err != nil {
t.Fatal(err)
}
if !strings.Contains(string(output), `"backend_id": ""`) ||
!strings.Contains(string(output), `"extra_params": {}`) {
t.Fatalf("expected explicit empty backend and object extra params, got %s", output)
}
if _, err := FormatProfileInspection(nil, OutputFormatText); err == nil {
t.Fatal("expected nil profile inspection error")
}
}

View File

@@ -14,14 +14,24 @@ import (
var ErrUnknownPreparedRunFormat = errors.New("unknown prepared run format")
// PreparedRunOutputFormat is the output format for prepared render data.
type PreparedRunOutputFormat string
// OutputFormat selects an application-owned textual or JSON representation.
type OutputFormat string
const (
PreparedRunFormatText PreparedRunOutputFormat = "text"
PreparedRunFormatJSON PreparedRunOutputFormat = "json"
OutputFormatText OutputFormat = "text"
OutputFormatJSON OutputFormat = "json"
DefaultPreparedRunOutputFormat PreparedRunOutputFormat = PreparedRunFormatText
DefaultOutputFormat OutputFormat = OutputFormatText
)
// PreparedRunOutputFormat is the output format for prepared render data.
type PreparedRunOutputFormat = OutputFormat
const (
PreparedRunFormatText = OutputFormatText
PreparedRunFormatJSON = OutputFormatJSON
DefaultPreparedRunOutputFormat = DefaultOutputFormat
)
// PreparedRunFormatter serializes a prepared run without performing use case work.
@@ -31,15 +41,20 @@ type PreparedRunFormatter interface {
// ParsePreparedRunOutputFormat parses a format name.
func ParsePreparedRunOutputFormat(raw string) (PreparedRunOutputFormat, error) {
switch PreparedRunOutputFormat(strings.ToLower(strings.TrimSpace(raw))) {
return ParseOutputFormat(raw)
}
// ParseOutputFormat parses the shared inspection and prepared-run format names.
func ParseOutputFormat(raw string) (OutputFormat, error) {
switch OutputFormat(strings.ToLower(strings.TrimSpace(raw))) {
case "":
return DefaultPreparedRunOutputFormat, nil
case PreparedRunFormatText:
return PreparedRunFormatText, nil
case PreparedRunFormatJSON:
return PreparedRunFormatJSON, nil
return DefaultOutputFormat, nil
case OutputFormatText:
return OutputFormatText, nil
case OutputFormatJSON:
return OutputFormatJSON, nil
default:
return "", fmt.Errorf("%w: %q (supported: %s, %s)", ErrUnknownPreparedRunFormat, raw, PreparedRunFormatText, PreparedRunFormatJSON)
return "", fmt.Errorf("%w: %q (supported: %s, %s)", ErrUnknownPreparedRunFormat, raw, OutputFormatText, OutputFormatJSON)
}
}
@@ -93,6 +108,9 @@ func (textPreparedRunFormatter) Format(prepared *promptkit.PreparedRun) ([]byte,
fmt.Fprintf(&b, "prompt: %s\n", prepared.PromptID)
fmt.Fprintf(&b, "prompt_version: %s\n", prepared.PromptVersion)
fmt.Fprintf(&b, "selected_profile_id: %s\n", prepared.SelectedProfileID)
if prepared.SelectedBackendID != "" {
fmt.Fprintf(&b, "selected_backend_id: %s\n", prepared.SelectedBackendID)
}
if prepared.PromptHash != "" {
fmt.Fprintf(&b, "prompt_hash: %s\n", prepared.PromptHash)
}

View File

@@ -22,6 +22,7 @@ func TestTextFormatterIncludesPreparedRunDetails(t *testing.T) {
"prompt: prompt.id",
"prompt_version: v1",
"selected_profile_id: local-fast",
"selected_backend_id: local",
"endpoint: http://llm/v1",
"model: gpt-test",
"temperature: 0.4",
@@ -346,7 +347,9 @@ func samplePreparedRun() *promptkit.PreparedRun {
PromptVersion: "v1",
PromptHash: "prompt-hash",
SelectedProfileID: "local-fast",
SelectedBackendID: "local",
EffectiveModelParams: promptkit.ExecutionTarget{
BackendID: "local",
Endpoint: "http://llm/v1",
Model: "gpt-test",
Temperature: 0.4,

12
testdata/promptkit-v0.9/config.yml vendored Normal file
View File

@@ -0,0 +1,12 @@
prompt_dir: ./testdata/promptkit-v0.9/prompts
profile_dir: ./testdata/promptkit-v0.9/profiles
schema_dir: ./testdata/promptkit-v0.9/schemas
backends:
fixture-custom:
endpoint: http://127.0.0.1:11434/v1
api_key_env: FIXTURE_BACKEND_API_KEY
extra_params:
backend_option:
enabled: true
concurrency_limit: 2
queue_capacity: 0

View File

@@ -0,0 +1 @@
The fixture source contains stable, synthetic material.

View File

@@ -0,0 +1,16 @@
id: custom-base
backend: fixture-custom
model: fixture-base-model
temperature: 0.25
max_tokens: 640
top_p: 0.9
timeout_seconds: 45
service_tier: flex
reasoning_effort: medium
api_key_env: FIXTURE_PROFILE_API_KEY
extra_params:
routing:
order:
- primary
- fallback
allow_fallback: true

View File

@@ -0,0 +1,4 @@
id: custom-derived
base_profile: custom-base
model: fixture-derived-model
reasoning_effort: high

View File

@@ -0,0 +1,10 @@
id: endpoint-only
endpoint: http://127.0.0.1:8000/v1
model: fixture-endpoint-model
temperature: 0.1
max_tokens: 128
top_p: 0.75
timeout_seconds: 30
extra_params:
endpoint_option:
nested: true

View File

@@ -0,0 +1,31 @@
id: compat.complete
version: "1.0.0"
description: Exercise the complete Promptkit v0.9 prompt-definition surface.
default_profile: custom-derived
session_id: 'fixture-{{.topic}}'
inputs:
- name: source
required: true
content_type: text/markdown
description: Source material rendered by a file-backed message.
- name: context
required: false
content_type: text/plain
description: Optional caller context.
messages:
- role: developer
content: 'Use the topic {{.topic}}.'
cache_control:
type: ephemeral
ttl: 1h
- role: system
content_file: ./messages/complete.system.md
- role: user
content_file: ./messages/complete.user.md
- role: assistant
content: Return one JSON object.
output:
format: json
validation_mode: json_schema
schema_path: complete.schema.json
repair_attempts: 2

View File

@@ -0,0 +1,10 @@
id: compat.complete
version: "2.0.0"
default_profile: endpoint-only
messages:
- role: user
content: Version two has no declared inputs.
output:
format: text
validation_mode: basic
repair_attempts: 1

View File

@@ -0,0 +1,10 @@
id: compat.json
version: "1.0.0"
default_profile: endpoint-only
messages:
- role: user
content: Return JSON.
output:
format: json
validation_mode: json
repair_attempts: 0

View File

@@ -0,0 +1 @@
Use the supplied source without inventing details.

View File

@@ -0,0 +1,4 @@
Topic: {{.topic}}
Source:
{{input "source"}}

View File

@@ -0,0 +1,10 @@
id: compat.none
version: "1.0.0"
default_profile: endpoint-only
messages:
- role: user
content: Return text.
output:
format: text
validation_mode: none
repair_attempts: 0

View File

@@ -0,0 +1,11 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"type": "object",
"required": ["summary"],
"properties": {
"summary": {
"type": "string"
}
},
"additionalProperties": false
}