18 Commits

Author SHA1 Message Date
f33af20524 Prepare the v0.13.0 release
All checks were successful
ci/woodpecker/tag/release Pipeline was successful
2026-08-29 15:36:01 +00:00
85d14784ca Mark Promptkit adoption roadmap complete 2026-08-29 15:16:40 +00:00
edca9fbbc1 Document inspection architecture 2026-08-29 15:16:25 +00:00
6c2541c1ee Unify inspection configuration handling 2026-08-29 15:16:04 +00:00
b9b98b2aec List profile inspection in CLI usage 2026-08-29 15:12:45 +00:00
8936ca7c18 Add Promptkit definition conformance coverage 2026-08-29 15:12:27 +00:00
3e5f5c5198 Complete Promptkit v0.9 adoption 2026-08-29 14:31:50 +00:00
68ebe9ee50 Add profile inspection command 2026-08-29 14:31:13 +00:00
36c7c5a358 Add prompt inspection command 2026-08-29 14:29:14 +00:00
157209f097 Expose backend identity and capacity errors 2026-08-29 14:26:04 +00:00
06b37c2bae Add session and reasoning controls 2026-08-29 14:22:45 +00:00
1a0f15e210 Add custom backend configuration 2026-08-29 14:18:29 +00:00
5f946a5a1f Allow prompt version selection without inputs 2026-08-29 14:12:51 +00:00
1806df9888 Adopt the Promptkit v0.9 compatibility baseline 2026-08-29 14:08:35 +00:00
c35ac5a0cb Document the Promptkit v0.9 adoption plan 2026-08-29 13:54:37 +00:00
40f8a1c628 Identify v0.12.0 as the first published application-only release 2026-07-28 15:59:23 -05:00
27d7ad5057 Retire the completed migration roadmaps 2026-07-28 20:04:24 +00:00
1f08a1a94c Record completion of the Promptkit migration 2026-07-28 20:03:11 +00:00
49 changed files with 3384 additions and 1283 deletions

View File

@@ -6,7 +6,7 @@ OpenAI-compatible model endpoints, and returns generated output with validation
metadata.
The application uses
[Promptkit v0.1.0](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.1.0/)
[Promptkit v0.9.0](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.9.0/)
for prompt, profile, schema, preparation, generation, and validation behavior.
Scriptorium owns executable configuration, CLI and HTTP mapping, process
behavior, output presentation, and HTTP artifact-containment policy.
@@ -38,8 +38,8 @@ a model. For complete invocation and output behavior, see the
- [Migration from the former Go package](docs/consumers/migrating-to-promptkit.md)
- [Subprocess integration](docs/integrations/subprocess.md)
- [Architecture policy](docs/policy/architecture.md)
- [Promptkit framework formats](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.1.0/docs/formats.md)
- [Promptkit Go consumer guide](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.1.0/docs/consumers/pkg-promptkit.md)
- [Promptkit framework formats](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.9.0/docs/formats.md)
- [Promptkit Go consumer guide](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.9.0/docs/consumers/pkg-promptkit.md)
## Examples

View File

@@ -0,0 +1,112 @@
# Preserve Definition Compatibility And A Simple Execution Model
## Status
Accepted.
## Date
2026-08-29.
## Context
Scriptorium is a user-facing application built on Promptkit. Promptkit is a Go
library with both declarative framework formats and public APIs intended for
applications that need to assemble specialized integrations or workflows.
Scriptorium needs a durable rule for deciding which upstream capabilities it
must support. Treating every new Promptkit public API as an application feature
would steadily expand Scriptorium into a workflow framework. Selecting an
arbitrary subset of prompt or profile fields would instead make valid Promptkit
definitions unexpectedly unusable through Scriptorium.
The intended product is narrower: Scriptorium provides simple executable
interfaces that select and execute predefined prompts and return their results
in a consistent, repeatable form. It is not intended to own conversations,
modify prompts dynamically, coordinate follow-up messages, or manage complex
multi-step workflows.
## Decision
Scriptorium will aim to support the complete set of features expressible in
valid Promptkit prompt and profile definitions for the Promptkit version it
selects. This includes the schema, backend, source, and request plumbing needed
to prepare and execute those definitions without Scriptorium imposing a
narrower field-level format contract.
Promptkit remains the canonical parser and validator for its definitions.
Scriptorium will pass its configured framework sources and mapped request
values through the supported Promptkit public facade rather than copying
Promptkit formats, defaults, or validation rules.
Promptkit library features that are not required to use valid prompt or profile
definitions are considered individually. They may be incorporated when they
serve Scriptorium's simple execution model and fit its application-owned CLI,
HTTP, configuration, presentation, security, and process boundaries. A public
Promptkit declaration does not by itself require a corresponding Scriptorium
interface.
Each execution request will remain self-contained from the caller's
perspective. A request must not depend on conversation or workflow state
retained from a completed request, and Scriptorium will not become a general
workflow or conversation system. Application-managed prompt modification,
appended or follow-up message flows, conversation state, prepared-handle
coordination, multi-step orchestration, and similar higher-level workflows are
outside its intended scope.
Scriptorium may, and when necessary must, maintain process-scoped operational
state shared by concurrent requests. This includes the configured Promptkit
engine and backend registry, capacity admission and queue accounting,
in-flight request coordination, cancellation, and ordinary server lifecycle
resources. The state must be scoped so configured backend limits apply across
all relevant in-flight requests. But this is operational implementation state,
not durable user workflow or conversation state.
## Alternatives Considered
### Expose Every Promptkit Public Feature
This would make the application surface track the library API closely, but it
would blur the boundary between a reusable Go library and a simple executable
consumer. Library-oriented lifecycle, source-construction, extension, and
workflow primitives would add commands, wire contracts, configuration, and
state without necessarily improving predefined prompt execution.
### Support A Curated Subset Of Definition Fields
This would keep the application small in the short term, but valid Promptkit
definitions could fail or behave differently solely because they were invoked
through Scriptorium. Maintaining a parallel field-level compatibility list
would also duplicate upstream format ownership and create recurring drift.
### Expand Scriptorium Into A Workflow Service
Scriptorium could own conversations, prompt changes, follow-up messages,
retries, checkpoints, and multi-step execution. That is a different product
with durable state, lifecycle, recovery, privacy, and operational requirements
that are not justified by Scriptorium's current purpose.
## Consequences
- Promptkit upgrades require an audit of the complete prompt and profile format
contracts, not only a source-compatibility build.
- Scriptorium adapters must not add unconditional request requirements that
reject a definition Promptkit can validly prepare or execute.
- Application configuration may need new plumbing, such as custom backend
registration, when it is necessary to use a valid profile feature.
- Definition parsing, validation, precedence, inheritance, defaults, and
built-in catalogs remain Promptkit responsibilities.
- A running HTTP server must share its appropriately scoped Promptkit engine
across requests so the engine's backend admission and concurrency limits
cannot be bypassed by per-request construction.
- Process-scoped counters, queues, in-flight request records, cancellation, and
lifecycle resources are permitted implementation state. They must be
concurrency-safe and do not create a durable resume or conversation
contract.
- Library-only source constructors, injected collaborators, opaque lifecycle
handles, and request-composition primitives remain optional Scriptorium
features rather than automatic requirements.
- New application features must be evaluated against the simple,
request-independent predefined-prompt execution model.
- Work that implements this decision remains tracked in roadmaps until it is
delivered and incorporated into current-state contracts.

View File

@@ -32,14 +32,11 @@ returns `413 response_too_large`.
### Request Body
The maintained [request example](../examples/http-run.json) is a complete
copyable shape. The smallest valid shape is:
copyable shape. At the HTTP adapter boundary, the smallest valid shape is:
```json
{
"prompt_id": "generic.markdown_summary",
"inputs": {
"transcript": {"type": "inline", "body": "Source text"}
}
"prompt_id": "generic.markdown_summary"
}
```
@@ -48,7 +45,8 @@ copyable shape. The smallest valid shape is:
| `prompt_id` | yes | Non-blank prompt ID. |
| `prompt_version` | no | Prompt version filter. |
| `profile_id` | no | Execution-profile ID; otherwise the prompt must set `default_profile`. |
| `inputs` | yes | Non-empty object mapping input names to references. |
| `session_id` | no | Optional direct, non-secret session identifier. |
| `inputs` | no | Optional object mapping input names to references. Promptkit decides whether the selected definition needs them. |
| `vars` | no | Object mapping template-variable names to strings. |
| `model` | no | Runtime model-override object. |
| `include_raw_output` | no | Include `raw_model_output` when true. |
@@ -67,20 +65,31 @@ The optional `model` object accepts `endpoint`, `model`, `temperature`,
`max_tokens`, `top_p`, `timeout_seconds`, `service_tier`,
`reasoning_effort`, `api_key_env`, and `extra_params`. Numeric ranges and
framework credential semantics are defined by the
[Promptkit format reference](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.1.0/docs/formats.md).
[Promptkit format reference](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.9.0/docs/formats.md).
Explicit zero values for the numeric fields are overrides; zero
`timeout_seconds` disables the per-generation deadline only, retaining the
request context and configured transport cap. The timeout layers are defined in
the [Promptkit outbound integration contract](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.1.0/docs/integrations/openai-compatible-chat.md#timeout-and-cancellation).
the [Promptkit outbound integration contract](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.9.0/docs/integrations/openai-compatible-chat.md#timeout-and-cancellation).
Raw API-key values are not accepted. `api_key` and any other unknown model
field cause `400 invalid_json`.
`model.reasoning_effort` is an optional JSON string with three states: omission
inherits the selected profile, a non-empty string replaces its value, and an
empty string explicitly clears it. JSON `null` is treated as omission.
`session_id` is passed directly to Promptkit. A nonblank value replaces a
definition-rendered session ID; omission or a blank value lets the definition
provide one. Promptkit trims direct values and limits them to 256 Unicode code
points. Session IDs are not credentials and may be included in prepared data,
results, and provider-facing requests, so use stable non-sensitive identifiers.
### Strict JSON
Request decoding rejects malformed JSON, unknown fields at every request level,
and trailing JSON tokens with `400 invalid_json`. A blank `prompt_id` or
empty `inputs` object returns `400 invalid_request`.
and trailing JSON tokens with `400 invalid_json`. A blank `prompt_id` returns
`400 invalid_request`. Omitted or empty `inputs` are passed to Promptkit, which
reports any definition-required or template-referenced inputs.
### Success Response
@@ -91,17 +100,26 @@ validation contract. The response contains:
optional `uri`;
- `validation`: `status`, `mode`, `repair_attempts`, `is_valid`, plus
optional `errors` and `schema_path`;
- `metadata`: run, prompt, rendered-prompt, profile, model, input-hash, usage,
- `metadata`: run, prompt, rendered-prompt, profile, optional backend identity, model, input-hash, usage,
timing, validation, and repair-attempt metadata; and
- optional `raw_model_output` when requested.
`metadata.model_params` has `endpoint`, `model`, `temperature`,
`max_tokens`, `top_p`, and `timeout_seconds`, plus optional
`service_tier`, `reasoning_effort`, `api_key_env`, and `extra_params`.
`backend_id`, `service_tier`, `reasoning_effort`, `api_key_env`, and `extra_params`.
`metadata.usage` always includes `prompt_tokens`, `completion_tokens`,
`total_tokens`, `cached_tokens`, and `cache_write_tokens`; unavailable
cache usage is reported as zero.
When Promptkit resolves a direct or definition-rendered session ID,
`metadata.session_id` contains that effective result value. It is omitted when
no effective session ID exists.
`metadata.selected_backend_id` and `metadata.model_params.backend_id` report
the corresponding Promptkit result fields independently when present. Both are
omitted for an endpoint-only profile; Scriptorium does not infer backend
identity from an endpoint.
A validation failure has `validation.status: "failed"`, `is_valid: false`,
and any available diagnostic errors, while still returning the artifact and
metadata.
@@ -137,6 +155,7 @@ Messages are concise and do not expose wrapped internal causes.
| `500` | `validation_runtime_failed` | Schema or validator runtime failure. |
| `500` | `internal_error` | Unclassified server failure. |
| `502` | `llm_failed` | Outbound model request failed. |
| `503` | `capacity_exceeded` | The selected model backend has no admission capacity. No retry timing is supplied. |
## Retry And Idempotency

View File

@@ -23,6 +23,8 @@ go run ./cmd/scriptorium render \
generated artifact.
- `scriptorium render`: prepare a prompt and write prepared-run output.
- `scriptorium serve`: start the HTTP server.
- `scriptorium inspect prompt`: inspect one prompt definition without model execution.
- `scriptorium inspect profile`: inspect one effective profile without model execution.
All commands accept `--config <path>` and reject positional arguments. An
effective `prompt_dir` is required for every command. Supply it through the
@@ -39,7 +41,6 @@ Required flags:
| Flag | Meaning |
| --- | --- |
| `--prompt <id>` | Prompt ID to execute. |
| `--input name=path` | Input file mapping; repeat or use comma-separated mappings. |
Optional flags:
@@ -49,7 +50,10 @@ Optional flags:
| `--prompt-dir <dir>` | Prompt-definition directory override. |
| `--profile-dir <dir>` | Custom profile-directory override. |
| `--schema-dir <dir>` | Schema base-directory override. |
| `--prompt-version <version>` | Optional prompt-definition version selector. |
| `--profile <id>` | Execution-profile override. |
| `--session-id <id>` | Optional direct session identifier. |
| `--input name=path` | Optional input file mapping; repeat or use comma-separated mappings. |
| `--var name=value` | Template-variable mapping; repeat or use comma-separated mappings. |
| `--out <path>` | Write generated content to this file instead of stdout. |
| `--llm-base-url <url>` | Runtime endpoint override. |
@@ -58,6 +62,7 @@ Optional flags:
| `--temperature <float>` | Runtime temperature override. |
| `--max-tokens <int>` | Runtime maximum-token override. |
| `--top-p <float>` | Runtime top-p override. |
| `--reasoning-effort <value>` | Runtime reasoning-effort override. |
| `--timeout <duration>` | Runtime timeout override using Go duration syntax. |
Deprecated aliases: `--prompt-id` for `--prompt`, and `--profile-id` for
@@ -69,25 +74,44 @@ only; the caller context and configured transport cap remain active. CLI
durations are converted to whole seconds by truncation toward zero, so any
duration whose absolute value is below one second becomes an explicit
zero-second override. The timeout layers are defined in the
[Promptkit outbound integration contract](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.1.0/docs/integrations/openai-compatible-chat.md#timeout-and-cancellation).
[Promptkit outbound integration contract](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.9.0/docs/integrations/openai-compatible-chat.md#timeout-and-cancellation).
There is no raw API-key flag. Use `--api-key-env`.
`--prompt-version` is passed directly to Promptkit. When it is omitted, the
selected prompt ID must have exactly one available version. `--input` is
optional at the CLI boundary: Promptkit decides whether the selected definition
requires declared inputs or template-referenced values.
`--session-id` supplies a direct, non-secret session identifier. A nonblank
value replaces a definition-rendered session ID; an omitted or blank value lets
the definition supply one. Promptkit trims direct values and limits them to 256
Unicode code points. Use stable, non-sensitive identifiers because effective
session IDs may appear in prepared output, run metadata, and provider-facing
requests.
`--reasoning-effort` is presence-aware: omitting it inherits the selected
profile value, a nonblank value replaces that value, and
`--reasoning-effort=` explicitly clears inherited reasoning. Promptkit treats
nonblank values as provider-specific opaque strings.
## `scriptorium render`
```text
scriptorium render [flags]
```
`--prompt <id>` and at least one `--input name=path` are required. The
following optional flags are supported: `--config`, `--prompt-dir`,
`--profile-dir`, `--profile`, `--var`, `--out`, `--llm-base-url`,
`--prompt <id>` is required. The following optional flags are supported:
`--config`, `--prompt-dir`, `--profile-dir`, `--prompt-version`, `--profile`,
`--input`, `--var`, `--out`, `--llm-base-url`,
`--model`, `--api-key-env`, `--temperature`, `--max-tokens`, `--top-p`,
`--timeout`, and `--format text|json`. Their meanings match the corresponding
`--reasoning-effort`, `--session-id`, `--timeout`, and `--format text|json`. Their meanings match the corresponding
`run` flags; `--format` selects prepared-run output and otherwise uses
`defaults.render_format`.
The same deprecated aliases and numeric/timeout behavior as `run` apply.
The same session and reasoning inheritance, replacement, and clearing behavior
also applies.
`render` does not accept `--schema-dir`; configure `schema_dir` through the
configuration file. It resolves profiles and schemas as part of preparation but
does not call an LLM.
@@ -115,6 +139,31 @@ Optional flags:
`serve` accepts no runtime model override flags. HTTP request fields, response
schemas, and error codes are defined in the [HTTP API reference](api.md).
## `scriptorium inspect prompt`
```text
scriptorium inspect prompt --prompt ID [--prompt-version VERSION]
[--config PATH] [--prompt-dir DIR] [--format text|json] [--out PATH]
```
`--prompt` is required. Inspection uses normal configuration discovery and a
`--prompt-dir` override, defaults to text regardless of `defaults.render_format`,
and writes to stdout unless `--out` is supplied. It loads and normalizes the
selected definition but does not resolve a profile, load a schema, render a
template, reserve backend capacity, or call a model.
## `scriptorium inspect profile`
```text
scriptorium inspect profile --profile ID
[--config PATH] [--profile-dir DIR] [--format text|json] [--out PATH]
```
`--profile` is required; built-in profiles need no prompt directory. Inspection
resolves profile inheritance and backend defaults, but never reads a credential
value, loads a prompt, reserves capacity, or calls a model. Unset provider
controls are shown as zero values when Promptkit leaves them unspecified.
## Input And Variable Syntax
`--input name=path` maps an input name to a local file; `--var name=value`
@@ -128,9 +177,12 @@ CLI inputs are file references. HTTP inline inputs are defined by the
## Output And Exit Behavior
- `run` writes generated content to stdout, or to `--out` when supplied, and
writes a concise summary to stderr.
writes a concise summary to stderr. The summary includes `backend=<id>` when
Promptkit selected a backend; endpoint-only profiles omit it.
- `render` writes prepared-run output to stdout, or to `--out` when supplied,
without a success summary.
without a success summary. Text output includes `selected_backend_id` after
`selected_profile_id` when Promptkit selected one; endpoint-only profiles
omit it.
- `serve` writes startup and server errors to stderr.
Exit statuses:
@@ -141,6 +193,10 @@ Exit statuses:
| `1` | Parse, configuration, loading, rendering, generation, output-write, or other runtime error. |
| `2` | `run` generated and wrote output, but validation failed. |
A backend admission rejection is a runtime error and prints `run error: model
backend capacity is exhausted`. The HTTP capacity response is defined in the
[HTTP API reference](api.md).
## Workflows And Examples
The [maintained render script](../examples/render-markdown-summary.sh) is a

View File

@@ -3,7 +3,7 @@
This is the canonical reference for Scriptorium application settings. Prompt,
profile, schema, execution-setting, built-in profile, and framework credential
semantics are defined by the
[Promptkit v0.1.0 format reference](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.1.0/docs/formats.md).
[Promptkit v0.9.0 format reference](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.9.0/docs/formats.md).
For command syntax, see the [CLI reference](cli.md); for HTTP request shapes and
outcomes, see the [HTTP API reference](api.md).
@@ -39,11 +39,39 @@ do not override a prior value. Raw API-key fields are not accepted.
| `server.max_artifact_bytes` | `16777216` | Maximum HTTP file-input artifact bytes; `0` disables the limit. |
| `server.max_response_bytes` | `16777216` | Maximum encoded HTTP response bytes; `0` disables the limit. |
| `defaults.render_format` | `text` | Default prepared-run output format: `text` or `json`. |
| `backends` | unset | Optional mapping of custom Promptkit backend IDs to engine-scoped connection and capacity settings. |
The size fields must be zero or greater. The [HTTP API](api.md) defines how
each limit is enforced and reported. `server.artifact_root` configures an HTTP
deployment boundary; see [operations](operations.md) for deployment handling.
## Custom Backends
Use `backends` when a profile selects an application-defined backend ID:
```yaml
backends:
local-gpu:
endpoint: http://localhost:11434/v1
api_key_env: LOCAL_GPU_API_KEY
extra_params:
provider_option: enabled
concurrency_limit: 2
queue_capacity: 0
```
Each mapping key is the case-sensitive backend ID. `endpoint` is required;
`api_key_env`, `extra_params`, `concurrency_limit`, and `queue_capacity` are
optional. `concurrency_limit: 0` leaves the backend unlimited. Omitting
`queue_capacity` lets Promptkit use its default for a limited backend, while
an explicit `queue_capacity: 0` disables queueing.
Configuration strictly owns the YAML shape and rejects unknown fields. Promptkit
validates backend IDs, endpoints, environment-variable names, extra parameters,
and capacity relationships when Scriptorium constructs its engine. There are no
backend command-line overrides. Store only an environment-variable name in
`api_key_env`; raw API-key fields are not accepted.
## Framework Source Mapping
Scriptorium passes `prompt_dir`, `profile_dir`, and `schema_dir` to Promptkit
@@ -58,7 +86,7 @@ parse those framework file formats.
contracts.
See the
[tagged Promptkit format reference](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.1.0/docs/formats.md)
[tagged Promptkit format reference](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.9.0/docs/formats.md)
for all of those definitions. The files under
[`examples/prompts`](../examples/prompts/),
[`examples/profiles`](../examples/profiles/), and
@@ -73,8 +101,20 @@ their names in configuration or framework source files. Do not place raw keys
in configuration, prompts, profiles, CLI arguments, examples, or HTTP
payloads.
An optional `api_key_env` whose environment value is absent or empty can result
in an unauthenticated provider request. A profile that declares credentials
required still fails as an invalid request when no credential source is
selected, and fails with `ErrAPIKeyEnvMissing` when its selected environment
source is absent or empty. Scriptorium never reads or emits the environment
value itself.
Unset optional provider controls are omitted from compatible provider requests.
A positive Promptkit `repair_attempts` budget can add provider calls, latency,
token use, and cost; see the tagged format reference for its permitted values
and validation-mode requirements.
Promptkit's
[OpenAI-compatible integration contract](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.1.0/docs/integrations/openai-compatible-chat.md)
[OpenAI-compatible integration contract](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.9.0/docs/integrations/openai-compatible-chat.md)
defines outbound authentication, provider request mapping, transport limits,
and timeout layering.
@@ -83,4 +123,4 @@ and timeout layering.
- [CLI reference](cli.md)
- [HTTP API reference](api.md)
- [Operations guide](operations.md)
- [Promptkit framework formats](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.1.0/docs/formats.md)
- [Promptkit framework formats](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.9.0/docs/formats.md)

View File

@@ -14,7 +14,7 @@ and the HTTP service according to the boundary your application needs.
Go applications that need an in-process prompt framework should import
Promptkit directly. The tagged
[Promptkit Go consumer guide](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.1.0/docs/consumers/pkg-promptkit.md)
[Promptkit Go consumer guide](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.9.0/docs/consumers/pkg-promptkit.md)
owns that interface; Scriptorium does not provide a Go library package.
Consumers arriving from the former Scriptorium Go API should follow the
[migration guide](migrating-to-promptkit.md).
@@ -35,4 +35,4 @@ Scriptorium does not persist run state. A retry can produce different output
and can incur another provider request. CLI exits belong to the
[CLI reference](../cli.md), HTTP status behavior belongs to the
[HTTP API reference](../api.md), and framework semantics belong to
[Promptkit v0.1.0](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.1.0/docs/formats.md).
[Promptkit v0.9.0](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.9.0/docs/formats.md).

View File

@@ -4,7 +4,7 @@
Scriptorium `v0.11.1` at
`gitea.maximumdirect.net/eric/scriptorium` is the final release that provides
the former in-process Go framework. Promptkit `v0.1.0` at
the former in-process Go framework. Promptkit `v0.9.0` at
`gitea.maximumdirect.net/eric/promptkit` is the destination for that framework
API. Scriptorium `v0.12.0` and later provide the CLI and HTTP application only.
@@ -19,7 +19,7 @@ Start from a clean consumer checkout and review the pending diff before
committing it. Add the published Promptkit module:
```sh
go get gitea.maximumdirect.net/eric/promptkit@v0.1.0
go get gitea.maximumdirect.net/eric/promptkit@v0.9.0
```
For an ordinary consumer that imports the former root package under its
@@ -51,14 +51,14 @@ go mod tidy
go test ./...
```
Confirm that `go.mod` selects Promptkit `v0.1.0` and that no Go file imports
Confirm that `go.mod` selects Promptkit `v0.9.0` and that no Go file imports
the former Scriptorium package:
```sh
test "$(
go list -m -f '{{.Path}}@{{.Version}}' \
gitea.maximumdirect.net/eric/promptkit
)" = 'gitea.maximumdirect.net/eric/promptkit@v0.1.0'
)" = 'gitea.maximumdirect.net/eric/promptkit@v0.9.0'
if git grep -n \
'gitea.maximumdirect.net/eric/scriptorium' \
-- '*.go'
@@ -73,20 +73,20 @@ fi
Promptkit preserves the established engine, request, result, profile,
source-option, model-client, artifact, validation-value, and public-error
shapes where practical. Exact declarations and current behavior belong to the
tagged [Promptkit consumer guide](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.1.0/docs/consumers/pkg-promptkit.md)
tagged [Promptkit consumer guide](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.9.0/docs/consumers/pkg-promptkit.md)
and Go source.
Promptkit also includes migration-relevant public contracts that were not in
Scriptorium `v0.11.1`:
- [`WithArtifactReader`](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.1.0/engine.go#L96-L105)
- [`WithArtifactReader`](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.9.0/engine.go#L182)
and the
[`ArtifactReader` declaration](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.1.0/types.go#L129-L135)
[`ArtifactReader` declaration](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.9.0/types.go#L304)
provide the artifact-reading extension described by the tagged
[extension-interface guide](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.1.0/docs/consumers/pkg-promptkit.md#extension-interfaces).
- [`ErrProfileRequired` and `ErrAPIKeyEnvMissing`](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.1.0/engine.go#L28-L40)
[extension-interface guide](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.9.0/docs/consumers/pkg-promptkit.md#extension-interfaces).
- [`ErrProfileRequired` and `ErrAPIKeyEnvMissing`](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.9.0/engine.go#L46-L61)
provide the specific identities described by the tagged
[error guide](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.1.0/docs/consumers/pkg-promptkit.md#errors).
[error guide](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.9.0/docs/consumers/pkg-promptkit.md#errors).
Use those tagged owners for exact signatures, wrapping guarantees, and
extension behavior.

View File

@@ -2,7 +2,7 @@
This is the contributor entry point for Scriptorium. Scriptorium is an
application that consumes the public
[Promptkit v0.1.0 package](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.1.0/docs/consumers/pkg-promptkit.md);
[Promptkit v0.9.0 package](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.9.0/docs/consumers/pkg-promptkit.md);
framework implementation work belongs in Promptkit.
## Initial Orientation
@@ -27,12 +27,12 @@ tests.
| HTTP routes, DTOs, limits, status mapping, or artifact policy | [HTTP API contract](api.md), [adapter internals](internal/adapters.md), and [source internals](internal/sources.md) |
| Application configuration or precedence | [Configuration contract](config.md), [adapter internals](internal/adapters.md), and [source internals](internal/sources.md) |
| Prepared-run presentation | [CLI contract](cli.md), [adapter internals](internal/adapters.md), and `internal/format` |
| Prompt, profile, schema, generation, or validation semantics | [Promptkit framework formats](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.1.0/docs/formats.md) and the [Promptkit consumer guide](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.1.0/docs/consumers/pkg-promptkit.md) |
| OpenAI-compatible outbound behavior or timeout layering | [Promptkit integration contract](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.1.0/docs/integrations/openai-compatible-chat.md) |
| Prompt, profile, schema, generation, or validation semantics | [Promptkit framework formats](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.9.0/docs/formats.md) and the [Promptkit consumer guide](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.9.0/docs/consumers/pkg-promptkit.md) |
| OpenAI-compatible outbound behavior or timeout layering | [Promptkit integration contract](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.9.0/docs/integrations/openai-compatible-chat.md) |
| Subprocess behavior | [Subprocess integration](integrations/subprocess.md) and [CLI contract](cli.md) |
| Runtime operation or recovery | [Operations](operations.md) |
| Release packaging or publication | The [release procedure](release.md), [hosted release workflow](../.woodpecker/release.yml), and [architecture policy](policy/architecture.md) |
| Examples or copyable assets | The owning Scriptorium contract, the relevant [Promptkit format contract](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.1.0/docs/formats.md), and the related files under `examples/` |
| Examples or copyable assets | The owning Scriptorium contract, the relevant [Promptkit format contract](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.9.0/docs/formats.md), and the related files under `examples/` |
| Architecture decisions or future work | The [documentation policy](policy/documentation.md), relevant accepted ADRs, and relevant roadmap documents |
Cross-project changes land and release in Promptkit before Scriptorium adopts

View File

@@ -9,7 +9,7 @@ own IO and presentation mechanics, not framework decisions.
External contracts are canonical in the [CLI reference](../cli.md) and
[HTTP API reference](../api.md). Promptkit's public engine contract is
described by its tagged
[Go consumer guide](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.1.0/docs/consumers/pkg-promptkit.md).
[Go consumer guide](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.9.0/docs/consumers/pkg-promptkit.md).
## Components And Collaborators
@@ -29,15 +29,30 @@ described by its tagged
### CLI
`run` calls `promptkit.Engine.Run`; `render` calls
`promptkit.Engine.Prepare`. Both share request mapping for prompt/profile
selection, file inputs, variables, and presence-aware execution overrides.
Omitted framework settings remain zero values so Promptkit resolves its own
defaults.
`promptkit.Engine.Prepare`. Both share request mapping for prompt ID/version
and profile selection, optional file inputs, variables, and presence-aware
execution overrides. Omitted framework settings remain zero values so Promptkit
resolves its own defaults and definition-required inputs.
`inspect prompt` maps the selected ID and optional version to
`promptkit.Engine.InspectPrompt`, then formats Scriptorium-owned inspection
data. It constructs the same configuration-aware engine but does not perform
preparation or generation.
`inspect profile` maps an explicit profile ID to `Engine.InspectProfile` and
formats a safe application-owned effective profile view. It intentionally does
not require prompt, schema, or artifact sources.
`serve` constructs Scriptorium's restricted HTTP artifact reader, injects it
with `promptkit.WithArtifactReader`, passes the engine through the HTTP
adapter's consumer-owned `Runner` interface, and starts the server.
All three CLI paths assemble the engine from the same resolved prompt, profile,
and schema directories plus configured custom backends. Each backend is mapped
to Promptkit's public `Backend` value and registered during engine construction,
so one constructed server engine retains one immutable backend registry and its
associated capacity state.
### HTTP
The handler enforces transport limits and strict JSON decoding before mapping

View File

@@ -11,8 +11,8 @@ public behavior belongs in the linked contracts.
| `internal/adapter/http` | Owns routes, DTOs, strict decoding, limits, Promptkit request/result mapping, public error mapping, and restricted HTTP artifact reading. | [HTTP API](../api.md), [adapter internals](adapters.md), [source internals](sources.md) |
| `internal/config` | Discovers and strictly decodes application configuration and applies built-in and CLI precedence. | [configuration contract](../config.md), [adapter internals](adapters.md) |
| `internal/defaults` | Holds Scriptorium-owned application and HTTP defaults. | [configuration contract](../config.md) |
| `internal/format` | Formats Promptkit prepared-run values for CLI text or JSON output. | [CLI contract](../cli.md), [adapter internals](adapters.md) |
| `internal/format` | Formats Promptkit prepared-run and prompt/profile inspection values for CLI text or JSON output. | [CLI contract](../cli.md), [adapter internals](adapters.md) |
Framework implementation packages are provided by
[Promptkit v0.1.0](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.1.0/docs/consumers/pkg-promptkit.md)
[Promptkit v0.9.0](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.9.0/docs/consumers/pkg-promptkit.md)
and are not part of this repository.

View File

@@ -5,14 +5,16 @@
This document covers Scriptorium-owned source locations and the restricted HTTP
artifact reader. Prompt, profile, schema, and ordinary artifact semantics are
owned by the tagged
[Promptkit format reference](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.1.0/docs/formats.md).
[Promptkit format reference](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.9.0/docs/formats.md).
## Application Source Locations
`internal/config` resolves `prompt_dir`, `profile_dir`, and `schema_dir` from
Scriptorium defaults, configuration files, and CLI overrides.
`internal/adapter/cli` passes those paths into `promptkit.Config` when
constructing the engine.
Scriptorium defaults, configuration files, and CLI overrides. It also resolves
the application-owned `backends` mapping into sorted engine settings.
`internal/adapter/cli` passes the directories into `promptkit.Config` and maps
each configured backend to Promptkit's public engine registration when
constructing an engine shared by the command path.
Scriptorium does not search, parse, validate, or overlay framework source files
itself. Promptkit owns prompt selection, profile built-ins and overlays, schema

View File

@@ -9,7 +9,7 @@ fields, or HTTP wire behavior.
- [CLI reference](cli.md): commands, output destinations, and exit codes.
- [Configuration reference](config.md): application settings, source
locations, defaults, and credential mapping.
- [Promptkit framework formats](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.1.0/docs/formats.md):
- [Promptkit framework formats](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.9.0/docs/formats.md):
prompt, profile, schema, execution-setting, and framework credential
contracts.
- [HTTP API reference](api.md): route, request/response schema, status codes,
@@ -94,6 +94,14 @@ protect request bodies, HTTP file artifacts, and encoded responses; configure
them through the [configuration reference](config.md) and rely on the
[HTTP API reference](api.md) for their response effects.
Configured backend concurrency and queue capacity are enforced per constructed
Promptkit engine. A `serve` process constructs one engine for its handler, so
concurrent HTTP requests share that transient admission state. Scriptorium does
not retain workflow state: capacity is neither durable nor a queue of resumable
runs. When admission is exhausted, HTTP returns `503 capacity_exceeded` without
retry timing; callers choose any retry policy that is safe for another model
call.
Before increasing a limit:
1. measure representative input, generated-output, and optional raw-output

View File

@@ -5,16 +5,18 @@ durable development boundaries.
## System Shape
Scriptorium is an executable application with three entry paths: CLI `run`, CLI
`render`, and the HTTP service started by `serve`. It does not expose a reusable
root Go package.
Scriptorium is an executable application with four command entry paths: CLI
`run`, CLI `render`, CLI `inspect`, and the HTTP service started by `serve`.
The `inspect` command has prompt and profile modes. Scriptorium does not expose
a reusable root Go package.
The application consumes
[Promptkit v0.1.0](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.1.0/docs/consumers/pkg-promptkit.md)
[Promptkit v0.9.0](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.9.0/docs/consumers/pkg-promptkit.md)
through its supported root package. Promptkit owns prompt execution,
preparation, source formats, built-in profiles, model-client behavior, and
validation. Scriptorium owns application configuration, executable adapters,
prepared-run presentation, process behavior, and HTTP deployment policy.
prepared-run and definition-inspection presentation, process behavior, and HTTP
deployment policy.
The concrete package inventory is maintained in the
[internal overview](../internal/overview.md).
@@ -33,6 +35,7 @@ gitea.maximumdirect.net/eric/promptkit
- Retained application packages may import Promptkit's root package.
- They must not import Promptkit `internal` packages.
- They must not import Promptkit catalog modules directly.
- They must not import the removed Scriptorium root facade or recreate former
framework package families.
- Adapter-owned interfaces use Promptkit public values when a consumer-side
@@ -52,11 +55,12 @@ invariants.
- `internal/config` owns discovery and strict decoding of Scriptorium
application configuration.
- `internal/defaults` owns Scriptorium application and HTTP defaults only.
- `internal/format` owns deterministic prepared-run text and JSON presentation.
- `internal/format` owns deterministic prepared-run and definition-inspection
text and JSON presentation.
- Promptkit owns framework orchestration and contracts. Its
[format reference](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.1.0/docs/formats.md)
[format reference](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.9.0/docs/formats.md)
and
[outbound integration contract](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.1.0/docs/integrations/openai-compatible-chat.md)
[outbound integration contract](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.9.0/docs/integrations/openai-compatible-chat.md)
are canonical.
## HTTP Artifact Security Boundary

View File

@@ -8,11 +8,11 @@ releases. The hosted
executables, publishes their SHA-256 checksums, and uses the matching file
under `docs/releases/` as the hosted release body.
`v0.12.0` is the selected version for the pending first application-only
release. It remains an unreleased candidate until its annotated tag is
published, the hosted workflow succeeds, and every published artifact is
verified. Later releases select a new `vMAJOR.MINOR.PATCH` version according to
the intended compatibility change.
`v0.12.0` is the first published application-only release. For each later
release, select a new `vMAJOR.MINOR.PATCH` version according to the intended
compatibility change. A selected version remains an unreleased candidate until
its annotated tag is published, the hosted workflow succeeds, and every
published artifact is verified.
Run this procedure from the Scriptorium repository root. A release must not
depend on a Go workspace, module replacement, vendor tree, sibling checkout,
@@ -20,14 +20,17 @@ unpublished dependency, or unpushed source commit.
## Establish The Candidate
For the pending application-only release, start a POSIX shell and select:
Start a POSIX shell, choose a semantic version that has not been published, and
export it as `RELEASE_VERSION`. For example, if `v0.12.1` is the intended next
version and remains unpublished, select:
```sh
export RELEASE_VERSION=v0.12.0
export RELEASE_VERSION=v0.12.1
```
For a later release, export its not-yet-published semantic version instead.
Then run the following guard in that same shell:
Use the version appropriate to the actual compatibility change rather than
assuming that the example is the next release. Then run the following guard in
that same shell:
```sh
set -eu
@@ -100,7 +103,7 @@ test "$(
)" = 'gitea.maximumdirect.net/eric/scriptorium 1.25.5'
```
Require Promptkit `v0.1.0` as both the direct module-graph edge and the selected
Require Promptkit `v0.9.0` as both the direct module-graph edge and the selected
module version:
```sh
@@ -114,11 +117,11 @@ direct_promptkit=$(
'
)
test "$direct_promptkit" = \
'gitea.maximumdirect.net/eric/promptkit@v0.1.0'
'gitea.maximumdirect.net/eric/promptkit@v0.9.0'
test "$(
GOWORK=off go list -m -f '{{.Path}}@{{.Version}}' \
gitea.maximumdirect.net/eric/promptkit
)" = 'gitea.maximumdirect.net/eric/promptkit@v0.1.0'
)" = 'gitea.maximumdirect.net/eric/promptkit@v0.9.0'
GOWORK=off go list -m all
```
@@ -169,11 +172,12 @@ unformatted=$(
test -z "$unformatted"
```
Run the maintained render script and smoke-test both maintained configuration
examples without a model call:
Run the maintained render and Promptkit v0.9 feature scripts, then smoke-test
both maintained configuration examples without a model call:
```sh
GOWORK=off ./examples/render-markdown-summary.sh
GOWORK=off ./examples/render-v0.9-features.sh
for config_file in examples/config.yml examples/config.full.yml
do
GOWORK=off go run ./cmd/scriptorium render \

59
docs/releases/v0.13.0.md Normal file
View File

@@ -0,0 +1,59 @@
# Scriptorium v0.13.0
## Promptkit v0.9 Adoption
Scriptorium now uses
[Promptkit `v0.9.0`](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.9.0/)
and supports the complete Promptkit v0.9 prompt and profile definition format
through its configured directory sources. This includes prompt versions,
optional inputs, all supported message roles, cache control, session templates,
output schemas and repair budgets, profile inheritance, backend selection,
provider controls, optional credential environment names, and JSON-compatible
extra parameters.
Promptkit remains the canonical owner of definition parsing and framework
validation. Scriptorium continues to own its CLI, HTTP, configuration,
presentation, and deployment contracts.
## Application Features
- `run` and `render` can select an explicit prompt version and can execute
prompts that do not declare or reference inputs.
- CLI and HTTP requests can provide a direct session ID. Reasoning controls can
inherit, replace, or explicitly clear the selected profile value.
- Application configuration can register custom OpenAI-compatible backends
with optional credential environment names, extra parameters, concurrency
limits, and queue policy.
- `inspect prompt` and `inspect profile` provide deterministic text or JSON
views without invoking a model. Profile inspection does not read credential
values or require a prompt definition.
- Prepared output, CLI summaries, and HTTP metadata report effective backend
identity when available.
- HTTP requests rejected by backend capacity policy return `503` with the
stable `capacity_exceeded` code. One server-scoped Promptkit engine enforces
those limits across concurrent requests.
See the versioned application contracts for exact behavior:
- [CLI reference](https://gitea.maximumdirect.net/eric/scriptorium/src/tag/v0.13.0/docs/cli.md)
- [HTTP API reference](https://gitea.maximumdirect.net/eric/scriptorium/src/tag/v0.13.0/docs/api.md)
- [Configuration reference](https://gitea.maximumdirect.net/eric/scriptorium/src/tag/v0.13.0/docs/config.md)
- [Operations guide](https://gitea.maximumdirect.net/eric/scriptorium/src/tag/v0.13.0/docs/operations.md)
- [Promptkit v0.9 format reference](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.9.0/docs/formats.md)
## Compatibility And Operations
There are no intentional removals from the v0.12 application interfaces. The
Promptkit upgrade does enforce its current definition-safety rules, including
supported message roles, absolute HTTP or HTTPS endpoints, contained content
and schema paths, bounded repair budgets, and JSON-compatible extra parameters.
Definitions rejected by these rules must be corrected before use.
Credential values remain outside configuration, definition files, CLI flags,
and HTTP payloads. Named environment sources are optional unless a selected
Promptkit profile explicitly requires a key. A positive repair budget permits
additional provider calls and can increase latency, token use, and cost.
The HTTP service retains only the shared operational state needed for backend
admission and in-flight request handling. Scriptorium does not retain durable
conversation, workflow, checkpoint, or resume state.

File diff suppressed because it is too large Load Diff

View File

@@ -1,341 +0,0 @@
# Promptkit Migration Roadmap
## Status
Accepted plan. Steps 1 through 8 are complete. Step 9 remains proposed and is
not yet implemented.
## Objective
Split the current repository into two projects:
- **Promptkit**: the reusable Go framework, public Go facade, execution engine,
source and validation support, OpenAI-compatible client, extension
interfaces, and built-in execution-profile registry.
- **Scriptorium**: a slim runnable application that imports Promptkit and
provides the CLI and HTTP interfaces.
Scriptorium will become another downstream Promptkit consumer rather than the
owner of the framework.
## Compatibility And Migration Policy
This is an intentionally breaking change.
- New and migrated Go consumers must import Promptkit instead of Scriptorium.
- Scriptorium will not retain type aliases, forwarding packages, deprecated
facade APIs, or other source-compatibility shims.
- Existing consumers may continue using a previously tagged Scriptorium module
version until they are migrated.
- The migration does not need to preserve compatibility between intermediate
development states. Each completed phase must instead leave the affected
repository internally consistent and tested.
- Promptkit should initially preserve the useful shape and behavior of the
current public Go facade where doing so reduces extraction risk. Broader API
redesign should follow the split unless required to establish the new
boundary.
## Target Ownership
Promptkit should own application-neutral framework behavior:
- public engine, request, result, option, extension, and error APIs;
- prompt-definition loading and rendering;
- execution profiles, overlays, and the built-in profile registry;
- artifact-loading interfaces and general-purpose `file` and `inline` support;
- schema loading and output validation;
- LLM client boundaries and the OpenAI-compatible implementation;
- preparation and execution orchestration;
- framework and execution defaults.
Scriptorium should own executable and transport concerns:
- the `scriptorium` command and its `run`, `render`, and `serve` interfaces;
- CLI parsing, output formatting, exit codes, and process behavior;
- application-config discovery and CLI precedence;
- HTTP routing, request and response DTOs, limits, and error/status mapping;
- HTTP artifact-root and deployment security policy;
- server and adapter defaults;
- executable examples, operations guidance, and transport documentation.
The intended dependency direction is:
```text
Scriptorium CLI and HTTP adapters
|
v
Promptkit
|
v
consumer-supplied sources and clients
```
Scriptorium must use Promptkit's public API. It must not depend on Promptkit
implementation packages or reproduce framework orchestration.
## Migration Steps
### Step 1: Refresh And Synchronize Documentation
Perform a repository-wide documentation refresh before migration development.
At minimum:
- reconcile all current-behavior documentation with the code, tests, examples,
defaults, and current public contracts;
- introduce the planned documentation-policy updates;
- establish an architecture decision record policy and canonical ADR location;
- resolve stale, duplicated, or misplaced material;
- validate documentation links and maintained examples;
- leave future migration behavior in `docs/roadmap/` until implemented.
**Gate:** Do not begin architectural migration work until the documentation
refresh and policy updates are merged and the repository has an agreed,
accurate baseline.
**Gate status:** Complete as of 2026-07-26. The completed documentation
refresh, follow-up verification, and layered-timeout correction remain recorded
in repository history. Step 1 remains complete after that validation.
### Step 2: Record The Architectural Decision And Detailed Boundary
Create an ADR, under the policy established in Step 1, that records:
- the decision to split Promptkit from Scriptorium;
- the target ownership and dependency direction;
- the selected Promptkit repository and Go module paths;
- the breaking-change and versioning policy;
- ownership of configuration fields and defaults;
- artifact-reader and HTTP containment responsibilities;
- local multi-repository development and release coordination;
- documentation ownership after the split.
Use the ADR to resolve any remaining public-boundary decisions before code is
moved.
**Gate:** The ADR is accepted, and every existing package, public contract,
configuration category, and maintained asset has a target owner.
**Gate status:** Complete as of 2026-07-26.
[ADR 0002: Split Promptkit From Scriptorium](../adr/0002-split-promptkit-from-scriptorium.md)
is accepted and records the required ownership and coordination decisions.
### Step 3: Characterize Existing Framework Behavior
Strengthen or add contract-focused tests where needed so extraction can be
verified without relying on package placement.
The completed Step 3 gate records the accepted implementation scope and
intended completion state.
Preserve coverage of:
- `Prepare` and `Run` behavior;
- prompt, profile, execution-default, and request-override precedence;
- presence-aware numeric overrides;
- built-in profile fallback and custom-profile overlays;
- strict YAML and JSON decoding;
- prompt, profile, schema, and artifact source behavior;
- structured-output requests and output validation;
- validation failures versus validation runtime errors;
- secret handling and redaction;
- public error classification;
- HTTP artifact restrictions and transport mappings.
**Gate:** Current framework and adapter contracts are represented by passing
tests sufficient to detect behavioral regressions during the split.
**Gate status:** Complete as of 2026-07-27. The framework contract corpus,
public `Engine` characterization, ownership audit, full test and vet suites,
temporary executable build, and maintained offline examples passed.
### Step 4: Make Scriptorium Adapters Consume The Public Facade
Within the current repository, refactor the CLI and HTTP adapters to use the
public framework facade rather than constructing or importing internal runner
components directly.
Add only the minimum public capabilities needed to support this boundary. These
may include:
- a small `Run`/`Prepare` consumer interface;
- injectable artifact-reading behavior for Scriptorium's restricted HTTP
policy;
- source options currently available only through internal constructors;
- prepared-run formatting based on public types;
- stable public error classification required by CLI and HTTP mappings.
Do not broadly export internal repositories, domain types, or use-case
implementations.
**Gate:** The CLI and HTTP adapters use only the public framework API for
framework behavior, and all tests and documented smoke commands pass.
**Gate status:** Complete as of 2026-07-28. CLI `run`, `render`, and `serve`,
the HTTP handler, and prepared-run formatting use the public facade; the
restricted HTTP reader is injected through the public extension point. The
post-implementation public-error, deterministic MIME, and recursive
dependency-guard corrections passed full tests, vet, build, race checks,
maintained examples, and configuration smoke checks.
### Step 5: Create The Promptkit Repository
Promptkit was established as an independent repository and Go module through
the completed out-of-band workflow recorded in repository history. Its
foundation includes:
- confirmed repository access, governance, origin, and default-branch tracking;
- module `gitea.maximumdirect.net/eric/promptkit` at Go `1.25.5`;
- a minimal root `promptkit` public package boundary with no placeholder
framework packages;
- library-specific development, architecture, documentation, testing, and
release policies;
- documented maintainer-run test, vet, build, formatting, documentation-link,
and repository-hygiene validation;
- source-commit and semantic Go module tag releases without runnable binaries
or binary packaging; and
- temporary workspace and uncommitted replacement workflows for coordinated
development without committed repository coupling.
[ADR 0003](../adr/0003-use-maintainer-run-validation-and-tag-only-releases-for-promptkit.md)
records the controlling Promptkit validation and release decision.
**Gate status:** Complete as of 2026-07-28. Promptkit passed its documented
validation independently, all maintained links and repository-hygiene checks
passed, and no workspace, replacement, CI configuration, binary, tag, command,
or placeholder package was added. The completed repository foundation
supported the Step 6 extraction.
### Step 6: Extract And Stabilize Promptkit
Move the application-neutral framework and built-in profile assets into
Promptkit. Preserve implementation packages as internal where practical.
The initial public API should remain focused on the established engine workflow
and the source and client extension points required by real consumers. Avoid
combining the extraction with unrelated API redesign.
Move or recreate the relevant:
- framework implementation;
- public package tests and framework contract tests;
- built-in profile assets and registry tests;
- Go consumer examples;
- framework, consumer, configuration-format, and integration documentation.
Verify that Promptkit can be built, tested, and consumed independently of the
Scriptorium repository.
**Gate:** Promptkit independently provides the agreed framework contract,
passes its documented validation, and has published its first versioned tag
before Scriptorium or another consumer adopts it, as required by
[ADR 0003](../adr/0003-use-maintainer-run-validation-and-tag-only-releases-for-promptkit.md).
**Gate status:** Complete as of 2026-07-28. Repository history records source
Scriptorium commit `c7263ab2a8e58f7fb97280082d327a820c7cece7`,
accepted Promptkit commit
`9e68a2bbf779545995270c47842048a3bc6c85dc`, independently passing acceptance,
published annotated tag `v0.1.0`, and successful remote-consumer validation.
Scriptorium remains unchanged at its pre-cutover boundary. Step 7 adoption of
the tagged module and removal of the duplicated framework is the next gate.
### Step 7: Slim Scriptorium And Adopt Promptkit
Update Scriptorium to import the tagged Promptkit module and remove the
framework implementation and public Go facade that Promptkit replaces.
Retain only Scriptorium-owned executable and transport behavior. In particular:
- wire CLI and HTTP requests through Promptkit's public API;
- keep application config and transport defaults in Scriptorium;
- keep restricted HTTP artifact policy in Scriptorium while injecting it
through Promptkit's supported boundary;
- remove obsolete framework packages, tests, and documentation;
- update Scriptorium examples and docs to describe the CLI and HTTP application;
- direct Go framework consumers to Promptkit without providing compatibility
aliases or forwarding APIs.
**Gate:** Scriptorium builds and passes all tests using a tagged Promptkit
dependency, contains no duplicate framework implementation, and its current
documentation describes only the slimmed application.
**Gate status:** Complete as of 2026-07-28. Scriptorium directly resolves
Promptkit `v0.1.0`, no longer contains the framework copy or public Go facade,
and retains only its application, adapter, configuration, presentation,
transport, packaging, and executable-example responsibilities. Release-grade
Scriptorium validation passed with a fresh remote dependency cache, and the
published Promptkit tag passed its documented validation independently. The
application is ready for the downstream-consumer migrations in Step 8.
### Step 8: Migrate Downstream Consumers To Promptkit
Inventory downstream Go consumers and migrate each from the Scriptorium package
to Promptkit. This work may occur in external repositories and must be tracked
explicitly.
For each consumer:
- update module imports and dependencies;
- adapt to any intentionally changed public API;
- run its tests and relevant integration or smoke checks;
- confirm configuration, source, validation, and error behavior;
- release or deploy the migrated consumer through its normal process.
Consumers that cannot migrate immediately may remain pinned to the last
framework-bearing Scriptorium tag. No compatibility work is required in the new
Scriptorium project for those consumers.
**Gate:** All in-scope downstream consumers are either migrated and verified or
explicitly recorded as remaining on the previous Scriptorium version with an
owner and follow-up plan. Do not declare the ecosystem migration complete until
the required out-of-band consumer changes are confirmed.
**Gate status:** Complete as of 2026-07-28. The maintainer confirmed that
Notarius was the only downstream consumer of Scriptorium's former Go package.
Its clean, synchronized main branch now directly requires Promptkit `v0.1.0`,
all relevant Go imports use Promptkit rather than Scriptorium, and its full Go
test suite passes. No downstream consumer remains to migrate or disposition.
### Step 9: Complete Release And Documentation Cutover
Complete the coordinated project transition:
- publish Promptkit before dependent Scriptorium releases;
- release the breaking Scriptorium version against the tagged Promptkit
dependency;
- publish migration guidance that maps the former Scriptorium Go API to
Promptkit;
- update cross-project links, examples, package documentation, and release
notes;
- verify that no release artifact depends on local workspaces or replacements;
- archive completed roadmap material according to the documentation policy in
effect at that time.
**Gate:** Promptkit and Scriptorium are independently releasable, their
documentation has distinct and accurate ownership, and the migration status of
all identified downstream consumers is recorded.
## Cross-Cutting Constraints
- Preserve the invariant that execution orchestration remains narrow and
application-neutral.
- Keep adapter-specific decisions out of Promptkit.
- Keep Scriptorium dependent only on Promptkit's supported public API.
- Preserve strict external decoding, error classification, validation
semantics, and secret redaction throughout the migration.
- Keep each repository buildable and testable at merged phase boundaries.
- Coordinate cross-repository changes through tagged dependencies and explicit
gates rather than assuming atomic commits across repositories.
- Document only implemented behavior outside roadmap files.
## Completion Criteria
The migration is complete when:
- Promptkit is the independent owner of the reusable framework and built-in
profiles;
- Scriptorium is a slim CLI and HTTP consumer of Promptkit;
- Scriptorium no longer exposes or maintains the former public Go framework;
- all required downstream migrations and external repository work have been
completed or explicitly dispositioned;
- both repositories build, test, document, version, and release independently.

View File

@@ -0,0 +1,351 @@
# Promptkit v0.9.0 Adoption Roadmap
## Status
Complete. This document records the delivered feature scope and targeted end
state. The completed delivery plan is maintained in
[implementation.md](implementation.md); current behavior remains defined by
the canonical application contracts linked below.
The durable product and integration boundary for this work is established by
[ADR 0004](../adr/0004-definition-boundary.md). This roadmap applies that
decision to Promptkit v0.9.0 and owns implementation scope and completion
status.
## Objective
Bring Scriptorium from Promptkit v0.1.0 to v0.9.0 and expose the newer
framework capabilities that fit Scriptorium's existing CLI, HTTP,
configuration, presentation, and process boundaries.
Scriptorium must accept the complete Promptkit v0.9.0 feature set expressed by
directory-backed prompt, profile, and schema definitions. Application adapters
must not narrow that format contract by requiring request values that the
selected definition does not require or by omitting selection values needed to
address a valid definition. Promptkit's tagged
[framework format reference](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.9.0/docs/formats.md)
remains canonical; Scriptorium must delegate parsing and framework validation
rather than reproduce those rules.
The completed application will retain Promptkit as the owner of reusable
prompt execution, source formats, profiles, backend definitions, capacity,
inspection, generation, and validation. Scriptorium will continue to own
application configuration, command and HTTP contracts, error and exit mapping,
output presentation, and deployment policy.
## Targeted End State
When this roadmap is complete:
- Scriptorium pins the released Promptkit v0.9.0 module without a workspace,
replacement, vendored source, direct catalog import, or Promptkit internal
import.
- Existing valid CLI, HTTP, configuration, prompt, profile, schema, and render
workflows remain supported subject to Promptkit's documented v0.2.0 through
v0.9.0 compatibility corrections.
- Every Promptkit v0.9.0 prompt and profile feature expressible in definitions
under Scriptorium's configured directory sources is usable through the
executable application, including prompts with no inputs and explicitly
selected prompt versions.
- CLI and HTTP requests can supply a direct non-secret session ID.
- HTTP requests and a presence-aware CLI flag can inherit, replace, or
explicitly clear reasoning effort.
- Prepared output, CLI run summaries, and HTTP result metadata expose effective
backend identity when Promptkit supplies one.
- HTTP capacity rejection has a stable Scriptorium-owned overload response.
- Application configuration can register reusable engine-scoped custom
backends, including bounded concurrency and queue policy, for selection by
Promptkit profiles.
- CLI users can inspect prompt and profile definitions without supplying
placeholder inputs or invoking a model.
- Scriptorium's contracts and examples point to Promptkit v0.9.0 and accurately
describe its current credential, validation-repair, provider-control,
message-role, backend, and source-validation semantics.
## Feature Scope
### Complete Prompt And Profile Definition Compatibility
Scriptorium will treat the tagged Promptkit v0.9.0 format contract as one
indivisible downstream compatibility boundary. This includes prompt identity
and version selection, optional and required input declarations, message and
content-file templates, supported roles, cache control, session templates,
default profiles, output contracts, schemas, and repair budgets. It also
includes standalone and inherited profiles, built-in and custom backend
selection, endpoint overrides, execution controls, optional credential
environment sources, and JSON-compatible extra parameters.
Most of this support is provided by passing configured prompt, profile, and
schema directories directly to Promptkit. Scriptorium will not independently
decode these framework definitions or maintain a field-level subset. Two
current adapter restrictions must be removed so the executable interfaces do
not reject definitions that Promptkit accepts:
- `run` and `render` will accept an optional prompt-version flag and map it to
`promptkit.RunRequest.PromptVersion`; and
- `run`, `render`, and `POST /v1/runs` will permit omitted or empty input maps.
Promptkit will decide whether an input is required by the selected
definition or referenced template.
The HTTP API already carries `prompt_version`; its mapping must remain covered
by adapter tests. CLI and HTTP callers may continue to supply extra inputs,
subject to their existing file, containment, and size policies. Removing the
application-level nonempty-input requirement does not weaken Promptkit's
declared-required-input or template-reference validation.
Profiles that name built-in backend IDs require no application registration.
Profiles that name other backend IDs become fully usable through the custom
backend configuration in this roadmap. Profile inheritance and all other
profile-field resolution remain Promptkit responsibilities.
This compatibility requirement concerns fields and behavior expressed in
Promptkit YAML definitions. Go-consumer construction alternatives such as
single-file sources, injected `fs.FS` sources, in-memory profiles, fallback
profile filesystems, and prepared-execution handles are library-integration
features and are not required merely to support the complete definition
format.
### Promptkit v0.9.0 Compatibility Baseline
Required dependency and source changes:
- update `go.mod` and `go.sum` to select Promptkit v0.9.0 and its selected
OpenRouter and Rakestrawhome catalog dependencies;
- change Scriptorium's inbound HTTP reasoning-effort representation to preserve
Promptkit's pointer semantics;
- retain keyed Promptkit public struct literals and confirm no removed
`RunRequest.Metadata` use exists;
- update tagged Promptkit links and version references throughout Scriptorium;
and
- preserve Scriptorium's architecture guard against replacements, workspaces,
vendored Promptkit code, former facade packages, and Promptkit internal
imports.
Required compatibility review:
- verify application and maintained example endpoints satisfy Promptkit's
absolute HTTP/HTTPS endpoint rules;
- verify prompt content paths, file artifacts, identities, JSON documents, and
JSON-compatible extra parameters satisfy the v0.6.0 safety boundaries;
- verify maintained prompt roles are limited to `developer`, `system`, `user`,
and `assistant`;
- verify every positive repair budget is no greater than three and is paired
with `basic`, `json`, or `json_schema` validation;
- document that positive repair budgets authorize additional provider calls,
latency, token use, and cost;
- document that unset optional provider controls are omitted and that a zero
effective value can represent an unspecified provider control unless an
explicit request override supplied it; and
- update credential guidance for Promptkit's optional environment lookup
behavior, profile `api_key_env` semantics, and Scriptorium's prohibition on
raw API-key inputs.
### Session And Reasoning Request Controls
The CLI `run` and `render` commands will accept a direct session-ID flag and map
it to `promptkit.RunRequest.SessionID`. The value remains non-secret
application correlation metadata and is subject to Promptkit's normalization
and length rules.
The HTTP run request will accept `session_id` and map it through the existing
strict DTO boundary. Successful HTTP metadata will report the effective
session ID when present.
The CLI `run` and `render` commands will also accept a presence-aware reasoning
effort flag. Omission inherits the selected profile, a nonblank value replaces
it, and an explicitly supplied empty value clears it. The existing HTTP
`model.reasoning_effort` field will gain the same three-state behavior while
remaining a JSON string when present.
These additions must not introduce raw direct API-key flags or fields.
### Backend Identity And Capacity Outcomes
Scriptorium presentation will expose `SelectedBackendID` and the effective
target's `BackendID` without deriving identity from endpoint text. Backend
identity will be included where applicable in:
- prepared-run text and JSON output;
- the CLI run summary; and
- HTTP response metadata and effective model parameters.
Endpoint-only profiles continue to have no backend ID, and empty identities
must remain distinguishable from registered built-in or custom backends.
The HTTP adapter will classify `promptkit.ErrCapacityExceeded` separately from
provider generation failures. The target public outcome is HTTP `503` with a
stable `capacity_exceeded` code and a generic message. Scriptorium will not
invent retry timing or expose an untrusted diagnostic. CLI capacity rejection
continues to be a runtime failure, with a clear safe diagnostic and the
existing runtime-error exit status.
The HTTP server will use one appropriately scoped Promptkit engine across its
requests so backend admission, active-generation limits, and queue capacity
apply across concurrent in-flight work. The server may retain the operational
state needed for those limits and for request cancellation and lifecycle
management. This does not create durable per-request, conversation, or resume
state.
Scriptorium will continue to return generic public model-generation failures.
Promptkit `GenerationError` provider code, type, and message values will not be
added to the public HTTP response because they are untrusted and potentially
sensitive.
### Custom Backend Configuration
The strict Scriptorium application configuration will gain an optional custom
backend collection. Each entry will support the application-owned mapping
needed to construct a public `promptkit.Backend`:
- backend ID;
- OpenAI-compatible endpoint;
- optional API-key environment-variable name;
- optional JSON-compatible request-wide extra parameters;
- non-negative concurrency limit; and
- queue capacity with presence preserved so omission and explicit zero remain
different.
Raw API-key values will remain invalid. Unknown fields will remain errors.
Configured backends will be registered through `promptkit.WithBackend` during
engine construction and will be immutable and engine-scoped. Promptkit will
remain the canonical validator for backend IDs, endpoints, reserved request
parameters, capacity bounds, duplicate or reserved IDs, environment-variable
names, and JSON-compatible values. Scriptorium will add context appropriate to
its configuration error boundary without copying those framework rules.
The collection will be configuration-file-owned. This roadmap does not add
per-backend CLI flags. Existing endpoint-only profiles remain valid, while
profiles may select configured custom backend IDs through Promptkit's existing
`backend` field. Promptkit's built-in OpenRouter and Rakestrawhome backends and
profiles remain available without Scriptorium registration.
The application configuration shape is a `backends` mapping whose keys are the
backend IDs registered with Promptkit:
```yaml
backends:
local-gpu:
endpoint: http://localhost:11434/v1
api_key_env: LOCAL_GPU_API_KEY
extra_params:
provider_option: enabled
concurrency_limit: 2
queue_capacity: 0
```
`endpoint` is required for each entry. The other fields are optional. Omitted
`queue_capacity` uses Promptkit's default queue policy when concurrency is
bounded, while an explicit zero disables queuing. The configuration contract
and complete maintained configuration example will document this shape when
the feature is implemented.
### Prompt And Profile Inspection CLI
Scriptorium will add a CLI inspection command with prompt and profile modes.
The intended command family is:
- `scriptorium inspect prompt`, backed by `Engine.InspectPrompt`; and
- `scriptorium inspect profile`, backed by `Engine.InspectProfile`.
Prompt inspection will select an ID and optional version and report the
normalized prompt identity, opaque prompt hash, declared default profile,
declared inputs, and normalized output contract. It will not require input
artifacts, resolve a profile, load a schema, render templates, reserve backend
capacity, or contact a model.
Profile inspection will select an explicit profile ID and report its resolved
effective target, backend identity, optional credential source name, and
whether a later request must provide credential configuration. It will not
read credential values, load a prompt, reserve capacity, or contact a model.
Inspection will support deterministic Scriptorium-owned text and JSON output
and normal output-file handling. Scriptorium will define explicit output DTOs
rather than treating Promptkit inspection structs as stable wire formats.
Inspection failures will use existing public Promptkit error identities and
Scriptorium-owned CLI diagnostics and exit behavior.
The prompt form accepts `--prompt`, optional `--prompt-version`, `--config`,
`--prompt-dir`, `--format text|json`, and `--out`. The profile form accepts
`--profile`, `--config`, `--profile-dir`, `--format text|json`, and `--out`.
Both forms reject positional arguments and default to text output independently
of `defaults.render_format`. Configuration discovery and CLI-over-file source
precedence match the existing command family. Prompt inspection requires an
effective prompt directory; profile inspection does not, because Promptkit's
built-in profiles remain inspectable without one.
## Documentation And Example Changes
Implementation of this roadmap requires coordinated updates to the canonical
owners of affected behavior:
- `docs/cli.md` for session and reasoning flags, inspection commands, output,
prompt-version selection, optional inputs, and exit behavior;
- `docs/api.md` for `session_id`, effective backend/session metadata, reasoning
presence semantics, optional inputs, and the capacity response;
- `docs/config.md` for custom backend configuration and current Promptkit
credential and provider-default semantics;
- `docs/operations.md` for capacity and custom-backend operational guidance
where deployment handling is affected;
- `docs/internal/adapters.md` and `docs/internal/sources.md` for implemented
mapping and engine-assembly behavior;
- `docs/policy/architecture.md` and `docs/internal/overview.md` only if concrete
component responsibilities or durable boundaries change; and
- maintained examples for valid v0.9.0 prompt/profile formats and custom
backend configuration.
Current-state documents must not describe these features as implemented until
the corresponding code lands.
## Validation And Completion Criteria
The roadmap is complete only when:
- ordinary and race-enabled tests pass against the tagged Promptkit v0.9.0
dependency;
- `go vet ./...` and `go build ./cmd/scriptorium` pass;
- Go formatting and `git diff --check` pass;
- the architecture guard passes with no workspace, replacement, vendor tree,
former facade, or Promptkit internal import;
- maintained render, custom-backend, session, reasoning, and inspection
examples execute successfully without real credentials or provider calls;
- CLI tests cover explicit and omitted prompt versions for prompt IDs with one
or multiple definitions;
- CLI and HTTP tests cover prompts with no declared inputs, omitted optional
inputs, missing required inputs, and template-referenced inputs;
- HTTP tests cover omitted, replacement, and clearing reasoning states;
- CLI and HTTP tests cover session propagation and invalid session handling;
- presentation tests cover registered and endpoint-only backend identity;
- HTTP tests cover capacity rejection independently from provider failure;
- concurrent HTTP tests demonstrate that one server's requests share backend
admission and cannot bypass capacity through per-request engine creation;
- configuration tests cover strict decoding, precedence, queue-capacity
presence, invalid backends, and secret-field rejection;
- inspection tests cover deterministic text and JSON, source selection,
output-file handling, safe credential presentation, and public error
mapping;
- all maintained prompt roles and repair budgets are valid under Promptkit
v0.9.0; and
- representative definition fixtures cover inline and file-backed messages,
cache control, session templates, every output validation mode, schemas,
profile inheritance, built-in and custom backends, execution controls,
credential environment names, and extra parameters without Scriptorium
independently parsing those fields; and
- affected documentation links and copyable examples validate successfully.
## Explicit Non-Goals
This roadmap does not include:
- appended request messages in either executable interface;
- retained or durable prepared-execution handles;
- HTTP prompt or profile inspection routes;
- embedded application fallback profiles;
- single-file, injected `fs.FS`, or in-memory prompt and profile source
configuration;
- request-level output-contract replacement through the executable interfaces;
- public disclosure of structured provider error details;
- direct imports or configuration of Promptkit's external catalog modules;
- raw API-key configuration, CLI flags, or HTTP fields;
- durable run state, conversations, retries, archives, checkpoints, or resume;
or
- changes to Promptkit itself.

View File

@@ -1,282 +0,0 @@
# Migration Step 9: Complete Release And Documentation Cutover
## Status
Proposed. Steps 1 through 8 of the
[migration roadmap](migration.md) are complete. This is the final migration
gate.
## Purpose
Finish the Promptkit split as a released, documented, and independently
maintainable project boundary. Step 9 turns the already-implemented repository
state into the supported public release state, gives former Scriptorium Go
consumers a durable migration path, reconciles release guidance in both
repositories, and retires the temporary migration records once their work is
complete.
This roadmap defines the intended end state.
## Release Baseline And Version Decisions
The coordinated release boundary is:
- Promptkit `v0.1.0`, already published from commit
`9e68a2bbf779545995270c47842048a3bc6c85dc`, is the framework version consumed
by Scriptorium;
- Scriptorium `v0.11.1` is the final published framework-bearing Scriptorium
release; and
- Scriptorium `v0.12.0` is the first slim application-only release.
The `v0.12.0` version satisfies
[ADR 0002](../adr/0002-split-promptkit-from-scriptorium.md), which requires
the first slim pre-`v1` Scriptorium release to advance the minor version beyond
the final framework-bearing release.
Promptkit does not need another tag merely to complete this migration.
Promptkit documentation corrections that do not change the library contract
may land on its main branch without changing Scriptorium's dependency. If Step
9 discovers that a Promptkit code or consumer-visible contract change is
required, Promptkit must instead publish an appropriate later semantic version
first, and Scriptorium must adopt and validate that tag before `v0.12.0` is
published.
No release may depend on a Go workspace, local module replacement, vendored
sibling source, unpublished commit, or an unpushed tag.
## Scriptorium Release Readiness
Scriptorium must have a durable, canonical release procedure appropriate to
its hosted tag-triggered binary workflow. The procedure and contributor
reading guide must collectively define:
- semantic-version selection and the clean-checkout preconditions;
- validation outside a Go workspace and without a module replacement;
- module identity, dependency resolution, module tidiness, tests, race tests,
vet, formatting, link, example, and repository-hygiene checks;
- release-equivalent Linux `amd64` and `arm64` builds;
- annotated tag creation and publication;
- observation of the hosted release workflow;
- verification of published binaries and checksums; and
- post-publication smoke checks using downloaded release artifacts where the
execution platform permits.
The documentation policy must assign the release procedure one canonical
owner, and `docs/development.md` must route release work to it. Operations
documentation should link to release material only when an operator task
requires it; it must not become a second release procedure.
The tag workflow must accurately describe the current application. In
particular, remove the obsolete linker assignment to the deleted
`internal/buildinfo.Version` symbol. The release continues to use stripped,
trimmed binaries unless a separate supported application-version interface is
introduced. Adding a new `--version` command or other product behavior is not
part of this migration.
Release validation must exercise the same build commands and artifact names as
the hosted workflow. The resulting `v0.12.0` release must contain the supported
Linux `amd64` and `arm64` Scriptorium binaries and published SHA-256 checksums,
with no framework source or Promptkit binary artifact.
## Promptkit Release Readiness
Promptkit remains governed by
[ADR 0003](../adr/0003-use-maintainer-run-validation-and-tag-only-releases-for-promptkit.md):
maintainer-run validation, semantic Go module tags, and no hosted CI or binary
release artifacts.
Its release procedure must be corrected from pre-release language to current,
reusable guidance. It must no longer describe `v0.1.0` as an uncreated planned
release or instruct a maintainer to create an existing tag. It should:
- acknowledge `v0.1.0` as the initial published release;
- use version-agnostic instructions for later releases;
- retain the clean-checkout, full-validation, tag-ordering, and remote-tag
verification requirements; and
- require pre-`v1` release notes to identify public API changes and consumer
migration requirements.
Before Scriptorium `v0.12.0` is tagged, independently reconfirm that the local
and remote Promptkit `v0.1.0` tags resolve to the expected source commit, that
the tagged module is available through ordinary Go module resolution, and that
Promptkit passes its documented release validation without Scriptorium or
sibling-repository state.
## Go Consumer Migration Guidance
Scriptorium must publish a permanent migration guide under `docs/consumers/`
for consumers of the former Go package. Scriptorium owns this guide because it
describes departure from Scriptorium's removed API; Promptkit's declarations,
GoDoc, format reference, and consumer guide remain canonical for the
destination contract.
The guide must identify the supported migration baseline:
- source: Scriptorium `v0.11.1` and import path
`gitea.maximumdirect.net/eric/scriptorium`;
- destination: Promptkit `v0.1.0` and import path
`gitea.maximumdirect.net/eric/promptkit`; and
- Scriptorium `v0.12.0` and later: executable application only, with no root Go
package or compatibility facade.
It must provide a minimal, copyable migration workflow:
1. replace the Scriptorium module requirement and Go imports with Promptkit
`v0.1.0`;
2. update package qualifiers from `scriptorium` to `promptkit`;
3. run `go mod tidy`;
4. compile and test the consuming project; and
5. verify prompt, profile, schema, credential, timeout, validation, injected
client, and error-handling behavior relevant to that consumer.
The guide should explain that the established engine, request, result, profile,
source-option, model-client, artifact, validation, and error shapes were
intentionally preserved where practical, while Promptkit also owns the
post-extraction public error identities and artifact-reader extension point.
It must direct exact API questions to Promptkit's tagged GoDoc and consumer
guide rather than duplicating the declaration reference.
The guide must also state the deliberate compatibility policy: there are no
Scriptorium aliases, forwarding packages, or deprecated wrappers. A consumer
that cannot migrate may remain pinned to `v0.11.1`, but it will not receive the
application-only Scriptorium line through that package API.
## Documentation And Project Identity Cutover
Review both repositories as separate products and reconcile every maintained
link, example, package comment, and current-state statement with the released
boundary.
Scriptorium documentation must:
- present Scriptorium as a CLI and HTTP application, not a Go framework;
- link in-process Go consumers and framework contract questions to tagged
Promptkit `v0.1.0` documentation;
- link former Scriptorium Go consumers to the migration guide;
- keep CLI, HTTP, application configuration, operations, subprocess, and
executable examples under Scriptorium ownership; and
- avoid reproducing Promptkit fields, defaults, public declarations, or
integration contracts.
Promptkit documentation must:
- present Promptkit as the reusable Go framework and owner of its root API,
file formats, built-in profiles, validation, and outbound integration;
- retain Scriptorium only as a downstream application example or related
project, not as a framework owner or dependency;
- link to Scriptorium only for executable CLI and HTTP workflows when that
navigation is useful; and
- contain no stale extraction, planned-first-release, or pre-cutover claims.
Cross-project links must point to the canonical owner. Scriptorium links that
define the framework version it consumes remain pinned to Promptkit `v0.1.0`;
general project-navigation links may point to the other repository's current
project entry point. Maintained examples must stay repository-local and must
not require a sibling checkout.
## Release Notes And Public Communication
The Scriptorium `v0.12.0` release notes must clearly identify the release as a
breaking project-boundary change. They must:
- state that Scriptorium is now an executable-only CLI and HTTP application;
- state that the former Go framework moved to Promptkit;
- link the Scriptorium migration guide and Promptkit `v0.1.0` consumer
documentation;
- identify `v0.11.1` as the final framework-bearing Scriptorium release;
- summarize the retained Scriptorium interfaces and the removed root package;
- record that all known downstream Go consumers were migrated before release;
and
- identify the Promptkit version used by the released binary.
Release notes must not serve as a duplicate CLI, HTTP, configuration, or
Promptkit API reference. They should route readers to the corresponding
canonical documents.
## Independent Release And Artifact Verification
The final acceptance run must treat the repositories as independent remote
projects:
- validate Promptkit from its exact published tag without Scriptorium;
- validate Scriptorium from its intended release commit outside any workspace
and with a fresh module and build cache that cannot read the sibling
Promptkit checkout;
- confirm the Scriptorium module graph selects the intended published Promptkit
tag;
- verify both working trees contain no tracked workspace, replacement, vendored
cross-project source, generated binary, credential, or temporary release
residue;
- publish and verify the annotated Scriptorium `v0.12.0` tag;
- verify the hosted release completes and publishes the expected binaries and
checksums;
- download the published artifacts into a temporary location, verify their
checksums, file types, target architectures, and basic executable behavior;
and
- recheck maintained local and cross-project documentation links after
publication.
Ordinary ignored developer files, including an ignored local Scriptorium
binary, do not fail repository hygiene. Acceptance concerns tracked content,
release inputs, generated files introduced by the release work, and published
artifacts.
## Roadmap Retirement
Roadmaps are temporary coordination documents. After every Step 9 completion
criterion is satisfied and durable release and migration records exist:
- mark Step 9 and the overall migration complete before cleanup;
- preserve any still-useful current contract in its canonical permanent owner;
- rely on ADRs, Git history, tags, release notes, and the migration guide for
durable decision and release history;
- remove completed migration, step, and implementation roadmaps rather than
retaining them as a second current-state reference; and
- repair every incoming link affected by that removal.
The roadmap files must remain until the out-of-band tag and hosted release have
been verified. Creating a release candidate or merging documentation is not
enough to declare the migration complete.
## Non-Goals
Step 9 does not:
- redesign Promptkit's public API or Scriptorium's CLI or HTTP contracts;
- restore a Scriptorium Go facade or add compatibility shims;
- add hosted CI or binary artifacts to Promptkit;
- add new Scriptorium target platforms beyond the existing Linux `amd64` and
`arm64` release policy;
- introduce an application version command solely to preserve a stale linker
flag;
- redo the completed downstream migration inventory; or
- require a new Promptkit release when no Promptkit contract change is needed.
## Completion Criteria
Step 9 is complete only when all of the following are true:
- Promptkit `v0.1.0` remains independently available, validated, and correctly
documented as the published framework dependency;
- Scriptorium has an accurate, canonical, and tested release procedure;
- the Scriptorium release workflow contains no reference to removed framework
or build-information packages and produces only the intended application
artifacts;
- the permanent Go-consumer migration guide is complete, copyable, and linked
from appropriate Scriptorium entry points;
- both repositories' permanent documentation, examples, package comments, and
cross-project links reflect distinct and canonical ownership;
- Scriptorium `v0.12.0` is published from a clean, independently validated
commit that directly requires a published Promptkit tag;
- the hosted Scriptorium release publishes verified Linux `amd64` and `arm64`
binaries and SHA-256 checksums;
- the `v0.12.0` release notes communicate the breaking package move and link to
the migration path;
- all known downstream Go consumers remain migrated or explicitly
dispositioned;
- neither release depends on local multi-repository state; and
- completed migration roadmaps are removed after their useful content and
completion evidence have durable owners.
When these criteria are satisfied, the Promptkit split is complete and both
projects can evolve, validate, version, document, and release independently.

View File

@@ -2,6 +2,15 @@ prompt_dir: ./examples/prompts
profile_dir: ./examples/profiles
schema_dir: ./examples/schemas
backends:
local-gpu:
endpoint: http://localhost:11434/v1
api_key_env: LOCAL_GPU_API_KEY
extra_params:
provider_option: enabled
concurrency_limit: 2
queue_capacity: 0
server:
addr: 127.0.0.1:8080
artifact_root: .

View File

@@ -0,0 +1,4 @@
id: local-gpu
backend: local-gpu
model: local-model
reasoning_effort: low

View File

@@ -0,0 +1,25 @@
#!/usr/bin/env bash
set -euo pipefail
repo_root="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
cd "$repo_root"
go run ./cmd/scriptorium render \
--config ./examples/config.full.yml \
--prompt generic.markdown_summary \
--prompt-version 1.0.0 \
--profile local-gpu \
--session-id example-session \
--reasoning-effort= \
--input transcript=./examples/fixtures/transcript.md \
--input glossary=./examples/fixtures/glossary.yml
go run ./cmd/scriptorium inspect prompt \
--config ./examples/config.full.yml \
--prompt generic.markdown_summary \
--format json
go run ./cmd/scriptorium inspect profile \
--config ./examples/config.full.yml \
--profile local-gpu \
--format json

4
go.mod
View File

@@ -3,11 +3,13 @@ module gitea.maximumdirect.net/eric/scriptorium
go 1.25.5
require (
gitea.maximumdirect.net/eric/promptkit v0.1.0
gitea.maximumdirect.net/eric/promptkit v0.9.0
gopkg.in/yaml.v3 v3.0.1
)
require (
gitea.maximumdirect.net/eric/promptkit-backend-openrouter v1.0.0 // indirect
gitea.maximumdirect.net/eric/promptkit-backend-rakestrawhome v1.0.0 // indirect
github.com/santhosh-tekuri/jsonschema/v6 v6.0.2 // indirect
golang.org/x/text v0.14.0 // indirect
)

8
go.sum
View File

@@ -1,5 +1,9 @@
gitea.maximumdirect.net/eric/promptkit v0.1.0 h1:vuKeBxkiY8E54LRFbLQFjlJJCiOfMvB1++DYBCrD/ug=
gitea.maximumdirect.net/eric/promptkit v0.1.0/go.mod h1:R95NM6fbMDGDC0/UomgnSBP6ui2ns+8SZb8bESNvrDQ=
gitea.maximumdirect.net/eric/promptkit v0.9.0 h1:IpvDRC8L6xRxQ9hpuyKOmMc5b6MeLTKYyx+h1YAjy08=
gitea.maximumdirect.net/eric/promptkit v0.9.0/go.mod h1:oMJ/WUJImUtwJ5e+6MAGECPYAErAkOaKel0G+3T/b4E=
gitea.maximumdirect.net/eric/promptkit-backend-openrouter v1.0.0 h1:lc062euk2qseO//D762i3JaFyulDNML3eQQX7DkYTho=
gitea.maximumdirect.net/eric/promptkit-backend-openrouter v1.0.0/go.mod h1:AIa7kAu2mfrRQgcspe4L+DW51WqgnALQT60lqkEywJI=
gitea.maximumdirect.net/eric/promptkit-backend-rakestrawhome v1.0.0 h1:j9YY7wsTVjzke2kHH4YAzpU0oUpM+x+nXwl1IeS+2eg=
gitea.maximumdirect.net/eric/promptkit-backend-rakestrawhome v1.0.0/go.mod h1:4RNS+LILDg4JbS4Ts9Lwy1C92wauXJIbeQaalps4Koo=
github.com/dlclark/regexp2 v1.11.0 h1:G/nrcoOa7ZXlpoa/91N3X7mM3r8eIlMBBJZvsz/mxKI=
github.com/dlclark/regexp2 v1.11.0/go.mod h1:DHkYz0B9wPfa6wondMfaivmHpzrQ3v9q8cnmRbL6yW8=
github.com/santhosh-tekuri/jsonschema/v6 v6.0.2 h1:KRzFb2m7YtdldCEkzs6KqmJw4nqEVZGK7IN2kJkjTuQ=

View File

@@ -0,0 +1,200 @@
package cli
import (
"context"
"errors"
"flag"
"fmt"
"io"
"io/fs"
"path/filepath"
"strings"
"gitea.maximumdirect.net/eric/promptkit"
appconfig "gitea.maximumdirect.net/eric/scriptorium/internal/config"
appformat "gitea.maximumdirect.net/eric/scriptorium/internal/format"
)
type promptInspectionConfig struct {
configPath, promptDir, promptID, promptVersion, outputPath string
configExplicit bool
outputFormat appformat.OutputFormat
}
type profileInspectionConfig struct {
configPath, profileDir, profileID, outputPath string
configExplicit bool
outputFormat appformat.OutputFormat
}
// emptyPromptDefinitionFS satisfies Promptkit's engine-level prompt-source
// requirement without exposing the caller's working directory. Profile
// inspection never reads this source.
type emptyPromptDefinitionFS struct{}
func (emptyPromptDefinitionFS) Open(name string) (fs.File, error) {
return nil, &fs.PathError{Op: "open", Path: name, Err: fs.ErrNotExist}
}
func inspectCommand(args []string, stdout, stderr io.Writer) int {
if len(args) == 0 {
fmt.Fprintln(stderr, "inspect parse error: inspection mode is required")
return ExitRuntimeError
}
if args[0] == "profile" {
return inspectProfileCommand(args[1:], stdout, stderr)
}
if args[0] != "prompt" {
fmt.Fprintln(stderr, "inspect parse error: unknown inspection mode")
return ExitRuntimeError
}
cfg, err := parsePromptInspectionArgs(args[1:])
if err != nil {
fmt.Fprintf(stderr, "inspect parse error: %v\n", err)
return ExitRuntimeError
}
settings, err := resolveAppSettingsForPromptInspection(cfg)
if err != nil {
fmt.Fprintf(stderr, "inspect error: %v\n", err)
return ExitRuntimeError
}
engine, err := newEngine(settings)
if err != nil {
fmt.Fprintf(stderr, "engine error: %v\n", err)
return ExitRuntimeError
}
inspection, err := engine.InspectPrompt(context.Background(), cfg.promptID, cfg.promptVersion)
if err != nil {
fmt.Fprintf(stderr, "inspect error: %v\n", err)
return ExitRuntimeError
}
data, err := appformat.FormatPromptInspection(inspection, cfg.outputFormat)
if err != nil {
fmt.Fprintf(stderr, "inspect error: %v\n", err)
return ExitRuntimeError
}
if err := writeOutput(stdout, cfg.outputPath, data); err != nil {
fmt.Fprintf(stderr, "output write error: %v\n", err)
return ExitRuntimeError
}
return ExitOK
}
func inspectProfileCommand(args []string, stdout, stderr io.Writer) int {
cfg, err := parseProfileInspectionArgs(args)
if err != nil {
fmt.Fprintf(stderr, "inspect parse error: %v\n", err)
return ExitRuntimeError
}
settings, err := resolveAppSettingsForProfileInspection(cfg)
if err != nil {
fmt.Fprintf(stderr, "inspect error: %v\n", err)
return ExitRuntimeError
}
engine, err := newEngine(settings, promptkit.WithPromptFS(emptyPromptDefinitionFS{}, "."))
if err != nil {
fmt.Fprintf(stderr, "engine error: %v\n", err)
return ExitRuntimeError
}
inspection, err := engine.InspectProfile(context.Background(), cfg.profileID)
if err != nil {
fmt.Fprintf(stderr, "inspect error: %v\n", err)
return ExitRuntimeError
}
data, err := appformat.FormatProfileInspection(inspection, cfg.outputFormat)
if err != nil {
fmt.Fprintf(stderr, "inspect error: %v\n", err)
return ExitRuntimeError
}
if err := writeOutput(stdout, cfg.outputPath, data); err != nil {
fmt.Fprintf(stderr, "output write error: %v\n", err)
return ExitRuntimeError
}
return ExitOK
}
func parseProfileInspectionArgs(args []string) (*profileInspectionConfig, error) {
cfg := &profileInspectionConfig{outputFormat: appformat.DefaultOutputFormat}
fs := flag.NewFlagSet("inspect profile", flag.ContinueOnError)
fs.SetOutput(io.Discard)
registerConfigPathFlag(fs, &cfg.configPath)
fs.StringVar(&cfg.profileDir, "profile-dir", "", "directory containing execution profiles")
fs.StringVar(&cfg.profileID, "profile", "", "profile ID to inspect")
rawFormat := ""
fs.StringVar(&rawFormat, "format", "", "output format: text or json")
fs.StringVar(&cfg.outputPath, "out", "", "optional output file path")
if err := fs.Parse(args); err != nil {
return nil, err
}
cfg.configExplicit = flagWasSet(fs, "config")
if fs.NArg() > 0 {
return nil, fmt.Errorf("unexpected positional args: %v", fs.Args())
}
if strings.TrimSpace(cfg.profileID) == "" {
return nil, errors.New("--profile is required")
}
format, err := appformat.ParseOutputFormat(rawFormat)
if err != nil {
return nil, err
}
cfg.outputFormat = format
if cfg.outputPath != "" {
cfg.outputPath = filepath.Clean(cfg.outputPath)
}
return cfg, nil
}
func resolveAppSettingsForProfileInspection(cfg *profileInspectionConfig) (engineSettings, error) {
settings, err := resolveAppSettingsWithConfigPresence(cfg.configPath, cfg.configExplicit, appconfig.CLIOverrides{
ProfileDir: cfg.profileDir,
})
if err != nil {
return engineSettings{}, err
}
return engineSettings{profileDir: settings.ProfileDir, backends: settings.Backends}, nil
}
func parsePromptInspectionArgs(args []string) (*promptInspectionConfig, error) {
cfg := &promptInspectionConfig{outputFormat: appformat.DefaultOutputFormat}
fs := flag.NewFlagSet("inspect prompt", flag.ContinueOnError)
fs.SetOutput(io.Discard)
registerConfigPathFlag(fs, &cfg.configPath)
fs.StringVar(&cfg.promptDir, "prompt-dir", "", "directory containing prompt definitions")
fs.StringVar(&cfg.promptID, "prompt", "", "prompt ID to inspect")
fs.StringVar(&cfg.promptVersion, "prompt-version", "", "optional prompt version")
rawFormat := ""
fs.StringVar(&rawFormat, "format", "", "output format: text or json")
fs.StringVar(&cfg.outputPath, "out", "", "optional output file path")
if err := fs.Parse(args); err != nil {
return nil, err
}
cfg.configExplicit = flagWasSet(fs, "config")
if fs.NArg() > 0 {
return nil, fmt.Errorf("unexpected positional args: %v", fs.Args())
}
if strings.TrimSpace(cfg.promptID) == "" {
return nil, errors.New("--prompt is required")
}
format, err := appformat.ParseOutputFormat(rawFormat)
if err != nil {
return nil, err
}
cfg.outputFormat = format
if cfg.outputPath != "" {
cfg.outputPath = filepath.Clean(cfg.outputPath)
}
return cfg, nil
}
func resolveAppSettingsForPromptInspection(cfg *promptInspectionConfig) (engineSettings, error) {
settings, err := resolveAppSettingsWithConfigPresence(cfg.configPath, cfg.configExplicit, appconfig.CLIOverrides{
PromptDir: cfg.promptDir,
})
if err != nil {
return engineSettings{}, err
}
if strings.TrimSpace(settings.PromptDir) == "" {
return engineSettings{}, errors.New(errPromptDirRequired)
}
return engineSettings{promptDir: settings.PromptDir, backends: settings.Backends}, nil
}

View File

@@ -35,7 +35,9 @@ type runConfig struct {
promptDir string
profileDir string
promptID string
promptVersion string
profileID string
sessionID string
inputRaw listFlag
varRaw listFlag
outputPath string
@@ -45,7 +47,9 @@ type runConfig struct {
temperature float64
maxTokens int
topP float64
reasoningEffort string
schemaDir string
backends []appconfig.BackendSettings
timeout time.Duration
defaultRenderFormat renderformat.PreparedRunOutputFormat
@@ -56,6 +60,7 @@ type runConfig struct {
temperatureSet bool
maxTokensSet bool
topPSet bool
reasoningEffortSet bool
timeoutSet bool
}
@@ -75,6 +80,7 @@ type serveConfig struct {
maxRequestBytes int64
maxArtifactBytes int64
maxResponseBytes int64
backends []appconfig.BackendSettings
}
type commonCommandSettings struct {
@@ -87,6 +93,7 @@ type commonCommandSettings struct {
maxArtifactBytes int64
maxResponseBytes int64
defaultRenderFormat renderformat.PreparedRunOutputFormat
backends []appconfig.BackendSettings
}
type listFlag []string
@@ -113,6 +120,8 @@ func Run(args []string, stdout, stderr io.Writer) int {
return renderCommand(args[1:], stdout, stderr)
case "serve":
return serveCommand(args[1:], stderr)
case "inspect":
return inspectCommand(args[1:], stdout, stderr)
default:
fmt.Fprintf(stderr, "unknown command %q\n", args[0])
printUsage(stderr)
@@ -133,7 +142,7 @@ func runCommand(args []string, stdout, stderr io.Writer) int {
return ExitRuntimeError
}
engine, err := newEngine(cfg)
engine, err := newEngine(cfg.engineSettings())
if err != nil {
fmt.Fprintf(stderr, "engine error: %v\n", err)
return ExitRuntimeError
@@ -141,7 +150,7 @@ func runCommand(args []string, stdout, stderr io.Writer) int {
res, runErr := engine.Run(context.Background(), req)
if runErr != nil {
fmt.Fprintf(stderr, "run error: %v\n", runErr)
fmt.Fprintln(stderr, runErrorMessage(runErr))
return ExitRuntimeError
}
@@ -167,7 +176,7 @@ func renderCommand(args []string, stdout, stderr io.Writer) int {
return ExitRuntimeError
}
engine, err := newEngine(&cfg.runConfig)
engine, err := newEngine(cfg.runConfig.engineSettings())
if err != nil {
fmt.Fprintf(stderr, "engine error: %v\n", err)
return ExitRuntimeError
@@ -205,11 +214,7 @@ func serveCommand(args []string, stderr io.Writer) int {
return ExitRuntimeError
}
engine, err := newEngine(&runConfig{
promptDir: cfg.promptDir,
profileDir: cfg.profileDir,
schemaDir: cfg.schemaDir,
}, promptkit.WithArtifactReader(artifactReader))
engine, err := newEngine(cfg.engineSettings(), promptkit.WithArtifactReader(artifactReader))
if err != nil {
fmt.Fprintf(stderr, "engine error: %v\n", err)
return ExitRuntimeError
@@ -329,6 +334,7 @@ func parseServeArgs(args []string) (*serveConfig, error) {
cfg.maxRequestBytes = settings.maxRequestBytes
cfg.maxArtifactBytes = settings.maxArtifactBytes
cfg.maxResponseBytes = settings.maxResponseBytes
cfg.backends = settings.backends
if err := validateRequiredLibraryDirs(cfg.promptDir); err != nil {
return nil, err
@@ -349,7 +355,9 @@ func registerExecutionRequestFlags(fs *flag.FlagSet, cfg *runConfig) {
fs.StringVar(&cfg.promptDir, "prompt-dir", "", "directory containing prompt definition YAML files")
fs.StringVar(&cfg.profileDir, "profile-dir", "", "directory containing execution profile YAML files")
fs.StringVar(&cfg.promptID, "prompt", "", "prompt ID to run")
fs.StringVar(&cfg.promptVersion, "prompt-version", "", "optional prompt definition version")
fs.StringVar(&cfg.profileID, "profile", "", "optional execution profile ID; if omitted, prompt default_profile is used")
fs.StringVar(&cfg.sessionID, "session-id", "", "optional session ID")
fs.Var(&cfg.inputRaw, "input", "input mapping(s): name=path (repeatable, comma-separated)")
fs.Var(&cfg.varRaw, "var", "variable mapping(s): name=value (repeatable, comma-separated)")
fs.StringVar(&cfg.outputPath, "out", "", "optional output file path")
@@ -359,6 +367,7 @@ func registerExecutionRequestFlags(fs *flag.FlagSet, cfg *runConfig) {
fs.Float64Var(&cfg.temperature, "temperature", 0, "optional temperature override")
fs.IntVar(&cfg.maxTokens, "max-tokens", 0, "optional max tokens override")
fs.Float64Var(&cfg.topP, "top-p", 0, "optional top_p override")
fs.StringVar(&cfg.reasoningEffort, "reasoning-effort", "", "optional reasoning effort override")
fs.DurationVar(&cfg.timeout, "timeout", 0, "LLM request timeout")
fs.StringVar(&cfg.promptID, "prompt-id", "", "deprecated alias for --prompt")
fs.StringVar(&cfg.profileID, "profile-id", "", "deprecated alias for --profile")
@@ -382,6 +391,7 @@ func finalizeExecutionRequestConfig(fs *flag.FlagSet, cfg *runConfig) error {
cfg.profileDir = settings.profileDir
cfg.schemaDir = settings.schemaDir
cfg.defaultRenderFormat = settings.defaultRenderFormat
cfg.backends = settings.backends
if err := validateRequiredLibraryDirs(cfg.promptDir); err != nil {
return err
@@ -389,9 +399,6 @@ func finalizeExecutionRequestConfig(fs *flag.FlagSet, cfg *runConfig) error {
if strings.TrimSpace(cfg.promptID) == "" {
return errors.New("--prompt is required")
}
if len(cfg.inputRaw) == 0 {
return errors.New("at least one --input is required")
}
cfg.promptDir = filepath.Clean(cfg.promptDir)
if strings.TrimSpace(cfg.profileDir) != "" {
cfg.profileDir = filepath.Clean(cfg.profileDir)
@@ -405,6 +412,7 @@ func finalizeExecutionRequestConfig(fs *flag.FlagSet, cfg *runConfig) error {
cfg.temperatureSet = flagWasSet(fs, "temperature")
cfg.maxTokensSet = flagWasSet(fs, "max-tokens")
cfg.topPSet = flagWasSet(fs, "top-p")
cfg.reasoningEffortSet = flagWasSet(fs, "reasoning-effort")
cfg.timeoutSet = flagWasSet(fs, "timeout")
return nil
}
@@ -500,7 +508,11 @@ func registerConfigPathFlag(fs *flag.FlagSet, target *string) {
}
func resolveAppSettings(fs *flag.FlagSet, configPath string, overrides appconfig.CLIOverrides) (appconfig.AppSettings, error) {
settings, err := appconfig.LoadConfig(configPath, flagWasSet(fs, "config"))
return resolveAppSettingsWithConfigPresence(configPath, flagWasSet(fs, "config"), overrides)
}
func resolveAppSettingsWithConfigPresence(configPath string, configExplicit bool, overrides appconfig.CLIOverrides) (appconfig.AppSettings, error) {
settings, err := appconfig.LoadConfig(configPath, configExplicit)
if err != nil {
return appconfig.AppSettings{}, fmt.Errorf("application config error: %w", err)
}
@@ -528,6 +540,7 @@ func resolveCommonSettings(fs *flag.FlagSet, configPath string, overrides appcon
maxArtifactBytes: settings.MaxArtifactBytes,
maxResponseBytes: settings.MaxResponseBytes,
defaultRenderFormat: settings.DefaultRenderFormat,
backends: settings.Backends,
}, nil
}
@@ -538,19 +551,54 @@ func validateRequiredLibraryDirs(promptDir string) error {
return nil
}
func newEngine(cfg *runConfig, options ...promptkit.Option) (*promptkit.Engine, error) {
return promptkit.NewEngine(promptkit.Config{
PromptDir: cfg.promptDir,
ProfileDir: cfg.profileDir,
SchemaDir: cfg.schemaDir,
}, options...)
type engineSettings struct {
promptDir string
profileDir string
schemaDir string
backends []appconfig.BackendSettings
}
func (c runConfig) engineSettings() engineSettings {
return engineSettings{promptDir: c.promptDir, profileDir: c.profileDir, schemaDir: c.schemaDir, backends: c.backends}
}
func (c serveConfig) engineSettings() engineSettings {
return engineSettings{promptDir: c.promptDir, profileDir: c.profileDir, schemaDir: c.schemaDir, backends: c.backends}
}
func newEngine(settings engineSettings, options ...promptkit.Option) (*promptkit.Engine, error) {
engineOptions := make([]promptkit.Option, 0, len(settings.backends)+len(options))
for _, configured := range settings.backends {
engineOptions = append(engineOptions, promptkit.WithBackend(promptkit.Backend{
ID: configured.ID,
Endpoint: configured.Endpoint,
APIKeyEnv: configured.APIKeyEnv,
ExtraParams: configured.ExtraParams,
ConcurrencyLimit: configured.ConcurrencyLimit,
QueueCapacity: configured.QueueCapacity,
}))
}
engineOptions = append(engineOptions, options...)
engine, err := promptkit.NewEngine(promptkit.Config{
PromptDir: settings.promptDir,
ProfileDir: settings.profileDir,
SchemaDir: settings.schemaDir,
}, engineOptions...)
if err != nil {
return nil, fmt.Errorf("engine initialization from application configuration: %w", err)
}
return engine, nil
}
func buildRunRequestFromConfig(cfg *runConfig) (promptkit.RunRequest, error) {
inputMappings, err := parseMappings(cfg.inputRaw, false)
var inputMappings map[string]string
var err error
if len(cfg.inputRaw) > 0 {
inputMappings, err = parseMappings(cfg.inputRaw, false)
if err != nil {
return promptkit.RunRequest{}, fmt.Errorf("input parse error: %w", err)
}
}
varMappings := map[string]string{}
if len(cfg.varRaw) > 0 {
@@ -560,13 +608,16 @@ func buildRunRequestFromConfig(cfg *runConfig) (promptkit.RunRequest, error) {
}
}
inputs := make(map[string]promptkit.ArtifactRef, len(inputMappings))
var inputs map[string]promptkit.ArtifactRef
if len(inputMappings) > 0 {
inputs = make(map[string]promptkit.ArtifactRef, len(inputMappings))
for name, path := range inputMappings {
inputs[name] = promptkit.File(path)
}
}
var modelOverride *promptkit.ExecutionTargetOverride
if cfg.llmBaseURLSet || cfg.modelSet || cfg.temperatureSet || cfg.maxTokensSet || cfg.topPSet || cfg.apiKeyEnvSet || cfg.timeoutSet {
if cfg.llmBaseURLSet || cfg.modelSet || cfg.temperatureSet || cfg.maxTokensSet || cfg.topPSet || cfg.reasoningEffortSet || cfg.apiKeyEnvSet || cfg.timeoutSet {
modelOverride = &promptkit.ExecutionTargetOverride{
Endpoint: cfg.llmBaseURL,
Model: cfg.model,
@@ -581,6 +632,9 @@ func buildRunRequestFromConfig(cfg *runConfig) (promptkit.RunRequest, error) {
if cfg.topPSet {
modelOverride.TopP = &cfg.topP
}
if cfg.reasoningEffortSet {
modelOverride.ReasoningEffort = &cfg.reasoningEffort
}
if cfg.timeoutSet {
timeoutSeconds := int(cfg.timeout.Seconds())
modelOverride.TimeoutSeconds = &timeoutSeconds
@@ -589,7 +643,9 @@ func buildRunRequestFromConfig(cfg *runConfig) (promptkit.RunRequest, error) {
return promptkit.RunRequest{
PromptID: cfg.promptID,
PromptVersion: cfg.promptVersion,
ProfileID: cfg.profileID,
SessionID: cfg.sessionID,
Inputs: inputs,
Vars: varMappings,
Execution: modelOverride,
@@ -682,12 +738,24 @@ func printSummary(stderr io.Writer, res *promptkit.RunResult) {
if res.Usage.CachedTokens != 0 || res.Usage.CacheWriteTokens != 0 {
fmt.Fprintf(stderr, " cached_tokens=%d cache_write_tokens=%d", res.Usage.CachedTokens, res.Usage.CacheWriteTokens)
}
if res.SelectedBackendID != "" {
fmt.Fprintf(stderr, " backend=%s", res.SelectedBackendID)
}
fmt.Fprintln(stderr)
}
func printUsage(w io.Writer) {
fmt.Fprintln(w, "usage: scriptorium <run|render|serve> ...")
fmt.Fprintln(w, " run: scriptorium run [--config PATH] [--prompt-dir DIR] [--profile-dir DIR] --prompt ID --input name=path [--input ...] [--profile ID] [--llm-base-url URL] [--model NAME] [--api-key-env ENV] [--temperature N] [--max-tokens N] [--top-p N] [--var k=v] [--out path] [--timeout 10m]")
fmt.Fprintln(w, " render: scriptorium render [--config PATH] [--prompt-dir DIR] [--profile-dir DIR] --prompt ID --input name=path [--input ...] [--profile ID] [--llm-base-url URL] [--model NAME] [--api-key-env ENV] [--temperature N] [--max-tokens N] [--top-p N] [--var k=v] [--format text|json] [--out path] [--timeout 10m]")
fmt.Fprintf(w, " serve: scriptorium serve [--config PATH] [--addr %s] [--prompt-dir DIR] [--profile-dir DIR] [--schema-dir DIR] [--artifact-root DIR] [--max-request-bytes N] [--max-artifact-bytes N] [--max-response-bytes N]\n", defaults.HTTPAddrDefault)
func runErrorMessage(err error) string {
if errors.Is(err, promptkit.ErrCapacityExceeded) {
return "run error: model backend capacity is exhausted"
}
return fmt.Sprintf("run error: %v", err)
}
func printUsage(w io.Writer) {
fmt.Fprintln(w, "usage: scriptorium <run|render|serve|inspect> ...")
fmt.Fprintln(w, " run: scriptorium run [--config PATH] [--prompt-dir DIR] [--profile-dir DIR] --prompt ID [--prompt-version VERSION] [--input name=path] [--profile ID] [--session-id ID] [--llm-base-url URL] [--model NAME] [--api-key-env ENV] [--temperature N] [--max-tokens N] [--top-p N] [--reasoning-effort VALUE] [--var k=v] [--out path] [--timeout 10m]")
fmt.Fprintln(w, " render: scriptorium render [--config PATH] [--prompt-dir DIR] [--profile-dir DIR] --prompt ID [--prompt-version VERSION] [--input name=path] [--profile ID] [--session-id ID] [--llm-base-url URL] [--model NAME] [--api-key-env ENV] [--temperature N] [--max-tokens N] [--top-p N] [--reasoning-effort VALUE] [--var k=v] [--format text|json] [--out path] [--timeout 10m]")
fmt.Fprintf(w, " serve: scriptorium serve [--config PATH] [--addr %s] [--prompt-dir DIR] [--profile-dir DIR] [--schema-dir DIR] [--artifact-root DIR] [--max-request-bytes N] [--max-artifact-bytes N] [--max-response-bytes N]\n", defaults.HTTPAddrDefault)
fmt.Fprintln(w, " inspect prompt: scriptorium inspect prompt --prompt ID [--prompt-version VERSION] [--config PATH] [--prompt-dir DIR] [--format text|json] [--out path]")
fmt.Fprintln(w, " inspect profile: scriptorium inspect profile --profile ID [--config PATH] [--profile-dir DIR] [--format text|json] [--out path]")
}

View File

@@ -87,9 +87,12 @@ func TestParseRunArgsRequiredFlags(t *testing.T) {
t.Fatal("expected missing --prompt error")
}
_, err = parseRunArgs([]string{"--config", configPath, "--prompt-dir", "./prompts", "--profile-dir", "./profiles", "--prompt", "p"})
if err == nil {
t.Fatal("expected missing --input error")
cfg, err = parseRunArgs([]string{"--config", configPath, "--prompt-dir", "./prompts", "--profile-dir", "./profiles", "--prompt", "p"})
if err != nil {
t.Fatalf("expected omitted --input to be accepted, got %v", err)
}
if len(cfg.inputRaw) != 0 {
t.Fatalf("expected no input mappings, got %#v", cfg.inputRaw)
}
}
@@ -98,13 +101,16 @@ func TestParseRunArgsFlagMapping(t *testing.T) {
"--prompt-dir", "./prompts",
"--profile-dir", "./profiles",
"--prompt", "prompt.a",
"--prompt-version", "2",
"--profile", "profile.a",
"--session-id", "session-1",
"--input", "a=b",
"--llm-base-url", "http://x/v1",
"--model", "m",
"--temperature", "0.7",
"--max-tokens", "111",
"--top-p", "0.8",
"--reasoning-effort", "medium",
"--timeout", "30s",
"--api-key-env", "SCRIPTORIUM_API_KEY",
})
@@ -114,8 +120,11 @@ func TestParseRunArgsFlagMapping(t *testing.T) {
if cfg.promptDir != filepath.Clean("./prompts") || cfg.profileDir != filepath.Clean("./profiles") {
t.Fatalf("unexpected dirs: prompt=%q profile=%q", cfg.promptDir, cfg.profileDir)
}
if cfg.promptID != "prompt.a" || cfg.profileID != "profile.a" {
t.Fatalf("unexpected prompt/profile ids: %q %q", cfg.promptID, cfg.profileID)
if cfg.promptID != "prompt.a" || cfg.promptVersion != "2" || cfg.profileID != "profile.a" {
t.Fatalf("unexpected prompt/version/profile ids: %q %q %q", cfg.promptID, cfg.promptVersion, cfg.profileID)
}
if cfg.sessionID != "session-1" || cfg.reasoningEffort != "medium" || !cfg.reasoningEffortSet {
t.Fatalf("unexpected session or reasoning configuration: %+v", cfg)
}
if !cfg.llmBaseURLSet || !cfg.modelSet || !cfg.temperatureSet || !cfg.maxTokensSet || !cfg.topPSet || !cfg.timeoutSet || !cfg.apiKeyEnvSet {
t.Fatalf("expected override flags set, got %+v", cfg)
@@ -192,7 +201,7 @@ func TestParseServeArgsRejectsRuntimeOverrideFlags(t *testing.T) {
}
}
func TestUsageIncludesServeFileAndSizeLimitFlags(t *testing.T) {
func TestUsageIncludesExecutionAndServeFlags(t *testing.T) {
var stderr bytes.Buffer
code := Run(nil, io.Discard, &stderr)
if code != ExitRuntimeError {
@@ -201,10 +210,15 @@ func TestUsageIncludesServeFileAndSizeLimitFlags(t *testing.T) {
usage := stderr.String()
for _, want := range []string{
"--prompt-version VERSION",
"--session-id ID",
"--reasoning-effort VALUE",
"--artifact-root",
"--max-request-bytes",
"--max-artifact-bytes",
"--max-response-bytes",
"inspect prompt",
"inspect profile",
} {
if !strings.Contains(usage, want) {
t.Fatalf("expected usage to include %q, got %q", want, usage)
@@ -689,6 +703,55 @@ func TestBuildRunRequestPreservesNumericOverridePresence(t *testing.T) {
}
}
func TestBuildRunRequestPreservesReasoningEffortPresenceAndSessionID(t *testing.T) {
omitted, err := buildRunRequestFromConfig(&runConfig{promptID: "prompt-1"})
if err != nil {
t.Fatalf("expected omitted request to build, got %v", err)
}
if omitted.Execution != nil {
t.Fatalf("expected omitted reasoning flag to leave execution nil, got %#v", omitted.Execution)
}
for _, tc := range []struct {
name string
value string
}{
{name: "replacement", value: "high"},
{name: "clear", value: ""},
} {
t.Run(tc.name, func(t *testing.T) {
req, err := buildRunRequestFromConfig(&runConfig{
promptID: "prompt-1",
sessionID: "session-1",
reasoningEffort: tc.value,
reasoningEffortSet: true,
})
if err != nil {
t.Fatalf("expected request to build, got %v", err)
}
if req.SessionID != "session-1" || req.Execution == nil || req.Execution.ReasoningEffort == nil || *req.Execution.ReasoningEffort != tc.value {
t.Fatalf("unexpected mapped request: %#v", req)
}
})
}
}
func TestBuildRunRequestAllowsOmittedInputsAndMapsPromptVersion(t *testing.T) {
req, err := buildRunRequestFromConfig(&runConfig{
promptID: "prompt-1",
promptVersion: "2",
})
if err != nil {
t.Fatalf("expected request without inputs to build, got %v", err)
}
if req.PromptVersion != "2" {
t.Fatalf("expected prompt version to be mapped, got %q", req.PromptVersion)
}
if req.Inputs != nil {
t.Fatalf("expected omitted inputs to remain nil, got %#v", req.Inputs)
}
}
func TestParseRunArgsFailsClearlyWhenNoEffectivePromptDir(t *testing.T) {
configPath := writeAppConfigFile(t, `
profile_dir: ./profiles
@@ -942,6 +1005,452 @@ profile_dir: %s
}
}
func TestRenderCommandUsesConfiguredCustomBackend(t *testing.T) {
lib := newCLITestLibrary(t)
writePromptDefinition(t, lib.promptDir, "custom.yaml", `id: custom
version: "1"
default_profile: local-gpu
messages:
- role: user
content: "hello"
output:
format: text
validation_mode: none
`)
if err := os.WriteFile(filepath.Join(lib.profileDir, "local-gpu.yaml"), []byte(`id: local-gpu
backend: local-gpu
model: local-model
`), 0o644); err != nil {
t.Fatalf("write profile fixture: %v", err)
}
configPath := writeAppConfigFile(t, fmt.Sprintf(`
prompt_dir: %s
profile_dir: %s
backends:
local-gpu:
endpoint: http://localhost:11434/v1
extra_params:
provider_option: enabled
concurrency_limit: 2
queue_capacity: 0
`, lib.promptDir, lib.profileDir))
code, stdout, stderr := runCLICommand(t, renderCommand, []string{
"--config", configPath,
"--prompt", "custom",
})
if code != ExitOK {
t.Fatalf("expected ExitOK, got %d stderr=%q", code, stderr)
}
if !strings.Contains(stdout, "selected_backend_id: local-gpu") {
t.Fatalf("expected configured backend in prepared output, got:\n%s", stdout)
}
}
func TestRenderCommandMapsReasoningEffortAndSessionID(t *testing.T) {
lib := newCLITestLibrary(t)
writePromptDefinition(t, lib.promptDir, "session.yaml", `id: session
version: "1"
default_profile: local
session_id: definition-session
messages:
- role: user
content: "hello"
output:
format: text
validation_mode: none
`)
if err := os.WriteFile(filepath.Join(lib.profileDir, "local.yaml"), []byte(`id: local
endpoint: http://127.0.0.1:1/v1
model: local-model
reasoning_effort: low
`), 0o644); err != nil {
t.Fatalf("write profile fixture: %v", err)
}
for _, tc := range []struct {
name string
args []string
wantReasoning string
wantSessionID string
absentReasoning bool
}{
{name: "omitted reasoning inherits profile", wantReasoning: "low", wantSessionID: "definition-session"},
{name: "nonblank reasoning replaces profile", args: []string{"--reasoning-effort", "high"}, wantReasoning: "high", wantSessionID: "definition-session"},
{name: "empty reasoning clears profile", args: []string{"--reasoning-effort="}, wantSessionID: "definition-session", absentReasoning: true},
{name: "direct session replaces definition", args: []string{"--session-id", "direct-session"}, wantReasoning: "low", wantSessionID: "direct-session"},
} {
t.Run(tc.name, func(t *testing.T) {
args := []string{"--prompt-dir", lib.promptDir, "--profile-dir", lib.profileDir, "--prompt", "session"}
args = append(args, tc.args...)
code, stdout, stderr := runCLICommand(t, renderCommand, args)
if code != ExitOK {
t.Fatalf("expected ExitOK, got %d stderr=%q", code, stderr)
}
if !strings.Contains(stdout, "session_id: "+tc.wantSessionID) {
t.Fatalf("expected session ID %q, got:\n%s", tc.wantSessionID, stdout)
}
hasReasoning := strings.Contains(stdout, "reasoning_effort:")
if tc.absentReasoning {
if hasReasoning {
t.Fatalf("expected cleared reasoning to be omitted, got:\n%s", stdout)
}
return
}
if !strings.Contains(stdout, "reasoning_effort: "+tc.wantReasoning) {
t.Fatalf("expected reasoning effort %q, got:\n%s", tc.wantReasoning, stdout)
}
})
}
}
func TestRenderCommandOmitsEmptyEffectiveSessionID(t *testing.T) {
lib := newCLITestLibrary(t)
writePromptDefinition(t, lib.promptDir, "plain.yaml", `id: plain
version: "1"
default_profile: local
messages:
- role: user
content: "hello"
output:
format: text
validation_mode: none
`)
writeProfileFile(t, lib.profileDir, "local", "http://127.0.0.1:1/v1", "local-model")
code, stdout, stderr := runCLICommand(t, renderCommand, []string{
"--prompt-dir", lib.promptDir,
"--profile-dir", lib.profileDir,
"--prompt", "plain",
})
if code != ExitOK {
t.Fatalf("expected ExitOK, got %d stderr=%q", code, stderr)
}
if strings.Contains(stdout, "session_id:") {
t.Fatalf("expected no effective session ID, got:\n%s", stdout)
}
}
func TestRenderCommandRejectsOverlongSessionID(t *testing.T) {
lib := newCLITestLibrary(t)
writePromptDefinition(t, lib.promptDir, "session.yaml", `id: session
version: "1"
default_profile: local
messages:
- role: user
content: "hello"
output:
format: text
validation_mode: none
`)
writeProfileFile(t, lib.profileDir, "local", "http://127.0.0.1:1/v1", "local-model")
code, _, stderr := runCLICommand(t, renderCommand, []string{
"--prompt-dir", lib.promptDir,
"--profile-dir", lib.profileDir,
"--prompt", "session",
"--session-id", strings.Repeat("x", 257),
})
if code != ExitRuntimeError {
t.Fatalf("expected runtime error, got %d stderr=%q", code, stderr)
}
if !strings.Contains(stderr, "invalid run request") {
t.Fatalf("expected invalid-request context, got %q", stderr)
}
}
func TestInspectPromptCommandFormatsDefinitionWithoutProfileOrGeneration(t *testing.T) {
lib := newCLITestLibrary(t)
writePromptDefinition(t, lib.promptDir, "inspect.yaml", `id: inspect
version: "1"
messages:
- role: user
content: "hello"
output:
format: text
validation_mode: none
`)
code, stdout, stderr := runCLICommand(t, inspectCommand, []string{"prompt", "--prompt-dir", lib.promptDir, "--prompt", "inspect", "--format", "json"})
if code != ExitOK {
t.Fatalf("expected ExitOK, got %d stderr=%q", code, stderr)
}
if !strings.Contains(stdout, `"prompt_id": "inspect"`) || !strings.Contains(stdout, `"inputs": []`) {
t.Fatalf("unexpected inspection output: %s", stdout)
}
code, stdout, stderr = runCLICommand(t, inspectCommand, []string{"prompt", "--prompt-dir", lib.promptDir, "--prompt", "missing"})
if code != ExitRuntimeError || stdout != "" || !strings.Contains(stderr, "inspect error") {
t.Fatalf("expected failed inspection without output, got code=%d stdout=%q stderr=%q", code, stdout, stderr)
}
}
func TestInspectionParsersPreserveExplicitEmptyConfigPath(t *testing.T) {
promptConfig, err := parsePromptInspectionArgs([]string{"--config=", "--prompt", "fixture"})
if err != nil {
t.Fatalf("parse prompt inspection: %v", err)
}
if promptConfig.configPath != "" || !promptConfig.configExplicit {
t.Fatalf("expected explicit empty prompt config path, got %+v", promptConfig)
}
profileConfig, err := parseProfileInspectionArgs([]string{"--config=", "--profile", "fixture"})
if err != nil {
t.Fatalf("parse profile inspection: %v", err)
}
if profileConfig.configPath != "" || !profileConfig.configExplicit {
t.Fatalf("expected explicit empty profile config path, got %+v", profileConfig)
}
}
func TestPromptkitV09DefinitionsRenderThroughCLI(t *testing.T) {
fixtureRoot := promptkitV09FixtureRoot(t)
configPath := writePromptkitV09Config(t, fixtureRoot, true)
inputPath := filepath.Join(fixtureRoot, "inputs", "source.md")
code, stdout, stderr := runCLICommand(t, renderCommand, []string{
"--config", configPath,
"--prompt", "compat.complete",
"--prompt-version", "1.0.0",
"--input", "source=" + inputPath,
"--var", "topic=testing",
"--format", "json",
})
if code != ExitOK {
t.Fatalf("expected ExitOK, got %d stderr=%q", code, stderr)
}
var payload map[string]any
if err := json.Unmarshal([]byte(stdout), &payload); err != nil {
t.Fatalf("decode rendered fixture: %v\nbody=%s", err, stdout)
}
if payload["prompt_version"] != "1.0.0" || payload["selected_profile_id"] != "custom-derived" || payload["selected_backend_id"] != "fixture-custom" {
t.Fatalf("unexpected selected definition and target: %#v", payload)
}
if payload["session_id"] != "fixture-testing" {
t.Fatalf("expected rendered session template, got %#v", payload["session_id"])
}
outputContract := payload["output_contract"].(map[string]any)
if outputContract["format"] != "json" || outputContract["validation_mode"] != "json_schema" || outputContract["repair_attempts"] != float64(2) {
t.Fatalf("unexpected output contract: %#v", outputContract)
}
if _, ok := payload["structured_output"].(map[string]any)["json_schema"]; !ok {
t.Fatalf("expected loaded JSON schema metadata, got %#v", payload["structured_output"])
}
messages := payload["messages"].([]any)
wantRoles := []string{"developer", "system", "user", "assistant"}
for i, wantRole := range wantRoles {
message := messages[i].(map[string]any)
if message["role"] != wantRole {
t.Fatalf("message %d: expected role %q, got %#v", i, wantRole, message["role"])
}
}
cacheControl := messages[0].(map[string]any)["cache_control"].(map[string]any)
if cacheControl["type"] != "ephemeral" || cacheControl["ttl"] != "1h" {
t.Fatalf("unexpected cache control: %#v", cacheControl)
}
if !strings.Contains(messages[2].(map[string]any)["content"].(string), "stable, synthetic material") {
t.Fatalf("file-backed input template was not rendered: %#v", messages[2])
}
inputHashes := payload["input_hashes"].(map[string]any)
if len(inputHashes) != 1 || inputHashes["source"] == "" {
t.Fatalf("required and omitted optional inputs were not preserved: %#v", inputHashes)
}
}
func TestPromptkitV09PromptInspectionSelectsVersionsAndContracts(t *testing.T) {
fixtureRoot := promptkitV09FixtureRoot(t)
configPath := writePromptkitV09Config(t, fixtureRoot, true)
tests := []struct {
promptID string
version string
format string
mode string
}{
{promptID: "compat.complete", version: "1.0.0", format: "json", mode: "json_schema"},
{promptID: "compat.complete", version: "2.0.0", format: "text", mode: "basic"},
{promptID: "compat.json", version: "1.0.0", format: "json", mode: "json"},
{promptID: "compat.none", version: "1.0.0", format: "text", mode: "none"},
}
for _, tc := range tests {
t.Run(tc.promptID+"@"+tc.version, func(t *testing.T) {
code, stdout, stderr := runCLICommand(t, inspectCommand, []string{
"prompt",
"--config", configPath,
"--prompt", tc.promptID,
"--prompt-version", tc.version,
"--format", "json",
})
if code != ExitOK {
t.Fatalf("expected ExitOK, got %d stderr=%q", code, stderr)
}
var inspection renderformat.PromptInspection
if err := json.Unmarshal([]byte(stdout), &inspection); err != nil {
t.Fatalf("decode prompt inspection: %v\nbody=%s", err, stdout)
}
if inspection.PromptVersion != tc.version || inspection.OutputContract.Format != tc.format || inspection.OutputContract.ValidationMode != tc.mode {
t.Fatalf("unexpected prompt inspection: %+v", inspection)
}
})
}
}
func TestPromptkitV09ProfileInspectionResolvesSupportedTargets(t *testing.T) {
const secret = "sentinel-profile-secret"
t.Setenv("FIXTURE_PROFILE_API_KEY", secret)
fixtureRoot := promptkitV09FixtureRoot(t)
configPath := writePromptkitV09Config(t, fixtureRoot, true)
tests := []struct {
name string
profileID string
wantBackend string
wantModel string
wantAPIKeyEnv string
}{
{name: "inherited custom backend", profileID: "custom-derived", wantBackend: "fixture-custom", wantModel: "fixture-derived-model", wantAPIKeyEnv: "FIXTURE_PROFILE_API_KEY"},
{name: "endpoint only", profileID: "endpoint-only", wantModel: "fixture-endpoint-model"},
{name: "built in", profileID: "deepseek-4-flash", wantBackend: "openrouter", wantModel: "deepseek/deepseek-v4-flash", wantAPIKeyEnv: "OPENROUTER_API_KEY"},
}
for _, tc := range tests {
t.Run(tc.name, func(t *testing.T) {
args := []string{"profile", "--config", configPath, "--profile", tc.profileID, "--format", "json"}
code, stdout, stderr := runCLICommand(t, inspectCommand, args)
if code != ExitOK {
t.Fatalf("expected ExitOK, got %d stderr=%q", code, stderr)
}
if strings.Contains(stdout, secret) || strings.Contains(stderr, secret) {
t.Fatalf("inspection exposed environment secret: stdout=%q stderr=%q", stdout, stderr)
}
var inspection renderformat.ProfileInspection
if err := json.Unmarshal([]byte(stdout), &inspection); err != nil {
t.Fatalf("decode profile inspection: %v\nbody=%s", err, stdout)
}
params := inspection.EffectiveModelParams
if inspection.ProfileID != tc.profileID || params.BackendID != tc.wantBackend || params.Model != tc.wantModel || params.APIKeyEnv != tc.wantAPIKeyEnv {
t.Fatalf("unexpected effective profile: %+v", inspection)
}
if tc.profileID == "custom-derived" {
if params.ServiceTier != "flex" || params.ReasoningEffort != "high" || params.TimeoutSeconds != 45 || len(params.ExtraParams) == 0 {
t.Fatalf("inherited profile controls were not resolved: %+v", params)
}
}
})
}
configWithoutPromptDir := writePromptkitV09Config(t, fixtureRoot, false)
code, _, stderr := runCLICommand(t, inspectCommand, []string{
"profile",
"--config", configWithoutPromptDir,
"--profile-dir", filepath.Join(fixtureRoot, "profiles"),
"--profile", "custom-derived",
})
if code != ExitOK {
t.Fatalf("profile inspection unexpectedly required a prompt directory: %q", stderr)
}
}
func TestProfileInspectionHonorsDirectoryPrecedenceOutputAndFailures(t *testing.T) {
fixtureRoot := promptkitV09FixtureRoot(t)
configPath := writePromptkitV09Config(t, fixtureRoot, true)
overrideDir := t.TempDir()
writeProfileFile(t, overrideDir, "custom-derived", "http://127.0.0.1:9000/v1", "override-model")
outPath := filepath.Join(t.TempDir(), "inspection.json")
code, stdout, stderr := runCLICommand(t, inspectCommand, []string{
"profile",
"--config", configPath,
"--profile-dir", overrideDir,
"--profile", "custom-derived",
"--format", "json",
"--out", outPath,
})
if code != ExitOK || stdout != "" {
t.Fatalf("expected file output, got code=%d stdout=%q stderr=%q", code, stdout, stderr)
}
output, err := os.ReadFile(outPath)
if err != nil {
t.Fatalf("read profile inspection output: %v", err)
}
if !strings.Contains(string(output), `"model": "override-model"`) || !strings.Contains(string(output), `"backend_id": ""`) {
t.Fatalf("profile directory override was not used: %s", output)
}
for _, profileID := range []string{"missing", "invalid"} {
t.Run(profileID, func(t *testing.T) {
profileDir := filepath.Join(fixtureRoot, "profiles")
if profileID == "invalid" {
profileDir = t.TempDir()
writePromptDefinition(t, profileDir, "invalid.yaml", "id: invalid\nendpoint: not-a-url\nmodel: fixture\n")
}
code, stdout, stderr := runCLICommand(t, inspectCommand, []string{
"profile",
"--config", configPath,
"--profile-dir", profileDir,
"--profile", profileID,
})
if code != ExitRuntimeError || stdout != "" || !strings.Contains(stderr, "inspect error") {
t.Fatalf("expected inspection failure without partial output, got code=%d stdout=%q stderr=%q", code, stdout, stderr)
}
})
}
}
func TestConfiguredBackendValidationComesFromPromptkit(t *testing.T) {
lib := newCLITestLibrary(t)
for _, tc := range []struct {
name string
backend string
}{
{
name: "reserved ID",
backend: `openrouter:
endpoint: http://localhost:11434/v1`,
},
{
name: "invalid endpoint",
backend: `local-gpu:
endpoint: not-a-url`,
},
{
name: "invalid capacity relationship",
backend: `local-gpu:
endpoint: http://localhost:11434/v1
queue_capacity: 0`,
},
{
name: "reserved extra parameter",
backend: `local-gpu:
endpoint: http://localhost:11434/v1
extra_params:
model: forbidden`,
},
} {
t.Run(tc.name, func(t *testing.T) {
configPath := writeAppConfigFile(t, fmt.Sprintf(`
prompt_dir: %s
profile_dir: %s
backends:
%s
`, lib.promptDir, lib.profileDir, tc.backend))
cfg, err := parseRenderArgs([]string{"--config", configPath, "--prompt", "custom"})
if err != nil {
t.Fatalf("expected config decoding to succeed, got %v", err)
}
_, err = newEngine(cfg.runConfig.engineSettings())
if !errors.Is(err, promptkit.ErrInvalidConfig) {
t.Fatalf("expected Promptkit ErrInvalidConfig, got %v", err)
}
if !strings.Contains(err.Error(), "engine initialization from application configuration") {
t.Fatalf("expected application context, got %v", err)
}
})
}
}
func TestRenderCommandExplicitTextFormatWorks(t *testing.T) {
lib := newCLITestLibrary(t)
inputPath := lib.writeInputFile(t, "transcript.md", "hello transcript")
@@ -1122,6 +1631,104 @@ func TestRenderCommandExplicitProfileOverridesPromptDefault(t *testing.T) {
}
}
func TestRenderCommandUsesDefinitionInputRulesAndPromptVersions(t *testing.T) {
lib := newCLITestLibrary(t)
writeProfileFile(t, lib.profileDir, "local", "http://127.0.0.1:1/v1", "model")
writePromptDefinition(t, lib.promptDir, "sole.yaml", `id: sole
version: "1"
default_profile: local
messages:
- role: user
content: "hello"
output:
format: text
validation_mode: none
`)
writePromptDefinition(t, lib.promptDir, "versioned-one.yaml", `id: versioned
version: "1"
default_profile: local
messages:
- role: user
content: "one"
output:
format: text
validation_mode: none
`)
writePromptDefinition(t, lib.promptDir, "versioned-two.yaml", `id: versioned
version: "2"
default_profile: local
messages:
- role: user
content: "two"
output:
format: text
validation_mode: none
`)
writePromptDefinition(t, lib.promptDir, "optional.yaml", `id: optional
version: "1"
default_profile: local
inputs:
- name: note
required: false
messages:
- role: user
content: "hello"
output:
format: text
validation_mode: none
`)
writePromptDefinition(t, lib.promptDir, "required.yaml", `id: required
version: "1"
default_profile: local
inputs:
- name: note
required: true
messages:
- role: user
content: "hello"
output:
format: text
validation_mode: none
`)
writePromptDefinition(t, lib.promptDir, "template.yaml", `id: template
version: "1"
default_profile: local
messages:
- role: user
content: '{{input "note"}}'
output:
format: text
validation_mode: none
`)
baseArgs := []string{"--prompt-dir", lib.promptDir, "--profile-dir", lib.profileDir}
for _, tc := range []struct {
name string
args []string
wantCode int
wantText string
}{
{name: "sole version selected when omitted", args: []string{"--prompt", "sole"}, wantCode: ExitOK, wantText: "prompt_version: 1"},
{name: "explicit version selected", args: []string{"--prompt", "versioned", "--prompt-version", "2"}, wantCode: ExitOK, wantText: "prompt_version: 2"},
{name: "multiple versions require selection", args: []string{"--prompt", "versioned"}, wantCode: ExitRuntimeError, wantText: "duplicate prompt definition id"},
{name: "no declared inputs", args: []string{"--prompt", "sole"}, wantCode: ExitOK, wantText: "prompt: sole"},
{name: "optional input omitted", args: []string{"--prompt", "optional"}, wantCode: ExitOK, wantText: "prompt: optional"},
{name: "required input omitted", args: []string{"--prompt", "required"}, wantCode: ExitRuntimeError, wantText: "required"},
{name: "template input omitted", args: []string{"--prompt", "template"}, wantCode: ExitRuntimeError, wantText: "note"},
} {
t.Run(tc.name, func(t *testing.T) {
code, stdout, stderr := runCLICommand(t, renderCommand, append(append([]string{}, baseArgs...), tc.args...))
if code != tc.wantCode {
t.Fatalf("expected exit %d, got %d stderr=%q", tc.wantCode, code, stderr)
}
if !strings.Contains(stdout+stderr, tc.wantText) {
t.Fatalf("expected output to contain %q, stdout=%q stderr=%q", tc.wantText, stdout, stderr)
}
})
}
}
func TestRunCommandPromptDefaultProfileWorksThroughCLIPath(t *testing.T) {
lib := newCLITestLibrary(t)
inputPath := lib.writeInputFile(t, "transcript.md", "hello")
@@ -1261,6 +1868,9 @@ func TestWriteOutputAndSummaryUseSeparateWriters(t *testing.T) {
if strings.Contains(stderr.String(), "cached_tokens=") || strings.Contains(stderr.String(), "cache_write_tokens=") {
t.Fatalf("expected zero cache usage to be omitted from summary, got %q", stderr.String())
}
if strings.Contains(stderr.String(), "backend=") {
t.Fatalf("expected endpoint-only backend to be omitted from summary, got %q", stderr.String())
}
}
func TestPrintSummaryIncludesCacheUsageWhenPresent(t *testing.T) {
@@ -1290,6 +1900,32 @@ func TestPrintSummaryIncludesCacheUsageWhenPresent(t *testing.T) {
if !strings.Contains(summary, "cached_tokens=0 cache_write_tokens=3") {
t.Fatalf("expected cache usage in summary, got %q", summary)
}
if strings.Contains(summary, "backend=") {
t.Fatalf("expected endpoint-only backend to be omitted from summary, got %q", summary)
}
}
func TestPrintSummaryIncludesBackendWhenPresent(t *testing.T) {
var stderr bytes.Buffer
printSummary(&stderr, &promptkit.RunResult{
PromptID: "p",
PromptVersion: "1",
SelectedProfileID: "exec",
SelectedBackendID: "local",
ModelName: "m",
Validation: promptkit.ValidationResult{Status: promptkit.ValidationPassed, Mode: promptkit.ValidationBasic},
RenderedPromptHash: "h",
})
if !strings.Contains(stderr.String(), "backend=local") {
t.Fatalf("expected backend in summary, got %q", stderr.String())
}
}
func TestRunErrorMessageDoesNotExposeCapacityDetails(t *testing.T) {
got := runErrorMessage(&promptkit.CapacityError{BackendID: "private-backend"})
if got != "run error: model backend capacity is exhausted" {
t.Fatalf("unexpected capacity diagnostic: %q", got)
}
}
type cliTestLibrary struct {
@@ -1335,11 +1971,48 @@ func runCLICommand(t *testing.T, command func([]string, io.Writer, io.Writer) in
return code, stdout.String(), stderr.String()
}
func promptkitV09FixtureRoot(t *testing.T) string {
t.Helper()
root, err := filepath.Abs(filepath.Join("..", "..", "..", "testdata", "promptkit-v0.9"))
if err != nil {
t.Fatalf("resolve Promptkit v0.9 fixture root: %v", err)
}
if _, err := os.Stat(root); err != nil {
t.Fatalf("stat Promptkit v0.9 fixture root: %v", err)
}
return root
}
func writePromptkitV09Config(t *testing.T, fixtureRoot string, includeSources bool) string {
t.Helper()
sources := ""
if includeSources {
sources = fmt.Sprintf("prompt_dir: %q\nprofile_dir: %q\nschema_dir: %q\n", filepath.Join(fixtureRoot, "prompts"), filepath.Join(fixtureRoot, "profiles"), filepath.Join(fixtureRoot, "schemas"))
}
return writeAppConfigFile(t, sources+`backends:
fixture-custom:
endpoint: http://127.0.0.1:11434/v1
api_key_env: FIXTURE_BACKEND_API_KEY
extra_params:
backend_option:
enabled: true
concurrency_limit: 2
queue_capacity: 0
`)
}
func writePromptFile(t *testing.T, dir, id, defaultProfile string) {
t.Helper()
writePromptFileWithTemplate(t, dir, id, defaultProfile, "Summarize: {{input \"transcript\"}}")
}
func writePromptDefinition(t *testing.T, dir, name, definition string) {
t.Helper()
if err := os.WriteFile(filepath.Join(dir, name), []byte(definition), 0o644); err != nil {
t.Fatalf("write prompt definition: %v", err)
}
}
func writePromptFileWithTemplate(t *testing.T, dir, id, defaultProfile, templateContent string) {
t.Helper()
data := fmt.Sprintf(`id: %s

View File

@@ -16,6 +16,8 @@ import (
const (
scriptoriumModulePath = "gitea.maximumdirect.net/eric/scriptorium"
promptkitInternalPath = "gitea.maximumdirect.net/eric/promptkit/internal"
promptkitOpenRouterCatalogPath = "gitea.maximumdirect.net/eric/promptkit-backend-openrouter"
promptkitRakestrawhomeCatalogPath = "gitea.maximumdirect.net/eric/promptkit-backend-rakestrawhome"
)
var (
@@ -120,6 +122,24 @@ func TestForbiddenImportScannerDetectsPromptkitInternalPackages(t *testing.T) {
}
}
func TestForbiddenImportScannerDetectsPromptkitCatalogPackages(t *testing.T) {
for _, importPath := range []string{
promptkitOpenRouterCatalogPath,
promptkitRakestrawhomeCatalogPath,
} {
t.Run(importPath, func(t *testing.T) {
root := t.TempDir()
sourcePath := writeGoSource(t, root, "nested/consumer/catalog.go", importPath)
violations, err := findForbiddenProductionImports(root)
if err != nil {
t.Fatalf("scan source fixture: %v", err)
}
assertSingleViolation(t, violations, sourcePath, importPath)
})
}
}
func TestForbiddenImportScannerAllowsRetainedApplicationPackages(t *testing.T) {
root := t.TempDir()
sourcePath := filepath.Join(root, "nested/consumer/application.go")
@@ -201,6 +221,12 @@ func isForbiddenProductionImport(importPath string) bool {
if importPath == promptkitInternalPath || strings.HasPrefix(importPath, promptkitInternalPath+"/") {
return true
}
if importPath == promptkitOpenRouterCatalogPath || strings.HasPrefix(importPath, promptkitOpenRouterCatalogPath+"/") {
return true
}
if importPath == promptkitRakestrawhomeCatalogPath || strings.HasPrefix(importPath, promptkitRakestrawhomeCatalogPath+"/") {
return true
}
for _, root := range removedFrameworkPackageRoots {
if importPath == root || strings.HasPrefix(importPath, root+"/") {
return true

View File

@@ -8,6 +8,7 @@ type runRequestDTO struct {
PromptID string `json:"prompt_id"`
PromptVersion string `json:"prompt_version,omitempty"`
ProfileID string `json:"profile_id,omitempty"`
SessionID string `json:"session_id,omitempty"`
Inputs map[string]inputRefDTO `json:"inputs"`
Vars map[string]string `json:"vars,omitempty"`
Model *modelOverrideRequestDTO `json:"model,omitempty"`
@@ -28,7 +29,7 @@ type modelOverrideRequestDTO struct {
TopP *float64 `json:"top_p,omitempty"`
TimeoutSeconds *int `json:"timeout_seconds,omitempty"`
ServiceTier string `json:"service_tier,omitempty"`
ReasoningEffort string `json:"reasoning_effort,omitempty"`
ReasoningEffort *string `json:"reasoning_effort,omitempty"`
APIKeyEnv string `json:"api_key_env,omitempty"`
ExtraParams map[string]any `json:"extra_params,omitempty"`
}
@@ -56,6 +57,8 @@ type metadataDTO struct {
PromptHash string `json:"prompt_hash"`
RenderedPromptHash string `json:"rendered_prompt_hash"`
SelectedProfileID string `json:"selected_profile_id"`
SelectedBackendID string `json:"selected_backend_id,omitempty"`
SessionID string `json:"session_id,omitempty"`
ModelName string `json:"model_name"`
Endpoint string `json:"endpoint"`
ModelParams modelParamsDTO `json:"model_params"`
@@ -71,6 +74,7 @@ type metadataDTO struct {
type modelParamsDTO struct {
Endpoint string `json:"endpoint"`
BackendID string `json:"backend_id,omitempty"`
Model string `json:"model"`
Temperature float64 `json:"temperature"`
MaxTokens int `json:"max_tokens"`

View File

@@ -76,12 +76,9 @@ func (h *Handler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
writeError(w, http.StatusBadRequest, "invalid_request", "prompt_id is required")
return
}
if len(req.Inputs) == 0 {
writeError(w, http.StatusBadRequest, "invalid_request", "inputs is required")
return
}
mappedInputs := make(map[string]promptkit.ArtifactRef, len(req.Inputs))
var mappedInputs map[string]promptkit.ArtifactRef
if len(req.Inputs) > 0 {
mappedInputs = make(map[string]promptkit.ArtifactRef, len(req.Inputs))
for name, in := range req.Inputs {
mappedInputs[name] = promptkit.ArtifactRef{
Type: promptkit.ArtifactRefType(in.Type),
@@ -89,6 +86,7 @@ func (h *Handler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
Body: in.Body,
}
}
}
var model *promptkit.ExecutionTargetOverride
if req.Model != nil {
@@ -99,6 +97,7 @@ func (h *Handler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
PromptID: req.PromptID,
PromptVersion: req.PromptVersion,
ProfileID: req.ProfileID,
SessionID: req.SessionID,
Inputs: mappedInputs,
Vars: req.Vars,
Execution: model,
@@ -126,6 +125,8 @@ func (h *Handler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
PromptHash: res.PromptHash,
RenderedPromptHash: res.RenderedPromptHash,
SelectedProfileID: res.SelectedProfileID,
SelectedBackendID: res.SelectedBackendID,
SessionID: res.SessionID,
ModelName: res.ModelName,
Endpoint: res.Endpoint,
ModelParams: modelParamsDTOFromExecutionTarget(res.EffectiveModelParams),
@@ -173,6 +174,7 @@ func executionTargetOverrideFromModelOverrideDTO(dto *modelOverrideRequestDTO) *
func modelParamsDTOFromExecutionTarget(target promptkit.ExecutionTarget) modelParamsDTO {
return modelParamsDTO{
Endpoint: target.Endpoint,
BackendID: target.BackendID,
Model: target.Model,
Temperature: target.Temperature,
MaxTokens: target.MaxTokens,
@@ -220,6 +222,8 @@ func mapRunError(err error) (int, string, string) {
return http.StatusBadRequest, "artifact_read_failed", "failed to read input artifact"
case errors.Is(err, promptkit.ErrPromptRender):
return http.StatusBadRequest, "prompt_render_failed", "failed to render prompt"
case errors.Is(err, promptkit.ErrCapacityExceeded):
return http.StatusServiceUnavailable, "capacity_exceeded", "model backend capacity is exhausted"
case errors.Is(err, promptkit.ErrLLMGenerate):
return http.StatusBadGateway, "llm_failed", "model generation request failed"
case errors.Is(err, promptkit.ErrValidation):

View File

@@ -11,6 +11,7 @@ import (
"path/filepath"
"reflect"
"strings"
"sync/atomic"
"testing"
"time"
@@ -69,6 +70,34 @@ func (handlerLLMClient) Generate(ctx context.Context, req promptkit.GenerateRequ
return &promptkit.GenerateResponse{Content: "ok"}, nil
}
type blockingLLMClient struct {
started chan struct{}
release chan struct{}
current int32
peak int32
}
func (c *blockingLLMClient) Generate(ctx context.Context, req promptkit.GenerateRequest) (*promptkit.GenerateResponse, error) {
current := atomic.AddInt32(&c.current, 1)
defer atomic.AddInt32(&c.current, -1)
for {
peak := atomic.LoadInt32(&c.peak)
if current <= peak || atomic.CompareAndSwapInt32(&c.peak, peak, current) {
break
}
}
select {
case c.started <- struct{}{}:
default:
}
select {
case <-c.release:
return &promptkit.GenerateResponse{Content: "ok"}, nil
case <-ctx.Done():
return nil, ctx.Err()
}
}
func TestHandlerPostRunsSuccessWithExplicitProfileID(t *testing.T) {
start := time.Now().UTC()
end := start.Add(2 * time.Second)
@@ -90,9 +119,11 @@ func TestHandlerPostRunsSuccessWithExplicitProfileID(t *testing.T) {
PromptHash: "phash",
RenderedPromptHash: "rhash",
SelectedProfileID: "exec-default",
SelectedBackendID: "local",
ModelName: "m1",
Endpoint: "http://llm/v1",
EffectiveModelParams: promptkit.ExecutionTarget{
BackendID: "local",
Endpoint: "http://llm/v1",
Model: "m1",
Temperature: 0.2,
@@ -151,6 +182,9 @@ func TestHandlerPostRunsSuccessWithExplicitProfileID(t *testing.T) {
if metadata["selected_profile_id"] != "exec-default" {
t.Fatalf("unexpected metadata.selected_profile_id: %#v", metadata["selected_profile_id"])
}
if metadata["selected_backend_id"] != "local" {
t.Fatalf("unexpected metadata.selected_backend_id: %#v", metadata["selected_backend_id"])
}
if metadata["model_name"] != "m1" || metadata["endpoint"] != "http://llm/v1" {
t.Fatalf("unexpected model metadata: name=%#v endpoint=%#v", metadata["model_name"], metadata["endpoint"])
}
@@ -162,6 +196,9 @@ func TestHandlerPostRunsSuccessWithExplicitProfileID(t *testing.T) {
t.Fatalf("unexpected cache usage metadata: %#v", usage)
}
modelParams := metadata["model_params"].(map[string]any)
if modelParams["backend_id"] != "local" {
t.Fatalf("expected model_params.backend_id=local, got %#v", modelParams["backend_id"])
}
if modelParams["api_key_env"] != envName {
t.Fatalf("expected model_params.api_key_env=%q, got %#v", envName, modelParams["api_key_env"])
}
@@ -192,6 +229,124 @@ func TestHandlerPostRunsSuccessWithExplicitProfileID(t *testing.T) {
}
}
func TestHandlerAllowsOmittedInputsAndMapsPromptVersion(t *testing.T) {
r := &fakeRunner{result: &promptkit.RunResult{
Artifact: promptkit.Artifact{Body: []byte("ok")},
Validation: promptkit.ValidationResult{Status: promptkit.ValidationPassed, Mode: promptkit.ValidationNone, IsValid: true},
EffectiveModelParams: promptkit.ExecutionTarget{Endpoint: "http://llm/v1", Model: "m1"},
}}
h := NewHandler(r)
for _, body := range []string{
`{"prompt_id":"prompt-1","prompt_version":"2"}`,
`{"prompt_id":"prompt-1","prompt_version":"2","inputs":{}}`,
} {
req := httptest.NewRequest(http.MethodPost, "/v1/runs", bytes.NewBufferString(body))
w := httptest.NewRecorder()
h.ServeHTTP(w, req)
if w.Code != http.StatusOK {
t.Fatalf("expected 200, got %d body=%s", w.Code, w.Body.String())
}
if r.last.PromptVersion != "2" {
t.Fatalf("expected prompt version to be mapped, got %q", r.last.PromptVersion)
}
if r.last.Inputs != nil {
t.Fatalf("expected omitted or empty inputs to remain nil, got %#v", r.last.Inputs)
}
}
}
func TestHandlerDelegatesDefinitionInputRequirements(t *testing.T) {
tests := []struct {
name string
definition string
wantStatus int
wantCode string
}{
{
name: "no declared inputs",
definition: `id: p
version: "1"
default_profile: exec
messages:
- role: user
content: "hello"
output:
format: text
validation_mode: none
`,
wantStatus: http.StatusOK,
},
{
name: "optional input omitted",
definition: `id: p
version: "1"
default_profile: exec
inputs:
- name: note
required: false
messages:
- role: user
content: "hello"
output:
format: text
validation_mode: none
`,
wantStatus: http.StatusOK,
},
{
name: "required input omitted",
definition: `id: p
version: "1"
default_profile: exec
inputs:
- name: note
required: true
messages:
- role: user
content: "hello"
output:
format: text
validation_mode: none
`,
wantStatus: http.StatusBadRequest,
wantCode: "prompt_render_failed",
},
{
name: "template input omitted",
definition: `id: p
version: "1"
default_profile: exec
messages:
- role: user
content: '{{input "note"}}'
output:
format: text
validation_mode: none
`,
wantStatus: http.StatusBadRequest,
wantCode: "prompt_render_failed",
},
}
for _, tc := range tests {
t.Run(tc.name, func(t *testing.T) {
h := newDefinitionHandler(t, tc.definition)
req := httptest.NewRequest(http.MethodPost, "/v1/runs", bytes.NewBufferString(`{"prompt_id":"p"}`))
w := httptest.NewRecorder()
h.ServeHTTP(w, req)
if w.Code != tc.wantStatus {
t.Fatalf("expected %d, got %d body=%s", tc.wantStatus, w.Code, w.Body.String())
}
if tc.wantCode != "" {
assertHTTPErrorCode(t, w, tc.wantStatus, tc.wantCode)
}
})
}
}
func TestHandlerInlineRefsWorkWithoutArtifactRoot(t *testing.T) {
h := newArtifactRootHandler(t, "")
@@ -365,10 +520,12 @@ func TestHandlerModelOverrideMapsAllSupportedExecutionFields(t *testing.T) {
if got.Endpoint != "http://override/v1" ||
got.Model != "override-model" ||
got.ServiceTier != "flex" ||
got.ReasoningEffort != "medium" ||
got.APIKeyEnv != "SCRIPTORIUM_API_KEY" {
t.Fatalf("unexpected mapped execution target: %+v", got)
}
if got.ReasoningEffort == nil || *got.ReasoningEffort != "medium" {
t.Fatalf("unexpected mapped reasoning_effort: %#v", got.ReasoningEffort)
}
if got.Temperature == nil || *got.Temperature != 0.6 {
t.Fatalf("unexpected mapped temperature: %#v", got.Temperature)
}
@@ -386,6 +543,251 @@ func TestHandlerModelOverrideMapsAllSupportedExecutionFields(t *testing.T) {
}
}
func TestHandlerModelOverridePreservesReasoningEffortPresence(t *testing.T) {
tests := []struct {
name string
model string
wantPresent bool
wantValue string
}{
{name: "omitted", model: `{}`},
{name: "nonblank", model: `{"reasoning_effort":"high"}`, wantPresent: true, wantValue: "high"},
{name: "explicit empty", model: `{"reasoning_effort":""}`, wantPresent: true},
{name: "null", model: `{"reasoning_effort":null}`},
}
for _, tc := range tests {
t.Run(tc.name, func(t *testing.T) {
r := &fakeRunner{result: &promptkit.RunResult{
Artifact: promptkit.Artifact{Body: []byte("ok")},
Validation: promptkit.ValidationResult{Status: promptkit.ValidationPassed, Mode: promptkit.ValidationBasic, IsValid: true},
EffectiveModelParams: promptkit.ExecutionTarget{Endpoint: "http://llm/v1", Model: "m1"},
}}
h := NewHandler(r)
req := httptest.NewRequest(http.MethodPost, "/v1/runs", bytes.NewBufferString(`{"prompt_id":"prompt-1","inputs":{"transcript":{"type":"file","uri":"./t.md"}},"model":`+tc.model+`}`))
w := httptest.NewRecorder()
h.ServeHTTP(w, req)
if w.Code != http.StatusOK {
t.Fatalf("expected 200, got %d body=%s", w.Code, w.Body.String())
}
if r.last.Execution == nil {
t.Fatal("expected execution override")
}
if (r.last.Execution.ReasoningEffort != nil) != tc.wantPresent {
t.Fatalf("unexpected reasoning_effort presence: %#v", r.last.Execution.ReasoningEffort)
}
if tc.wantPresent && *r.last.Execution.ReasoningEffort != tc.wantValue {
t.Fatalf("unexpected reasoning_effort: got %q want %q", *r.last.Execution.ReasoningEffort, tc.wantValue)
}
})
}
}
func TestHandlerMapsSessionIDAndReportsEffectiveResultSessionID(t *testing.T) {
r := &fakeRunner{result: &promptkit.RunResult{
Artifact: promptkit.Artifact{Body: []byte("ok")},
Validation: promptkit.ValidationResult{Status: promptkit.ValidationPassed, Mode: promptkit.ValidationBasic, IsValid: true},
EffectiveModelParams: promptkit.ExecutionTarget{Endpoint: "http://llm/v1", Model: "m1"},
SessionID: "effective-session",
}}
h := NewHandler(r)
req := httptest.NewRequest(http.MethodPost, "/v1/runs", bytes.NewBufferString(`{"prompt_id":"prompt-1","session_id":"request-session"}`))
w := httptest.NewRecorder()
h.ServeHTTP(w, req)
if w.Code != http.StatusOK {
t.Fatalf("expected 200, got %d body=%s", w.Code, w.Body.String())
}
if r.last.SessionID != "request-session" {
t.Fatalf("expected session ID in run request, got %q", r.last.SessionID)
}
var resp map[string]any
if err := json.Unmarshal(w.Body.Bytes(), &resp); err != nil {
t.Fatalf("invalid JSON response: %v", err)
}
metadata := resp["metadata"].(map[string]any)
if metadata["session_id"] != "effective-session" {
t.Fatalf("expected effective session ID in response, got %#v", metadata["session_id"])
}
}
func TestHandlerOmitsEmptyEffectiveSessionID(t *testing.T) {
r := &fakeRunner{result: &promptkit.RunResult{
Artifact: promptkit.Artifact{Body: []byte("ok")},
Validation: promptkit.ValidationResult{Status: promptkit.ValidationPassed, Mode: promptkit.ValidationBasic, IsValid: true},
EffectiveModelParams: promptkit.ExecutionTarget{Endpoint: "http://llm/v1", Model: "m1"},
}}
h := NewHandler(r)
req := httptest.NewRequest(http.MethodPost, "/v1/runs", bytes.NewBufferString(`{"prompt_id":"prompt-1"}`))
w := httptest.NewRecorder()
h.ServeHTTP(w, req)
if w.Code != http.StatusOK {
t.Fatalf("expected 200, got %d body=%s", w.Code, w.Body.String())
}
var resp map[string]any
if err := json.Unmarshal(w.Body.Bytes(), &resp); err != nil {
t.Fatalf("invalid JSON response: %v", err)
}
metadata := resp["metadata"].(map[string]any)
if _, ok := metadata["session_id"]; ok {
t.Fatalf("expected empty effective session ID to be omitted, got %#v", metadata)
}
}
func TestHandlerSessionIDUsesPromptkitResolution(t *testing.T) {
for _, tc := range []struct {
name string
definitionSession string
requestSession string
wantSession string
}{
{name: "definition session", definitionSession: "definition-session", wantSession: "definition-session"},
{name: "direct session", definitionSession: "definition-session", requestSession: "direct-session", wantSession: "direct-session"},
{name: "no effective session"},
} {
t.Run(tc.name, func(t *testing.T) {
promptDir := t.TempDir()
profileDir := t.TempDir()
sessionLine := ""
if tc.definitionSession != "" {
sessionLine = "session_id: " + tc.definitionSession + "\n"
}
if err := os.WriteFile(filepath.Join(promptDir, "prompt.yaml"), []byte("id: p\nversion: \"1\"\ndefault_profile: exec\n"+sessionLine+`messages:
- role: user
content: "hi"
output:
format: text
validation_mode: none
`), 0o644); err != nil {
t.Fatalf("write prompt fixture: %v", err)
}
if err := os.WriteFile(filepath.Join(profileDir, "profile.yaml"), []byte("id: exec\nendpoint: http://127.0.0.1:1/v1\nmodel: test\n"), 0o644); err != nil {
t.Fatalf("write profile fixture: %v", err)
}
engine, err := promptkit.NewEngine(promptkit.Config{PromptDir: promptDir, ProfileDir: profileDir}, promptkit.WithLLMClient(handlerLLMClient{}))
if err != nil {
t.Fatalf("new engine: %v", err)
}
body := `{"prompt_id":"p"}`
if tc.requestSession != "" {
body = `{"prompt_id":"p","session_id":"` + tc.requestSession + `"}`
}
w := httptest.NewRecorder()
NewHandler(engine).ServeHTTP(w, httptest.NewRequest(http.MethodPost, "/v1/runs", bytes.NewBufferString(body)))
if w.Code != http.StatusOK {
t.Fatalf("expected 200, got %d body=%s", w.Code, w.Body.String())
}
var resp map[string]any
if err := json.Unmarshal(w.Body.Bytes(), &resp); err != nil {
t.Fatalf("invalid JSON response: %v", err)
}
metadata := resp["metadata"].(map[string]any)
if tc.wantSession == "" {
if _, ok := metadata["session_id"]; ok {
t.Fatalf("expected session_id to be omitted, got %#v", metadata)
}
return
}
if metadata["session_id"] != tc.wantSession {
t.Fatalf("expected effective session ID %q, got %#v", tc.wantSession, metadata["session_id"])
}
})
}
}
func TestHandlerSharesBackendCapacityAcrossConcurrentRequests(t *testing.T) {
promptDir := t.TempDir()
profileDir := t.TempDir()
if err := os.WriteFile(filepath.Join(promptDir, "prompt.yaml"), []byte(`id: p
version: "1"
default_profile: limited
messages:
- role: user
content: "hi"
output:
format: text
validation_mode: none
`), 0o644); err != nil {
t.Fatalf("write prompt fixture: %v", err)
}
if err := os.WriteFile(filepath.Join(profileDir, "profile.yaml"), []byte(`id: limited
backend: limited
model: test
`), 0o644); err != nil {
t.Fatalf("write profile fixture: %v", err)
}
queueCapacity := 0
client := &blockingLLMClient{started: make(chan struct{}, 1), release: make(chan struct{})}
engine, err := promptkit.NewEngine(
promptkit.Config{PromptDir: promptDir, ProfileDir: profileDir},
promptkit.WithBackend(promptkit.Backend{
ID: "limited",
Endpoint: "http://127.0.0.1:1/v1",
ConcurrencyLimit: 1,
QueueCapacity: &queueCapacity,
}),
promptkit.WithLLMClient(client),
)
if err != nil {
t.Fatalf("new engine: %v", err)
}
h := NewHandler(engine)
first := httptest.NewRecorder()
firstDone := make(chan struct{})
go func() {
defer close(firstDone)
h.ServeHTTP(first, httptest.NewRequest(http.MethodPost, "/v1/runs", bytes.NewBufferString(`{"prompt_id":"p"}`)))
}()
select {
case <-client.started:
case <-time.After(time.Second):
t.Fatal("first request did not reach generation")
}
second := httptest.NewRecorder()
h.ServeHTTP(second, httptest.NewRequest(http.MethodPost, "/v1/runs", bytes.NewBufferString(`{"prompt_id":"p"}`)))
assertHTTPErrorCode(t, second, http.StatusServiceUnavailable, "capacity_exceeded")
if second.Header().Get("Retry-After") != "" {
t.Fatalf("expected no Retry-After header, got %q", second.Header().Get("Retry-After"))
}
if strings.Contains(second.Body.String(), "limited") {
t.Fatalf("capacity response leaked backend details: %s", second.Body.String())
}
close(client.release)
select {
case <-firstDone:
case <-time.After(time.Second):
t.Fatal("first request did not complete")
}
if first.Code != http.StatusOK {
t.Fatalf("expected first request to succeed, got %d body=%s", first.Code, first.Body.String())
}
if atomic.LoadInt32(&client.peak) != 1 {
t.Fatalf("expected peak generation concurrency of one, got %d", atomic.LoadInt32(&client.peak))
}
}
func TestHandlerRejectsOverlongSessionIDAndNonStringReasoningEffort(t *testing.T) {
engine := newHandlerEngine(t)
for _, body := range []string{
`{"prompt_id":"p","session_id":"` + strings.Repeat("x", 257) + `"}`,
`{"prompt_id":"p","model":{"reasoning_effort":1}}`,
} {
w := httptest.NewRecorder()
NewHandler(engine).ServeHTTP(w, httptest.NewRequest(http.MethodPost, "/v1/runs", bytes.NewBufferString(body)))
if w.Code != http.StatusBadRequest {
t.Fatalf("expected 400, got %d body=%s", w.Code, w.Body.String())
}
}
}
func TestHandlerModelOverrideAcceptsJSONCompatibleExtraParams(t *testing.T) {
r := &fakeRunner{result: &promptkit.RunResult{
Artifact: promptkit.Artifact{Body: []byte("ok")},
@@ -722,6 +1124,7 @@ func TestHandlerPublicErrorMapping(t *testing.T) {
{name: "file too large", err: ErrFileTooLarge, status: http.StatusRequestEntityTooLarge, code: "artifact_too_large", message: "file input artifact is too large"},
{name: "artifact", err: wrap(promptkit.ErrArtifactLoad, fmt.Errorf("read failed")), status: http.StatusBadRequest, code: "artifact_read_failed", message: "failed to read input artifact", avoidCause: "read failed"},
{name: "prompt render", err: wrap(promptkit.ErrPromptRender, fmt.Errorf("render failed")), status: http.StatusBadRequest, code: "prompt_render_failed", message: "failed to render prompt", avoidCause: "render failed"},
{name: "capacity", err: &promptkit.CapacityError{BackendID: "private-backend"}, status: http.StatusServiceUnavailable, code: "capacity_exceeded", message: "model backend capacity is exhausted", avoidCause: "private-backend"},
{name: "llm", err: wrap(promptkit.ErrLLMGenerate, fmt.Errorf("llm failed")), status: http.StatusBadGateway, code: "llm_failed", message: "model generation request failed", avoidCause: "llm failed"},
{name: "validation runtime", err: wrap(promptkit.ErrValidation, fmt.Errorf("validator broke")), status: http.StatusInternalServerError, code: "validation_runtime_failed", message: "validation runtime failed", avoidCause: "validator broke"},
}
@@ -889,6 +1292,31 @@ model: model
return engine
}
func newDefinitionHandler(t *testing.T, definition string) *Handler {
t.Helper()
promptDir := t.TempDir()
profileDir := t.TempDir()
if err := os.WriteFile(filepath.Join(promptDir, "prompt.yaml"), []byte(definition), 0o644); err != nil {
t.Fatalf("write prompt fixture: %v", err)
}
if err := os.WriteFile(filepath.Join(profileDir, "profile.yaml"), []byte(`id: exec
endpoint: http://example.invalid/v1
model: model
`), 0o644); err != nil {
t.Fatalf("write profile fixture: %v", err)
}
engine, err := promptkit.NewEngine(promptkit.Config{
PromptDir: promptDir,
ProfileDir: profileDir,
}, promptkit.WithLLMClient(handlerLLMClient{}))
if err != nil {
t.Fatalf("construct public engine: %v", err)
}
return NewHandler(engine)
}
func assertHTTPErrorCode(t *testing.T, w *httptest.ResponseRecorder, status int, code string) {
t.Helper()

View File

@@ -7,6 +7,7 @@ import (
"io"
"os"
"path/filepath"
"sort"
"strings"
"gitea.maximumdirect.net/eric/scriptorium/internal/defaults"
@@ -37,6 +38,24 @@ type Config struct {
SchemaDir string `yaml:"schema_dir"`
Server ServerConfig `yaml:"server"`
Defaults DefaultsConfig `yaml:"defaults"`
Backends map[string]BackendConfig `yaml:"backends"`
}
type BackendConfig struct {
Endpoint string `yaml:"endpoint"`
APIKeyEnv string `yaml:"api_key_env"`
ExtraParams map[string]any `yaml:"extra_params"`
ConcurrencyLimit int `yaml:"concurrency_limit"`
QueueCapacity *int `yaml:"queue_capacity"`
}
type BackendSettings struct {
ID string
Endpoint string
APIKeyEnv string
ExtraParams map[string]any
ConcurrencyLimit int
QueueCapacity *int
}
type ServerConfig struct {
@@ -62,6 +81,7 @@ type AppSettings struct {
MaxArtifactBytes int64
MaxResponseBytes int64
DefaultRenderFormat renderformat.PreparedRunOutputFormat
Backends []BackendSettings
}
// CLIOverrides can be applied after config load to enforce precedence.
@@ -245,6 +265,25 @@ func applyConfig(base AppSettings, cfg Config) (AppSettings, error) {
}
out.DefaultRenderFormat = parsed
}
if len(cfg.Backends) > 0 {
ids := make([]string, 0, len(cfg.Backends))
for id := range cfg.Backends {
ids = append(ids, id)
}
sort.Strings(ids)
out.Backends = make([]BackendSettings, 0, len(ids))
for _, id := range ids {
backend := cfg.Backends[id]
out.Backends = append(out.Backends, BackendSettings{
ID: id,
Endpoint: backend.Endpoint,
APIKeyEnv: backend.APIKeyEnv,
ExtraParams: backend.ExtraParams,
ConcurrencyLimit: backend.ConcurrencyLimit,
QueueCapacity: backend.QueueCapacity,
})
}
}
return out, nil
}

View File

@@ -4,6 +4,7 @@ import (
"errors"
"os"
"path/filepath"
"reflect"
"testing"
"gitea.maximumdirect.net/eric/scriptorium/internal/defaults"
@@ -20,7 +21,7 @@ func TestLoadConfigMissingImplicitPathUsesBuiltInDefaults(t *testing.T) {
}
want := BuiltInDefaults()
if got != want {
if !reflect.DeepEqual(got, want) {
t.Fatalf("unexpected settings: got=%+v want=%+v", got, want)
}
}
@@ -100,6 +101,78 @@ func TestLoadConfigAPIKeyFieldIsRejectedAsUnknown(t *testing.T) {
}
}
func TestLoadConfigBackendsRetainsResolvedSettingsInSortedOrder(t *testing.T) {
path := writeConfigFile(t, "config.yml", `
backends:
zebra:
endpoint: https://zebra.example/v1
api_key_env: ZEBRA_API_KEY
extra_params:
provider_option: enabled
nested:
enabled: true
attempts: 2
concurrency_limit: 2
alpha:
endpoint: http://alpha.example/v1
queue_capacity: 0
`)
got, err := LoadConfig(path, true)
if err != nil {
t.Fatalf("expected no error, got %v", err)
}
if len(got.Backends) != 2 {
t.Fatalf("expected two backends, got %#v", got.Backends)
}
if got.Backends[0].ID != "alpha" || got.Backends[1].ID != "zebra" {
t.Fatalf("expected sorted backend IDs, got %#v", got.Backends)
}
if got.Backends[0].QueueCapacity == nil || *got.Backends[0].QueueCapacity != 0 {
t.Fatalf("expected explicit zero queue capacity, got %#v", got.Backends[0].QueueCapacity)
}
if got.Backends[1].QueueCapacity != nil {
t.Fatalf("expected omitted queue capacity to remain nil, got %#v", got.Backends[1].QueueCapacity)
}
wantParams := map[string]any{
"provider_option": "enabled",
"nested": map[string]any{
"enabled": true,
"attempts": 2,
},
}
if !reflect.DeepEqual(got.Backends[1].ExtraParams, wantParams) {
t.Fatalf("unexpected extra params: got=%#v want=%#v", got.Backends[1].ExtraParams, wantParams)
}
if got.Backends[1].Endpoint != "https://zebra.example/v1" || got.Backends[1].APIKeyEnv != "ZEBRA_API_KEY" || got.Backends[1].ConcurrencyLimit != 2 {
t.Fatalf("unexpected zebra backend: %#v", got.Backends[1])
}
}
func TestLoadConfigRejectsUnknownOrSecretBackendFields(t *testing.T) {
for name, body := range map[string]string{
"unknown": "backends:\n local:\n endpoint: http://localhost:11434/v1\n unexpected: value\n",
"secret": "backends:\n local:\n endpoint: http://localhost:11434/v1\n api_key: secret\n",
} {
t.Run(name, func(t *testing.T) {
path := writeConfigFile(t, "config.yml", body)
_, err := LoadConfig(path, true)
if !errors.Is(err, ErrInvalidConfigYAML) {
t.Fatalf("expected ErrInvalidConfigYAML, got %v", err)
}
})
}
}
func TestLoadConfigRejectsInvalidBackendFieldTypes(t *testing.T) {
path := writeConfigFile(t, "config.yml", "backends:\n local:\n endpoint: http://localhost:11434/v1\n queue_capacity: not-a-number\n")
_, err := LoadConfig(path, true)
if !errors.Is(err, ErrInvalidConfigYAML) {
t.Fatalf("expected ErrInvalidConfigYAML, got %v", err)
}
}
func TestLoadConfigValidConfigSetsDirectoriesAndServerAddr(t *testing.T) {
path := writeConfigFile(t, "config.yml", `
prompt_dir: ./prompts
@@ -196,7 +269,7 @@ func TestLoadConfigEmptyFileResolvesToBuiltInDefaults(t *testing.T) {
}
want := BuiltInDefaults()
if got != want {
if !reflect.DeepEqual(got, want) {
t.Fatalf("unexpected settings: got=%+v want=%+v", got, want)
}
}

View File

@@ -0,0 +1,122 @@
package format
import (
"bytes"
"encoding/json"
"errors"
"fmt"
"strconv"
"gitea.maximumdirect.net/eric/promptkit"
)
type PromptInspection struct {
PromptID string `json:"prompt_id"`
PromptVersion string `json:"prompt_version"`
PromptHash string `json:"prompt_hash"`
DefaultProfileID string `json:"default_profile_id"`
Inputs []PromptInspectionInput `json:"inputs"`
OutputContract OutputContract `json:"output_contract"`
}
type PromptInspectionInput struct {
Name string `json:"name"`
Required bool `json:"required"`
ContentType string `json:"content_type"`
Description string `json:"description"`
}
type OutputContract struct {
Format string `json:"format"`
ValidationMode string `json:"validation_mode"`
SchemaPath string `json:"schema_path"`
RepairAttempts int `json:"repair_attempts"`
}
type ProfileInspection struct {
ProfileID string `json:"profile_id"`
EffectiveModelParams ProfileModelParams `json:"effective_model_params"`
APIKeyRequired bool `json:"api_key_required"`
}
type ProfileModelParams struct {
BackendID string `json:"backend_id"`
Endpoint string `json:"endpoint"`
Model string `json:"model"`
Temperature float64 `json:"temperature"`
MaxTokens int `json:"max_tokens"`
TopP float64 `json:"top_p"`
TimeoutSeconds int `json:"timeout_seconds"`
ServiceTier string `json:"service_tier"`
ReasoningEffort string `json:"reasoning_effort"`
APIKeyEnv string `json:"api_key_env"`
ExtraParams map[string]any `json:"extra_params"`
}
func FormatPromptInspection(value *promptkit.PromptInspection, outputFormat OutputFormat) ([]byte, error) {
if value == nil {
return nil, errors.New("prompt inspection is nil")
}
dto := PromptInspection{
PromptID: value.PromptID, PromptVersion: value.PromptVersion, PromptHash: value.PromptHash, DefaultProfileID: value.DefaultProfileID,
Inputs: make([]PromptInspectionInput, len(value.Inputs)),
OutputContract: OutputContract{Format: string(value.OutputContract.Format), ValidationMode: string(value.OutputContract.ValidationMode), SchemaPath: value.OutputContract.SchemaPath, RepairAttempts: value.OutputContract.RepairAttempts},
}
for i, input := range value.Inputs {
dto.Inputs[i] = PromptInspectionInput{Name: input.Name, Required: input.Required, ContentType: input.ContentType, Description: input.Description}
}
switch outputFormat {
case OutputFormatJSON:
data, err := json.MarshalIndent(dto, "", " ")
if err != nil {
return nil, err
}
return append(data, '\n'), nil
case OutputFormatText:
var b bytes.Buffer
fmt.Fprintf(&b, "prompt_id: %s\nprompt_version: %s\nprompt_hash: %s\ndefault_profile_id: %s\ninputs:", dto.PromptID, dto.PromptVersion, dto.PromptHash, dto.DefaultProfileID)
if len(dto.Inputs) == 0 {
fmt.Fprintln(&b, " []")
} else {
fmt.Fprintln(&b)
for _, input := range dto.Inputs {
fmt.Fprintf(&b, " - name: %s\n required: %t\n content_type: %s\n description: %s\n", input.Name, input.Required, input.ContentType, strconv.Quote(input.Description))
}
}
fmt.Fprintf(&b, "output_contract:\n format: %s\n validation_mode: %s\n schema_path: %s\n repair_attempts: %d\n", dto.OutputContract.Format, dto.OutputContract.ValidationMode, dto.OutputContract.SchemaPath, dto.OutputContract.RepairAttempts)
return b.Bytes(), nil
default:
return nil, fmt.Errorf("%w: %q", ErrUnknownPreparedRunFormat, outputFormat)
}
}
func FormatProfileInspection(value *promptkit.ProfileInspection, outputFormat OutputFormat) ([]byte, error) {
if value == nil {
return nil, errors.New("profile inspection is nil")
}
target := value.EffectiveModelParams
params := map[string]any{}
for key, item := range target.ExtraParams {
params[key] = item
}
dto := ProfileInspection{ProfileID: value.ProfileID, APIKeyRequired: value.APIKeyRequired, EffectiveModelParams: ProfileModelParams{BackendID: target.BackendID, Endpoint: target.Endpoint, Model: target.Model, Temperature: target.Temperature, MaxTokens: target.MaxTokens, TopP: target.TopP, TimeoutSeconds: target.TimeoutSeconds, ServiceTier: target.ServiceTier, ReasoningEffort: target.ReasoningEffort, APIKeyEnv: target.APIKeyEnv, ExtraParams: params}}
switch outputFormat {
case OutputFormatJSON:
data, err := json.MarshalIndent(dto, "", " ")
if err != nil {
return nil, err
}
return append(data, '\n'), nil
case OutputFormatText:
var b bytes.Buffer
fmt.Fprintf(&b, "profile_id: %s\neffective_model_params:\n backend_id: %s\n endpoint: %s\n model: %s\n temperature: %g\n max_tokens: %d\n top_p: %g\n timeout_seconds: %d\n service_tier: %s\n reasoning_effort: %s\n api_key_env: %s\n extra_params: ", dto.ProfileID, dto.EffectiveModelParams.BackendID, dto.EffectiveModelParams.Endpoint, dto.EffectiveModelParams.Model, dto.EffectiveModelParams.Temperature, dto.EffectiveModelParams.MaxTokens, dto.EffectiveModelParams.TopP, dto.EffectiveModelParams.TimeoutSeconds, dto.EffectiveModelParams.ServiceTier, dto.EffectiveModelParams.ReasoningEffort, dto.EffectiveModelParams.APIKeyEnv)
paramsJSON, err := json.Marshal(dto.EffectiveModelParams.ExtraParams)
if err != nil {
return nil, err
}
fmt.Fprintf(&b, "%s\napi_key_required: %t\n", paramsJSON, dto.APIKeyRequired)
return b.Bytes(), nil
default:
return nil, fmt.Errorf("%w: %q", ErrUnknownPreparedRunFormat, outputFormat)
}
}

View File

@@ -0,0 +1,124 @@
package format
import (
"encoding/json"
"strings"
"testing"
"gitea.maximumdirect.net/eric/promptkit"
)
func TestFormatPromptInspectionPreservesDeclaredOrderAndEmptyValues(t *testing.T) {
value := &promptkit.PromptInspection{PromptID: "p", PromptVersion: "1", PromptHash: "hash", Inputs: []promptkit.PromptInputDefinition{{Name: "first", Required: true, ContentType: "text/plain", Description: "first input"}, {Name: "second"}}, OutputContract: promptkit.OutputContract{Format: "text", ValidationMode: promptkit.ValidationNone}}
text, err := FormatPromptInspection(value, OutputFormatText)
if err != nil {
t.Fatal(err)
}
if !strings.Contains(string(text), "default_profile_id: \ninputs:\n - name: first") || strings.Index(string(text), "name: first") > strings.Index(string(text), "name: second") {
t.Fatalf("unexpected text inspection: %s", text)
}
jsonOutput, err := FormatPromptInspection(value, OutputFormatJSON)
if err != nil {
t.Fatal(err)
}
if !strings.HasSuffix(string(jsonOutput), "\n") || !strings.Contains(string(jsonOutput), `"default_profile_id": ""`) {
t.Fatalf("unexpected json inspection: %s", jsonOutput)
}
}
func TestFormatPromptInspectionRejectsNil(t *testing.T) {
if _, err := FormatPromptInspection(nil, OutputFormatText); err == nil {
t.Fatal("expected nil inspection error")
}
}
func TestFormatProfileInspectionPreservesSafeEffectiveValues(t *testing.T) {
const secret = "sentinel-secret-must-not-appear"
t.Setenv("FIXTURE_PROFILE_API_KEY", secret)
value := &promptkit.ProfileInspection{
ProfileID: "custom-derived",
EffectiveModelParams: promptkit.ExecutionTarget{
BackendID: "fixture-custom",
Endpoint: "http://127.0.0.1:11434/v1",
Model: "fixture-model",
Temperature: 0.25,
MaxTokens: 640,
TopP: 0.9,
TimeoutSeconds: 45,
ServiceTier: "flex",
ReasoningEffort: "high",
APIKeyEnv: "FIXTURE_PROFILE_API_KEY",
ExtraParams: map[string]any{
"zeta": true,
"alpha": map[string]any{"nested": []any{"first", 2.0}},
},
},
APIKeyRequired: true,
}
jsonOutput, err := FormatProfileInspection(value, OutputFormatJSON)
if err != nil {
t.Fatal(err)
}
secondJSONOutput, err := FormatProfileInspection(value, OutputFormatJSON)
if err != nil {
t.Fatal(err)
}
if string(jsonOutput) != string(secondJSONOutput) || !strings.HasSuffix(string(jsonOutput), "\n") {
t.Fatalf("expected deterministic newline-terminated JSON, got %q", jsonOutput)
}
if strings.Contains(string(jsonOutput), secret) {
t.Fatalf("inspection exposed an environment secret: %s", jsonOutput)
}
var decoded ProfileInspection
if err := json.Unmarshal(jsonOutput, &decoded); err != nil {
t.Fatalf("decode profile inspection: %v", err)
}
if decoded.ProfileID != "custom-derived" || decoded.EffectiveModelParams.BackendID != "fixture-custom" {
t.Fatalf("unexpected profile identity: %+v", decoded)
}
if decoded.EffectiveModelParams.APIKeyEnv != "FIXTURE_PROFILE_API_KEY" || !decoded.APIKeyRequired {
t.Fatalf("unexpected credential metadata: %+v", decoded)
}
alpha, ok := decoded.EffectiveModelParams.ExtraParams["alpha"].(map[string]any)
if !ok || len(alpha["nested"].([]any)) != 2 {
t.Fatalf("nested extra parameters were not preserved: %#v", decoded.EffectiveModelParams.ExtraParams)
}
textOutput, err := FormatProfileInspection(value, OutputFormatText)
if err != nil {
t.Fatal(err)
}
textValue := string(textOutput)
if !strings.Contains(textValue, "backend_id: fixture-custom") ||
!strings.Contains(textValue, "api_key_env: FIXTURE_PROFILE_API_KEY") ||
!strings.Contains(textValue, `extra_params: {"alpha":{"nested":["first",2]},"zeta":true}`) ||
!strings.Contains(textValue, "api_key_required: true") {
t.Fatalf("unexpected text inspection: %s", textValue)
}
if strings.Contains(textValue, secret) {
t.Fatalf("text inspection exposed an environment secret: %s", textValue)
}
}
func TestFormatProfileInspectionPreservesEmptyBackendAndRejectsNil(t *testing.T) {
output, err := FormatProfileInspection(&promptkit.ProfileInspection{
ProfileID: "endpoint-only",
EffectiveModelParams: promptkit.ExecutionTarget{
Endpoint: "http://127.0.0.1:8000/v1",
Model: "fixture-model",
},
}, OutputFormatJSON)
if err != nil {
t.Fatal(err)
}
if !strings.Contains(string(output), `"backend_id": ""`) ||
!strings.Contains(string(output), `"extra_params": {}`) {
t.Fatalf("expected explicit empty backend and object extra params, got %s", output)
}
if _, err := FormatProfileInspection(nil, OutputFormatText); err == nil {
t.Fatal("expected nil profile inspection error")
}
}

View File

@@ -14,14 +14,24 @@ import (
var ErrUnknownPreparedRunFormat = errors.New("unknown prepared run format")
// PreparedRunOutputFormat is the output format for prepared render data.
type PreparedRunOutputFormat string
// OutputFormat selects an application-owned textual or JSON representation.
type OutputFormat string
const (
PreparedRunFormatText PreparedRunOutputFormat = "text"
PreparedRunFormatJSON PreparedRunOutputFormat = "json"
OutputFormatText OutputFormat = "text"
OutputFormatJSON OutputFormat = "json"
DefaultPreparedRunOutputFormat PreparedRunOutputFormat = PreparedRunFormatText
DefaultOutputFormat OutputFormat = OutputFormatText
)
// PreparedRunOutputFormat is the output format for prepared render data.
type PreparedRunOutputFormat = OutputFormat
const (
PreparedRunFormatText = OutputFormatText
PreparedRunFormatJSON = OutputFormatJSON
DefaultPreparedRunOutputFormat = DefaultOutputFormat
)
// PreparedRunFormatter serializes a prepared run without performing use case work.
@@ -31,15 +41,20 @@ type PreparedRunFormatter interface {
// ParsePreparedRunOutputFormat parses a format name.
func ParsePreparedRunOutputFormat(raw string) (PreparedRunOutputFormat, error) {
switch PreparedRunOutputFormat(strings.ToLower(strings.TrimSpace(raw))) {
return ParseOutputFormat(raw)
}
// ParseOutputFormat parses the shared inspection and prepared-run format names.
func ParseOutputFormat(raw string) (OutputFormat, error) {
switch OutputFormat(strings.ToLower(strings.TrimSpace(raw))) {
case "":
return DefaultPreparedRunOutputFormat, nil
case PreparedRunFormatText:
return PreparedRunFormatText, nil
case PreparedRunFormatJSON:
return PreparedRunFormatJSON, nil
return DefaultOutputFormat, nil
case OutputFormatText:
return OutputFormatText, nil
case OutputFormatJSON:
return OutputFormatJSON, nil
default:
return "", fmt.Errorf("%w: %q (supported: %s, %s)", ErrUnknownPreparedRunFormat, raw, PreparedRunFormatText, PreparedRunFormatJSON)
return "", fmt.Errorf("%w: %q (supported: %s, %s)", ErrUnknownPreparedRunFormat, raw, OutputFormatText, OutputFormatJSON)
}
}
@@ -93,6 +108,9 @@ func (textPreparedRunFormatter) Format(prepared *promptkit.PreparedRun) ([]byte,
fmt.Fprintf(&b, "prompt: %s\n", prepared.PromptID)
fmt.Fprintf(&b, "prompt_version: %s\n", prepared.PromptVersion)
fmt.Fprintf(&b, "selected_profile_id: %s\n", prepared.SelectedProfileID)
if prepared.SelectedBackendID != "" {
fmt.Fprintf(&b, "selected_backend_id: %s\n", prepared.SelectedBackendID)
}
if prepared.PromptHash != "" {
fmt.Fprintf(&b, "prompt_hash: %s\n", prepared.PromptHash)
}

View File

@@ -22,6 +22,7 @@ func TestTextFormatterIncludesPreparedRunDetails(t *testing.T) {
"prompt: prompt.id",
"prompt_version: v1",
"selected_profile_id: local-fast",
"selected_backend_id: local",
"endpoint: http://llm/v1",
"model: gpt-test",
"temperature: 0.4",
@@ -346,7 +347,9 @@ func samplePreparedRun() *promptkit.PreparedRun {
PromptVersion: "v1",
PromptHash: "prompt-hash",
SelectedProfileID: "local-fast",
SelectedBackendID: "local",
EffectiveModelParams: promptkit.ExecutionTarget{
BackendID: "local",
Endpoint: "http://llm/v1",
Model: "gpt-test",
Temperature: 0.4,

12
testdata/promptkit-v0.9/config.yml vendored Normal file
View File

@@ -0,0 +1,12 @@
prompt_dir: ./testdata/promptkit-v0.9/prompts
profile_dir: ./testdata/promptkit-v0.9/profiles
schema_dir: ./testdata/promptkit-v0.9/schemas
backends:
fixture-custom:
endpoint: http://127.0.0.1:11434/v1
api_key_env: FIXTURE_BACKEND_API_KEY
extra_params:
backend_option:
enabled: true
concurrency_limit: 2
queue_capacity: 0

View File

@@ -0,0 +1 @@
The fixture source contains stable, synthetic material.

View File

@@ -0,0 +1,16 @@
id: custom-base
backend: fixture-custom
model: fixture-base-model
temperature: 0.25
max_tokens: 640
top_p: 0.9
timeout_seconds: 45
service_tier: flex
reasoning_effort: medium
api_key_env: FIXTURE_PROFILE_API_KEY
extra_params:
routing:
order:
- primary
- fallback
allow_fallback: true

View File

@@ -0,0 +1,4 @@
id: custom-derived
base_profile: custom-base
model: fixture-derived-model
reasoning_effort: high

View File

@@ -0,0 +1,10 @@
id: endpoint-only
endpoint: http://127.0.0.1:8000/v1
model: fixture-endpoint-model
temperature: 0.1
max_tokens: 128
top_p: 0.75
timeout_seconds: 30
extra_params:
endpoint_option:
nested: true

View File

@@ -0,0 +1,31 @@
id: compat.complete
version: "1.0.0"
description: Exercise the complete Promptkit v0.9 prompt-definition surface.
default_profile: custom-derived
session_id: 'fixture-{{.topic}}'
inputs:
- name: source
required: true
content_type: text/markdown
description: Source material rendered by a file-backed message.
- name: context
required: false
content_type: text/plain
description: Optional caller context.
messages:
- role: developer
content: 'Use the topic {{.topic}}.'
cache_control:
type: ephemeral
ttl: 1h
- role: system
content_file: ./messages/complete.system.md
- role: user
content_file: ./messages/complete.user.md
- role: assistant
content: Return one JSON object.
output:
format: json
validation_mode: json_schema
schema_path: complete.schema.json
repair_attempts: 2

View File

@@ -0,0 +1,10 @@
id: compat.complete
version: "2.0.0"
default_profile: endpoint-only
messages:
- role: user
content: Version two has no declared inputs.
output:
format: text
validation_mode: basic
repair_attempts: 1

View File

@@ -0,0 +1,10 @@
id: compat.json
version: "1.0.0"
default_profile: endpoint-only
messages:
- role: user
content: Return JSON.
output:
format: json
validation_mode: json
repair_attempts: 0

View File

@@ -0,0 +1 @@
Use the supplied source without inventing details.

View File

@@ -0,0 +1,4 @@
Topic: {{.topic}}
Source:
{{input "source"}}

View File

@@ -0,0 +1,10 @@
id: compat.none
version: "1.0.0"
default_profile: endpoint-only
messages:
- role: user
content: Return text.
output:
format: text
validation_mode: none
repair_attempts: 0

View File

@@ -0,0 +1,11 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"type": "object",
"required": ["summary"],
"properties": {
"summary": {
"type": "string"
}
},
"additionalProperties": false
}