scriptorium/docs/roadmap/promptkit-v0.9.0-adoption.md

# Promptkit v0.9.0 Adoption Roadmap

## Status

Complete. This document records the delivered feature scope and targeted end
state. The completed delivery plan is maintained in
[implementation.md](implementation.md); current behavior remains defined by
the canonical application contracts linked below.

The durable product and integration boundary for this work is established by
[ADR 0004](../adr/0004-definition-boundary.md). This roadmap applies that
decision to Promptkit v0.9.0 and owns implementation scope and completion
status.

## Objective

Bring Scriptorium from Promptkit v0.1.0 to v0.9.0 and expose the newer
framework capabilities that fit Scriptorium's existing CLI, HTTP,
configuration, presentation, and process boundaries.

Scriptorium must accept the complete Promptkit v0.9.0 feature set expressed by
directory-backed prompt, profile, and schema definitions. Application adapters
must not narrow that format contract by requiring request values that the
selected definition does not require or by omitting selection values needed to
address a valid definition. Promptkit's tagged
[framework format reference](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.9.0/docs/formats.md)
remains canonical; Scriptorium must delegate parsing and framework validation
rather than reproduce those rules.

The completed application will retain Promptkit as the owner of reusable
prompt execution, source formats, profiles, backend definitions, capacity,
inspection, generation, and validation. Scriptorium will continue to own
application configuration, command and HTTP contracts, error and exit mapping,
output presentation, and deployment policy.

## Targeted End State

When this roadmap is complete:

- Scriptorium pins the released Promptkit v0.9.0 module without a workspace,
  replacement, vendored source, direct catalog import, or Promptkit internal
  import.
- Existing valid CLI, HTTP, configuration, prompt, profile, schema, and render
  workflows remain supported subject to Promptkit's documented v0.2.0 through
  v0.9.0 compatibility corrections.
- Every Promptkit v0.9.0 prompt and profile feature expressible in definitions
  under Scriptorium's configured directory sources is usable through the
  executable application, including prompts with no inputs and explicitly
  selected prompt versions.
- CLI and HTTP requests can supply a direct non-secret session ID.
- HTTP requests and a presence-aware CLI flag can inherit, replace, or
  explicitly clear reasoning effort.
- Prepared output, CLI run summaries, and HTTP result metadata expose effective
  backend identity when Promptkit supplies one.
- HTTP capacity rejection has a stable Scriptorium-owned overload response.
- Application configuration can register reusable engine-scoped custom
  backends, including bounded concurrency and queue policy, for selection by
  Promptkit profiles.
- CLI users can inspect prompt and profile definitions without supplying
  placeholder inputs or invoking a model.
- Scriptorium's contracts and examples point to Promptkit v0.9.0 and accurately
  describe its current credential, validation-repair, provider-control,
  message-role, backend, and source-validation semantics.

## Feature Scope

### Complete Prompt And Profile Definition Compatibility

Scriptorium will treat the tagged Promptkit v0.9.0 format contract as one
indivisible downstream compatibility boundary. This includes prompt identity
and version selection, optional and required input declarations, message and
content-file templates, supported roles, cache control, session templates,
default profiles, output contracts, schemas, and repair budgets. It also
includes standalone and inherited profiles, built-in and custom backend
selection, endpoint overrides, execution controls, optional credential
environment sources, and JSON-compatible extra parameters.

Most of this support is provided by passing configured prompt, profile, and
schema directories directly to Promptkit. Scriptorium will not independently
decode these framework definitions or maintain a field-level subset. Two
current adapter restrictions must be removed so the executable interfaces do
not reject definitions that Promptkit accepts:

- `run` and `render` will accept an optional prompt-version flag and map it to
  `promptkit.RunRequest.PromptVersion`; and
- `run`, `render`, and `POST /v1/runs` will permit omitted or empty input maps.
  Promptkit will decide whether an input is required by the selected
  definition or referenced template.

The HTTP API already carries `prompt_version`; its mapping must remain covered
by adapter tests. CLI and HTTP callers may continue to supply extra inputs,
subject to their existing file, containment, and size policies. Removing the
application-level nonempty-input requirement does not weaken Promptkit's
declared-required-input or template-reference validation.

Profiles that name built-in backend IDs require no application registration.
Profiles that name other backend IDs become fully usable through the custom
backend configuration in this roadmap. Profile inheritance and all other
profile-field resolution remain Promptkit responsibilities.

This compatibility requirement concerns fields and behavior expressed in
Promptkit YAML definitions. Go-consumer construction alternatives such as
single-file sources, injected `fs.FS` sources, in-memory profiles, fallback
profile filesystems, and prepared-execution handles are library-integration
features and are not required merely to support the complete definition
format.

### Promptkit v0.9.0 Compatibility Baseline

Required dependency and source changes:

- update `go.mod` and `go.sum` to select Promptkit v0.9.0 and its selected
  OpenRouter and Rakestrawhome catalog dependencies;
- change Scriptorium's inbound HTTP reasoning-effort representation to preserve
  Promptkit's pointer semantics;
- retain keyed Promptkit public struct literals and confirm no removed
  `RunRequest.Metadata` use exists;
- update tagged Promptkit links and version references throughout Scriptorium;
  and
- preserve Scriptorium's architecture guard against replacements, workspaces,
  vendored Promptkit code, former facade packages, and Promptkit internal
  imports.

Required compatibility review:

- verify application and maintained example endpoints satisfy Promptkit's
  absolute HTTP/HTTPS endpoint rules;
- verify prompt content paths, file artifacts, identities, JSON documents, and
  JSON-compatible extra parameters satisfy the v0.6.0 safety boundaries;
- verify maintained prompt roles are limited to `developer`, `system`, `user`,
  and `assistant`;
- verify every positive repair budget is no greater than three and is paired
  with `basic`, `json`, or `json_schema` validation;
- document that positive repair budgets authorize additional provider calls,
  latency, token use, and cost;
- document that unset optional provider controls are omitted and that a zero
  effective value can represent an unspecified provider control unless an
  explicit request override supplied it; and
- update credential guidance for Promptkit's optional environment lookup
  behavior, profile `api_key_env` semantics, and Scriptorium's prohibition on
  raw API-key inputs.

### Session And Reasoning Request Controls

The CLI `run` and `render` commands will accept a direct session-ID flag and map
it to `promptkit.RunRequest.SessionID`. The value remains non-secret
application correlation metadata and is subject to Promptkit's normalization
and length rules.

The HTTP run request will accept `session_id` and map it through the existing
strict DTO boundary. Successful HTTP metadata will report the effective
session ID when present.

The CLI `run` and `render` commands will also accept a presence-aware reasoning
effort flag. Omission inherits the selected profile, a nonblank value replaces
it, and an explicitly supplied empty value clears it. The existing HTTP
`model.reasoning_effort` field will gain the same three-state behavior while
remaining a JSON string when present.

These additions must not introduce raw direct API-key flags or fields.

### Backend Identity And Capacity Outcomes

Scriptorium presentation will expose `SelectedBackendID` and the effective
target's `BackendID` without deriving identity from endpoint text. Backend
identity will be included where applicable in:

- prepared-run text and JSON output;
- the CLI run summary; and
- HTTP response metadata and effective model parameters.

Endpoint-only profiles continue to have no backend ID, and empty identities
must remain distinguishable from registered built-in or custom backends.

The HTTP adapter will classify `promptkit.ErrCapacityExceeded` separately from
provider generation failures. The target public outcome is HTTP `503` with a
stable `capacity_exceeded` code and a generic message. Scriptorium will not
invent retry timing or expose an untrusted diagnostic. CLI capacity rejection
continues to be a runtime failure, with a clear safe diagnostic and the
existing runtime-error exit status.

The HTTP server will use one appropriately scoped Promptkit engine across its
requests so backend admission, active-generation limits, and queue capacity
apply across concurrent in-flight work. The server may retain the operational
state needed for those limits and for request cancellation and lifecycle
management. This does not create durable per-request, conversation, or resume
state.

Scriptorium will continue to return generic public model-generation failures.
Promptkit `GenerationError` provider code, type, and message values will not be
added to the public HTTP response because they are untrusted and potentially
sensitive.

### Custom Backend Configuration

The strict Scriptorium application configuration will gain an optional custom
backend collection. Each entry will support the application-owned mapping
needed to construct a public `promptkit.Backend`:

- backend ID;
- OpenAI-compatible endpoint;
- optional API-key environment-variable name;
- optional JSON-compatible request-wide extra parameters;
- non-negative concurrency limit; and
- queue capacity with presence preserved so omission and explicit zero remain
  different.

Raw API-key values will remain invalid. Unknown fields will remain errors.
Configured backends will be registered through `promptkit.WithBackend` during
engine construction and will be immutable and engine-scoped. Promptkit will
remain the canonical validator for backend IDs, endpoints, reserved request
parameters, capacity bounds, duplicate or reserved IDs, environment-variable
names, and JSON-compatible values. Scriptorium will add context appropriate to
its configuration error boundary without copying those framework rules.

The collection will be configuration-file-owned. This roadmap does not add
per-backend CLI flags. Existing endpoint-only profiles remain valid, while
profiles may select configured custom backend IDs through Promptkit's existing
`backend` field. Promptkit's built-in OpenRouter and Rakestrawhome backends and
profiles remain available without Scriptorium registration.

The application configuration shape is a `backends` mapping whose keys are the
backend IDs registered with Promptkit:

```yaml
backends:
  local-gpu:
    endpoint: http://localhost:11434/v1
    api_key_env: LOCAL_GPU_API_KEY
    extra_params:
      provider_option: enabled
    concurrency_limit: 2
    queue_capacity: 0
```

`endpoint` is required for each entry. The other fields are optional. Omitted
`queue_capacity` uses Promptkit's default queue policy when concurrency is
bounded, while an explicit zero disables queuing. The configuration contract
and complete maintained configuration example will document this shape when
the feature is implemented.

### Prompt And Profile Inspection CLI

Scriptorium will add a CLI inspection command with prompt and profile modes.
The intended command family is:

- `scriptorium inspect prompt`, backed by `Engine.InspectPrompt`; and
- `scriptorium inspect profile`, backed by `Engine.InspectProfile`.

Prompt inspection will select an ID and optional version and report the
normalized prompt identity, opaque prompt hash, declared default profile,
declared inputs, and normalized output contract. It will not require input
artifacts, resolve a profile, load a schema, render templates, reserve backend
capacity, or contact a model.

Profile inspection will select an explicit profile ID and report its resolved
effective target, backend identity, optional credential source name, and
whether a later request must provide credential configuration. It will not
read credential values, load a prompt, reserve capacity, or contact a model.

Inspection will support deterministic Scriptorium-owned text and JSON output
and normal output-file handling. Scriptorium will define explicit output DTOs
rather than treating Promptkit inspection structs as stable wire formats.
Inspection failures will use existing public Promptkit error identities and
Scriptorium-owned CLI diagnostics and exit behavior.

The prompt form accepts `--prompt`, optional `--prompt-version`, `--config`,
`--prompt-dir`, `--format text|json`, and `--out`. The profile form accepts
`--profile`, `--config`, `--profile-dir`, `--format text|json`, and `--out`.
Both forms reject positional arguments and default to text output independently
of `defaults.render_format`. Configuration discovery and CLI-over-file source
precedence match the existing command family. Prompt inspection requires an
effective prompt directory; profile inspection does not, because Promptkit's
built-in profiles remain inspectable without one.

## Documentation And Example Changes

Implementation of this roadmap requires coordinated updates to the canonical
owners of affected behavior:

- `docs/cli.md` for session and reasoning flags, inspection commands, output,
  prompt-version selection, optional inputs, and exit behavior;
- `docs/api.md` for `session_id`, effective backend/session metadata, reasoning
  presence semantics, optional inputs, and the capacity response;
- `docs/config.md` for custom backend configuration and current Promptkit
  credential and provider-default semantics;
- `docs/operations.md` for capacity and custom-backend operational guidance
  where deployment handling is affected;
- `docs/internal/adapters.md` and `docs/internal/sources.md` for implemented
  mapping and engine-assembly behavior;
- `docs/policy/architecture.md` and `docs/internal/overview.md` only if concrete
  component responsibilities or durable boundaries change; and
- maintained examples for valid v0.9.0 prompt/profile formats and custom
  backend configuration.

Current-state documents must not describe these features as implemented until
the corresponding code lands.

## Validation And Completion Criteria

The roadmap is complete only when:

- ordinary and race-enabled tests pass against the tagged Promptkit v0.9.0
  dependency;
- `go vet ./...` and `go build ./cmd/scriptorium` pass;
- Go formatting and `git diff --check` pass;
- the architecture guard passes with no workspace, replacement, vendor tree,
  former facade, or Promptkit internal import;
- maintained render, custom-backend, session, reasoning, and inspection
  examples execute successfully without real credentials or provider calls;
- CLI tests cover explicit and omitted prompt versions for prompt IDs with one
  or multiple definitions;
- CLI and HTTP tests cover prompts with no declared inputs, omitted optional
  inputs, missing required inputs, and template-referenced inputs;
- HTTP tests cover omitted, replacement, and clearing reasoning states;
- CLI and HTTP tests cover session propagation and invalid session handling;
- presentation tests cover registered and endpoint-only backend identity;
- HTTP tests cover capacity rejection independently from provider failure;
- concurrent HTTP tests demonstrate that one server's requests share backend
  admission and cannot bypass capacity through per-request engine creation;
- configuration tests cover strict decoding, precedence, queue-capacity
  presence, invalid backends, and secret-field rejection;
- inspection tests cover deterministic text and JSON, source selection,
  output-file handling, safe credential presentation, and public error
  mapping;
- all maintained prompt roles and repair budgets are valid under Promptkit
  v0.9.0; and
- representative definition fixtures cover inline and file-backed messages,
  cache control, session templates, every output validation mode, schemas,
  profile inheritance, built-in and custom backends, execution controls,
  credential environment names, and extra parameters without Scriptorium
  independently parsing those fields; and
- affected documentation links and copyable examples validate successfully.

## Explicit Non-Goals

This roadmap does not include:

- appended request messages in either executable interface;
- retained or durable prepared-execution handles;
- HTTP prompt or profile inspection routes;
- embedded application fallback profiles;
- single-file, injected `fs.FS`, or in-memory prompt and profile source
  configuration;
- request-level output-contract replacement through the executable interfaces;
- public disclosure of structured provider error details;
- direct imports or configuration of Promptkit's external catalog modules;
- raw API-key configuration, CLI flags, or HTTP fields;
- durable run state, conversations, retries, archives, checkpoints, or resume;
  or
- changes to Promptkit itself.