diff --git a/docs/adr/0004-definition-boundary.md b/docs/adr/0004-definition-boundary.md new file mode 100644 index 0000000..0e77981 --- /dev/null +++ b/docs/adr/0004-definition-boundary.md @@ -0,0 +1,112 @@ +# Preserve Definition Compatibility And A Simple Execution Model + +## Status + +Accepted. + +## Date + +2026-08-29. + +## Context + +Scriptorium is a user-facing application built on Promptkit. Promptkit is a Go +library with both declarative framework formats and public APIs intended for +applications that need to assemble specialized integrations or workflows. + +Scriptorium needs a durable rule for deciding which upstream capabilities it +must support. Treating every new Promptkit public API as an application feature +would steadily expand Scriptorium into a workflow framework. Selecting an +arbitrary subset of prompt or profile fields would instead make valid Promptkit +definitions unexpectedly unusable through Scriptorium. + +The intended product is narrower: Scriptorium provides simple executable +interfaces that select and execute predefined prompts and return their results +in a consistent, repeatable form. It is not intended to own conversations, +modify prompts dynamically, coordinate follow-up messages, or manage complex +multi-step workflows. + +## Decision + +Scriptorium will aim to support the complete set of features expressible in +valid Promptkit prompt and profile definitions for the Promptkit version it +selects. This includes the schema, backend, source, and request plumbing needed +to prepare and execute those definitions without Scriptorium imposing a +narrower field-level format contract. + +Promptkit remains the canonical parser and validator for its definitions. +Scriptorium will pass its configured framework sources and mapped request +values through the supported Promptkit public facade rather than copying +Promptkit formats, defaults, or validation rules. + +Promptkit library features that are not required to use valid prompt or profile +definitions are considered individually. They may be incorporated when they +serve Scriptorium's simple execution model and fit its application-owned CLI, +HTTP, configuration, presentation, security, and process boundaries. A public +Promptkit declaration does not by itself require a corresponding Scriptorium +interface. + +Each execution request will remain self-contained from the caller's +perspective. A request must not depend on conversation or workflow state +retained from a completed request, and Scriptorium will not become a general +workflow or conversation system. Application-managed prompt modification, +appended or follow-up message flows, conversation state, prepared-handle +coordination, multi-step orchestration, and similar higher-level workflows are +outside its intended scope. + +Scriptorium may, and when necessary must, maintain process-scoped operational +state shared by concurrent requests. This includes the configured Promptkit +engine and backend registry, capacity admission and queue accounting, +in-flight request coordination, cancellation, and ordinary server lifecycle +resources. The state must be scoped so configured backend limits apply across +all relevant in-flight requests. But this is operational implementation state, +not durable user workflow or conversation state. + +## Alternatives Considered + +### Expose Every Promptkit Public Feature + +This would make the application surface track the library API closely, but it +would blur the boundary between a reusable Go library and a simple executable +consumer. Library-oriented lifecycle, source-construction, extension, and +workflow primitives would add commands, wire contracts, configuration, and +state without necessarily improving predefined prompt execution. + +### Support A Curated Subset Of Definition Fields + +This would keep the application small in the short term, but valid Promptkit +definitions could fail or behave differently solely because they were invoked +through Scriptorium. Maintaining a parallel field-level compatibility list +would also duplicate upstream format ownership and create recurring drift. + +### Expand Scriptorium Into A Workflow Service + +Scriptorium could own conversations, prompt changes, follow-up messages, +retries, checkpoints, and multi-step execution. That is a different product +with durable state, lifecycle, recovery, privacy, and operational requirements +that are not justified by Scriptorium's current purpose. + +## Consequences + +- Promptkit upgrades require an audit of the complete prompt and profile format + contracts, not only a source-compatibility build. +- Scriptorium adapters must not add unconditional request requirements that + reject a definition Promptkit can validly prepare or execute. +- Application configuration may need new plumbing, such as custom backend + registration, when it is necessary to use a valid profile feature. +- Definition parsing, validation, precedence, inheritance, defaults, and + built-in catalogs remain Promptkit responsibilities. +- A running HTTP server must share its appropriately scoped Promptkit engine + across requests so the engine's backend admission and concurrency limits + cannot be bypassed by per-request construction. +- Process-scoped counters, queues, in-flight request records, cancellation, and + lifecycle resources are permitted implementation state. They must be + concurrency-safe and do not create a durable resume or conversation + contract. +- Library-only source constructors, injected collaborators, opaque lifecycle + handles, and request-composition primitives remain optional Scriptorium + features rather than automatic requirements. +- New application features must be evaluated against the simple, + request-independent predefined-prompt execution model. +- Work that implements this decision remains tracked in roadmaps until it is + delivered and incorporated into current-state contracts. diff --git a/docs/roadmap/implementation.md b/docs/roadmap/implementation.md new file mode 100644 index 0000000..b9c51b2 --- /dev/null +++ b/docs/roadmap/implementation.md @@ -0,0 +1,585 @@ +# Promptkit v0.9.0 Adoption Implementation Plan + +## Status And Authority + +Planned. Implement the stages below in numeric order. The accepted feature +scope and target state are defined by the +[Promptkit v0.9.0 adoption roadmap](promptkit-v0.9.0-adoption.md), and the +durable product boundary is defined by +[ADR 0004](../adr/0004-definition-boundary.md). If this plan and either source +conflict, the ADR governs product boundaries and the feature roadmap governs +scope. + +Follow the architecture, documentation, and testing policies under +`docs/policy/` throughout the work. In particular: + +- use only Promptkit's public root package and its released v0.9.0 tag; +- do not add a workspace, module replacement, vendored Promptkit source, + direct catalog import, or Scriptorium reimplementation of Promptkit parsing + and validation; +- keep default tests deterministic, offline, and free of real credentials or + provider calls; +- update each current-state contract in the same stage that implements its + behavior; and +- prefer a small number of boundary and integration tests over duplicating + Promptkit's own framework test matrix. + +At the start of every stage, inspect the working tree and preserve unrelated +changes. At the end of every stage, run the focused tests named in that stage, +`gofmt` changed Go files, and run `git diff --check`. Do not proceed while a +stage's completion conditions are unmet. + +## Stage 1: Adopt The Promptkit v0.9.0 Compatibility Baseline + +Make the released dependency the build baseline before adding new application +surface area. + +### Implementation + +1. Update `go.mod` to require + `gitea.maximumdirect.net/eric/promptkit v0.9.0`, then run `go mod tidy` so + `go.mod` and `go.sum` contain the module graph selected by that tag. Accept + Promptkit's OpenRouter and Rakestrawhome catalog modules only as transitive + dependencies; do not import them from Scriptorium. +2. In the HTTP request DTO, change `model.reasoning_effort` from `string` to + `*string`, and pass the pointer directly to + `promptkit.ExecutionTargetOverride.ReasoningEffort`. This is required by the + v0.9.0 API and preserves omitted versus explicit-empty input. Keep the JSON + field itself a string when present. +3. Compile every Promptkit public struct literal against v0.9.0. Keep literals + keyed, do not restore the removed `RunRequest.Metadata` field, and do not + add compatibility wrappers around Promptkit. +4. Audit maintained prompt, profile, schema, configuration, and HTTP examples + against v0.9.0. Correct only actual incompatibilities: endpoints must be + absolute HTTP(S) URLs, roles must be `developer`, `system`, `user`, or + `assistant`, positive repair budgets must be at most three and paired with + `basic`, `json`, or `json_schema`, content and schema paths must remain + within their Promptkit source rules, and extra parameters must be + JSON-compatible and avoid reserved provider keys. +5. Replace v0.1.0 links and current-version statements in current-state + documentation with tagged v0.9.0 links. Do not rewrite historical version + statements in accepted ADRs. Update the owning documentation to state the + v0.9.0 credential and provider behavior: an optional missing environment + credential may result in an unauthenticated provider request, while a + profile that declares credentials required still fails as an invalid request + when no source is supplied and with `ErrAPIKeyEnvMissing` when its selected + environment source is empty; unset optional provider controls are omitted; + and a positive repair budget can add provider calls, latency, token use, and + cost. +6. Preserve the architecture guard for forbidden replacements, workspaces, + vendor trees, former facade packages, and Promptkit `internal` imports. + Extend it only if the dependency graph introduces a new realistic bypass + that the existing assertions do not cover. + +### Verification + +- Update HTTP mapping tests for the pointer-valued reasoning field, including + separate omitted and explicit-empty cases, and update `docs/api.md` with the + HTTP three-state behavior in this stage. Stage 4 adds the corresponding CLI + control and session-ID support. +- Run `go test ./...`, `go test -race ./...`, `go vet ./...`, and + `go build ./cmd/scriptorium` against the tagged module with no local + replacement. +- Run the existing maintained render example and any example-validation tests. +- Confirm that a search for `v0.1.0` finds only legitimate historical material, + and that searches for `replace`, `go.work`, and Promptkit `/internal/` + imports do not reveal a prohibited dependency path. + +### Completion Conditions + +The repository builds and all existing behavior tests pass on Promptkit v0.9.0; +current documentation identifies v0.9.0 as the framework contract; no local or +private Promptkit integration mechanism has been introduced. + +## Stage 2: Remove Definition-Compatibility Restrictions In Run And Render + +Allow the executable adapters to address every valid directory-backed prompt +definition without imposing input requirements of their own. + +### Implementation + +1. Add `promptVersion string` to the shared run/render request configuration. + Register `--prompt-version ` for both `run` and `render`, include it + in usage text, and map it without local normalization to + `promptkit.RunRequest.PromptVersion`. +2. Make `--input` optional for `run` and `render`. Do not weaken + `parseMappings` for callers that actually supplied mappings; instead, skip + that parser when no input flag was supplied and pass a nil or empty input + map to Promptkit. Promptkit remains responsible for declared-required and + template-referenced input failures. +3. In `POST /v1/runs`, remove the adapter check that rejects a nil or empty + `inputs` object. Preserve strict JSON decoding and all validation of input + references that are present. Continue mapping the already-supported + `prompt_version` field exactly once into `RunRequest.PromptVersion`. +4. Keep extra supplied inputs legal and retain the CLI file-reader and HTTP + artifact containment and size policies. +5. Update `docs/cli.md` and `docs/api.md` in this stage: prompt ID remains + required, prompt version is optional, inputs are optional at the adapter + boundary, and Promptkit decides whether the selected definition requires + them. Update the shortest request examples only where necessary to avoid + implying that all prompts need inputs. + +### Verification + +- Add CLI behavior tests proving that an omitted version selects a sole prompt + definition, an explicit version selects the requested definition, and an + omitted version fails clearly when a prompt ID has multiple versions. +- Add CLI and HTTP adapter coverage for: a prompt with no declared inputs, + omitted optional inputs, a missing declared-required input, and a missing + template-referenced input. Use a capturing runner for pure request mapping + and one real Promptkit engine integration fixture for the definition-driven + failures; do not reproduce Promptkit's parser tests. +- Retain tests for malformed supplied mappings, invalid HTTP input references, + artifact containment, and request-size limits. +- Run `go test ./internal/adapter/cli ./internal/adapter/http` and the maintained + render workflow. + +### Completion Conditions + +Both executable interfaces can select a specific prompt version and can submit +no inputs; only Promptkit rejects missing definition-required data. + +## Stage 3: Add Engine-Scoped Custom Backend Configuration + +Add the application-owned configuration needed for Promptkit profiles to select +custom backend IDs and capacity policies. + +### Configuration Contract + +Use this exact strict-YAML shape: + +```yaml +backends: + local-gpu: + endpoint: http://localhost:11434/v1 + api_key_env: LOCAL_GPU_API_KEY + extra_params: + provider_option: enabled + concurrency_limit: 2 + queue_capacity: 0 +``` + +The mapping key is the backend ID. `endpoint` is required by the effective +contract. `api_key_env`, `extra_params`, `concurrency_limit`, and +`queue_capacity` are optional. `concurrency_limit` uses zero as Promptkit's +unlimited value. Represent `queue_capacity` as `*int` so omission uses +Promptkit's default queue capacity and explicit zero disables queuing. There +are no backend CLI overrides and no raw API-key field. + +### Implementation + +1. Add `Backends map[string]BackendConfig` to the on-disk `config.Config` and a + `BackendConfig` with exactly the five YAML fields above. Add a resolved + `BackendSettings` value containing `ID` plus the corresponding fields, and + carry `[]BackendSettings` on `AppSettings`. +2. During config application, sort the YAML map keys and build the resolved + slice in that order. Preserve the `queue_capacity` pointer and the complete + `extra_params` value. The deterministic order makes multi-error diagnostics + and tests stable; it is not backend precedence. +3. Do not duplicate Promptkit's backend validators in `internal/config`. + Strict YAML decoding owns unknown-field and type errors. Promptkit + `NewEngine` owns blank/reserved/duplicate IDs, endpoint and environment-name + rules, reserved or invalid extra parameters, and capacity relationships. +4. Refactor CLI engine construction around a small shared engine-settings value + containing prompt, profile, and schema directories plus resolved backends. + Convert every `BackendSettings` to `promptkit.Backend`, register it with + `promptkit.WithBackend`, and use the same constructor for `run`, `render`, + and `serve`. Wrap construction failures with application-configuration and + engine-initialization context while preserving `errors.Is` identities. +5. Retain one immutable backend registry per constructed engine. Do not create + registries per HTTP request, and do not add direct imports of Promptkit's + catalog modules; Promptkit's built-ins remain available automatically. +6. Document `backends` in `docs/config.md`, including key identity, + queue-capacity presence, validation ownership, optional environment + credentials, and the raw-secret prohibition. Update + `docs/internal/adapters.md` and `docs/internal/sources.md` to describe the + implemented mapping and engine assembly. Add the exact shape to + `examples/config.full.yml`; keep the minimal config minimal. + +### Verification + +- In `internal/config`, test strict decoding of all fields, unknown and raw + secret-field rejection, deterministic resolved ordering, JSON-compatible + nested `extra_params`, and omitted versus explicit-zero `queue_capacity`. +- At the CLI/engine boundary, test a valid custom backend selected by a custom + profile through `render`, and representative Promptkit validation failures: + reserved backend ID, invalid endpoint, invalid capacity relationship, and a + reserved `extra_params` key. Assert the stable error identity or a small + diagnostic fragment, not Promptkit's full text. +- Test that endpoint-only profiles and built-in profiles still work without a + `backends` section. +- Run `go test ./internal/config ./internal/adapter/cli` and render with the + complete example without contacting a provider. + +### Completion Conditions + +Every command constructs its engine from the same resolved custom-backend +configuration, Promptkit remains the validator, and profiles can select custom, +built-in, or endpoint-only execution targets. + +## Stage 4: Expose Session And Presence-Aware Reasoning Controls + +Complete the request mapping for direct session identifiers and the v0.9.0 +reasoning override semantics. + +### Implementation + +1. Add `--session-id ` and `--reasoning-effort ` to both `run` and + `render`, with no aliases. Map the session value to + `RunRequest.SessionID`. +2. Track flag presence for `--reasoning-effort`. Omission leaves + `ExecutionTargetOverride.ReasoningEffort` nil, a nonblank value supplies a + pointer to that value, and `--reasoning-effort=` supplies a pointer to the + empty string. Include reasoning presence when deciding whether to allocate + the enclosing execution override. +3. Add optional `session_id` to the strict HTTP run-request DTO and map it to + `RunRequest.SessionID` without application-level normalization. Keep + `model.reasoning_effort` pointer-valued so omitted, nonblank, and empty JSON + strings remain distinct. +4. Add optional `session_id` to successful HTTP metadata and populate it from + the effective `RunResult.SessionID`, not directly from the request. Use + `omitempty`; absence means no effective session ID. +5. Continue to reject unknown HTTP fields and do not add `api_key`, appended + messages, or request-level output-contract fields. +6. Update `docs/cli.md` and `docs/api.md` with the exact flag/field names, + inheritance/replacement/clearing semantics, the non-secret nature and + Promptkit length rules of session IDs, and effective-session response + behavior. + +### Verification + +- CLI tests must distinguish an omitted reasoning flag, a nonblank replacement, + and an explicit empty clear by observing the resulting prepared run through + `render`. Add a mapping assertion that an explicit empty value still creates + an execution override. +- HTTP tests must capture and distinguish the same three reasoning states and + continue to reject non-string values under the strict DTO contract. Treat + JSON `null` like omission, matching ordinary pointer-field decoding; the + documented explicit clear remains the empty JSON string. +- CLI and HTTP tests must cover a direct session ID, a definition-rendered + session ID when no direct value is supplied, no effective session ID, and an + invalid overlong direct ID. Assert that successful HTTP metadata reports the + engine's effective result. +- Run `go test ./internal/adapter/cli ./internal/adapter/http ./internal/format`. + +### Completion Conditions + +CLI and HTTP callers can inherit, replace, or clear reasoning effort and can +supply a direct session ID, with Promptkit retaining normalization and +validation ownership. + +## Stage 5: Present Backend Identity And Map Capacity Outcomes + +Expose Promptkit's selected routing identity and make overload behavior a +stable application contract. + +### Implementation + +1. In prepared-run text output, add `selected_backend_id` immediately after + `selected_profile_id` when it is nonempty. Preserve direct JSON marshaling of + `promptkit.PreparedRun`; v0.9.0 already supplies `selected_backend_id` and + `effective_model_params.backend_id` JSON fields. +2. In the CLI run summary, append `backend=` when + `RunResult.SelectedBackendID` is nonempty. Omit the token for endpoint-only + profiles rather than deriving an ID from the endpoint. +3. Add `selected_backend_id,omitempty` to HTTP metadata and + `backend_id,omitempty` to `metadata.model_params`, mapping them from + `RunResult.SelectedBackendID` and `ExecutionTarget.BackendID` respectively. + Keep the two values independently mapped so an upstream inconsistency is not + hidden by Scriptorium. +4. In HTTP error mapping, classify `promptkit.ErrCapacityExceeded` before the + general generation failure and return HTTP 503 with code + `capacity_exceeded` and message `model backend capacity is exhausted`. Do + not emit `Retry-After` and do not expose `CapacityError` details. +5. In CLI `run`, classify the same identity as the existing runtime-error exit + status 1 and print the stable safe diagnostic + `run error: model backend capacity is exhausted`. Keep all other runtime + error handling unchanged. +6. Preserve `serveCommand`'s one-engine/one-handler construction outside the + request path. Do not add per-request engine construction; that would split + capacity state and violate ADR 0004. +7. Update `docs/cli.md`, `docs/api.md`, and `docs/operations.md` with backend + identity, endpoint-only omission, capacity status, lack of retry timing, + shared per-engine admission, and the distinction between transient + in-flight state and durable workflow state. + +### Verification + +- Formatter, CLI, and HTTP tests must cover a registered backend and an + endpoint-only profile. Assert supplied backend IDs and empty/omitted identity + rather than endpoint-derived guesses. +- Add HTTP error-table coverage proving capacity is 503 + `capacity_exceeded`, provider failure remains 502 `llm_failed`, and neither + response leaks wrapped provider or capacity diagnostics. +- Add one offline concurrent HTTP integration test using a single real + Promptkit engine, a custom backend with `concurrency_limit: 1` and explicit + `queue_capacity: 0`, and a blocking fake LLM client. Hold the first request + in generation, issue a second request, assert the second receives the 503 + contract, release the first, assert it succeeds, and assert observed peak + generation concurrency is one. This test owns the Scriptorium shared-engine + interaction; do not replicate Promptkit's FIFO or broader scheduler suite. +- Run `go test -race ./internal/adapter/http ./internal/adapter/cli ./internal/format`. + +### Completion Conditions + +All presentations report actual backend identity when present, endpoint-only +profiles remain identity-free, and one server instance enforces one engine's +backend capacity across concurrent requests. + +## Stage 6: Add Stable Prompt Inspection Output And CLI + +Introduce reusable application-owned inspection presentation, then expose +prompt inspection without model execution. + +### Presentation Contract + +1. Generalize the internal prepared-run format selector to an internal + `OutputFormat` with `text` and `json` values, a text default, and one parser. + Update config and prepared-run call sites to use it without changing + existing prepared-run bytes or `defaults.render_format` behavior. +2. Define a Scriptorium-owned prompt-inspection DTO; never marshal + `promptkit.PromptInspection` directly. JSON contains these fields in this + shape, including empty strings and an empty array where applicable: + + ```json + { + "prompt_id": "example", + "prompt_version": "1", + "prompt_hash": "opaque", + "default_profile_id": "", + "inputs": [ + { + "name": "source", + "required": true, + "content_type": "text/plain", + "description": "Input text" + } + ], + "output_contract": { + "format": "text", + "validation_mode": "none", + "schema_path": "", + "repair_attempts": 0 + } + } + ``` + +3. Preserve Promptkit's declared input order. Emit JSON with deterministic + indentation and one trailing newline. Text output uses the same field order, + renders `inputs: []` when empty, lists input records in declaration order, + quotes free-form descriptions safely, includes empty declared values, and + ends in one newline. The prompt hash is opaque and must not be interpreted. + +### CLI Contract And Implementation + +1. Add `inspect` to top-level dispatch and usage, with implementation in a new + focused CLI file rather than further expanding `run.go`. +2. Implement: + + ```text + scriptorium inspect prompt --prompt ID [--prompt-version VERSION] + [--config PATH] [--prompt-dir DIR] [--format text|json] [--out PATH] + ``` + + `--prompt` is required. All other flags are optional. Positional arguments + and unknown inspection modes are errors. There are no deprecated aliases. +3. Apply ordinary config discovery and CLI-over-file precedence. Require an + effective prompt directory, pass the optional version directly to + `Engine.InspectPrompt`, and construct the engine through the shared settings + path so configured backends are validated consistently. Do not require a + profile or schema directory for this operation. +4. Default inspection format to text independently of + `defaults.render_format`; `--format` is the only inspection format override. + Reuse ordinary output-file handling. Write output only after successful + inspection and formatting. Use exit 0 for success and exit 1 for parsing, + configuration, load, formatting, or write failure. +5. Update `docs/cli.md`, `docs/internal/adapters.md`, and the internal format + component description. Make clear that inspection loads and normalizes a + definition but does not resolve a profile, load a schema, render templates, + reserve backend capacity, or call a model. + +### Verification + +- Formatter tests cover deterministic text and JSON, declared order, empty + inputs/default profile/schema path, all nonempty fields, and nil inspection + handling. Prefer structural JSON assertions plus a few stable text fragments + over large snapshots. +- CLI tests cover required flags, rejected positional arguments, config source + selection, `--prompt-dir` precedence, sole and explicit multi-version + selection, default and explicit formats, stdout versus `--out`, no partial + output on failure, and Promptkit prompt-not-found/load identities. +- Use a fake or deliberately failing model client to prove inspection performs + no generation; do not assert Promptkit's internal call sequence. +- Run `go test ./internal/format ./internal/adapter/cli`. + +### Completion Conditions + +`inspect prompt` produces stable Scriptorium text or JSON for every valid +directory-backed v0.9.0 prompt definition without inputs or model access, and +prepared-run presentation remains backward compatible. + +## Stage 7: Add Stable Profile Inspection Output And CLI + +Complete the inspection command family with inherited, built-in, custom, and +endpoint-only profile support. + +### Presentation Contract + +1. Define a Scriptorium-owned profile-inspection DTO; never marshal + `promptkit.ProfileInspection` directly. JSON has the following complete + shape. Keep empty strings and zero numeric values because Promptkit reports + unresolved optional controls as zero, and normalize absent `extra_params` + to an empty object: + + ```json + { + "profile_id": "example", + "effective_model_params": { + "backend_id": "", + "endpoint": "https://example.test/v1", + "model": "model-name", + "temperature": 0, + "max_tokens": 0, + "top_p": 0, + "timeout_seconds": 0, + "service_tier": "", + "reasoning_effort": "", + "api_key_env": "", + "extra_params": {} + }, + "api_key_required": false + } + ``` + +2. Text output follows the same order, nests effective model parameters, + encodes extra-parameter values as deterministic compact JSON with sorted + keys, includes an empty `backend_id` for endpoint-only profiles, and ends in + one newline. Never read or print an environment-variable value or direct + API key. + +### CLI Contract And Implementation + +1. Implement the second mode: + + ```text + scriptorium inspect profile --profile ID + [--config PATH] [--profile-dir DIR] [--format text|json] [--out PATH] + ``` + + `--profile` is required. The common inspection parse, format, output, and + exit rules from Stage 6 apply. No prompt, prompt directory, schema directory, + or input is required, so built-in profiles are inspectable with only their + ID. +2. Resolve config and `--profile-dir` with normal precedence, construct the + engine through the shared settings path so custom backends participate, and + call `Engine.InspectProfile` exactly once. +3. Update `docs/cli.md`, `docs/config.md`, and internal adapter/source docs. + Explain that profile inspection resolves inheritance and backend defaults + but does not read credentials, load a prompt, reserve capacity, or call a + model. Document that zero provider-control values may mean unspecified, + consistent with Promptkit v0.9.0. + +### Verification + +- Formatter tests cover deterministic text and JSON, registered and empty + backend identity, optional environment-variable names, `api_key_required`, + nested extra parameters, and the absence of credential values. +- CLI tests cover built-in, inherited custom, custom-backend, and endpoint-only + profiles; config and `--profile-dir` precedence; default and explicit format; + stdout and `--out`; missing and invalid profiles; no partial output; and no + requirement for `prompt_dir`. +- Include a sentinel environment secret in a test and assert it appears in no + inspection output or diagnostic while its variable name may appear. +- Run `go test ./internal/format ./internal/adapter/cli ./internal/config`. + +### Completion Conditions + +`inspect profile` reports the complete safe effective profile view for every +valid directory-backed v0.9.0 profile and configured backend combination +without requiring a prompt or contacting a model. + +## Stage 8: Complete Cross-Feature Conformance And Release Validation + +Close the roadmap with representative definition-level integration coverage, +copyable examples, documentation alignment, and full validation. Do not defer +contract documentation from earlier stages to this stage; this is a final +cross-check. + +### Implementation And Documentation + +1. Maintain one compact shared fixture set under + `testdata/promptkit-v0.9/` with `prompts`, `profiles`, `schemas`, and any + message content files needed by CLI and HTTP integration tests. Through a + small set of composable definitions, cover: + + - inline and file-backed messages; + - `developer`, `system`, `user`, and `assistant` roles and message cache + control; + - required and optional inputs plus template-referenced values; + - prompt versions, default profiles, and session-ID templates; + - text and JSON output plus `none`, `basic`, `json`, and `json_schema` + validation, a schema path, and a valid positive repair budget; + - standalone and inherited profiles; + - built-in, configured custom, and endpoint-only backends; + - model controls, service tier, reasoning effort, timeout, optional + credential environment names, and nested JSON-compatible extra + parameters. + + Exercise these fixtures through Scriptorium's `render` and inspection + boundaries. Assert representative effective outcomes; do not independently + decode YAML or exhaustively retest every Promptkit field combination. +2. Add a copyable `examples/render-v0.9-features.sh` workflow that uses only + `render` and `inspect` operations to demonstrate prompt-version selection, + a direct session ID, reasoning replacement or clearing, a configured custom + backend, and both inspection modes without provider access. Keep + `examples/config.full.yml` complete and keep raw credentials out of every + example. +3. Review every canonical current-state owner named by the feature roadmap. + Remove stale claims that inputs are always required, that every command + needs `prompt_dir`, or that v0.1.0 semantics still apply. Ensure CLI, API, + configuration, operations, internal adapter/source, development, and + overview documents link instead of duplicating one another's contracts. + Change architecture policy or internal component boundaries only if the + implementation actually changed those durable responsibilities. +4. Verify that all explicit non-goals remain absent: appended messages, + retained prepared handles, HTTP inspection routes, alternative Promptkit + source kinds, request output-contract replacement, public provider details, + direct catalog imports, raw API keys, and durable workflow state. +5. After every completion criterion below passes, change the feature roadmap's + Status from `Planned` to `Complete`. Do not mark it complete for a partial + implementation. + +### Final Validation + +Run all of the following from the repository root: + +```bash +gofmt -w $(git ls-files '*.go') +go test ./... +go test -race ./... +go vet ./... +go build ./cmd/scriptorium +go run ./cmd/scriptorium render --config ./examples/config.yml --prompt generic.markdown_summary --input transcript=./examples/fixtures/transcript.md --input glossary=./examples/fixtures/glossary.yml +bash ./examples/render-v0.9-features.sh +git diff --check +``` + +Also verify the architecture guard, strict example decoding, documentation +links, executable permissions on shell examples, and absence of secrets. Review +the suite for redundant upstream-semantic tests and retain only tests that +protect Scriptorium mappings, presentation, configuration, concurrency +integration, or complete workflows. + +### Completion Conditions + +Every roadmap validation criterion is satisfied on the tagged dependency; all +current-state documentation and examples describe implemented behavior; the +full application supports every Promptkit v0.9.0 feature expressible in its +directory-backed prompt and profile definitions; and the ADR 0004 non-goals +remain intact. + +## Open Questions + +None. The external syntax, application mappings, validation ownership, +inspection wire shapes, concurrency behavior, testing boundaries, and stage +order required for implementation are fixed above. diff --git a/docs/roadmap/promptkit-v0.9.0-adoption.md b/docs/roadmap/promptkit-v0.9.0-adoption.md new file mode 100644 index 0000000..46bef88 --- /dev/null +++ b/docs/roadmap/promptkit-v0.9.0-adoption.md @@ -0,0 +1,350 @@ +# Promptkit v0.9.0 Adoption Roadmap + +## Status + +Planned. This document records accepted feature scope and the intended end +state. It does not describe implemented behavior. The concrete delivery plan is +maintained in [implementation.md](implementation.md). + +The durable product and integration boundary for this work is established by +[ADR 0004](../adr/0004-definition-boundary.md). This roadmap applies that +decision to Promptkit v0.9.0 and owns implementation scope and completion +status. + +## Objective + +Bring Scriptorium from Promptkit v0.1.0 to v0.9.0 and expose the newer +framework capabilities that fit Scriptorium's existing CLI, HTTP, +configuration, presentation, and process boundaries. + +Scriptorium must accept the complete Promptkit v0.9.0 feature set expressed by +directory-backed prompt, profile, and schema definitions. Application adapters +must not narrow that format contract by requiring request values that the +selected definition does not require or by omitting selection values needed to +address a valid definition. Promptkit's tagged +[framework format reference](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.9.0/docs/formats.md) +remains canonical; Scriptorium must delegate parsing and framework validation +rather than reproduce those rules. + +The completed application will retain Promptkit as the owner of reusable +prompt execution, source formats, profiles, backend definitions, capacity, +inspection, generation, and validation. Scriptorium will continue to own +application configuration, command and HTTP contracts, error and exit mapping, +output presentation, and deployment policy. + +## Targeted End State + +When this roadmap is complete: + +- Scriptorium pins the released Promptkit v0.9.0 module without a workspace, + replacement, vendored source, direct catalog import, or Promptkit internal + import. +- Existing valid CLI, HTTP, configuration, prompt, profile, schema, and render + workflows remain supported subject to Promptkit's documented v0.2.0 through + v0.9.0 compatibility corrections. +- Every Promptkit v0.9.0 prompt and profile feature expressible in definitions + under Scriptorium's configured directory sources is usable through the + executable application, including prompts with no inputs and explicitly + selected prompt versions. +- CLI and HTTP requests can supply a direct non-secret session ID. +- HTTP requests and a presence-aware CLI flag can inherit, replace, or + explicitly clear reasoning effort. +- Prepared output, CLI run summaries, and HTTP result metadata expose effective + backend identity when Promptkit supplies one. +- HTTP capacity rejection has a stable Scriptorium-owned overload response. +- Application configuration can register reusable engine-scoped custom + backends, including bounded concurrency and queue policy, for selection by + Promptkit profiles. +- CLI users can inspect prompt and profile definitions without supplying + placeholder inputs or invoking a model. +- Scriptorium's contracts and examples point to Promptkit v0.9.0 and accurately + describe its current credential, validation-repair, provider-control, + message-role, backend, and source-validation semantics. + +## Feature Scope + +### Complete Prompt And Profile Definition Compatibility + +Scriptorium will treat the tagged Promptkit v0.9.0 format contract as one +indivisible downstream compatibility boundary. This includes prompt identity +and version selection, optional and required input declarations, message and +content-file templates, supported roles, cache control, session templates, +default profiles, output contracts, schemas, and repair budgets. It also +includes standalone and inherited profiles, built-in and custom backend +selection, endpoint overrides, execution controls, optional credential +environment sources, and JSON-compatible extra parameters. + +Most of this support is provided by passing configured prompt, profile, and +schema directories directly to Promptkit. Scriptorium will not independently +decode these framework definitions or maintain a field-level subset. Two +current adapter restrictions must be removed so the executable interfaces do +not reject definitions that Promptkit accepts: + +- `run` and `render` will accept an optional prompt-version flag and map it to + `promptkit.RunRequest.PromptVersion`; and +- `run`, `render`, and `POST /v1/runs` will permit omitted or empty input maps. + Promptkit will decide whether an input is required by the selected + definition or referenced template. + +The HTTP API already carries `prompt_version`; its mapping must remain covered +by adapter tests. CLI and HTTP callers may continue to supply extra inputs, +subject to their existing file, containment, and size policies. Removing the +application-level nonempty-input requirement does not weaken Promptkit's +declared-required-input or template-reference validation. + +Profiles that name built-in backend IDs require no application registration. +Profiles that name other backend IDs become fully usable through the custom +backend configuration in this roadmap. Profile inheritance and all other +profile-field resolution remain Promptkit responsibilities. + +This compatibility requirement concerns fields and behavior expressed in +Promptkit YAML definitions. Go-consumer construction alternatives such as +single-file sources, injected `fs.FS` sources, in-memory profiles, fallback +profile filesystems, and prepared-execution handles are library-integration +features and are not required merely to support the complete definition +format. + +### Promptkit v0.9.0 Compatibility Baseline + +Required dependency and source changes: + +- update `go.mod` and `go.sum` to select Promptkit v0.9.0 and its selected + OpenRouter and Rakestrawhome catalog dependencies; +- change Scriptorium's inbound HTTP reasoning-effort representation to preserve + Promptkit's pointer semantics; +- retain keyed Promptkit public struct literals and confirm no removed + `RunRequest.Metadata` use exists; +- update tagged Promptkit links and version references throughout Scriptorium; + and +- preserve Scriptorium's architecture guard against replacements, workspaces, + vendored Promptkit code, former facade packages, and Promptkit internal + imports. + +Required compatibility review: + +- verify application and maintained example endpoints satisfy Promptkit's + absolute HTTP/HTTPS endpoint rules; +- verify prompt content paths, file artifacts, identities, JSON documents, and + JSON-compatible extra parameters satisfy the v0.6.0 safety boundaries; +- verify maintained prompt roles are limited to `developer`, `system`, `user`, + and `assistant`; +- verify every positive repair budget is no greater than three and is paired + with `basic`, `json`, or `json_schema` validation; +- document that positive repair budgets authorize additional provider calls, + latency, token use, and cost; +- document that unset optional provider controls are omitted and that a zero + effective value can represent an unspecified provider control unless an + explicit request override supplied it; and +- update credential guidance for Promptkit's optional environment lookup + behavior, profile `api_key_env` semantics, and Scriptorium's prohibition on + raw API-key inputs. + +### Session And Reasoning Request Controls + +The CLI `run` and `render` commands will accept a direct session-ID flag and map +it to `promptkit.RunRequest.SessionID`. The value remains non-secret +application correlation metadata and is subject to Promptkit's normalization +and length rules. + +The HTTP run request will accept `session_id` and map it through the existing +strict DTO boundary. Successful HTTP metadata will report the effective +session ID when present. + +The CLI `run` and `render` commands will also accept a presence-aware reasoning +effort flag. Omission inherits the selected profile, a nonblank value replaces +it, and an explicitly supplied empty value clears it. The existing HTTP +`model.reasoning_effort` field will gain the same three-state behavior while +remaining a JSON string when present. + +These additions must not introduce raw direct API-key flags or fields. + +### Backend Identity And Capacity Outcomes + +Scriptorium presentation will expose `SelectedBackendID` and the effective +target's `BackendID` without deriving identity from endpoint text. Backend +identity will be included where applicable in: + +- prepared-run text and JSON output; +- the CLI run summary; and +- HTTP response metadata and effective model parameters. + +Endpoint-only profiles continue to have no backend ID, and empty identities +must remain distinguishable from registered built-in or custom backends. + +The HTTP adapter will classify `promptkit.ErrCapacityExceeded` separately from +provider generation failures. The target public outcome is HTTP `503` with a +stable `capacity_exceeded` code and a generic message. Scriptorium will not +invent retry timing or expose an untrusted diagnostic. CLI capacity rejection +continues to be a runtime failure, with a clear safe diagnostic and the +existing runtime-error exit status. + +The HTTP server will use one appropriately scoped Promptkit engine across its +requests so backend admission, active-generation limits, and queue capacity +apply across concurrent in-flight work. The server may retain the operational +state needed for those limits and for request cancellation and lifecycle +management. This does not create durable per-request, conversation, or resume +state. + +Scriptorium will continue to return generic public model-generation failures. +Promptkit `GenerationError` provider code, type, and message values will not be +added to the public HTTP response because they are untrusted and potentially +sensitive. + +### Custom Backend Configuration + +The strict Scriptorium application configuration will gain an optional custom +backend collection. Each entry will support the application-owned mapping +needed to construct a public `promptkit.Backend`: + +- backend ID; +- OpenAI-compatible endpoint; +- optional API-key environment-variable name; +- optional JSON-compatible request-wide extra parameters; +- non-negative concurrency limit; and +- queue capacity with presence preserved so omission and explicit zero remain + different. + +Raw API-key values will remain invalid. Unknown fields will remain errors. +Configured backends will be registered through `promptkit.WithBackend` during +engine construction and will be immutable and engine-scoped. Promptkit will +remain the canonical validator for backend IDs, endpoints, reserved request +parameters, capacity bounds, duplicate or reserved IDs, environment-variable +names, and JSON-compatible values. Scriptorium will add context appropriate to +its configuration error boundary without copying those framework rules. + +The collection will be configuration-file-owned. This roadmap does not add +per-backend CLI flags. Existing endpoint-only profiles remain valid, while +profiles may select configured custom backend IDs through Promptkit's existing +`backend` field. Promptkit's built-in OpenRouter and Rakestrawhome backends and +profiles remain available without Scriptorium registration. + +The application configuration shape is a `backends` mapping whose keys are the +backend IDs registered with Promptkit: + +```yaml +backends: + local-gpu: + endpoint: http://localhost:11434/v1 + api_key_env: LOCAL_GPU_API_KEY + extra_params: + provider_option: enabled + concurrency_limit: 2 + queue_capacity: 0 +``` + +`endpoint` is required for each entry. The other fields are optional. Omitted +`queue_capacity` uses Promptkit's default queue policy when concurrency is +bounded, while an explicit zero disables queuing. The configuration contract +and complete maintained configuration example will document this shape when +the feature is implemented. + +### Prompt And Profile Inspection CLI + +Scriptorium will add a CLI inspection command with prompt and profile modes. +The intended command family is: + +- `scriptorium inspect prompt`, backed by `Engine.InspectPrompt`; and +- `scriptorium inspect profile`, backed by `Engine.InspectProfile`. + +Prompt inspection will select an ID and optional version and report the +normalized prompt identity, opaque prompt hash, declared default profile, +declared inputs, and normalized output contract. It will not require input +artifacts, resolve a profile, load a schema, render templates, reserve backend +capacity, or contact a model. + +Profile inspection will select an explicit profile ID and report its resolved +effective target, backend identity, optional credential source name, and +whether a later request must provide credential configuration. It will not +read credential values, load a prompt, reserve capacity, or contact a model. + +Inspection will support deterministic Scriptorium-owned text and JSON output +and normal output-file handling. Scriptorium will define explicit output DTOs +rather than treating Promptkit inspection structs as stable wire formats. +Inspection failures will use existing public Promptkit error identities and +Scriptorium-owned CLI diagnostics and exit behavior. + +The prompt form accepts `--prompt`, optional `--prompt-version`, `--config`, +`--prompt-dir`, `--format text|json`, and `--out`. The profile form accepts +`--profile`, `--config`, `--profile-dir`, `--format text|json`, and `--out`. +Both forms reject positional arguments and default to text output independently +of `defaults.render_format`. Configuration discovery and CLI-over-file source +precedence match the existing command family. Prompt inspection requires an +effective prompt directory; profile inspection does not, because Promptkit's +built-in profiles remain inspectable without one. + +## Documentation And Example Changes + +Implementation of this roadmap requires coordinated updates to the canonical +owners of affected behavior: + +- `docs/cli.md` for session and reasoning flags, inspection commands, output, + prompt-version selection, optional inputs, and exit behavior; +- `docs/api.md` for `session_id`, effective backend/session metadata, reasoning + presence semantics, optional inputs, and the capacity response; +- `docs/config.md` for custom backend configuration and current Promptkit + credential and provider-default semantics; +- `docs/operations.md` for capacity and custom-backend operational guidance + where deployment handling is affected; +- `docs/internal/adapters.md` and `docs/internal/sources.md` for implemented + mapping and engine-assembly behavior; +- `docs/policy/architecture.md` and `docs/internal/overview.md` only if concrete + component responsibilities or durable boundaries change; and +- maintained examples for valid v0.9.0 prompt/profile formats and custom + backend configuration. + +Current-state documents must not describe these features as implemented until +the corresponding code lands. + +## Validation And Completion Criteria + +The roadmap is complete only when: + +- ordinary and race-enabled tests pass against the tagged Promptkit v0.9.0 + dependency; +- `go vet ./...` and `go build ./cmd/scriptorium` pass; +- Go formatting and `git diff --check` pass; +- the architecture guard passes with no workspace, replacement, vendor tree, + former facade, or Promptkit internal import; +- maintained render, custom-backend, session, reasoning, and inspection + examples execute successfully without real credentials or provider calls; +- CLI tests cover explicit and omitted prompt versions for prompt IDs with one + or multiple definitions; +- CLI and HTTP tests cover prompts with no declared inputs, omitted optional + inputs, missing required inputs, and template-referenced inputs; +- HTTP tests cover omitted, replacement, and clearing reasoning states; +- CLI and HTTP tests cover session propagation and invalid session handling; +- presentation tests cover registered and endpoint-only backend identity; +- HTTP tests cover capacity rejection independently from provider failure; +- concurrent HTTP tests demonstrate that one server's requests share backend + admission and cannot bypass capacity through per-request engine creation; +- configuration tests cover strict decoding, precedence, queue-capacity + presence, invalid backends, and secret-field rejection; +- inspection tests cover deterministic text and JSON, source selection, + output-file handling, safe credential presentation, and public error + mapping; +- all maintained prompt roles and repair budgets are valid under Promptkit + v0.9.0; and +- representative definition fixtures cover inline and file-backed messages, + cache control, session templates, every output validation mode, schemas, + profile inheritance, built-in and custom backends, execution controls, + credential environment names, and extra parameters without Scriptorium + independently parsing those fields; and +- affected documentation links and copyable examples validate successfully. + +## Explicit Non-Goals + +This roadmap does not include: + +- appended request messages in either executable interface; +- retained or durable prepared-execution handles; +- HTTP prompt or profile inspection routes; +- embedded application fallback profiles; +- single-file, injected `fs.FS`, or in-memory prompt and profile source + configuration; +- request-level output-contract replacement through the executable interfaces; +- public disclosure of structured provider error details; +- direct imports or configuration of Promptkit's external catalog modules; +- raw API-key configuration, CLI flags, or HTTP fields; +- durable run state, conversations, retries, archives, checkpoints, or resume; + or +- changes to Promptkit itself.