scriptorium/docs/roadmap/implementation.md

Promptkit v0.9.0 Adoption Implementation Plan

Status And Authority

Planned. Implement the stages below in numeric order. The accepted feature scope and target state are defined by the Promptkit v0.9.0 adoption roadmap, and the durable product boundary is defined by ADR 0004. If this plan and either source conflict, the ADR governs product boundaries and the feature roadmap governs scope.

Follow the architecture, documentation, and testing policies under docs/policy/ throughout the work. In particular:

• use only Promptkit's public root package and its released v0.9.0 tag;
• do not add a workspace, module replacement, vendored Promptkit source, direct catalog import, or Scriptorium reimplementation of Promptkit parsing and validation;
• keep default tests deterministic, offline, and free of real credentials or provider calls;
• update each current-state contract in the same stage that implements its behavior; and
• prefer a small number of boundary and integration tests over duplicating Promptkit's own framework test matrix.

At the start of every stage, inspect the working tree and preserve unrelated changes. At the end of every stage, run the focused tests named in that stage, gofmt changed Go files, and run git diff --check. Do not proceed while a stage's completion conditions are unmet.

Stage 1: Adopt The Promptkit v0.9.0 Compatibility Baseline

Completion: Complete.

Make the released dependency the build baseline before adding new application surface area.

Implementation

1. Update go.mod to require gitea.maximumdirect.net/eric/promptkit v0.9.0, then run go mod tidy so go.mod and go.sum contain the module graph selected by that tag. Accept Promptkit's OpenRouter and Rakestrawhome catalog modules only as transitive dependencies; do not import them from Scriptorium.
2. In the HTTP request DTO, change model.reasoning_effort from string to *string, and pass the pointer directly to promptkit.ExecutionTargetOverride.ReasoningEffort. This is required by the v0.9.0 API and preserves omitted versus explicit-empty input. Keep the JSON field itself a string when present.
3. Compile every Promptkit public struct literal against v0.9.0. Keep literals keyed, do not restore the removed RunRequest.Metadata field, and do not add compatibility wrappers around Promptkit.
4. Audit maintained prompt, profile, schema, configuration, and HTTP examples against v0.9.0. Correct only actual incompatibilities: endpoints must be absolute HTTP(S) URLs, roles must be developer, system, user, or assistant, positive repair budgets must be at most three and paired with basic, json, or json_schema, content and schema paths must remain within their Promptkit source rules, and extra parameters must be JSON-compatible and avoid reserved provider keys.
5. Replace v0.1.0 links and current-version statements in current-state documentation with tagged v0.9.0 links. Do not rewrite historical version statements in accepted ADRs. Update the owning documentation to state the v0.9.0 credential and provider behavior: an optional missing environment credential may result in an unauthenticated provider request, while a profile that declares credentials required still fails as an invalid request when no source is supplied and with ErrAPIKeyEnvMissing when its selected environment source is empty; unset optional provider controls are omitted; and a positive repair budget can add provider calls, latency, token use, and cost.
6. Preserve the architecture guard for forbidden replacements, workspaces, vendor trees, former facade packages, and Promptkit internal imports. Extend it only if the dependency graph introduces a new realistic bypass that the existing assertions do not cover.

Verification

• Update HTTP mapping tests for the pointer-valued reasoning field, including separate omitted and explicit-empty cases, and update docs/api.md with the HTTP three-state behavior in this stage. Stage 4 adds the corresponding CLI control and session-ID support.
• Run go test ./..., go test -race ./..., go vet ./..., and go build ./cmd/scriptorium against the tagged module with no local replacement.
• Run the existing maintained render example and any example-validation tests.
• Confirm that a search for v0.1.0 finds only legitimate historical material, and that searches for replace, go.work, and Promptkit /internal/ imports do not reveal a prohibited dependency path.

Completion Conditions

The repository builds and all existing behavior tests pass on Promptkit v0.9.0; current documentation identifies v0.9.0 as the framework contract; no local or private Promptkit integration mechanism has been introduced.

Stage 2: Remove Definition-Compatibility Restrictions In Run And Render

Completion: Complete.

Allow the executable adapters to address every valid directory-backed prompt definition without imposing input requirements of their own.

Implementation

1. Add promptVersion string to the shared run/render request configuration. Register --prompt-version <version> for both run and render, include it in usage text, and map it without local normalization to promptkit.RunRequest.PromptVersion.
2. Make --input optional for run and render. Do not weaken parseMappings for callers that actually supplied mappings; instead, skip that parser when no input flag was supplied and pass a nil or empty input map to Promptkit. Promptkit remains responsible for declared-required and template-referenced input failures.
3. In POST /v1/runs, remove the adapter check that rejects a nil or empty inputs object. Preserve strict JSON decoding and all validation of input references that are present. Continue mapping the already-supported prompt_version field exactly once into RunRequest.PromptVersion.
4. Keep extra supplied inputs legal and retain the CLI file-reader and HTTP artifact containment and size policies.
5. Update docs/cli.md and docs/api.md in this stage: prompt ID remains required, prompt version is optional, inputs are optional at the adapter boundary, and Promptkit decides whether the selected definition requires them. Update the shortest request examples only where necessary to avoid implying that all prompts need inputs.

Verification

• Add CLI behavior tests proving that an omitted version selects a sole prompt definition, an explicit version selects the requested definition, and an omitted version fails clearly when a prompt ID has multiple versions.
• Add CLI and HTTP adapter coverage for: a prompt with no declared inputs, omitted optional inputs, a missing declared-required input, and a missing template-referenced input. Use a capturing runner for pure request mapping and one real Promptkit engine integration fixture for the definition-driven failures; do not reproduce Promptkit's parser tests.
• Retain tests for malformed supplied mappings, invalid HTTP input references, artifact containment, and request-size limits.
• Run go test ./internal/adapter/cli ./internal/adapter/http and the maintained render workflow.

Completion Conditions

Both executable interfaces can select a specific prompt version and can submit no inputs; only Promptkit rejects missing definition-required data.

Stage 3: Add Engine-Scoped Custom Backend Configuration

Completion: Complete.

Add the application-owned configuration needed for Promptkit profiles to select custom backend IDs and capacity policies.

Configuration Contract

Use this exact strict-YAML shape:

backends:
  local-gpu:
    endpoint: http://localhost:11434/v1
    api_key_env: LOCAL_GPU_API_KEY
    extra_params:
      provider_option: enabled
    concurrency_limit: 2
    queue_capacity: 0

The mapping key is the backend ID. endpoint is required by the effective contract. api_key_env, extra_params, concurrency_limit, and queue_capacity are optional. concurrency_limit uses zero as Promptkit's unlimited value. Represent queue_capacity as *int so omission uses Promptkit's default queue capacity and explicit zero disables queuing. There are no backend CLI overrides and no raw API-key field.

Implementation

1. Add Backends map[string]BackendConfig to the on-disk config.Config and a BackendConfig with exactly the five YAML fields above. Add a resolved BackendSettings value containing ID plus the corresponding fields, and carry []BackendSettings on AppSettings.
2. During config application, sort the YAML map keys and build the resolved slice in that order. Preserve the queue_capacity pointer and the complete extra_params value. The deterministic order makes multi-error diagnostics and tests stable; it is not backend precedence.
3. Do not duplicate Promptkit's backend validators in internal/config. Strict YAML decoding owns unknown-field and type errors. Promptkit NewEngine owns blank/reserved/duplicate IDs, endpoint and environment-name rules, reserved or invalid extra parameters, and capacity relationships.
4. Refactor CLI engine construction around a small shared engine-settings value containing prompt, profile, and schema directories plus resolved backends. Convert every BackendSettings to promptkit.Backend, register it with promptkit.WithBackend, and use the same constructor for run, render, and serve. Wrap construction failures with application-configuration and engine-initialization context while preserving errors.Is identities.
5. Retain one immutable backend registry per constructed engine. Do not create registries per HTTP request, and do not add direct imports of Promptkit's catalog modules; Promptkit's built-ins remain available automatically.
6. Document backends in docs/config.md, including key identity, queue-capacity presence, validation ownership, optional environment credentials, and the raw-secret prohibition. Update docs/internal/adapters.md and docs/internal/sources.md to describe the implemented mapping and engine assembly. Add the exact shape to examples/config.full.yml; keep the minimal config minimal.

Verification

• In internal/config, test strict decoding of all fields, unknown and raw secret-field rejection, deterministic resolved ordering, JSON-compatible nested extra_params, and omitted versus explicit-zero queue_capacity.
• At the CLI/engine boundary, test a valid custom backend selected by a custom profile through render, and representative Promptkit validation failures: reserved backend ID, invalid endpoint, invalid capacity relationship, and a reserved extra_params key. Assert the stable error identity or a small diagnostic fragment, not Promptkit's full text.
• Test that endpoint-only profiles and built-in profiles still work without a backends section.
• Run go test ./internal/config ./internal/adapter/cli and render with the complete example without contacting a provider.

Completion Conditions

Every command constructs its engine from the same resolved custom-backend configuration, Promptkit remains the validator, and profiles can select custom, built-in, or endpoint-only execution targets.

Stage 4: Expose Session And Presence-Aware Reasoning Controls

Completion: Complete.

Complete the request mapping for direct session identifiers and the v0.9.0 reasoning override semantics.

Implementation

1. Add --session-id <id> and --reasoning-effort <value> to both run and render, with no aliases. Map the session value to RunRequest.SessionID.
2. Track flag presence for --reasoning-effort. Omission leaves ExecutionTargetOverride.ReasoningEffort nil, a nonblank value supplies a pointer to that value, and --reasoning-effort= supplies a pointer to the empty string. Include reasoning presence when deciding whether to allocate the enclosing execution override.
3. Add optional session_id to the strict HTTP run-request DTO and map it to RunRequest.SessionID without application-level normalization. Keep model.reasoning_effort pointer-valued so omitted, nonblank, and empty JSON strings remain distinct.
4. Add optional session_id to successful HTTP metadata and populate it from the effective RunResult.SessionID, not directly from the request. Use omitempty; absence means no effective session ID.
5. Continue to reject unknown HTTP fields and do not add api_key, appended messages, or request-level output-contract fields.
6. Update docs/cli.md and docs/api.md with the exact flag/field names, inheritance/replacement/clearing semantics, the non-secret nature and Promptkit length rules of session IDs, and effective-session response behavior.

Verification

• CLI tests must distinguish an omitted reasoning flag, a nonblank replacement, and an explicit empty clear by observing the resulting prepared run through render. Add a mapping assertion that an explicit empty value still creates an execution override.
• HTTP tests must capture and distinguish the same three reasoning states and continue to reject non-string values under the strict DTO contract. Treat JSON null like omission, matching ordinary pointer-field decoding; the documented explicit clear remains the empty JSON string.
• CLI and HTTP tests must cover a direct session ID, a definition-rendered session ID when no direct value is supplied, no effective session ID, and an invalid overlong direct ID. Assert that successful HTTP metadata reports the engine's effective result.
• Run go test ./internal/adapter/cli ./internal/adapter/http ./internal/format.

Completion Conditions

CLI and HTTP callers can inherit, replace, or clear reasoning effort and can supply a direct session ID, with Promptkit retaining normalization and validation ownership.

Stage 5: Present Backend Identity And Map Capacity Outcomes

Completion: Complete.

Expose Promptkit's selected routing identity and make overload behavior a stable application contract.

Implementation

1. In prepared-run text output, add selected_backend_id immediately after selected_profile_id when it is nonempty. Preserve direct JSON marshaling of promptkit.PreparedRun; v0.9.0 already supplies selected_backend_id and effective_model_params.backend_id JSON fields.
2. In the CLI run summary, append backend=<id> when RunResult.SelectedBackendID is nonempty. Omit the token for endpoint-only profiles rather than deriving an ID from the endpoint.
3. Add selected_backend_id,omitempty to HTTP metadata and backend_id,omitempty to metadata.model_params, mapping them from RunResult.SelectedBackendID and ExecutionTarget.BackendID respectively. Keep the two values independently mapped so an upstream inconsistency is not hidden by Scriptorium.
4. In HTTP error mapping, classify promptkit.ErrCapacityExceeded before the general generation failure and return HTTP 503 with code capacity_exceeded and message model backend capacity is exhausted. Do not emit Retry-After and do not expose CapacityError details.
5. In CLI run, classify the same identity as the existing runtime-error exit status 1 and print the stable safe diagnostic run error: model backend capacity is exhausted. Keep all other runtime error handling unchanged.
6. Preserve serveCommand's one-engine/one-handler construction outside the request path. Do not add per-request engine construction; that would split capacity state and violate ADR 0004.
7. Update docs/cli.md, docs/api.md, and docs/operations.md with backend identity, endpoint-only omission, capacity status, lack of retry timing, shared per-engine admission, and the distinction between transient in-flight state and durable workflow state.

Verification

• Formatter, CLI, and HTTP tests must cover a registered backend and an endpoint-only profile. Assert supplied backend IDs and empty/omitted identity rather than endpoint-derived guesses.
• Add HTTP error-table coverage proving capacity is 503 capacity_exceeded, provider failure remains 502 llm_failed, and neither response leaks wrapped provider or capacity diagnostics.
• Add one offline concurrent HTTP integration test using a single real Promptkit engine, a custom backend with concurrency_limit: 1 and explicit queue_capacity: 0, and a blocking fake LLM client. Hold the first request in generation, issue a second request, assert the second receives the 503 contract, release the first, assert it succeeds, and assert observed peak generation concurrency is one. This test owns the Scriptorium shared-engine interaction; do not replicate Promptkit's FIFO or broader scheduler suite.
• Run go test -race ./internal/adapter/http ./internal/adapter/cli ./internal/format.

Completion Conditions

All presentations report actual backend identity when present, endpoint-only profiles remain identity-free, and one server instance enforces one engine's backend capacity across concurrent requests.

Stage 6: Add Stable Prompt Inspection Output And CLI

Completion: Complete.

Introduce reusable application-owned inspection presentation, then expose prompt inspection without model execution.

Presentation Contract

1. Generalize the internal prepared-run format selector to an internal OutputFormat with text and json values, a text default, and one parser. Update config and prepared-run call sites to use it without changing existing prepared-run bytes or defaults.render_format behavior.

2. Define a Scriptorium-owned prompt-inspection DTO; never marshal promptkit.PromptInspection directly. JSON contains these fields in this shape, including empty strings and an empty array where applicable:

   {
     "prompt_id": "example",
     "prompt_version": "1",
     "prompt_hash": "opaque",
     "default_profile_id": "",
     "inputs": [
       {
         "name": "source",
         "required": true,
         "content_type": "text/plain",
         "description": "Input text"
       }
     ],
     "output_contract": {
       "format": "text",
       "validation_mode": "none",
       "schema_path": "",
       "repair_attempts": 0
     }
   }
   

3. Preserve Promptkit's declared input order. Emit JSON with deterministic indentation and one trailing newline. Text output uses the same field order, renders inputs: [] when empty, lists input records in declaration order, quotes free-form descriptions safely, includes empty declared values, and ends in one newline. The prompt hash is opaque and must not be interpreted.

CLI Contract And Implementation

1. Add inspect to top-level dispatch and usage, with implementation in a new focused CLI file rather than further expanding run.go.

2. Implement:

   scriptorium inspect prompt --prompt ID [--prompt-version VERSION]
       [--config PATH] [--prompt-dir DIR] [--format text|json] [--out PATH]
   

   --prompt is required. All other flags are optional. Positional arguments and unknown inspection modes are errors. There are no deprecated aliases.

3. Apply ordinary config discovery and CLI-over-file precedence. Require an effective prompt directory, pass the optional version directly to Engine.InspectPrompt, and construct the engine through the shared settings path so configured backends are validated consistently. Do not require a profile or schema directory for this operation.

4. Default inspection format to text independently of defaults.render_format; --format is the only inspection format override. Reuse ordinary output-file handling. Write output only after successful inspection and formatting. Use exit 0 for success and exit 1 for parsing, configuration, load, formatting, or write failure.

5. Update docs/cli.md, docs/internal/adapters.md, and the internal format component description. Make clear that inspection loads and normalizes a definition but does not resolve a profile, load a schema, render templates, reserve backend capacity, or call a model.

Verification

• Formatter tests cover deterministic text and JSON, declared order, empty inputs/default profile/schema path, all nonempty fields, and nil inspection handling. Prefer structural JSON assertions plus a few stable text fragments over large snapshots.
• CLI tests cover required flags, rejected positional arguments, config source selection, --prompt-dir precedence, sole and explicit multi-version selection, default and explicit formats, stdout versus --out, no partial output on failure, and Promptkit prompt-not-found/load identities.
• Use a fake or deliberately failing model client to prove inspection performs no generation; do not assert Promptkit's internal call sequence.
• Run go test ./internal/format ./internal/adapter/cli.

Completion Conditions

inspect prompt produces stable Scriptorium text or JSON for every valid directory-backed v0.9.0 prompt definition without inputs or model access, and prepared-run presentation remains backward compatible.

Stage 7: Add Stable Profile Inspection Output And CLI

Completion: Complete.

Complete the inspection command family with inherited, built-in, custom, and endpoint-only profile support.

Presentation Contract

1. Define a Scriptorium-owned profile-inspection DTO; never marshal promptkit.ProfileInspection directly. JSON has the following complete shape. Keep empty strings and zero numeric values because Promptkit reports unresolved optional controls as zero, and normalize absent extra_params to an empty object:

   {
     "profile_id": "example",
     "effective_model_params": {
       "backend_id": "",
       "endpoint": "https://example.test/v1",
       "model": "model-name",
       "temperature": 0,
       "max_tokens": 0,
       "top_p": 0,
       "timeout_seconds": 0,
       "service_tier": "",
       "reasoning_effort": "",
       "api_key_env": "",
       "extra_params": {}
     },
     "api_key_required": false
   }
   

2. Text output follows the same order, nests effective model parameters, encodes extra-parameter values as deterministic compact JSON with sorted keys, includes an empty backend_id for endpoint-only profiles, and ends in one newline. Never read or print an environment-variable value or direct API key.

CLI Contract And Implementation

1. Implement the second mode:

   scriptorium inspect profile --profile ID
       [--config PATH] [--profile-dir DIR] [--format text|json] [--out PATH]
   

   --profile is required. The common inspection parse, format, output, and exit rules from Stage 6 apply. No prompt, prompt directory, schema directory, or input is required, so built-in profiles are inspectable with only their ID.

2. Resolve config and --profile-dir with normal precedence, construct the engine through the shared settings path so custom backends participate, and call Engine.InspectProfile exactly once.

3. Update docs/cli.md, docs/config.md, and internal adapter/source docs. Explain that profile inspection resolves inheritance and backend defaults but does not read credentials, load a prompt, reserve capacity, or call a model. Document that zero provider-control values may mean unspecified, consistent with Promptkit v0.9.0.

Verification

• Formatter tests cover deterministic text and JSON, registered and empty backend identity, optional environment-variable names, api_key_required, nested extra parameters, and the absence of credential values.
• CLI tests cover built-in, inherited custom, custom-backend, and endpoint-only profiles; config and --profile-dir precedence; default and explicit format; stdout and --out; missing and invalid profiles; no partial output; and no requirement for prompt_dir.
• Include a sentinel environment secret in a test and assert it appears in no inspection output or diagnostic while its variable name may appear.
• Run go test ./internal/format ./internal/adapter/cli ./internal/config.

Completion Conditions

inspect profile reports the complete safe effective profile view for every valid directory-backed v0.9.0 profile and configured backend combination without requiring a prompt or contacting a model.

Stage 8: Complete Cross-Feature Conformance And Release Validation

Close the roadmap with representative definition-level integration coverage, copyable examples, documentation alignment, and full validation. Do not defer contract documentation from earlier stages to this stage; this is a final cross-check.

Implementation And Documentation

1. Maintain one compact shared fixture set under testdata/promptkit-v0.9/ with prompts, profiles, schemas, and any message content files needed by CLI and HTTP integration tests. Through a small set of composable definitions, cover:

   • inline and file-backed messages;
   • developer, system, user, and assistant roles and message cache control;
   • required and optional inputs plus template-referenced values;
   • prompt versions, default profiles, and session-ID templates;
   • text and JSON output plus none, basic, json, and json_schema validation, a schema path, and a valid positive repair budget;
   • standalone and inherited profiles;
   • built-in, configured custom, and endpoint-only backends;
   • model controls, service tier, reasoning effort, timeout, optional credential environment names, and nested JSON-compatible extra parameters.

   Exercise these fixtures through Scriptorium's render and inspection boundaries. Assert representative effective outcomes; do not independently decode YAML or exhaustively retest every Promptkit field combination.

2. Add a copyable examples/render-v0.9-features.sh workflow that uses only render and inspect operations to demonstrate prompt-version selection, a direct session ID, reasoning replacement or clearing, a configured custom backend, and both inspection modes without provider access. Keep examples/config.full.yml complete and keep raw credentials out of every example.

3. Review every canonical current-state owner named by the feature roadmap. Remove stale claims that inputs are always required, that every command needs prompt_dir, or that v0.1.0 semantics still apply. Ensure CLI, API, configuration, operations, internal adapter/source, development, and overview documents link instead of duplicating one another's contracts. Change architecture policy or internal component boundaries only if the implementation actually changed those durable responsibilities.

4. Verify that all explicit non-goals remain absent: appended messages, retained prepared handles, HTTP inspection routes, alternative Promptkit source kinds, request output-contract replacement, public provider details, direct catalog imports, raw API keys, and durable workflow state.

5. After every completion criterion below passes, change the feature roadmap's Status from Planned to Complete. Do not mark it complete for a partial implementation.

Final Validation

Run all of the following from the repository root:

gofmt -w $(git ls-files '*.go')
go test ./...
go test -race ./...
go vet ./...
go build ./cmd/scriptorium
go run ./cmd/scriptorium render --config ./examples/config.yml --prompt generic.markdown_summary --input transcript=./examples/fixtures/transcript.md --input glossary=./examples/fixtures/glossary.yml
bash ./examples/render-v0.9-features.sh
git diff --check

Also verify the architecture guard, strict example decoding, documentation links, executable permissions on shell examples, and absence of secrets. Review the suite for redundant upstream-semantic tests and retain only tests that protect Scriptorium mappings, presentation, configuration, concurrency integration, or complete workflows.

Completion Conditions

Every roadmap validation criterion is satisfied on the tagged dependency; all current-state documentation and examples describe implemented behavior; the full application supports every Promptkit v0.9.0 feature expressible in its directory-backed prompt and profile definitions; and the ADR 0004 non-goals remain intact.

Open Questions

None. The external syntax, application mappings, validation ownership, inspection wire shapes, concurrency behavior, testing boundaries, and stage order required for implementation are fixed above.