diff --git a/docs/roadmap/implementation.md b/docs/roadmap/implementation.md index 3d54ada..33cd388 100644 --- a/docs/roadmap/implementation.md +++ b/docs/roadmap/implementation.md @@ -1,601 +1,491 @@ -# Promptkit v0.9.0 Adoption Implementation Plan +# Promptkit v0.11 Adoption Implementation Plan ## Status And Authority -Complete. Implement the stages below in numeric order. The accepted feature -scope and target state are defined by the -[Promptkit v0.9.0 adoption roadmap](promptkit-v0.9.0-adoption.md), and the -durable product boundary is defined by -[ADR 0004](../adr/0004-definition-boundary.md). If this plan and either source -conflict, the ADR governs product boundaries and the feature roadmap governs -scope. - -Follow the architecture, documentation, and testing policies under -`docs/policy/` throughout the work. In particular: - -- use only Promptkit's public root package and its released v0.9.0 tag; -- do not add a workspace, module replacement, vendored Promptkit source, - direct catalog import, or Scriptorium reimplementation of Promptkit parsing - and validation; -- keep default tests deterministic, offline, and free of real credentials or - provider calls; -- update each current-state contract in the same stage that implements its - behavior; and -- prefer a small number of boundary and integration tests over duplicating - Promptkit's own framework test matrix. - -At the start of every stage, inspect the working tree and preserve unrelated -changes. At the end of every stage, run the focused tests named in that stage, -`gofmt` changed Go files, and run `git diff --check`. Do not proceed while a -stage's completion conditions are unmet. - -## Stage 1: Adopt The Promptkit v0.9.0 Compatibility Baseline - -**Completion: Complete.** - -Make the released dependency the build baseline before adding new application -surface area. - -### Implementation - -1. Update `go.mod` to require - `gitea.maximumdirect.net/eric/promptkit v0.9.0`, then run `go mod tidy` so - `go.mod` and `go.sum` contain the module graph selected by that tag. Accept - Promptkit's OpenRouter and Rakestrawhome catalog modules only as transitive - dependencies; do not import them from Scriptorium. -2. In the HTTP request DTO, change `model.reasoning_effort` from `string` to - `*string`, and pass the pointer directly to - `promptkit.ExecutionTargetOverride.ReasoningEffort`. This is required by the - v0.9.0 API and preserves omitted versus explicit-empty input. Keep the JSON - field itself a string when present. -3. Compile every Promptkit public struct literal against v0.9.0. Keep literals - keyed, do not restore the removed `RunRequest.Metadata` field, and do not - add compatibility wrappers around Promptkit. -4. Audit maintained prompt, profile, schema, configuration, and HTTP examples - against v0.9.0. Correct only actual incompatibilities: endpoints must be - absolute HTTP(S) URLs, roles must be `developer`, `system`, `user`, or - `assistant`, positive repair budgets must be at most three and paired with - `basic`, `json`, or `json_schema`, content and schema paths must remain - within their Promptkit source rules, and extra parameters must be - JSON-compatible and avoid reserved provider keys. -5. Replace v0.1.0 links and current-version statements in current-state - documentation with tagged v0.9.0 links. Do not rewrite historical version - statements in accepted ADRs. Update the owning documentation to state the - v0.9.0 credential and provider behavior: an optional missing environment - credential may result in an unauthenticated provider request, while a - profile that declares credentials required still fails as an invalid request - when no source is supplied and with `ErrAPIKeyEnvMissing` when its selected - environment source is empty; unset optional provider controls are omitted; - and a positive repair budget can add provider calls, latency, token use, and - cost. -6. Preserve the architecture guard for forbidden replacements, workspaces, - vendor trees, former facade packages, and Promptkit `internal` imports. - Extend it only if the dependency graph introduces a new realistic bypass - that the existing assertions do not cover. - -### Verification - -- Update HTTP mapping tests for the pointer-valued reasoning field, including - separate omitted and explicit-empty cases, and update `docs/api.md` with the - HTTP three-state behavior in this stage. Stage 4 adds the corresponding CLI - control and session-ID support. -- Run `go test ./...`, `go test -race ./...`, `go vet ./...`, and - `go build ./cmd/scriptorium` against the tagged module with no local - replacement. -- Run the existing maintained render example and any example-validation tests. -- Confirm that a search for `v0.1.0` finds only legitimate historical material, - and that searches for `replace`, `go.work`, and Promptkit `/internal/` - imports do not reveal a prohibited dependency path. - -### Completion Conditions - -The repository builds and all existing behavior tests pass on Promptkit v0.9.0; -current documentation identifies v0.9.0 as the framework contract; no local or -private Promptkit integration mechanism has been introduced. - -## Stage 2: Remove Definition-Compatibility Restrictions In Run And Render - -**Completion: Complete.** - -Allow the executable adapters to address every valid directory-backed prompt -definition without imposing input requirements of their own. - -### Implementation - -1. Add `promptVersion string` to the shared run/render request configuration. - Register `--prompt-version ` for both `run` and `render`, include it - in usage text, and map it without local normalization to - `promptkit.RunRequest.PromptVersion`. -2. Make `--input` optional for `run` and `render`. Do not weaken - `parseMappings` for callers that actually supplied mappings; instead, skip - that parser when no input flag was supplied and pass a nil or empty input - map to Promptkit. Promptkit remains responsible for declared-required and - template-referenced input failures. -3. In `POST /v1/runs`, remove the adapter check that rejects a nil or empty - `inputs` object. Preserve strict JSON decoding and all validation of input - references that are present. Continue mapping the already-supported - `prompt_version` field exactly once into `RunRequest.PromptVersion`. -4. Keep extra supplied inputs legal and retain the CLI file-reader and HTTP - artifact containment and size policies. -5. Update `docs/cli.md` and `docs/api.md` in this stage: prompt ID remains - required, prompt version is optional, inputs are optional at the adapter - boundary, and Promptkit decides whether the selected definition requires - them. Update the shortest request examples only where necessary to avoid - implying that all prompts need inputs. - -### Verification - -- Add CLI behavior tests proving that an omitted version selects a sole prompt - definition, an explicit version selects the requested definition, and an - omitted version fails clearly when a prompt ID has multiple versions. -- Add CLI and HTTP adapter coverage for: a prompt with no declared inputs, - omitted optional inputs, a missing declared-required input, and a missing - template-referenced input. Use a capturing runner for pure request mapping - and one real Promptkit engine integration fixture for the definition-driven - failures; do not reproduce Promptkit's parser tests. -- Retain tests for malformed supplied mappings, invalid HTTP input references, - artifact containment, and request-size limits. -- Run `go test ./internal/adapter/cli ./internal/adapter/http` and the maintained - render workflow. - -### Completion Conditions - -Both executable interfaces can select a specific prompt version and can submit -no inputs; only Promptkit rejects missing definition-required data. - -## Stage 3: Add Engine-Scoped Custom Backend Configuration - -**Completion: Complete.** - -Add the application-owned configuration needed for Promptkit profiles to select -custom backend IDs and capacity policies. - -### Configuration Contract - -Use this exact strict-YAML shape: - -```yaml -backends: - local-gpu: - endpoint: http://localhost:11434/v1 - api_key_env: LOCAL_GPU_API_KEY - extra_params: - provider_option: enabled - concurrency_limit: 2 - queue_capacity: 0 -``` - -The mapping key is the backend ID. `endpoint` is required by the effective -contract. `api_key_env`, `extra_params`, `concurrency_limit`, and -`queue_capacity` are optional. `concurrency_limit` uses zero as Promptkit's -unlimited value. Represent `queue_capacity` as `*int` so omission uses -Promptkit's default queue capacity and explicit zero disables queuing. There -are no backend CLI overrides and no raw API-key field. - -### Implementation - -1. Add `Backends map[string]BackendConfig` to the on-disk `config.Config` and a - `BackendConfig` with exactly the five YAML fields above. Add a resolved - `BackendSettings` value containing `ID` plus the corresponding fields, and - carry `[]BackendSettings` on `AppSettings`. -2. During config application, sort the YAML map keys and build the resolved - slice in that order. Preserve the `queue_capacity` pointer and the complete - `extra_params` value. The deterministic order makes multi-error diagnostics - and tests stable; it is not backend precedence. -3. Do not duplicate Promptkit's backend validators in `internal/config`. - Strict YAML decoding owns unknown-field and type errors. Promptkit - `NewEngine` owns blank/reserved/duplicate IDs, endpoint and environment-name - rules, reserved or invalid extra parameters, and capacity relationships. -4. Refactor CLI engine construction around a small shared engine-settings value - containing prompt, profile, and schema directories plus resolved backends. - Convert every `BackendSettings` to `promptkit.Backend`, register it with - `promptkit.WithBackend`, and use the same constructor for `run`, `render`, - and `serve`. Wrap construction failures with application-configuration and - engine-initialization context while preserving `errors.Is` identities. -5. Retain one immutable backend registry per constructed engine. Do not create - registries per HTTP request, and do not add direct imports of Promptkit's - catalog modules; Promptkit's built-ins remain available automatically. -6. Document `backends` in `docs/config.md`, including key identity, - queue-capacity presence, validation ownership, optional environment - credentials, and the raw-secret prohibition. Update - `docs/internal/adapters.md` and `docs/internal/sources.md` to describe the - implemented mapping and engine assembly. Add the exact shape to - `examples/config.full.yml`; keep the minimal config minimal. - -### Verification - -- In `internal/config`, test strict decoding of all fields, unknown and raw - secret-field rejection, deterministic resolved ordering, JSON-compatible - nested `extra_params`, and omitted versus explicit-zero `queue_capacity`. -- At the CLI/engine boundary, test a valid custom backend selected by a custom - profile through `render`, and representative Promptkit validation failures: - reserved backend ID, invalid endpoint, invalid capacity relationship, and a - reserved `extra_params` key. Assert the stable error identity or a small - diagnostic fragment, not Promptkit's full text. -- Test that endpoint-only profiles and built-in profiles still work without a - `backends` section. -- Run `go test ./internal/config ./internal/adapter/cli` and render with the - complete example without contacting a provider. - -### Completion Conditions - -Every command constructs its engine from the same resolved custom-backend -configuration, Promptkit remains the validator, and profiles can select custom, -built-in, or endpoint-only execution targets. - -## Stage 4: Expose Session And Presence-Aware Reasoning Controls - -**Completion: Complete.** - -Complete the request mapping for direct session identifiers and the v0.9.0 -reasoning override semantics. - -### Implementation - -1. Add `--session-id ` and `--reasoning-effort ` to both `run` and - `render`, with no aliases. Map the session value to - `RunRequest.SessionID`. -2. Track flag presence for `--reasoning-effort`. Omission leaves - `ExecutionTargetOverride.ReasoningEffort` nil, a nonblank value supplies a - pointer to that value, and `--reasoning-effort=` supplies a pointer to the - empty string. Include reasoning presence when deciding whether to allocate - the enclosing execution override. -3. Add optional `session_id` to the strict HTTP run-request DTO and map it to - `RunRequest.SessionID` without application-level normalization. Keep - `model.reasoning_effort` pointer-valued so omitted, nonblank, and empty JSON - strings remain distinct. -4. Add optional `session_id` to successful HTTP metadata and populate it from - the effective `RunResult.SessionID`, not directly from the request. Use - `omitempty`; absence means no effective session ID. -5. Continue to reject unknown HTTP fields and do not add `api_key`, appended - messages, or request-level output-contract fields. -6. Update `docs/cli.md` and `docs/api.md` with the exact flag/field names, - inheritance/replacement/clearing semantics, the non-secret nature and - Promptkit length rules of session IDs, and effective-session response - behavior. - -### Verification - -- CLI tests must distinguish an omitted reasoning flag, a nonblank replacement, - and an explicit empty clear by observing the resulting prepared run through - `render`. Add a mapping assertion that an explicit empty value still creates - an execution override. -- HTTP tests must capture and distinguish the same three reasoning states and - continue to reject non-string values under the strict DTO contract. Treat - JSON `null` like omission, matching ordinary pointer-field decoding; the - documented explicit clear remains the empty JSON string. -- CLI and HTTP tests must cover a direct session ID, a definition-rendered - session ID when no direct value is supplied, no effective session ID, and an - invalid overlong direct ID. Assert that successful HTTP metadata reports the - engine's effective result. -- Run `go test ./internal/adapter/cli ./internal/adapter/http ./internal/format`. - -### Completion Conditions - -CLI and HTTP callers can inherit, replace, or clear reasoning effort and can -supply a direct session ID, with Promptkit retaining normalization and -validation ownership. - -## Stage 5: Present Backend Identity And Map Capacity Outcomes - -**Completion: Complete.** - -Expose Promptkit's selected routing identity and make overload behavior a -stable application contract. - -### Implementation - -1. In prepared-run text output, add `selected_backend_id` immediately after - `selected_profile_id` when it is nonempty. Preserve direct JSON marshaling of - `promptkit.PreparedRun`; v0.9.0 already supplies `selected_backend_id` and - `effective_model_params.backend_id` JSON fields. -2. In the CLI run summary, append `backend=` when - `RunResult.SelectedBackendID` is nonempty. Omit the token for endpoint-only - profiles rather than deriving an ID from the endpoint. -3. Add `selected_backend_id,omitempty` to HTTP metadata and - `backend_id,omitempty` to `metadata.model_params`, mapping them from - `RunResult.SelectedBackendID` and `ExecutionTarget.BackendID` respectively. - Keep the two values independently mapped so an upstream inconsistency is not - hidden by Scriptorium. -4. In HTTP error mapping, classify `promptkit.ErrCapacityExceeded` before the - general generation failure and return HTTP 503 with code - `capacity_exceeded` and message `model backend capacity is exhausted`. Do - not emit `Retry-After` and do not expose `CapacityError` details. -5. In CLI `run`, classify the same identity as the existing runtime-error exit - status 1 and print the stable safe diagnostic - `run error: model backend capacity is exhausted`. Keep all other runtime - error handling unchanged. -6. Preserve `serveCommand`'s one-engine/one-handler construction outside the - request path. Do not add per-request engine construction; that would split - capacity state and violate ADR 0004. -7. Update `docs/cli.md`, `docs/api.md`, and `docs/operations.md` with backend - identity, endpoint-only omission, capacity status, lack of retry timing, - shared per-engine admission, and the distinction between transient - in-flight state and durable workflow state. - -### Verification - -- Formatter, CLI, and HTTP tests must cover a registered backend and an - endpoint-only profile. Assert supplied backend IDs and empty/omitted identity - rather than endpoint-derived guesses. -- Add HTTP error-table coverage proving capacity is 503 - `capacity_exceeded`, provider failure remains 502 `llm_failed`, and neither - response leaks wrapped provider or capacity diagnostics. -- Add one offline concurrent HTTP integration test using a single real - Promptkit engine, a custom backend with `concurrency_limit: 1` and explicit - `queue_capacity: 0`, and a blocking fake LLM client. Hold the first request - in generation, issue a second request, assert the second receives the 503 - contract, release the first, assert it succeeds, and assert observed peak - generation concurrency is one. This test owns the Scriptorium shared-engine - interaction; do not replicate Promptkit's FIFO or broader scheduler suite. -- Run `go test -race ./internal/adapter/http ./internal/adapter/cli ./internal/format`. - -### Completion Conditions - -All presentations report actual backend identity when present, endpoint-only -profiles remain identity-free, and one server instance enforces one engine's -backend capacity across concurrent requests. - -## Stage 6: Add Stable Prompt Inspection Output And CLI - -**Completion: Complete.** - -Introduce reusable application-owned inspection presentation, then expose -prompt inspection without model execution. - -### Presentation Contract - -1. Generalize the internal prepared-run format selector to an internal - `OutputFormat` with `text` and `json` values, a text default, and one parser. - Update config and prepared-run call sites to use it without changing - existing prepared-run bytes or `defaults.render_format` behavior. -2. Define a Scriptorium-owned prompt-inspection DTO; never marshal - `promptkit.PromptInspection` directly. JSON contains these fields in this - shape, including empty strings and an empty array where applicable: - - ```json - { - "prompt_id": "example", - "prompt_version": "1", - "prompt_hash": "opaque", - "default_profile_id": "", - "inputs": [ - { - "name": "source", - "required": true, - "content_type": "text/plain", - "description": "Input text" - } - ], - "output_contract": { - "format": "text", - "validation_mode": "none", - "schema_path": "", - "repair_attempts": 0 - } - } - ``` - -3. Preserve Promptkit's declared input order. Emit JSON with deterministic - indentation and one trailing newline. Text output uses the same field order, - renders `inputs: []` when empty, lists input records in declaration order, - quotes free-form descriptions safely, includes empty declared values, and - ends in one newline. The prompt hash is opaque and must not be interpreted. - -### CLI Contract And Implementation - -1. Add `inspect` to top-level dispatch and usage, with implementation in a new - focused CLI file rather than further expanding `run.go`. -2. Implement: - - ```text - scriptorium inspect prompt --prompt ID [--prompt-version VERSION] - [--config PATH] [--prompt-dir DIR] [--format text|json] [--out PATH] - ``` - - `--prompt` is required. All other flags are optional. Positional arguments - and unknown inspection modes are errors. There are no deprecated aliases. -3. Apply ordinary config discovery and CLI-over-file precedence. Require an - effective prompt directory, pass the optional version directly to - `Engine.InspectPrompt`, and construct the engine through the shared settings - path so configured backends are validated consistently. Do not require a - profile or schema directory for this operation. -4. Default inspection format to text independently of - `defaults.render_format`; `--format` is the only inspection format override. - Reuse ordinary output-file handling. Write output only after successful - inspection and formatting. Use exit 0 for success and exit 1 for parsing, - configuration, load, formatting, or write failure. -5. Update `docs/cli.md`, `docs/internal/adapters.md`, and the internal format - component description. Make clear that inspection loads and normalizes a - definition but does not resolve a profile, load a schema, render templates, - reserve backend capacity, or call a model. - -### Verification - -- Formatter tests cover deterministic text and JSON, declared order, empty - inputs/default profile/schema path, all nonempty fields, and nil inspection - handling. Prefer structural JSON assertions plus a few stable text fragments - over large snapshots. -- CLI tests cover required flags, rejected positional arguments, config source - selection, `--prompt-dir` precedence, sole and explicit multi-version - selection, default and explicit formats, stdout versus `--out`, no partial - output on failure, and Promptkit prompt-not-found/load identities. -- Use a fake or deliberately failing model client to prove inspection performs - no generation; do not assert Promptkit's internal call sequence. -- Run `go test ./internal/format ./internal/adapter/cli`. - -### Completion Conditions - -`inspect prompt` produces stable Scriptorium text or JSON for every valid -directory-backed v0.9.0 prompt definition without inputs or model access, and -prepared-run presentation remains backward compatible. - -## Stage 7: Add Stable Profile Inspection Output And CLI - -**Completion: Complete.** - -Complete the inspection command family with inherited, built-in, custom, and -endpoint-only profile support. - -### Presentation Contract - -1. Define a Scriptorium-owned profile-inspection DTO; never marshal - `promptkit.ProfileInspection` directly. JSON has the following complete - shape. Keep empty strings and zero numeric values because Promptkit reports - unresolved optional controls as zero, and normalize absent `extra_params` - to an empty object: - - ```json - { - "profile_id": "example", - "effective_model_params": { - "backend_id": "", - "endpoint": "https://example.test/v1", - "model": "model-name", - "temperature": 0, - "max_tokens": 0, - "top_p": 0, - "timeout_seconds": 0, - "service_tier": "", - "reasoning_effort": "", - "api_key_env": "", - "extra_params": {} - }, - "api_key_required": false - } - ``` - -2. Text output follows the same order, nests effective model parameters, - encodes extra-parameter values as deterministic compact JSON with sorted - keys, includes an empty `backend_id` for endpoint-only profiles, and ends in - one newline. Never read or print an environment-variable value or direct - API key. - -### CLI Contract And Implementation - -1. Implement the second mode: - - ```text - scriptorium inspect profile --profile ID - [--config PATH] [--profile-dir DIR] [--format text|json] [--out PATH] - ``` - - `--profile` is required. The common inspection parse, format, output, and - exit rules from Stage 6 apply. No prompt, prompt directory, schema directory, - or input is required, so built-in profiles are inspectable with only their - ID. -2. Resolve config and `--profile-dir` with normal precedence, construct the - engine through the shared settings path so custom backends participate, and - call `Engine.InspectProfile` exactly once. -3. Update `docs/cli.md`, `docs/config.md`, and internal adapter/source docs. - Explain that profile inspection resolves inheritance and backend defaults - but does not read credentials, load a prompt, reserve capacity, or call a - model. Document that zero provider-control values may mean unspecified, - consistent with Promptkit v0.9.0. - -### Verification - -- Formatter tests cover deterministic text and JSON, registered and empty - backend identity, optional environment-variable names, `api_key_required`, - nested extra parameters, and the absence of credential values. -- CLI tests cover built-in, inherited custom, custom-backend, and endpoint-only - profiles; config and `--profile-dir` precedence; default and explicit format; - stdout and `--out`; missing and invalid profiles; no partial output; and no - requirement for `prompt_dir`. -- Include a sentinel environment secret in a test and assert it appears in no - inspection output or diagnostic while its variable name may appear. -- Run `go test ./internal/format ./internal/adapter/cli ./internal/config`. - -### Completion Conditions - -`inspect profile` reports the complete safe effective profile view for every -valid directory-backed v0.9.0 profile and configured backend combination -without requiring a prompt or contacting a model. - -## Stage 8: Complete Cross-Feature Conformance And Release Validation - -**Completion: Complete.** - -Close the roadmap with representative definition-level integration coverage, -copyable examples, documentation alignment, and full validation. Do not defer -contract documentation from earlier stages to this stage; this is a final -cross-check. - -### Implementation And Documentation - -1. Maintain one compact shared fixture set under - `testdata/promptkit-v0.9/` with `prompts`, `profiles`, `schemas`, and any - message content files needed by CLI and HTTP integration tests. Through a - small set of composable definitions, cover: - - - inline and file-backed messages; - - `developer`, `system`, `user`, and `assistant` roles and message cache - control; - - required and optional inputs plus template-referenced values; - - prompt versions, default profiles, and session-ID templates; - - text and JSON output plus `none`, `basic`, `json`, and `json_schema` - validation, a schema path, and a valid positive repair budget; - - standalone and inherited profiles; - - built-in, configured custom, and endpoint-only backends; - - model controls, service tier, reasoning effort, timeout, optional - credential environment names, and nested JSON-compatible extra - parameters. - - Exercise these fixtures through Scriptorium's `render` and inspection - boundaries. Assert representative effective outcomes; do not independently - decode YAML or exhaustively retest every Promptkit field combination. -2. Add a copyable `examples/render-v0.9-features.sh` workflow that uses only - `render` and `inspect` operations to demonstrate prompt-version selection, - a direct session ID, reasoning replacement or clearing, a configured custom - backend, and both inspection modes without provider access. Keep - `examples/config.full.yml` complete and keep raw credentials out of every - example. -3. Review every canonical current-state owner named by the feature roadmap. - Remove stale claims that inputs are always required, that every command - needs `prompt_dir`, or that v0.1.0 semantics still apply. Ensure CLI, API, - configuration, operations, internal adapter/source, development, and - overview documents link instead of duplicating one another's contracts. - Change architecture policy or internal component boundaries only if the - implementation actually changed those durable responsibilities. -4. Verify that all explicit non-goals remain absent: appended messages, - retained prepared handles, HTTP inspection routes, alternative Promptkit - source kinds, request output-contract replacement, public provider details, - direct catalog imports, raw API keys, and durable workflow state. -5. After every completion criterion below passes, change the feature roadmap's - Status from `Planned` to `Complete`. Do not mark it complete for a partial - implementation. - -### Final Validation - -Run all of the following from the repository root: - -```bash -gofmt -w $(git ls-files '*.go') -go test ./... -go test -race ./... -go vet ./... -go build ./cmd/scriptorium -go run ./cmd/scriptorium render --config ./examples/config.yml --prompt generic.markdown_summary --input transcript=./examples/fixtures/transcript.md --input glossary=./examples/fixtures/glossary.yml -bash ./examples/render-v0.9-features.sh -git diff --check -``` - -Also verify the architecture guard, strict example decoding, documentation -links, executable permissions on shell examples, and absence of secrets. Review -the suite for redundant upstream-semantic tests and retain only tests that -protect Scriptorium mappings, presentation, configuration, concurrency -integration, or complete workflows. - -### Completion Conditions - -Every roadmap validation criterion is satisfied on the tagged dependency; all -current-state documentation and examples describe implemented behavior; the -full application supports every Promptkit v0.9.0 feature expressible in its -directory-backed prompt and profile definitions; and the ADR 0004 non-goals -remain intact. +Planned on 2026-09-26; no stage in this plan is implemented yet. Execute Stage 1 +through Stage 10 in order. Each stage is intended to fit one focused +gpt-5.6-terra implementation task, including its tests and documentation. + +The [feature roadmap](upgrade-kit.md) owns desired scope and end state. +[ADR 0004](../adr/0004-definition-boundary.md) governs the product boundary. +Read [development](../development.md) and all five documents under +[policy](../policy/agents.md) before starting; follow the task-specific reading +guide for each stage. Policies and the ADR take precedence over implementation +mechanics here. This plan resolves the interface decisions left by the feature +roadmap; it does not introduce new architectural invariants. + +This replaces the completed v0.9 adoption plan. Its +[historical version](https://gitea.maximumdirect.net/eric/scriptorium/src/commit/e4405966c87c8a7843b340146e90cc1e515bef42/docs/roadmap/implementation.md) +remains the record for links from the older adoption roadmap. Do not repeat +that migration. + +## Baseline And Working Rules + +The reviewed baseline is commit `e440596`: Promptkit v0.9.0, v1 backend +catalogs, and Scriptorium's completed definition-facing architecture. The +upgrade target is the released Promptkit v0.11.0, not whatever upstream +`main` contains when implementation starts. The feature roadmap links the +tagged upstream migration, formats, integration, and observation contracts. + +The principal change points are: + +- `internal/config/config.go`: strict application YAML and resolved settings. +- `internal/adapter/cli/run.go`: shared settings/engine construction, + run/render request mapping, flags, output, and server startup. +- `internal/adapter/http/dto.go` and `handler.go`: request mapping, + caller contexts, response envelopes, and error/size handling. +- A small new `internal/report` package: shared terminal-observation projection, + not an execution facade or telemetry framework. +- Existing adapter, configuration, definition-conformance, and architecture + tests: retain useful coverage and extend it at the owning boundary. + +For each stage: + +1. Inspect the worktree and the relevant implementation/tests. Use the code + graph for code discovery as directed by repository instructions. +2. Implement only that stage and justified supporting refactoring. Retain one + shared server engine and its backend admission state. Do not add an outer + retry loop, new admission queue, conversation state, or copied profile schema. +3. Update the owning current-state contracts in the same change, describing + only delivered behavior. Link rather than copy schemas into internal docs. +4. Run focused tests, format changed Go files, and check the diff. For code + changes, also follow the baseline validation in development.md. Reuse checks + already run for unchanged content; do not create additional release gates. +5. Record completion and any remaining issues here, then commit the coherent + unit under the agent policy. Do not mark later stages complete by inference. + +Tests use local fixtures, real internal collaborators where practical, and +controlled provider boundaries. No paid calls or real credentials. Integration +tests of Promptkit's built-in retry path must use a local HTTP server or a +controlled HTTP transport, not `WithLLMClient`: an injected LLM client bypasses +that built-in retry policy. Do not duplicate upstream's complete status, +backoff, parser, or repair test matrices. + +Reports are included in this implementation sprint, not deferred by default. +Stage 9 has one explicit prerequisite in Open Questions. Stages 1–8 do not +depend on its answer, but the distribution work and release-readiness claim do. +No stage authorizes a tag, push, publication, or changes to historical releases. + +## Stage 1: Adopt The Released Dependency And Catalog Baseline + +**Status: Pending.** + +1. Upgrade `gitea.maximumdirect.net/eric/promptkit` to v0.11.0 and tidy the + module graph. Review the resulting go.mod/go.sum changes. Expected indirect + catalogs are OpenRouter `/v2` v2.20260922.1 and Rakestrawhome `/v2` + v2.20260922.0. Use released modules; commit no workspace, replacement, + vendor tree, direct catalog imports, or upstream implementation. +2. Update the built-in case of + `TestPromptkitV09ProfileInspectionResolvesSupportedTargets`, which currently + uses removed `deepseek-4-flash`. Rename tests that now misleadingly describe + only v0.9. Keep custom/inline target cases and the existing conformance suite. +3. Add compact offline coverage of both maintained names: + `promptkit-openrouter-standard` and `promptkit-openrouter-auto`. Exercise + profile inspection and representative prompt-default, explicit-selection, + inheritance, and custom-source override paths. Assert resolution/precedence, + not a copied snapshot of the whole catalog. Do not make either name an + application default. +4. Review maintained examples and fixtures for `deepseek-3-2`, + `deepseek-4-flash`, and `deepseek-4-pro`. Replace active obsolete references + where necessary without inventing aliases. Keep application-owned examples + whose purpose is fixed settings. Leave historical release notes unchanged. +5. Inspect architecture guards for version-sensitive catalog paths; ensure + `/v2` imports remain covered by the existing direct-import prohibition. + Extend only an actual gap, not an exact-version release gate. +6. Update current upstream-contract links from v0.9.0 to the selected v0.11.0 + contracts, including development, architecture, source, and integration + references. Update a policy's last-updated date if editing it, but retain + its base template version. Document the already-effective upstream retry + default and timeout-per-attempt change without claiming the later + Scriptorium controls exist yet. + +Acceptance: the full existing suite passes against the released dependency; +maintained names resolve offline through Scriptorium; obsolete live references +are handled; no new runtime flags or report schema are claimed in current docs. + +## Stage 2: Wire Retry Configuration And CLI Overrides + +**Status: Pending.** + +Selected interface: top-level application YAML `transient_retries`, and +`--transient-retries ` on `run` and `render`. This is not a model +parameter or a prompt/profile definition field. Serve and inspect consume the +application setting through normal engine construction; they gain no new +retry flag. + +1. Add a presence-aware optional integer to strict config, resolved application + settings, common command settings, and engine settings. Omitted or YAML + null remains nil. Reject negative values; malformed types follow existing + configuration error handling. There is no new arbitrary maximum. +2. Map the engine value to `promptkit.Config.TransientRetries` in shared + construction, including serve. Do not store Promptkit's default in + `internal/defaults`: nil delegates to upstream. +3. Register the CLI override with the shared execution-request flags. Preserve + presence, including explicit zero, into + `promptkit.RunRequest.TransientRetries`. The override belongs to the + request; it must not mutate the engine setting. +4. Render accepts and validates the same override for preparation consistency, + but performs no generation and adds no synthetic retry field to prepared + output. The report/deadline flags in later stages remain run-only. +5. Update config and CLI contracts, adapter internals, and operational retry + guidance. Explain precedence: request override, then application setting, + then upstream default (currently two additional attempts). Zero disables + additional transient attempts. Each repair generation has its own transient + budget; structural repair counts remain separate. + +Acceptance: focused config/CLI tests cover omission, null where applicable, +zero overriding a positive setting, positive values, and negative/malformed +values. Exercise common engine construction through CLI execution and server +settings without duplicating all config cases in every command. Offline +rendering continues to work. No application-owned retry/backoff loop exists. + +## Stage 3: Expose HTTP Retry Control And Verify Shared Admission + +**Status: Pending.** + +Selected interface: optional top-level JSON `transient_retries` integer on +`POST /v1/runs`; omission/null inherits the engine, zero disables additional +attempts, positive values override, negatives are invalid. + +1. Add a pointer integer to the request DTO and map it directly to the public + RunRequest field. Keep the strict decoder. Type/representation errors use + existing `400 invalid_json`; a negative number uses `400 invalid_request` + and does not invoke the runner. +2. Preserve the existing shared engine. Requests with different retry budgets + must not change each other's settings. +3. Extend safe error-mapping tests using wrapped public errors: + transport, malformed response, and retry-exhausted generation errors retain + `502 llm_failed`; capacity and validation classifications stay unchanged. + Use error identities, not error-message parsing. Optional report detail + will come from terminal events, not invented partial RunResults. +4. Add a representative built-in HTTP retry integration scenario: a transient + provider status followed by success. Demonstrate inherited retries and + explicit zero at the adapter boundary, without testing every eligible + status or asserting exact jitter. +5. Reuse or extend the server capacity test to verify concurrent requests share + the configured backend limit when retry behavior is enabled. Use barriers + or channels to measure active calls relative to a test-controlled limit. + Do not assert unrelated scheduler ordering or reimplement upstream admission. +6. Update the API contract and relevant operational guidance. + +Acceptance: HTTP presence/strictness tests pass; the real built-in retry path +is exercised offline; configured backend limits remain shared; no new error +body text exposes provider diagnostics or secrets. + +## Stage 4: Add The CLI Whole-Run Deadline + +**Status: Pending.** + +Selected interface: `run --run-timeout `. Omission or zero adds +no deadline, a positive duration retains full duration precision, and a negative +duration is invalid. No application-config default is added. Existing +`--timeout` retains its conversion and per-generation-attempt meaning. + +1. Add the run-only flag and validation, separately from existing timeout + conversion. Render/inspect/serve do not acquire a total-run flag. +2. Immediately before engine.Run, derive a context with the selected timeout + from the command's existing execution context (currently Background). + Release its timer with cancel. Do not add signal/lifecycle redesign as a + prerequisite. If a parent context seam is introduced for tests, keep it + private and preserve any earlier parent deadline. +3. The timed interval is the engine invocation: preparation, capacity waits, + retry waits, generation attempts, and repairs to the extent they honor + context. It does not include CLI parsing, engine construction, or output + file writes. Do not promise immediate interruption of synchronous code. +4. On a failed run whose execution context has expired, emit a safe deadline + diagnostic and exit 1. Preserve existing classifications otherwise: + success 0, operational/output error 1, completed invalid output 2. Do not + reclassify a successful returned result merely because the timer expires + after completion. +5. Update CLI, subprocess integration, and operations docs with timeout + layering. Provider Retry-After can exceed the ordinary backoff cap; + per-attempt timeouts do not bound the whole invocation. + +Acceptance: CLI parsing preserves subsecond values and rejects negatives; +omission/zero adds no deadline. A controlled waiting runner or HTTP boundary +demonstrates cancellation and exit 1 without fragile elapsed-time assertions. +Existing per-attempt timeout tests and generated-output behavior remain intact. + +## Stage 5: Add HTTP Whole-Run Deadlines And Context Outcomes + +**Status: Pending.** + +Selected interface: top-level `run_timeout_seconds`, an optional signed +64-bit integer. Omission/null/zero adds no deadline; positive whole seconds +create a child timeout; negatives are `400 invalid_request`. Reject values +whose conversion would overflow time.Duration, not an arbitrary operational +cap. JSON type or int64 representation errors remain `400 invalid_json`. + +1. Decode and validate before invocation. Derive the timeout from r.Context() + immediately before Run and defer cancellation. Parent cancellation or an + earlier parent deadline always wins. Body decoding and response writing are + outside this engine-invocation deadline. +2. For a failed Run, inspect the execution context before calling its deferred + cancel. If that context is deadline-exceeded, return + `504 deadline_exceeded`; if canceled, return `408 request_canceled`. + Use fixed safe messages. These are deliberate context outcomes; do not + infer them solely from a nested context error in an upstream generation + error. A provider-attempt timeout while the execution context is live + retains the existing generation-failure mapping. +3. Otherwise use existing error mapping, including capacity and validation + handling. A successful result remains successful even if cancellation + races with subsequent response formatting. No response-delivery guarantee + is possible after a client disconnect. +4. Cover child/parent context propagation and overflow/negative input at the + handler boundary. Add representative real-engine cancellation during a + retry wait and a capacity wait; share useful fixtures with Stage 3 instead + of duplicating the upstream scheduler suite. Synchronize entry into the + wait; avoid exact backoff timing assertions. +5. Update the API error table, request contract, and operations links. + +Acceptance: omission preserves the parent context, deadlines shorten rather +than extend its lifetime, a live-context attempt timeout still maps to 502, +and canceled waiters release capacity through Promptkit. The server has no +extra timer queue or admission manager. + +## Stage 6: Implement A Shared Terminal Report Projection + +**Status: Pending.** + +Create `internal/report` with a small Scriptorium-owned DTO and per-invocation +collector. It depends only on Promptkit's supported root types and the standard +library. It neither invokes the engine nor writes output. + +1. Provide a collector callback compatible with ExecutionObserver. Ignore + nonterminal events and project the execution-finished event into a value + owned by that collector. Read it after synchronous Run returns. Retain no + event history; no callback I/O, goroutines, queues, or global collector. + Different invocations always use distinct collectors. No panic-recovery + contract or synthetic terminal event is added. +2. Use the same report object for CLI JSON files and the HTTP `execution` + field. Explicitly map fields; do not marshal ExecutionEvent wholesale. + The selected JSON shape is: + + | Field | Representation and source | + | --- | --- | + | `run_id`, `prompt_id`, `prompt_version`, `profile_id`, `backend_id`, `model` | Strings from terminal identity fields; omit unavailable empty strings. | + | `outcome` | String matching a known upstream terminal outcome. | + | `failure` | String matching a known upstream failure kind, including `none`. | + | `status_code` | Provider status integer; omit when upstream reports zero/unavailable. | + | `retries_exhausted` | Boolean, always present and independent of failure category. | + | `attempts`, `transient_retries`, `repair_calls` | Unsigned terminal aggregate counts, always present. | + | `duration_ms`, `capacity_wait_ms`, `generation_duration_ms`, `retry_delay_ms` | Integer milliseconds from the corresponding terminal durations, always present. | + | `usage` | Object with the five token-count objects below, always present. | + + The usage keys are `prompt_tokens`, `completion_tokens`, `total_tokens`, + `cached_tokens`, and `cache_write_tokens`. Each is an object containing + `value` (integer), `known` (boolean), and `complete` (boolean), all always + present. Copy all three upstream fields without substituting unknown + values with known zero or complete totals. +3. Map the v0.11 public outcome/failure constants explicitly; use `other` for + an unrecognized value, not arbitrary new upstream text. Empty upstream + no-failure values, if any, map to `none`. Durations are observed components, + not a promise that their sum equals wall time. +4. Copy terminal aggregates once. Do not sum attempt events, add terminal + usage again, infer dispatch counts from retry budgets, or derive missing + values from successful-result metadata. No terminal event means no report. +5. Exclude inputs, outputs, session identifiers, headers, endpoint URLs, + credentials, raw errors, and provider diagnostic text. Selected prompt, + profile, backend, model, and run identifiers can still be sensitive. +6. Test projection, omission of unavailable optional fields, known-zero versus + unknown usage, incomplete totals, aggregate non-duplication, terminal + absence, and separate collector isolation. Synthetic public events suit + mapping tests; one real-engine terminal event proves wiring compatibility. +7. Document the implemented package in internal overview/adapters. Do not + advertise a CLI or HTTP opt-in until its stage lands. + +Acceptance: a small bounded-by-one-event collector produces only the selected +fields and preserves upstream meaning on both success and failure. It creates +no new orchestration layer or durable state. + +## Stage 7: Expose CLI Execution Report Files + +**Status: Pending.** + +Selected interface: run-only `--execution-report `, writing one +newline-terminated JSON report to a separate file. A nonempty path opts in; +an explicitly empty path is invalid. Paths use normal filesystem semantics; +`-` has no special stdout/stderr meaning. No parent directories are created +automatically. Existing files are replaced using ordinary output-file behavior; +there is no atomicity, rollback, or crash-durability promise. + +1. Add the flag and attach a fresh collector only when requested. No report + destination means no observer and unchanged default streams. +2. Keep generated content on stdout or --out. Reject a report/output collision + before Run when it is detectable from equal cleaned absolute paths or + os.SameFile for existing destinations. This prevents one selected output + overwriting the other; it is not a filesystem sandbox or protection + against concurrent path replacement. +3. Do not create/truncate the report during parsing or engine setup. After + Run, write it if a terminal event exists, even when Run failed or validation + failed. Adapter failures before invocation leave the report path untouched. + If no terminal event exists, leave the path untouched and give a concise + diagnostic that no report was produced; do not fabricate an empty report + or a RunResult. An existing report file may therefore be from an earlier run. +4. On successful Run, attempt the normal artifact output first, then attempt + report writing even if artifact output failed. On failed Run, report the + safe execution error and still attempt report writing. Any artifact or + report write failure yields exit 1; preserve the execution diagnostic if + both execution and report writing fail. Otherwise retain exit 0 or 2 from + the actual result. Absence of a terminal event alone does not override the + engine result's exit classification. Already-written output is not undone. +5. Reports describe engine execution, not later adapter output failures. + Keep ordinary summary behavior when outputs succeed. Use standard JSON + encoding and file writes consistent with existing CLI output permissions. +6. Test success, validation failure, operational failure with terminal data, + pre-invocation failure, report-write failure, output collision, and no-report + defaults. Cover the consequential output-failure combinations without + building an exhaustive filesystem-failure harness. +7. Update CLI, subprocess, and operations contracts, including sensitive + identifiers, caller-owned retention, and nontransactional output handling. + Initially document the report fields in the CLI contract; Stage 8 moves + that shared schema to the API contract once both adapters expose it. + +Acceptance: reporting never replaces or contaminates generated stdout; +terminal failed invocations can yield reports; write failures are visible and +do not erase the original run failure. Render and inspect remain unchanged. + +## Stage 8: Expose HTTP Execution Reports + +**Status: Pending.** + +Selected interface: optional top-level request boolean +`include_execution_report`. Omission, null, or false leaves reporting off, +consistent with the existing include_raw_output decoder behavior. True opts in. +Both success and operational-error envelopes may then contain an optional +top-level `execution` object using Stage 6's DTO. + +1. Add the request flag and omitempty report pointers to success/error DTOs. + Preserve existing artifact, validation, metadata, raw-output, and error + fields. In particular, do not change metadata.usage to observed aggregate + usage. The error envelope remains an error, not a partial success response. +2. Allocate a collector per opted-in invocation and attach its callback. + Populate execution only when a terminal observation exists. Adapter decode + or validation failures before Run have no report. A failing invocation can + still have terminal information even if no provider call was dispatched. +3. Use Stage 5 context classifications and existing typed-error mappings. + Preserve failure category and retries_exhausted separately in reports. + Do not derive category labels from raw messages. +4. Apply MaxResponseBytes to the complete serialized response including its + report, for success and report-bearing error responses. On overflow return + the existing compact `413 response_too_large` error without execution; + do not recursively try to fit a report into that fallback. Preserve the + existing compact-error fallback behavior rather than inventing a new + minimum response limit. +5. Test omitted-by-default shape, successful/invalid-output reporting, + operational failures, no-terminal omission, and report-bearing response + limits. Use a representative real-engine failure to ensure terminal data + survives the adapter. Concurrent requests must return only their own + identifiers/counters; exercise this under the race detector. +6. Move the shared report-schema definition from the CLI contract to the API + contract, removing the duplicate; CLI docs link to it for field meanings + while owning file behavior. + Update adapter internals and operations without duplicating the schema. + +Acceptance: opt-in works for success and errors, reports participate in response +limits, defaults remain compatible, and concurrent runs do not share collectors. +Diagnostics include no disallowed content even when an upstream error contains it. + +## Stage 9: Resolve And Implement Binary Distribution Materials + +**Status: Pending; requires the decision in Open Questions.** + +Do not turn the recommendation below into an assertion of legal sufficiency. +Once the maintainer resolves the distribution approach, record that decision +here before implementing its packaging mechanics. + +1. Inventory licenses and notices for the selected module graph, including the + two catalog modules, against the actual tagged sources. Keep Scriptorium's + own license unchanged. Record dependency notices and license texts in a + maintained third-party location and link it from the appropriate user and + release documentation. +2. For the recommended static-binary/source-material approach, provide an + accompanying versioned source/material archive tied to the exact candidate: + Scriptorium sources, selected dependency sources and license texts, + go.mod/go.sum, and instructions/toolchain information for rebuilding with + a modified Promptkit. Include additional materials required by the resolved + approach. Build this as a release artifact, not a committed vendor tree + or runtime replacement. +3. Update the hosted workflow's asset selection/checksum generation to include + the chosen materials with the binaries. Preserve published-asset immutability + and consistent manifests on partial-upload recovery. Update the release + procedure's deliverable list and packaging instructions. Do not add CI + polling, consumer-installation checks, or routine human approval gates to + source publication. +4. Exercise archive construction and the documented rebuild procedure locally + using prepared dependencies and no paid calls. Verify included versions, + source availability, notices, and association with the binary source + revision. Record what was actually validated, not a blanket legal guarantee. +5. If the chosen approach differs, revise this stage's concrete packaging + steps before coding. A source-only approach must explicitly reconcile the + existing asynchronous-binary workflow; simply leaving automatic binary + publication enabled would not implement that decision. + +Acceptance: the adoption-specific distribution question is resolved and the +chosen materials/workflow exist and are tested. Do not publish upgraded +binaries while it is unresolved. This is not a new recurring release signoff. + +## Stage 10: Verify The Integrated End State And Prepare The Handoff + +**Status: Pending.** + +1. Review the feature roadmap against implemented code and stage evidence. + Confirm definition conformance, maintained profiles, retry inheritance, + whole-run versus attempt timeouts, safe terminal reports, shared capacity, + and distribution materials. Add only tests for a concrete uncovered risk; + do not require every combination of these features. +2. Run the repository's development validation on the final code: + `go test ./...`, `go test -race ./...`, `go vet ./...`, and + `go build ./cmd/scriptorium`. Check formatting and `git diff --check`. + Validate affected offline examples and documentation links. Ensure no + build output, local credentials, workspace, replacement, or generated + packaging scratch data is staged. +3. Check all current contracts and orientation links against v0.11 and delivered + behavior. Preserve immutable accepted ADR content and historical release + notes. The plan's interface tables cease to be the current contract once + their canonical docs are implemented; retain this file as completion history. +4. Record release-note handoff items here: removed profiles and migration + choices, upstream retry default/cost implications and explicit zero, + timeout layering and new context outcomes, opt-in reports and their limits, + and the selected distribution approach. The next release should be a minor + version under the release policy. Version selection and creation of the + version-matched release note belong to release preparation, not an assumed + version or publication in this implementation task. +5. Mark stages and the feature roadmap complete only to the delivered boundary. + If anything remains deferred or blocked, name it and do not claim full + adoption/release readiness. Commit the final coherent documentation/validation + changes and report remaining work. + +Acceptance: the integrated implementation satisfies the feature roadmap, normal +validation passes, and the release handoff is accurate. Actual source +publication still follows the existing release policy and requires authorization. ## Open Questions -None. The external syntax, application mappings, validation ownership, -inspection wire shapes, concurrency behavior, testing boundaries, and stage -order required for implementation are fixed above. +### Which Distribution Approach Will Satisfy The Upgraded Dependency Licenses? + +The runtime interfaces above are settled for implementation. The outstanding +decision is the applicable material/permission requirements for distributing +Scriptorium's statically linked binaries with LGPL-3.0-or-later Promptkit and +the other selected dependencies. The roadmap identified this as a prerequisite; +this plan cannot establish legal sufficiency from a dependency version alone. + +**Recommended approach:** retain the current static Linux binary delivery and +add a version-matched source/license/rebuild-material archive as outlined in +Stage 9. Have the maintainer confirm the applicable requirements and whether +those materials and permissions are sufficient before finalizing packaging. +This preserves the established installation model and gives recipients an +auditable route to rebuild with a modified library. It is an adoption-specific +decision, not a proposal for a manual gate on every release. + +**Viable alternative:** defer upgraded binary distribution and deliver source +only while the requirements are resolved. This avoids claiming that an +unreviewed binary package is ready, but reduces installation convenience and +requires an explicit change to the current binary-delivery workflow and its +documentation. Another possible route is obtaining suitable additional +distribution permission from the relevant rights holders; its availability +must be confirmed, not assumed. + +**Impact:** settle this before Stage 9 and before any tag that would trigger +upgraded binary distribution. No runtime implementation stage needs to wait, +but the full distribution end state must remain incomplete until resolved. diff --git a/docs/roadmap/upgrade-kit.md b/docs/roadmap/upgrade-kit.md index 5cc29c1..2b60efa 100644 --- a/docs/roadmap/upgrade-kit.md +++ b/docs/roadmap/upgrade-kit.md @@ -5,15 +5,16 @@ Accepted feature direction, recorded on 2026-09-26; implementation is pending. Upgrade directly from Promptkit v0.9.0 to v0.11.0, including the intervening v0.10.0 catalog transition. This document defines the desired end state and -the changes from the current baseline, not a staged coding-agent plan. +the changes from the current baseline. The [implementation plan](implementation.md) +owns sequencing, concrete interface choices, verification steps, and unresolved +implementation decisions. The target is complete definition compatibility, controllable transient retries, an optional whole-run deadline, and modest opt-in execution diagnostics. This is a focused adapter and configuration update, not another architecture -migration. Optional diagnostics are included in the desired feature set but -need not block the core compatibility upgrade; they remain separately tracked -until delivered. “Optional” means callers opt in, not that they appear in -default output. +migration. Optional diagnostics are included in the desired feature set. +“Optional” means callers opt in, not that diagnostics may be silently omitted +from the completed adoption or appear in default output. [ADR 0004](../adr/0004-definition-boundary.md) and the [architecture policy](../policy/architecture.md) govern scope. Promptkit remains @@ -77,24 +78,18 @@ catalog entries can change or disappear across upgrades; inheriting from one does not create a frozen copy. Complete application-owned profiles remain the appropriate choice when an application owns its model and settings. -### Required Changes +### Change From Baseline -- Update `go.mod` and `go.sum`, retaining tagged upstream dependencies and no - committed workspace, replacement, vendor tree, or direct catalog imports. -- Replace the obsolete built-in inspection test and add focused coverage for - the maintained names, prompt/default selection, inheritance, and overrides. - Preserve existing definition conformance coverage rather than rebuilding - Promptkit's parser tests in Scriptorium. -- Review maintained prompts, profile references, examples, and fixtures for - removed catalog IDs. The selected upstream catalog removes `deepseek-3-2`, - `deepseek-4-flash`, and `deepseek-4-pro`. Newer DeepSeek names are not automatic - compatibility aliases, and Scriptorium must not silently invent aliases. -- Explain migration options: choose an available catalog profile, choose a - maintained Promptkit name, or supply a complete application-owned definition. - Keep the existing application-owned example profiles where fixed settings - are the instructional intent. -- Audit dependency-boundary tests for the `/v2` module paths without allowing - direct catalog imports or duplicating catalog contents. +The module graph and active definition fixtures reflect the released v0.11 +catalogs rather than v0.9's catalogs. The selected catalog removes +`deepseek-3-2`, `deepseek-4-flash`, and `deepseek-4-pro`; newer DeepSeek names +are not compatibility aliases. Migration guidance offers an available catalog +profile, a maintained Promptkit name, or a complete application-owned definition. +Application-owned examples retain fixed settings where that is their purpose. + +Definition conformance and focused resolution coverage protect Scriptorium's +adapter boundary without duplicating the upstream parser or catalog. Existing +dependency boundaries also apply to the new `/v2` module paths. ## Controllable Transient Retries @@ -121,23 +116,16 @@ loop. Transient retries and structural repairs have distinct budgets: each generation, including each repair generation, receives its transient budget. Retries do not imply exactly-once generation or complete billing knowledge. -### Required Changes +### Change From Baseline -- Extend strict application configuration and engine settings, mapping the - optional value to `promptkit.Config.TransientRetries` in shared engine - construction, including the server engine. -- Add presence-aware CLI and HTTP request mapping to - `promptkit.RunRequest.TransientRetries`. Keep this an execution setting, not - a copied prompt/profile schema field or provider model parameter. -- Validate malformed and negative values at the appropriate configuration or - request boundary, using Promptkit for framework semantics. -- Update CLI, HTTP, configuration, operations, and migration guidance to make - the default change, cost/latency implications, zero override, and independent - repair budget explicit. -- Protect inheritance and explicit-zero mapping with focused tests; include a - representative built-in HTTP retry path and verify that cancellation and - shared backend limits remain effective. Do not duplicate the entire upstream - status/backoff test matrix. +Application settings reach `promptkit.Config.TransientRetries`, including the +shared server engine, and request overrides reach +`promptkit.RunRequest.TransientRetries`. This adds execution controls, not +prompt/profile schema fields or provider model parameters. Strict decoding and +presence-aware mapping preserve inheritance and explicit zero. User and +operations guidance makes the changed default, latency/cost implications, and +independent repair budget visible. Cancellation and shared backend limits +remain effective during retrying executions. ## Optional Whole-Run Deadline @@ -155,21 +143,15 @@ context from the caller's existing context, so a caller cancellation or earlier deadline still wins. This is a cancellation boundary, not a guarantee of instantaneous interruption of arbitrary callback or filesystem code. -### Required Changes +### Change From Baseline -- Replace the CLI's unconditional background-context execution with a - context carrying the selected whole-run deadline when requested. -- Apply an HTTP execution deadline as a child of the incoming request context, - retaining disconnect cancellation and existing server lifecycle behavior. -- Keep total-run duration handling separate from per-attempt timeout conversion; - do not reuse names or fields that would reinterpret existing requests. -- Document that provider retry timing can exceed local backoff limits and that - per-attempt timeouts do not cap the complete run. -- Test deadline/cancellation propagation through a representative retry wait - and capacity wait, without adding another admission queue or retry engine. - -Exact option names, units, and zero-value semantics belong in the subsequent -implementation plan and must be settled before coding this interface. +CLI execution can carry a selected deadline instead of always using an +unbounded background context. HTTP execution can shorten the incoming context +without losing parent cancellation or changing server lifecycle behavior. +Total-run duration handling is separate from existing attempt-timeout conversion. +The interval covers the engine invocation, not adapter parsing, engine creation, +or output writing. Provider retry timing can exceed local backoff limits; +per-attempt timeouts do not cap that complete invocation. ## Opt-In Execution Reports @@ -193,26 +175,17 @@ Existing successful-result usage semantics remain unchanged. A terminal total is an aggregate, not another attempt to add to the bill; no report is a billing ledger. -### Required Changes +### Change From Baseline -- Add an opt-in reporting interface and explicit application-owned report DTOs. -- Collect per-request observations without shared mutable report state, durable - storage, or blocking work inside synchronous callbacks. Shared operational - infrastructure, if used, must remain concurrency-safe. -- Format a concise terminal report rather than introducing event streaming, - an event archive, or a general telemetry framework. -- Define success and failure response/report handling without manufacturing a - partial `RunResult` when Promptkit returns an operational error. -- Exclude prompt and response bodies, credentials, headers, endpoint URLs, - provider diagnostic text, and raw errors. Treat selected identifiers as - potentially sensitive even when the upstream event is content-free. -- Test omission by default, success/failure reporting, unknown versus known-zero - usage, incomplete totals, no aggregate double counting, and concurrent-request - isolation at the adapter/report boundary. +Both adapters have opt-in reporting and explicit application-owned report DTOs. +Per-request collection adds no shared mutable report state, durable storage, +callback I/O, or event archive. Operational failures can carry reports without +manufacturing partial `RunResult` values. Default output remains unchanged. -The report destination, wire fields, report-write failure handling, and precise -scope of reported metadata remain implementation-plan decisions. Full event -streaming and durable history are explicitly outside this feature. +Reports exclude prompt and response bodies, credentials, headers, endpoint URLs, +provider diagnostic text, and raw errors. Selected identifiers remain potentially +sensitive even when observations are content-free. Reports are concise terminal +metadata, not a general telemetry framework, event stream, or durable history. ## Errors And Compatibility @@ -226,11 +199,11 @@ failure details in execution diagnostics, including actual attempts where available. Retry exhaustion and the final cause are distinct facts; do not lose the underlying failure category by flattening them into one label. -Review mapping precedence and add representative wrapped-error tests while -retaining existing capacity and validation behavior. Specify total-deadline -outcomes in the implementation plan rather than accidentally treating every -context error as an unclassified failure. No breaking replacement of existing -error codes is required by this upgrade. +Capacity and validation behavior remains intact. Whole-run deadline and caller +cancellation outcomes are deliberate adapter classifications, distinct from a +provider-attempt timeout while the caller context is still live. The implementation +plan owns the exact presentation choices. Existing generation-error codes do +not need a breaking replacement for this upgrade. ## Licensing And Distribution Review @@ -253,18 +226,16 @@ approval gate for every future release. ## Documentation And Delivery -Update current documentation to link to the selected v0.11.0 contracts when -the implementation lands. CLI, HTTP, configuration, and operations documents -own their respective new behavior; internal docs explain adapter mapping; -release notes summarize compatibility impact. Preserve historical release notes -and ADR decision content. Revise current examples and migration guidance where -needed rather than globally replacing version strings in historical material. +Current documentation links to the selected v0.11.0 contracts once implemented. +CLI, HTTP, configuration, and operations documents own their respective new +behavior; internal docs explain adapter mapping; release notes summarize +compatibility impact. Historical release notes and ADR decision content remain +unchanged. Current examples and migration guidance reflect the selected version. -Deliver the upgrade as a minor Scriptorium release, not a compatibility patch. -Prominently explain removed catalog profiles, changed retry behavior, the -explicit single-attempt option, and timeout layering. Optional diagnostics may -follow in a separate feature release without representing unfinished reports -as already available. +The upgrade merits a minor Scriptorium release, not a compatibility patch, +with prominent migration guidance for removed profiles, changed retry behavior, +the explicit single-attempt option, and timeout layering. Any later decision to +defer part of the target leaves that feature visibly incomplete until delivered. Use the existing [release policy](../policy/release.md) and [runbook](../release.md). This roadmap authorizes no publication. No new ADR is @@ -283,7 +254,7 @@ boundaries. per-attempt timeouts or disabling parent cancellation. - Opt-in terminal reports provide safe, isolated, presence-aware execution metadata without changing default output or creating durable workflow state. - If deferred to a follow-up release, this item remains visibly incomplete. + Any deferred reporting work remains visibly incomplete. - Existing error classifications remain compatible, with deliberate handling of the new public identities and context outcomes. - The licensing/distribution approach is resolved before upgraded binary @@ -292,17 +263,7 @@ boundaries. implemented behavior. Relevant tests remain offline and credential-free, and normal development and release validation pass. -## Decisions To Resolve In The Implementation Plan - -- Exact placement/naming of retry overrides and which preparation-only CLI - options should accept them for consistency without implying execution. -- Whole-run deadline option names, units, zero handling, and external deadline - error presentation; omission must impose no new deadline. -- Report destinations and opt-in request shape, report DTO fields, failure - reporting, and report-output error behavior; defaults must remain unchanged. -- Whether reports ship with the core upgrade or in a clearly tracked follow-up. -- The concrete licensing/distribution approach and any resulting packaging work. - -These are bounded interface and delivery decisions, not permission to expand +Concrete interface decisions and the remaining distribution question are recorded +in the [implementation plan](implementation.md). They do not expand the product into conversations, follow-up messages, application-owned semantic retry loops, prepared-handle coordination, or multi-step workflows.