scriptorium/docs/roadmap/implementation.md

Promptkit v0.11 Adoption Implementation Plan

Status And Authority

Planned on 2026-09-26. Execute Stage 1 through Stage 10 in order; each section records its own completion status. Each stage is intended to fit one focused gpt-5.6-terra implementation task, including its tests and documentation.

The feature roadmap owns desired scope and end state. ADR 0004 governs the product boundary. Read development and all five documents under policy before starting; follow the task-specific reading guide for each stage. Policies and the ADR take precedence over implementation mechanics here. This plan resolves the interface decisions left by the feature roadmap; it does not introduce new architectural invariants.

This replaces the completed v0.9 adoption plan. Its historical version remains the record for links from the older adoption roadmap. Do not repeat that migration.

Baseline And Working Rules

The reviewed baseline is commit e440596: Promptkit v0.9.0, v1 backend catalogs, and Scriptorium's completed definition-facing architecture. The upgrade target is the released Promptkit v0.11.0, not whatever upstream main contains when implementation starts. The feature roadmap links the tagged upstream migration, formats, integration, and observation contracts.

The principal change points are:

• internal/config/config.go: strict application YAML and resolved settings.
• internal/adapter/cli/run.go: shared settings/engine construction, run/render request mapping, flags, output, and server startup.
• internal/adapter/http/dto.go and handler.go: request mapping, caller contexts, response envelopes, and error/size handling.
• A small new internal/report package: shared terminal-observation projection, not an execution facade or telemetry framework.
• Existing adapter, configuration, definition-conformance, and architecture tests: retain useful coverage and extend it at the owning boundary.

For each stage:

1. Inspect the worktree and the relevant implementation/tests. Use the code graph for code discovery as directed by repository instructions.
2. Implement only that stage and justified supporting refactoring. Retain one shared server engine and its backend admission state. Do not add an outer retry loop, new admission queue, conversation state, or copied profile schema.
3. Update the owning current-state contracts in the same change, describing only delivered behavior. Link rather than copy schemas into internal docs.
4. Run focused tests, format changed Go files, and check the diff. For code changes, also follow the baseline validation in development.md. Reuse checks already run for unchanged content; do not create additional release gates.
5. Record completion and any remaining issues here, then commit the coherent unit under the agent policy. Do not mark later stages complete by inference.

Tests use local fixtures, real internal collaborators where practical, and controlled provider boundaries. No paid calls or real credentials. Integration tests of Promptkit's built-in retry path must use a local HTTP server or a controlled HTTP transport, not WithLLMClient: an injected LLM client bypasses that built-in retry policy. Do not duplicate upstream's complete status, backoff, parser, or repair test matrices.

Reports are included in this implementation sprint, not deferred by default. The distribution decision is resolved: Stage 9 implements repository-hosted attribution and license texts, made discoverable through release-note footers. The OSS attribution policy roadmap generalizes that approach for future upstream template adoption; that separate adoption is not a prerequisite. No stage authorizes a tag, push, publication, or changes to historical releases.

Stage 1: Adopt The Released Dependency And Catalog Baseline

Status: Complete.

1. Upgrade gitea.maximumdirect.net/eric/promptkit to v0.11.0 and tidy the module graph. Review the resulting go.mod/go.sum changes. Expected indirect catalogs are OpenRouter /v2 v2.20260922.1 and Rakestrawhome /v2 v2.20260922.0. Use released modules; commit no workspace, replacement, vendor tree, direct catalog imports, or upstream implementation.
2. Update the built-in case of TestPromptkitV09ProfileInspectionResolvesSupportedTargets, which currently uses removed deepseek-4-flash. Rename tests that now misleadingly describe only v0.9. Keep custom/inline target cases and the existing conformance suite.
3. Add compact offline coverage of both maintained names: promptkit-openrouter-standard and promptkit-openrouter-auto. Exercise profile inspection and representative prompt-default, explicit-selection, inheritance, and custom-source override paths. Assert resolution/precedence, not a copied snapshot of the whole catalog. Do not make either name an application default.
4. Review maintained examples and fixtures for deepseek-3-2, deepseek-4-flash, and deepseek-4-pro. Replace active obsolete references where necessary without inventing aliases. Keep application-owned examples whose purpose is fixed settings. Leave historical release notes unchanged.
5. Inspect architecture guards for version-sensitive catalog paths; ensure /v2 imports remain covered by the existing direct-import prohibition. Extend only an actual gap, not an exact-version release gate.
6. Update current upstream-contract links from v0.9.0 to the selected v0.11.0 contracts, including development, architecture, source, and integration references. Update a policy's last-updated date if editing it, but retain its base template version. Document the already-effective upstream retry default and timeout-per-attempt change without claiming the later Scriptorium controls exist yet.

Acceptance: the full existing suite passes against the released dependency; maintained names resolve offline through Scriptorium; obsolete live references are handled; no new runtime flags or report schema are claimed in current docs.

Stage 2: Wire Retry Configuration And CLI Overrides

Status: Complete.

Selected interface: top-level application YAML transient_retries, and --transient-retries <integer> on run and render. This is not a model parameter or a prompt/profile definition field. Serve and inspect consume the application setting through normal engine construction; they gain no new retry flag.

1. Add a presence-aware optional integer to strict config, resolved application settings, common command settings, and engine settings. Omitted or YAML null remains nil. Reject negative values; malformed types follow existing configuration error handling. There is no new arbitrary maximum.
2. Map the engine value to promptkit.Config.TransientRetries in shared construction, including serve. Do not store Promptkit's default in internal/defaults: nil delegates to upstream.
3. Register the CLI override with the shared execution-request flags. Preserve presence, including explicit zero, into promptkit.RunRequest.TransientRetries. The override belongs to the request; it must not mutate the engine setting.
4. Render accepts and validates the same override for preparation consistency, but performs no generation and adds no synthetic retry field to prepared output. The report/deadline flags in later stages remain run-only.
5. Update config and CLI contracts, adapter internals, and operational retry guidance. Explain precedence: request override, then application setting, then upstream default (currently two additional attempts). Zero disables additional transient attempts. Each repair generation has its own transient budget; structural repair counts remain separate.

Acceptance: focused config/CLI tests cover omission, null where applicable, zero overriding a positive setting, positive values, and negative/malformed values. Exercise common engine construction through CLI execution and server settings without duplicating all config cases in every command. Offline rendering continues to work. No application-owned retry/backoff loop exists.

Stage 3: Expose HTTP Retry Control And Verify Shared Admission

Status: Complete.

Selected interface: optional top-level JSON transient_retries integer on POST /v1/runs; omission/null inherits the engine, zero disables additional attempts, positive values override, negatives are invalid.

1. Add a pointer integer to the request DTO and map it directly to the public RunRequest field. Keep the strict decoder. Type/representation errors use existing 400 invalid_json; a negative number uses 400 invalid_request and does not invoke the runner.
2. Preserve the existing shared engine. Requests with different retry budgets must not change each other's settings.
3. Extend safe error-mapping tests using wrapped public errors: transport, malformed response, and retry-exhausted generation errors retain 502 llm_failed; capacity and validation classifications stay unchanged. Use error identities, not error-message parsing. Optional report detail will come from terminal events, not invented partial RunResults.
4. Add a representative built-in HTTP retry integration scenario: a transient provider status followed by success. Demonstrate inherited retries and explicit zero at the adapter boundary, without testing every eligible status or asserting exact jitter.
5. Reuse or extend the server capacity test to verify concurrent requests share the configured backend limit when retry behavior is enabled. Use barriers or channels to measure active calls relative to a test-controlled limit. Do not assert unrelated scheduler ordering or reimplement upstream admission.
6. Update the API contract and relevant operational guidance.

Acceptance: HTTP presence/strictness tests pass; the real built-in retry path is exercised offline; configured backend limits remain shared; no new error body text exposes provider diagnostics or secrets.

Stage 4: Add The CLI Whole-Run Deadline

Status: Complete.

Selected interface: run --run-timeout <Go duration>. Omission or zero adds no deadline, a positive duration retains full duration precision, and a negative duration is invalid. No application-config default is added. Existing --timeout retains its conversion and per-generation-attempt meaning.

1. Add the run-only flag and validation, separately from existing timeout conversion. Render/inspect/serve do not acquire a total-run flag.
2. Immediately before engine.Run, derive a context with the selected timeout from the command's existing execution context (currently Background). Release its timer with cancel. Do not add signal/lifecycle redesign as a prerequisite. If a parent context seam is introduced for tests, keep it private and preserve any earlier parent deadline.
3. The timed interval is the engine invocation: preparation, capacity waits, retry waits, generation attempts, and repairs to the extent they honor context. It does not include CLI parsing, engine construction, or output file writes. Do not promise immediate interruption of synchronous code.
4. On a failed run whose execution context has expired, emit a safe deadline diagnostic and exit 1. Preserve existing classifications otherwise: success 0, operational/output error 1, completed invalid output 2. Do not reclassify a successful returned result merely because the timer expires after completion.
5. Update CLI, subprocess integration, and operations docs with timeout layering. Provider Retry-After can exceed the ordinary backoff cap; per-attempt timeouts do not bound the whole invocation.

Acceptance: CLI parsing preserves subsecond values and rejects negatives; omission/zero adds no deadline. A controlled waiting runner or HTTP boundary demonstrates cancellation and exit 1 without fragile elapsed-time assertions. Existing per-attempt timeout tests and generated-output behavior remain intact.

Stage 5: Add HTTP Whole-Run Deadlines And Context Outcomes

Status: Complete.

Selected interface: top-level run_timeout_seconds, an optional signed 64-bit integer. Omission/null/zero adds no deadline; positive whole seconds create a child timeout; negatives are 400 invalid_request. Reject values whose conversion would overflow time.Duration, not an arbitrary operational cap. JSON type or int64 representation errors remain 400 invalid_json.

1. Decode and validate before invocation. Derive the timeout from r.Context() immediately before Run and defer cancellation. Parent cancellation or an earlier parent deadline always wins. Body decoding and response writing are outside this engine-invocation deadline.
2. For a failed Run, inspect the execution context before calling its deferred cancel. If that context is deadline-exceeded, return 504 deadline_exceeded; if canceled, return 408 request_canceled. Use fixed safe messages. These are deliberate context outcomes; do not infer them solely from a nested context error in an upstream generation error. A provider-attempt timeout while the execution context is live retains the existing generation-failure mapping.
3. Otherwise use existing error mapping, including capacity and validation handling. A successful result remains successful even if cancellation races with subsequent response formatting. No response-delivery guarantee is possible after a client disconnect.
4. Cover child/parent context propagation and overflow/negative input at the handler boundary. Add representative real-engine cancellation during a retry wait and a capacity wait; share useful fixtures with Stage 3 instead of duplicating the upstream scheduler suite. Synchronize entry into the wait; avoid exact backoff timing assertions.
5. Update the API error table, request contract, and operations links.

Acceptance: omission preserves the parent context, deadlines shorten rather than extend its lifetime, a live-context attempt timeout still maps to 502, and canceled waiters release capacity through Promptkit. The server has no extra timer queue or admission manager.

Stage 6: Implement A Shared Terminal Report Projection

Status: Complete.

Create internal/report with a small Scriptorium-owned DTO and per-invocation collector. It depends only on Promptkit's supported root types and the standard library. It neither invokes the engine nor writes output.

1. Provide a collector callback compatible with ExecutionObserver. Ignore nonterminal events and project the execution-finished event into a value owned by that collector. Read it after synchronous Run returns. Retain no event history; no callback I/O, goroutines, queues, or global collector. Different invocations always use distinct collectors. No panic-recovery contract or synthetic terminal event is added.

2. Use the same report object for CLI JSON files and the HTTP execution field. Explicitly map fields; do not marshal ExecutionEvent wholesale. The selected JSON shape is:

   Field	Representation and source
   run_id, prompt_id, prompt_version, profile_id, backend_id, model	Strings from terminal identity fields; omit unavailable empty strings.
   outcome	String matching a known upstream terminal outcome.
   failure	String matching a known upstream failure kind, including none.
   status_code	Provider status integer; omit when upstream reports zero/unavailable.
   retries_exhausted	Boolean, always present and independent of failure category.
   attempts, transient_retries, repair_calls	Unsigned terminal aggregate counts, always present.
   duration_ms, capacity_wait_ms, generation_duration_ms, retry_delay_ms	Integer milliseconds from the corresponding terminal durations, always present.
   usage	Object with the five token-count objects below, always present.

   The usage keys are prompt_tokens, completion_tokens, total_tokens, cached_tokens, and cache_write_tokens. Each is an object containing value (integer), known (boolean), and complete (boolean), all always present. Copy all three upstream fields without substituting unknown values with known zero or complete totals.

3. Map the v0.11 public outcome/failure constants explicitly; use other for an unrecognized value, not arbitrary new upstream text. Empty upstream no-failure values, if any, map to none. Durations are observed components, not a promise that their sum equals wall time.

4. Copy terminal aggregates once. Do not sum attempt events, add terminal usage again, infer dispatch counts from retry budgets, or derive missing values from successful-result metadata. No terminal event means no report.

5. Exclude inputs, outputs, session identifiers, headers, endpoint URLs, credentials, raw errors, and provider diagnostic text. Selected prompt, profile, backend, model, and run identifiers can still be sensitive.

6. Test projection, omission of unavailable optional fields, known-zero versus unknown usage, incomplete totals, aggregate non-duplication, terminal absence, and separate collector isolation. Synthetic public events suit mapping tests; one real-engine terminal event proves wiring compatibility.

7. Document the implemented package in internal overview/adapters. Do not advertise a CLI or HTTP opt-in until its stage lands.

Acceptance: a small bounded-by-one-event collector produces only the selected fields and preserves upstream meaning on both success and failure. It creates no new orchestration layer or durable state.

Stage 7: Expose CLI Execution Report Files

Status: Complete.

Selected interface: run-only --execution-report <path>, writing one newline-terminated JSON report to a separate file. A nonempty path opts in; an explicitly empty path is invalid. Paths use normal filesystem semantics; - has no special stdout/stderr meaning. No parent directories are created automatically. Existing files are replaced using ordinary output-file behavior; there is no atomicity, rollback, or crash-durability promise.

1. Add the flag and attach a fresh collector only when requested. No report destination means no observer and unchanged default streams.
2. Keep generated content on stdout or --out. Reject a report/output collision before Run when it is detectable from equal cleaned absolute paths or os.SameFile for existing destinations. This prevents one selected output overwriting the other; it is not a filesystem sandbox or protection against concurrent path replacement.
3. Do not create/truncate the report during parsing or engine setup. After Run, write it if a terminal event exists, even when Run failed or validation failed. Adapter failures before invocation leave the report path untouched. If no terminal event exists, leave the path untouched and give a concise diagnostic that no report was produced; do not fabricate an empty report or a RunResult. An existing report file may therefore be from an earlier run.
4. On successful Run, attempt the normal artifact output first, then attempt report writing even if artifact output failed. On failed Run, report the safe execution error and still attempt report writing. Any artifact or report write failure yields exit 1; preserve the execution diagnostic if both execution and report writing fail. Otherwise retain exit 0 or 2 from the actual result. Absence of a terminal event alone does not override the engine result's exit classification. Already-written output is not undone.
5. Reports describe engine execution, not later adapter output failures. Keep ordinary summary behavior when outputs succeed. Use standard JSON encoding and file writes consistent with existing CLI output permissions.
6. Test success, validation failure, operational failure with terminal data, pre-invocation failure, report-write failure, output collision, and no-report defaults. Cover the consequential output-failure combinations without building an exhaustive filesystem-failure harness.
7. Update CLI, subprocess, and operations contracts, including sensitive identifiers, caller-owned retention, and nontransactional output handling. Initially document the report fields in the CLI contract; Stage 8 moves that shared schema to the API contract once both adapters expose it.

Acceptance: reporting never replaces or contaminates generated stdout; terminal failed invocations can yield reports; write failures are visible and do not erase the original run failure. Render and inspect remain unchanged.

Stage 8: Expose HTTP Execution Reports

Status: Complete.

Selected interface: optional top-level request boolean include_execution_report. Omission, null, or false leaves reporting off, consistent with the existing include_raw_output decoder behavior. True opts in. Both success and operational-error envelopes may then contain an optional top-level execution object using Stage 6's DTO.

1. Add the request flag and omitempty report pointers to success/error DTOs. Preserve existing artifact, validation, metadata, raw-output, and error fields. In particular, do not change metadata.usage to observed aggregate usage. The error envelope remains an error, not a partial success response.
2. Allocate a collector per opted-in invocation and attach its callback. Populate execution only when a terminal observation exists. Adapter decode or validation failures before Run have no report. A failing invocation can still have terminal information even if no provider call was dispatched.
3. Use Stage 5 context classifications and existing typed-error mappings. Preserve failure category and retries_exhausted separately in reports. Do not derive category labels from raw messages.
4. Apply MaxResponseBytes to the complete serialized response including its report, for success and report-bearing error responses. On overflow return the existing compact 413 response_too_large error without execution; do not recursively try to fit a report into that fallback. Preserve the existing compact-error fallback behavior rather than inventing a new minimum response limit.
5. Test omitted-by-default shape, successful/invalid-output reporting, operational failures, no-terminal omission, and report-bearing response limits. Use a representative real-engine failure to ensure terminal data survives the adapter. Concurrent requests must return only their own identifiers/counters; exercise this under the race detector.
6. Move the shared report-schema definition from the CLI contract to the API contract, removing the duplicate; CLI docs link to it for field meanings while owning file behavior. Update adapter internals and operations without duplicating the schema.

Acceptance: opt-in works for success and errors, reports participate in response limits, defaults remain compatible, and concurrent runs do not share collectors. Diagnostics include no disallowed content even when an upstream error contains it.

Stage 9: Add OSS Attribution And Release Links

Status: Pending; distribution approach settled.

Retain the existing static Linux binaries and source repositories. The selected approach is repository-hosted license texts and attribution, with a LICENSE | ATTRIBUTION footer in each new release document and hosted release page. This is the maintainer's selected arrangement, not a general assertion that repository co-location satisfies every license or distribution model.

1. Add verbatim GPLv3 and LGPLv3 texts under docs/oss/licenses/, using GPL-3.0.txt and LGPL-3.0.txt. Obtain the texts from the selected Promptkit release's COPYING and license text; retain the complete license text without folding Promptkit's project-specific preamble into the generic LGPL file. Keep Scriptorium's own LICENSE unchanged.
2. Add docs/oss/attribution.md with a Promptkit heading and three attribution lines: Copyright (C) 2026 Eric Rakestraw; SPDX-License-Identifier: LGPL-3.0-or-later, linked to the local LGPL text; and a link to the Promptkit source repository. Link the corresponding v0.11.0 tag as the source destination for this adoption. Make the companion GPL text reachable from the document's general license-text navigation without expanding the three-line Promptkit entry into a legal essay.
3. Preserve any additional notices required for other distributed components, including catalog dependencies, based on their actual selected licenses. The agreed Promptkit entry does not determine other components' terms. Keep attribution concise; do not add a dependency audit framework.
4. Update the release policy and runbook to require the footer on future release notes, with absolute URLs targeting that Scriptorium release tag's root LICENSE and docs/oss/attribution.md. The visible text is LICENSE | ATTRIBUTION. Update the last-updated date of any edited policy, retaining its base template version. Do not edit published notes.
5. Keep the checked-in release document as the source of the hosted release body. The existing workflow already uses it; no new asset packaging or separately maintained hosted notice is needed. During the next release preparation, add the footer with the selected version. Do not invent a release version or create a release solely to exercise this stage.
6. Link the attribution document from README and assign ownership of attribution/license files in the documentation policy. Check local license links and the release-footer URL construction against the tagged-file layout and existing note-to-hosted-body flow. A not-yet-published tag need not resolve online before publication. Review the copied texts for fidelity. No live publication or network-dependent release gate is required.

The full Scriptorium and Promptkit sources, applicable permissions, and existing build materials are the basis for modification and recombination in this arrangement. Do not add a separate source/material archive, standalone license download assets, special modified-library rebuild instructions or verification exercise, dynamic linking, or a recurring legal-approval gate merely for this adoption. This does not excuse addressing an actual missing source/build material or a distinct dependency obligation if one is found.

Acceptance: attribution and both license texts exist and are linked; future release preparation specifies the version-specific footer; the existing workflow carries that footer to the hosted page. Binary packaging, checksum behavior, source-publication completion, and historical releases stay intact. The broader upstream policy template remains separate future work.

Stage 10: Verify The Integrated End State And Prepare The Handoff

Status: Pending.

1. Review the feature roadmap against implemented code and stage evidence. Confirm definition conformance, maintained profiles, retry inheritance, whole-run versus attempt timeouts, safe terminal reports, shared capacity, and distribution materials. Add only tests for a concrete uncovered risk; do not require every combination of these features.
2. Run the repository's development validation on the final code: go test ./..., go test -race ./..., go vet ./..., and go build ./cmd/scriptorium. Check formatting and git diff --check. Validate affected offline examples and documentation links. Ensure no build output, local credentials, workspace, replacement, or generated packaging scratch data is staged.
3. Check all current contracts and orientation links against v0.11 and delivered behavior. Preserve immutable accepted ADR content and historical release notes. The plan's interface tables cease to be the current contract once their canonical docs are implemented; retain this file as completion history.
4. Record release-note handoff items here: removed profiles and migration choices, upstream retry default/cost implications and explicit zero, timeout layering and new context outcomes, opt-in reports and their limits, and the selected distribution approach. The next release should be a minor version under the release policy. Version selection and creation of the version-matched release note belong to release preparation, not an assumed version or publication in this implementation task.
5. Mark stages and the feature roadmap complete only to the delivered boundary. If anything remains deferred or blocked, name it and do not claim full adoption/release readiness. Commit the final coherent documentation/validation changes and report remaining work.

Acceptance: the integrated implementation satisfies the feature roadmap, normal validation passes, and the release handoff is accurate. Actual source publication still follows the existing release policy and requires authorization.

Open Questions

None currently. The previously open distribution question is resolved by Stage 9's repository-hosted attribution/license texts and release footer. Its implementation remains pending; resolution of the decision does not imply that the notices or footer already exist.