309 lines
17 KiB
Markdown
309 lines
17 KiB
Markdown
# Promptkit v0.11 Adoption
|
|
|
|
## Status And Intent
|
|
|
|
Accepted feature direction, recorded on 2026-09-26; implementation is pending.
|
|
Upgrade directly from Promptkit v0.9.0 to v0.11.0, including the intervening
|
|
v0.10.0 catalog transition. This document defines the desired end state and
|
|
the changes from the current baseline, not a staged coding-agent plan.
|
|
|
|
The target is complete definition compatibility, controllable transient
|
|
retries, an optional whole-run deadline, and modest opt-in execution diagnostics.
|
|
This is a focused adapter and configuration update, not another architecture
|
|
migration. Optional diagnostics are included in the desired feature set but
|
|
need not block the core compatibility upgrade; they remain separately tracked
|
|
until delivered. “Optional” means callers opt in, not that they appear in
|
|
default output.
|
|
|
|
[ADR 0004](../adr/0004-definition-boundary.md) and the
|
|
[architecture policy](../policy/architecture.md) govern scope. Promptkit remains
|
|
the parser, validator, execution engine, and owner of framework defaults.
|
|
Scriptorium remains a simple executable application for independent prompt
|
|
executions, with shared operational state where required for backend limits.
|
|
|
|
## Baseline And Upstream Changes
|
|
|
|
Scriptorium currently selects Promptkit v0.9.0 and the v1.0.0 backend catalog
|
|
modules. It already supports CLI execution, rendering, prompt/profile
|
|
inspection, HTTP execution, configured backends, and presence-aware execution
|
|
overrides. It does not expose transient retry settings, a whole-run deadline,
|
|
or execution observers. CLI execution currently supplies a background context.
|
|
|
|
The upstream review covered the local tagged release v0.11.0 at commit
|
|
`db05b56166ca2d8650dfaf660e1cf0eed3932e1d`. Its canonical references are:
|
|
|
|
- [v0.10.0 migration guide](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.10.0/docs/releases/v0.10.0.md);
|
|
- [v0.11.0 release notes](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.11.0/docs/releases/v0.11.0.md);
|
|
- [definition formats](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.11.0/docs/formats.md);
|
|
- [execution and retry integration contract](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.11.0/docs/integrations/openai-compatible-chat.md);
|
|
- [consumer guide](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.11.0/docs/consumers/pkg-promptkit.md); and
|
|
- [observation declarations](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.11.0/observations.go).
|
|
|
|
A diagnostic run using a temporary module file compiled Scriptorium against
|
|
v0.11.0. The existing test suite reported one failing profile-inspection case
|
|
that selects the removed `deepseek-4-flash` profile; other packages passed.
|
|
This establishes an initial migration issue, not proof of complete behavioral
|
|
compatibility or coverage of the new capabilities. The diagnostic did not
|
|
change the repository dependency pin.
|
|
|
|
Neither intervening release adds declarative prompt/profile fields requiring
|
|
new adapter plumbing. The definition-facing changes concern catalog contents
|
|
and resolution precedence. Retry and observer APIs are execution controls;
|
|
their adoption is a deliberate case-by-case choice under ADR 0004.
|
|
|
|
## Dependency And Definition Compatibility
|
|
|
|
### Target End State
|
|
|
|
Scriptorium uses tagged Promptkit v0.11.0 through its supported root package,
|
|
with the catalog versions selected by that release:
|
|
|
|
- `promptkit-backend-openrouter/v2` v2.20260922.1;
|
|
- `promptkit-backend-rakestrawhome/v2` v2.20260922.0.
|
|
|
|
Valid definitions for the selected Promptkit version remain usable through
|
|
Scriptorium's existing sources and adapters, subject to its established HTTP
|
|
artifact and deployment boundaries. Rendering and inspection remain offline
|
|
operations that do not generate model output.
|
|
|
|
The maintained names `promptkit-openrouter-standard` and
|
|
`promptkit-openrouter-auto` work wherever a profile ID is accepted, including
|
|
prompt defaults, explicit selection, inheritance, and profile inspection.
|
|
They are opt-in choices, not new Scriptorium defaults. Ordinary custom-source
|
|
precedence remains intact.
|
|
|
|
A stable maintained name is not a frozen model/settings selection. External
|
|
catalog entries can change or disappear across upgrades; inheriting from one
|
|
does not create a frozen copy. Complete application-owned profiles remain the
|
|
appropriate choice when an application owns its model and settings.
|
|
|
|
### Required Changes
|
|
|
|
- Update `go.mod` and `go.sum`, retaining tagged upstream dependencies and no
|
|
committed workspace, replacement, vendor tree, or direct catalog imports.
|
|
- Replace the obsolete built-in inspection test and add focused coverage for
|
|
the maintained names, prompt/default selection, inheritance, and overrides.
|
|
Preserve existing definition conformance coverage rather than rebuilding
|
|
Promptkit's parser tests in Scriptorium.
|
|
- Review maintained prompts, profile references, examples, and fixtures for
|
|
removed catalog IDs. The selected upstream catalog removes `deepseek-3-2`,
|
|
`deepseek-4-flash`, and `deepseek-4-pro`. Newer DeepSeek names are not automatic
|
|
compatibility aliases, and Scriptorium must not silently invent aliases.
|
|
- Explain migration options: choose an available catalog profile, choose a
|
|
maintained Promptkit name, or supply a complete application-owned definition.
|
|
Keep the existing application-owned example profiles where fixed settings
|
|
are the instructional intent.
|
|
- Audit dependency-boundary tests for the `/v2` module paths without allowing
|
|
direct catalog imports or duplicating catalog contents.
|
|
|
|
## Controllable Transient Retries
|
|
|
|
### Target End State
|
|
|
|
Expose `transient_retries` in application configuration, with a CLI execution
|
|
override and an HTTP request override. Preserve value presence throughout:
|
|
|
|
| Value | Meaning |
|
|
| --- | --- |
|
|
| Omitted | Inherit the engine setting, ultimately Promptkit's default. |
|
|
| Zero | Disable additional transient attempts. |
|
|
| Positive integer | Permit that many additional transient attempts. |
|
|
| Negative integer | Invalid configuration or request. |
|
|
|
|
Adopt Promptkit's default of two additional attempts rather than silently
|
|
pinning the previous single-attempt behavior. Users must be able to select zero
|
|
explicitly. Request overrides take precedence over application configuration;
|
|
omission must not accidentally become an explicit zero.
|
|
|
|
Promptkit alone owns eligible statuses, attempt execution, backoff,
|
|
`Retry-After`, and capacity coordination. Scriptorium adds no outer model retry
|
|
loop. Transient retries and structural repairs have distinct budgets: each
|
|
generation, including each repair generation, receives its transient budget.
|
|
Retries do not imply exactly-once generation or complete billing knowledge.
|
|
|
|
### Required Changes
|
|
|
|
- Extend strict application configuration and engine settings, mapping the
|
|
optional value to `promptkit.Config.TransientRetries` in shared engine
|
|
construction, including the server engine.
|
|
- Add presence-aware CLI and HTTP request mapping to
|
|
`promptkit.RunRequest.TransientRetries`. Keep this an execution setting, not
|
|
a copied prompt/profile schema field or provider model parameter.
|
|
- Validate malformed and negative values at the appropriate configuration or
|
|
request boundary, using Promptkit for framework semantics.
|
|
- Update CLI, HTTP, configuration, operations, and migration guidance to make
|
|
the default change, cost/latency implications, zero override, and independent
|
|
repair budget explicit.
|
|
- Protect inheritance and explicit-zero mapping with focused tests; include a
|
|
representative built-in HTTP retry path and verify that cancellation and
|
|
shared backend limits remain effective. Do not duplicate the entire upstream
|
|
status/backoff test matrix.
|
|
|
|
## Optional Whole-Run Deadline
|
|
|
|
### Target End State
|
|
|
|
Callers can opt into a total execution deadline independently of the existing
|
|
generation timeout. No new default deadline is imposed. Existing timeout
|
|
controls must not silently change meaning: in v0.11.0 they apply per generation
|
|
attempt, whereas a whole-run deadline bounds the overall operation, including
|
|
preparation, capacity waiting, retry delays, attempts, and repairs as supported
|
|
by the upstream context contract.
|
|
|
|
Expose this choice through CLI execution and HTTP execution. Derive any timeout
|
|
context from the caller's existing context, so a caller cancellation or earlier
|
|
deadline still wins. This is a cancellation boundary, not a guarantee of
|
|
instantaneous interruption of arbitrary callback or filesystem code.
|
|
|
|
### Required Changes
|
|
|
|
- Replace the CLI's unconditional background-context execution with a
|
|
context carrying the selected whole-run deadline when requested.
|
|
- Apply an HTTP execution deadline as a child of the incoming request context,
|
|
retaining disconnect cancellation and existing server lifecycle behavior.
|
|
- Keep total-run duration handling separate from per-attempt timeout conversion;
|
|
do not reuse names or fields that would reinterpret existing requests.
|
|
- Document that provider retry timing can exceed local backoff limits and that
|
|
per-attempt timeouts do not cap the complete run.
|
|
- Test deadline/cancellation propagation through a representative retry wait
|
|
and capacity wait, without adding another admission queue or retry engine.
|
|
|
|
Exact option names, units, and zero-value semantics belong in the subsequent
|
|
implementation plan and must be settled before coding this interface.
|
|
|
|
## Opt-In Execution Reports
|
|
|
|
### Target End State
|
|
|
|
Provide a modest terminal execution report for CLI and HTTP callers who opt
|
|
in. Capture selected metadata through `RunRequest.Observer`, including attempt,
|
|
retry, repair, timing, outcome, and presence-aware usage information. Reports
|
|
should be available for completed successful and failed execution invocations
|
|
when Promptkit supplies terminal observations. Do not invent execution data for
|
|
failures that occur before invocation.
|
|
|
|
Keep generated CLI content on stdout or the existing output path. Execution
|
|
reports use a separate destination. Ordinary CLI output and HTTP response
|
|
shapes remain unchanged when reporting is not requested.
|
|
|
|
The report has a Scriptorium-owned schema rather than serializing upstream
|
|
events directly: `ExecutionEvent` has no stable JSON contract. Preserve unknown
|
|
usage versus reported zero, and incomplete versus complete observed totals.
|
|
Existing successful-result usage semantics remain unchanged. A terminal total
|
|
is an aggregate, not another attempt to add to the bill; no report is a billing
|
|
ledger.
|
|
|
|
### Required Changes
|
|
|
|
- Add an opt-in reporting interface and explicit application-owned report DTOs.
|
|
- Collect per-request observations without shared mutable report state, durable
|
|
storage, or blocking work inside synchronous callbacks. Shared operational
|
|
infrastructure, if used, must remain concurrency-safe.
|
|
- Format a concise terminal report rather than introducing event streaming,
|
|
an event archive, or a general telemetry framework.
|
|
- Define success and failure response/report handling without manufacturing a
|
|
partial `RunResult` when Promptkit returns an operational error.
|
|
- Exclude prompt and response bodies, credentials, headers, endpoint URLs,
|
|
provider diagnostic text, and raw errors. Treat selected identifiers as
|
|
potentially sensitive even when the upstream event is content-free.
|
|
- Test omission by default, success/failure reporting, unknown versus known-zero
|
|
usage, incomplete totals, no aggregate double counting, and concurrent-request
|
|
isolation at the adapter/report boundary.
|
|
|
|
The report destination, wire fields, report-write failure handling, and precise
|
|
scope of reported metadata remain implementation-plan decisions. Full event
|
|
streaming and durable history are explicitly outside this feature.
|
|
|
|
## Errors And Compatibility
|
|
|
|
Preserve existing CLI exit classifications and HTTP `502 / llm_failed`
|
|
classification for generation failures. Keep safe public messages rather than
|
|
exposing provider error bodies or parsing diagnostic strings.
|
|
|
|
Promptkit adds `ErrTransport`, `ErrMalformedResponse`, `ErrRetriesExhausted`, and
|
|
`RetryExhaustedError`. Use their public identities to support optional structured
|
|
failure details in execution diagnostics, including actual attempts where
|
|
available. Retry exhaustion and the final cause are distinct facts; do not lose
|
|
the underlying failure category by flattening them into one label.
|
|
|
|
Review mapping precedence and add representative wrapped-error tests while
|
|
retaining existing capacity and validation behavior. Specify total-deadline
|
|
outcomes in the implementation plan rather than accidentally treating every
|
|
context error as an unclassified failure. No breaking replacement of existing
|
|
error codes is required by this upgrade.
|
|
|
|
## Licensing And Distribution Review
|
|
|
|
Promptkit v0.11.0 changes from GPLv3 to LGPL-3.0-or-later. Scriptorium's current
|
|
binary workflow produces statically linked Go executables, and upstream's
|
|
[license summary](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.11.0/README.md#license)
|
|
explicitly calls out distribution considerations for that model. Catalog and
|
|
other dependency licenses remain separate.
|
|
|
|
Before distributing the upgraded binaries, resolve the applicable notices,
|
|
license texts, source availability, and any required materials or permissions
|
|
for recombination with a modified library. Record the chosen distribution
|
|
approach and make any corresponding packaging/documentation changes. This
|
|
roadmap is not a legal determination or a complete compliance checklist.
|
|
|
|
Do not automatically relicense Scriptorium, assume that library source alone
|
|
settles every distribution requirement, or alter historical release contents.
|
|
Licensing review is a concrete adoption prerequisite, not a new blanket manual
|
|
approval gate for every future release.
|
|
|
|
## Documentation And Delivery
|
|
|
|
Update current documentation to link to the selected v0.11.0 contracts when
|
|
the implementation lands. CLI, HTTP, configuration, and operations documents
|
|
own their respective new behavior; internal docs explain adapter mapping;
|
|
release notes summarize compatibility impact. Preserve historical release notes
|
|
and ADR decision content. Revise current examples and migration guidance where
|
|
needed rather than globally replacing version strings in historical material.
|
|
|
|
Deliver the upgrade as a minor Scriptorium release, not a compatibility patch.
|
|
Prominently explain removed catalog profiles, changed retry behavior, the
|
|
explicit single-attempt option, and timeout layering. Optional diagnostics may
|
|
follow in a separate feature release without representing unfinished reports
|
|
as already available.
|
|
|
|
Use the existing [release policy](../policy/release.md) and
|
|
[runbook](../release.md). This roadmap authorizes no publication. No new ADR is
|
|
needed unless a later decision changes the established product or compatibility
|
|
boundaries.
|
|
|
|
## Completion Criteria
|
|
|
|
- The tagged dependency and selected catalog modules are adopted without
|
|
unsupported imports or local dependency substitutions.
|
|
- Existing definition conformance tests and focused maintained-profile tests
|
|
pass; removed profile IDs have explicit migration guidance, not silent aliases.
|
|
- Retry settings preserve omission and zero across configuration, CLI, and HTTP;
|
|
Promptkit remains the sole retry/capacity implementation.
|
|
- Optional total deadlines bound the caller operation without redefining
|
|
per-attempt timeouts or disabling parent cancellation.
|
|
- Opt-in terminal reports provide safe, isolated, presence-aware execution
|
|
metadata without changing default output or creating durable workflow state.
|
|
If deferred to a follow-up release, this item remains visibly incomplete.
|
|
- Existing error classifications remain compatible, with deliberate handling
|
|
of the new public identities and context outcomes.
|
|
- The licensing/distribution approach is resolved before upgraded binary
|
|
distribution, and any required materials are included in the chosen workflow.
|
|
- Affected contracts, examples, migration guidance, and release notes match
|
|
implemented behavior. Relevant tests remain offline and credential-free,
|
|
and normal development and release validation pass.
|
|
|
|
## Decisions To Resolve In The Implementation Plan
|
|
|
|
- Exact placement/naming of retry overrides and which preparation-only CLI
|
|
options should accept them for consistency without implying execution.
|
|
- Whole-run deadline option names, units, zero handling, and external deadline
|
|
error presentation; omission must impose no new deadline.
|
|
- Report destinations and opt-in request shape, report DTO fields, failure
|
|
reporting, and report-output error behavior; defaults must remain unchanged.
|
|
- Whether reports ship with the core upgrade or in a clearly tracked follow-up.
|
|
- The concrete licensing/distribution approach and any resulting packaging work.
|
|
|
|
These are bounded interface and delivery decisions, not permission to expand
|
|
into conversations, follow-up messages, application-owned semantic retry loops,
|
|
prepared-handle coordination, or multi-step workflows.
|