Plan the ConfigKit configuration cutover
This commit is contained in:
@@ -1,516 +1,482 @@
|
||||
# Promptkit v0.11 Adoption Implementation Plan
|
||||
# ConfigKit Adoption Implementation Plan
|
||||
|
||||
## Status And Authority
|
||||
## Status, Authority, And Historical Plans
|
||||
|
||||
Planned on 2026-09-26. Execute Stage 1 through Stage 10 in order; each section
|
||||
records its own completion status. Each stage is intended to fit one focused
|
||||
gpt-5.6-terra implementation task, including its tests and documentation.
|
||||
Planned on 2026-09-26 against Scriptorium commit
|
||||
`43eea5c7c41d2a8134b3dece60f3af5c98b8fd1c`. No stage is implemented by this
|
||||
planning change. Execute Stage 1 through Stage 7 in order. Each stage is sized
|
||||
as one focused gpt-5.6-terra implementation task, including tests and current
|
||||
documentation. The maintainer has resolved the inspection-policy question:
|
||||
`extra_params` remain visible; only API-key values require secret protection.
|
||||
|
||||
The [feature roadmap](upgrade-kit.md) owns desired scope and end state.
|
||||
[ADR 0004](../adr/0004-definition-boundary.md) governs the product boundary.
|
||||
Read [development](../development.md) and all five documents under
|
||||
[policy](../policy/agents.md) before starting; follow the task-specific reading
|
||||
guide for each stage. Policies and the ADR take precedence over implementation
|
||||
mechanics here. This plan resolves the interface decisions left by the feature
|
||||
roadmap; it does not introduce new architectural invariants.
|
||||
The [feature roadmap](configkit.md) owns scope, target behavior, and user intent.
|
||||
This plan owns implementation steps and the interface choices needed to execute
|
||||
them. Read [development](../development.md) and every policy before work:
|
||||
[agents](../policy/agents.md), [architecture](../policy/architecture.md),
|
||||
[documentation](../policy/documentation.md), [testing](../policy/testing.md),
|
||||
[release](../policy/release.md), and [OSS](../policy/oss.md).
|
||||
[ADR 0004](../adr/0004-definition-boundary.md) continues to govern Promptkit's
|
||||
definition boundary. Do not rewrite accepted ADR decisions.
|
||||
|
||||
This replaces the completed v0.9 adoption plan. Its
|
||||
[historical version](https://gitea.maximumdirect.net/eric/scriptorium/src/commit/e4405966c87c8a7843b340146e90cc1e515bef42/docs/roadmap/implementation.md)
|
||||
remains the record for links from the older adoption roadmap. Do not repeat
|
||||
that migration.
|
||||
This filename previously contained completed Promptkit plans. Older roadmap
|
||||
links to this file refer to these historical records, not the ConfigKit stages:
|
||||
|
||||
## Baseline And Working Rules
|
||||
- Completed v0.11 implementation and release handoff:
|
||||
`git show 43eea5c7c41d2a8134b3dece60f3af5c98b8fd1c:docs/roadmap/implementation.md`.
|
||||
- Completed v0.9 implementation:
|
||||
`git show e4405966c87c8a7843b340146e90cc1e515bef42:docs/roadmap/implementation.md`.
|
||||
|
||||
The reviewed baseline is commit `e440596`: Promptkit v0.9.0, v1 backend
|
||||
catalogs, and Scriptorium's completed definition-facing architecture. The
|
||||
upgrade target is the released Promptkit v0.11.0, not whatever upstream
|
||||
`main` contains when implementation starts. The feature roadmap links the
|
||||
tagged upstream migration, formats, integration, and observation contracts.
|
||||
Those deliveries must not be repeated or lost from upcoming release preparation.
|
||||
These Git records are available locally without publishing the recent commits.
|
||||
|
||||
The principal change points are:
|
||||
## Working Rules And Selected Design
|
||||
|
||||
- `internal/config/config.go`: strict application YAML and resolved settings.
|
||||
- `internal/adapter/cli/run.go`: shared settings/engine construction,
|
||||
run/render request mapping, flags, output, and server startup.
|
||||
- `internal/adapter/http/dto.go` and `handler.go`: request mapping,
|
||||
caller contexts, response envelopes, and error/size handling.
|
||||
- A small new `internal/report` package: shared terminal-observation projection,
|
||||
not an execution facade or telemetry framework.
|
||||
- Existing adapter, configuration, definition-conformance, and architecture
|
||||
tests: retain useful coverage and extend it at the owning boundary.
|
||||
The dependency target is `gitea.maximumdirect.net/eric/ConfigKit v0.4.0`,
|
||||
commit `e6ced0d3d1194b58e0b3dc8970089471ba175258`. Use its public root package,
|
||||
imported as `configkit`. The roadmap links the exact upstream contracts.
|
||||
Do not opportunistically upgrade Promptkit or other dependencies, commit a
|
||||
workspace/replacement, or depend on sibling source at runtime.
|
||||
|
||||
For each stage:
|
||||
This is a clean configuration cutover, not a compatibility project. Native
|
||||
ConfigKit syntax, presence, nullability, resource limits, and precedence apply.
|
||||
Do not add a pre-parser, YAML serialization round trip, alternate merge engine,
|
||||
legacy mode, generic provider abstraction, or a new CLI framework.
|
||||
|
||||
1. Inspect the worktree and the relevant implementation/tests. Use the code
|
||||
graph for code discovery as directed by repository instructions.
|
||||
2. Implement only that stage and justified supporting refactoring. Retain one
|
||||
shared server engine and its backend admission state. Do not add an outer
|
||||
retry loop, new admission queue, conversation state, or copied profile schema.
|
||||
3. Update the owning current-state contracts in the same change, describing
|
||||
only delivered behavior. Link rather than copy schemas into internal docs.
|
||||
4. Run focused tests, format changed Go files, and check the diff. For code
|
||||
changes, also follow the baseline validation in development.md. Reuse checks
|
||||
already run for unchanged content; do not create additional release gates.
|
||||
5. Record completion and any remaining issues here, then commit the coherent
|
||||
unit under the agent policy. Do not mark later stages complete by inference.
|
||||
Keep application source discovery and domain rules in `internal/config`;
|
||||
CLI parsing/output stays in `internal/adapter/cli`. Promptkit remains responsible
|
||||
for prompt/profile/schema parsing, backend semantics, credentials, and engine
|
||||
behavior. Keep one shared server engine and existing HTTP artifact policy.
|
||||
No HTTP schema, execution-report, request-timeout, or retry-orchestration change
|
||||
is part of this plan.
|
||||
|
||||
Tests use local fixtures, real internal collaborators where practical, and
|
||||
controlled provider boundaries. No paid calls or real credentials. Integration
|
||||
tests of Promptkit's built-in retry path must use a local HTTP server or a
|
||||
controlled HTTP transport, not `WithLLMClient`: an injected LLM client bypasses
|
||||
that built-in retry policy. Do not duplicate upstream's complete status,
|
||||
backoff, parser, or repair test matrices.
|
||||
For every stage:
|
||||
|
||||
Reports are included in this implementation sprint, not deferred by default.
|
||||
The distribution decision is resolved: Stage 9 implements repository-hosted
|
||||
attribution and license texts, made discoverable through release-note footers.
|
||||
The [OSS attribution policy roadmap](policy.md) generalizes that approach for
|
||||
future upstream template adoption; that separate adoption is not a prerequisite.
|
||||
No stage authorizes a tag, push, publication, or changes to historical releases.
|
||||
1. Inspect the working tree and relevant code/tests through the repository's
|
||||
code-discovery guidance. Preserve unrelated changes.
|
||||
2. Implement the listed outcome and only supporting refactoring it needs.
|
||||
Internal names below are selected implementation guidance, not public APIs
|
||||
or immutable package-layout contracts.
|
||||
3. Update canonical current-state docs when behavior lands. Until Stage 3 the
|
||||
new loader is internal preparation, not an advertised executable feature.
|
||||
Never claim later stages are implemented.
|
||||
4. Run focused tests and the development guide's baseline validation for code
|
||||
changes. Tests stay offline, deterministic, and free of real credentials.
|
||||
Reuse existing coverage rather than cloning ConfigKit's parser test matrix.
|
||||
5. Format changed Go files, inspect the diff, validate affected docs/examples,
|
||||
record stage status/evidence here, and commit the coherent unit. Do not push,
|
||||
tag, publish, or add new release gates.
|
||||
|
||||
## Stage 1: Adopt The Released Dependency And Catalog Baseline
|
||||
### Application Configuration Definition
|
||||
|
||||
**Status: Complete.**
|
||||
Use a new `Document` type in `internal/config` for the native configuration
|
||||
shape. It replaces the old on-disk `Config` after CLI cutover. Keep the familiar
|
||||
YAML field names:
|
||||
|
||||
1. Upgrade `gitea.maximumdirect.net/eric/promptkit` to v0.11.0 and tidy the
|
||||
module graph. Review the resulting go.mod/go.sum changes. Expected indirect
|
||||
catalogs are OpenRouter `/v2` v2.20260922.1 and Rakestrawhome `/v2`
|
||||
v2.20260922.0. Use released modules; commit no workspace, replacement,
|
||||
vendor tree, direct catalog imports, or upstream implementation.
|
||||
2. Update the built-in case of
|
||||
`TestPromptkitV09ProfileInspectionResolvesSupportedTargets`, which currently
|
||||
uses removed `deepseek-4-flash`. Rename tests that now misleadingly describe
|
||||
only v0.9. Keep custom/inline target cases and the existing conformance suite.
|
||||
3. Add compact offline coverage of both maintained names:
|
||||
`promptkit-openrouter-standard` and `promptkit-openrouter-auto`. Exercise
|
||||
profile inspection and representative prompt-default, explicit-selection,
|
||||
inheritance, and custom-source override paths. Assert resolution/precedence,
|
||||
not a copied snapshot of the whole catalog. Do not make either name an
|
||||
application default.
|
||||
4. Review maintained examples and fixtures for `deepseek-3-2`,
|
||||
`deepseek-4-flash`, and `deepseek-4-pro`. Replace active obsolete references
|
||||
where necessary without inventing aliases. Keep application-owned examples
|
||||
whose purpose is fixed settings. Leave historical release notes unchanged.
|
||||
5. Inspect architecture guards for version-sensitive catalog paths; ensure
|
||||
`/v2` imports remain covered by the existing direct-import prohibition.
|
||||
Extend only an actual gap, not an exact-version release gate.
|
||||
6. Update current upstream-contract links from v0.9.0 to the selected v0.11.0
|
||||
contracts, including development, architecture, source, and integration
|
||||
references. Update a policy's last-updated date if editing it, but retain
|
||||
its base template version. Document the already-effective upstream retry
|
||||
default and timeout-per-attempt change without claiming the later
|
||||
Scriptorium controls exist yet.
|
||||
| YAML path | Go shape | Default and selected meaning |
|
||||
| --- | --- | --- |
|
||||
| `prompt_dir`, `profile_dir` | `string` | Empty; no source configured. |
|
||||
| `schema_dir` | `string` | Existing schema-directory default. Empty remains empty at the application boundary; Promptkit owns any subsequent framework default. |
|
||||
| `transient_retries` | `*int` | Nil; omitted/null delegates, zero disables additional attempts. |
|
||||
| `server` | Non-pointer struct | Its children receive typed defaults; null is invalid. |
|
||||
| `server.addr` | `string` | Existing listen-address default when absent; explicit empty is passed unchanged to the standard HTTP server, not replaced with the Scriptorium default. |
|
||||
| `server.artifact_root` | `string` | Empty; HTTP file input remains disabled. |
|
||||
| Three `server.max_*_bytes` fields | `int64` | Existing defaults; explicit zero disables the respective limit; null is invalid. |
|
||||
| `defaults` | Non-pointer struct | Contains `render_format`; null is invalid. |
|
||||
| `defaults.render_format` | `string` | Existing text default; resolved value must be text or json. Explicit empty is invalid, not defaulted. |
|
||||
| `backends` | `map[string]BackendDocument` | Nil; null is permitted, an empty mapping follows ConfigKit semantics. |
|
||||
|
||||
Acceptance: the full existing suite passes against the released dependency;
|
||||
maintained names resolve offline through Scriptorium; obsolete live references
|
||||
are handled; no new runtime flags or report schema are claimed in current docs.
|
||||
`BackendDocument` keeps `endpoint string`, `api_key_env string`,
|
||||
`extra_params map[string]any`, `concurrency_limit int`, and
|
||||
`queue_capacity *int`. Scalar nulls are invalid; nullable maps/pointers retain
|
||||
native semantics. No catalog IDs or Promptkit fields are added.
|
||||
|
||||
## Stage 2: Wire Retry Configuration And CLI Overrides
|
||||
The fresh defaults factory reads existing `internal/defaults` and format
|
||||
constants, not a duplicated set of literals. Normalization trims surrounding
|
||||
whitespace from application directory strings, artifact root, and server
|
||||
address; cleans only nonempty application paths; and lowercases/trims
|
||||
`defaults.render_format`. Empty paths remain empty: never let
|
||||
`filepath.Clean("")` turn disabled/unset into `.`. Record actual changes with
|
||||
`configkit.Normalize`, using value-free reasons. Do not normalize backend
|
||||
fields or parameter values on Promptkit's behalf.
|
||||
|
||||
**Status: Complete.**
|
||||
The read-only validator reports negative application size limits, negative
|
||||
nonnull application retries, and invalid render format through
|
||||
`Validation.Error` at their field paths. It does not print rejected values.
|
||||
Do not call the existing format parser's empty-as-default branch for this field.
|
||||
Do not add endpoint, backend ID, credential-presence, directory-existence,
|
||||
address-format, or positive-only-capacity checks. Existing Promptkit and
|
||||
command-specific validation retains those responsibilities where applicable.
|
||||
|
||||
Selected interface: top-level application YAML `transient_retries`, and
|
||||
`--transient-retries <integer>` on `run` and `render`. This is not a model
|
||||
parameter or a prompt/profile definition field. Serve and inspect consume the
|
||||
application setting through normal engine construction; they gain no new
|
||||
retry flag.
|
||||
## Stage 1: Add The Dependency And Native Application Definition
|
||||
|
||||
1. Add a presence-aware optional integer to strict config, resolved application
|
||||
settings, common command settings, and engine settings. Omitted or YAML
|
||||
null remains nil. Reject negative values; malformed types follow existing
|
||||
configuration error handling. There is no new arbitrary maximum.
|
||||
2. Map the engine value to `promptkit.Config.TransientRetries` in shared
|
||||
construction, including serve. Do not store Promptkit's default in
|
||||
`internal/defaults`: nil delegates to upstream.
|
||||
3. Register the CLI override with the shared execution-request flags. Preserve
|
||||
presence, including explicit zero, into
|
||||
`promptkit.RunRequest.TransientRetries`. The override belongs to the
|
||||
request; it must not mutate the engine setting.
|
||||
4. Render accepts and validates the same override for preparation consistency,
|
||||
but performs no generation and adds no synthetic retry field to prepared
|
||||
output. The report/deadline flags in later stages remain run-only.
|
||||
5. Update config and CLI contracts, adapter internals, and operational retry
|
||||
guidance. Explain precedence: request override, then application setting,
|
||||
then upstream default (currently two additional attempts). Zero disables
|
||||
additional transient attempts. Each repair generation has its own transient
|
||||
budget; structural repair counts remain separate.
|
||||
**Status: Not started.**
|
||||
|
||||
Acceptance: focused config/CLI tests cover omission, null where applicable,
|
||||
zero overriding a positive setting, positive values, and negative/malformed
|
||||
values. Exercise common engine construction through CLI execution and server
|
||||
settings without duplicating all config cases in every command. Offline
|
||||
rendering continues to work. No application-owned retry/backoff loop exists.
|
||||
Primary files: `go.mod`, `go.sum`, new focused files under `internal/config`,
|
||||
and `docs/oss/attribution.md`.
|
||||
|
||||
## Stage 3: Expose HTTP Retry Control And Verify Shared Admission
|
||||
1. Add ConfigKit at the selected release. Retain Promptkit v0.11.0 and its
|
||||
selected catalogs. Provision dependencies outside offline test execution.
|
||||
ConfigKit's yaml/v4 release-candidate dependency is the reviewed upstream
|
||||
choice, not an instruction to upgrade its parser independently.
|
||||
2. Implement `Document`, `BackendDocument`, the fresh defaults factory,
|
||||
recorded resolver, and read-only validator described above. Keep the old
|
||||
loader operational until Stage 3 so this stage remains independently
|
||||
buildable; this temporary development state is not a shipped compatibility
|
||||
mode or a second user-selectable configuration path.
|
||||
3. Test the new definition using real `configkit.Load[Document]` with
|
||||
`Bytes` and `NoSource`. Cover explicit zero/empty/null distinctions,
|
||||
retry/queue nil-versus-zero, safe application validation, and a normalized
|
||||
path whose effective output agrees with the returned value. Include nested
|
||||
unknown fields and a representative dynamic numeric parameter.
|
||||
4. Add ConfigKit and yaml/v4 attribution from their selected source material.
|
||||
Resolve the imported yaml/v4 commit from release/module metadata. Preserve
|
||||
actual copyright statements; do not infer a project statement from the
|
||||
FSF license-document copyright. Use applicable SPDX expressions, local
|
||||
standard texts, and commit-pinned upstream license/notice links. Reuse GPL,
|
||||
LGPL, MIT, and Apache texts where applicable, including companion links.
|
||||
Retain yaml.v3 attribution because Promptkit still incorporates it.
|
||||
5. Document the internal definition as preparation in the relevant internal
|
||||
guide without claiming the CLI has switched. No current configuration
|
||||
contract changes yet.
|
||||
|
||||
**Status: Complete.**
|
||||
Acceptance: the dependency and definition compile, focused tests demonstrate
|
||||
native semantics and audited normalization, and attribution follows the OSS
|
||||
policy. Existing executable behavior still passes its tests.
|
||||
|
||||
Selected interface: optional top-level JSON `transient_retries` integer on
|
||||
`POST /v1/runs`; omission/null inherits the engine, zero disables additional
|
||||
attempts, positive values override, negatives are invalid.
|
||||
## Stage 2: Build Discovery And The Shared Loading Boundary
|
||||
|
||||
1. Add a pointer integer to the request DTO and map it directly to the public
|
||||
RunRequest field. Keep the strict decoder. Type/representation errors use
|
||||
existing `400 invalid_json`; a negative number uses `400 invalid_request`
|
||||
and does not invoke the runner.
|
||||
2. Preserve the existing shared engine. Requests with different retry budgets
|
||||
must not change each other's settings.
|
||||
3. Extend safe error-mapping tests using wrapped public errors:
|
||||
transport, malformed response, and retry-exhausted generation errors retain
|
||||
`502 llm_failed`; capacity and validation classifications stay unchanged.
|
||||
Use error identities, not error-message parsing. Optional report detail
|
||||
will come from terminal events, not invented partial RunResults.
|
||||
4. Add a representative built-in HTTP retry integration scenario: a transient
|
||||
provider status followed by success. Demonstrate inherited retries and
|
||||
explicit zero at the adapter boundary, without testing every eligible
|
||||
status or asserting exact jitter.
|
||||
5. Reuse or extend the server capacity test to verify concurrent requests share
|
||||
the configured backend limit when retry behavior is enabled. Use barriers
|
||||
or channels to measure active calls relative to a test-controlled limit.
|
||||
Do not assert unrelated scheduler ordering or reimplement upstream admission.
|
||||
6. Update the API contract and relevant operational guidance.
|
||||
**Status: Not started.**
|
||||
|
||||
Acceptance: HTTP presence/strictness tests pass; the real built-in retry path
|
||||
is exercised offline; configured backend limits remain shared; no new error
|
||||
body text exposes provider diagnostics or secrets.
|
||||
Primary files: `internal/config` and its tests; `docs/internal/sources.md`.
|
||||
|
||||
## Stage 4: Add The CLI Whole-Run Deadline
|
||||
1. Add `LoadOptions` with `Path string`, `Explicit bool`, `Overlay string`,
|
||||
and `Overrides []configkit.Override`, plus
|
||||
`Load(LoadOptions) (*configkit.Result[Document], error)`.
|
||||
Discovery chooses a source; the one ConfigKit call owns actual file reading
|
||||
and decoding. Return the result so consumers retain the frozen inspection
|
||||
snapshot as well as the typed configuration.
|
||||
2. With `Explicit=true`, pass the selected path to `configkit.File`;
|
||||
an explicitly empty path is invalid, not implicit discovery or a fallback
|
||||
to the old system filename. Do not silently trim a source filename into a
|
||||
different filename. ConfigKit validates and resolves file sources.
|
||||
3. With `Explicit=false`, ignore the option's path value and examine the
|
||||
existing ordered implicit search locations. Only nonexistence permits
|
||||
examining the next candidate. Other stat failures are errors. Select the
|
||||
first existing candidate and let ConfigKit load it. A selected file's later
|
||||
disappearance/read/parse failure does not restart discovery.
|
||||
4. If neither implicit candidate exists, use `NoSource`. Initially leave
|
||||
imports disabled and the overlay empty; Stage 4 enables composition.
|
||||
Defaults-only loads still accept explicit application overrides. Never
|
||||
reinterpret an empty file as `NoSource`.
|
||||
5. Pass overrides into the ConfigKit call before decoding/validation; do not
|
||||
validate a separately merged lower configuration first. Native parsing and
|
||||
composition failures still follow the upstream lifecycle and are not
|
||||
bypassed by CLI overrides.
|
||||
6. Add a small `Settings(Document) AppSettings` conversion for existing
|
||||
executable wiring. It copies resolved fields, maps the already-validated
|
||||
format, and converts the backend map into ID-sorted backend settings.
|
||||
It performs no file IO, normalization, credential lookup, semantic backend
|
||||
checks, or additional defaults. Dynamic values retain their decoded shapes.
|
||||
7. Preserve ConfigKit's typed `*LoadError` through wrapping so the CLI can
|
||||
render safe structured diagnostics. Discovery-only errors are concise and
|
||||
quote paths safely; no new public error taxonomy is needed.
|
||||
8. Test ordered discovery with temporary paths rather than host system files,
|
||||
explicit missing/empty paths, empty-file rejection versus `{}`,
|
||||
defaults-only loading, source records, override-before-validation behavior,
|
||||
sorted backend conversion, and preservation of zero/nullable values.
|
||||
Remove no existing entry point until all its CLI callers migrate in Stage 3.
|
||||
|
||||
**Status: Complete.**
|
||||
Acceptance: real files and defaults-only inputs produce one reusable result,
|
||||
runtime conversion agrees with it, and tests do not depend on machine-wide
|
||||
configuration or secrets. The old and new internals are temporarily buildable,
|
||||
with no new user-visible mode.
|
||||
|
||||
Selected interface: `run --run-timeout <Go duration>`. Omission or zero adds
|
||||
no deadline, a positive duration retains full duration precision, and a negative
|
||||
duration is invalid. No application-config default is added. Existing
|
||||
`--timeout` retains its conversion and per-generation-attempt meaning.
|
||||
## Stage 3: Cut Existing Commands Over And Remove The Legacy Loader
|
||||
|
||||
1. Add the run-only flag and validation, separately from existing timeout
|
||||
conversion. Render/inspect/serve do not acquire a total-run flag.
|
||||
2. Immediately before engine.Run, derive a context with the selected timeout
|
||||
from the command's existing execution context (currently Background).
|
||||
Release its timer with cancel. Do not add signal/lifecycle redesign as a
|
||||
prerequisite. If a parent context seam is introduced for tests, keep it
|
||||
private and preserve any earlier parent deadline.
|
||||
3. The timed interval is the engine invocation: preparation, capacity waits,
|
||||
retry waits, generation attempts, and repairs to the extent they honor
|
||||
context. It does not include CLI parsing, engine construction, or output
|
||||
file writes. Do not promise immediate interruption of synchronous code.
|
||||
4. On a failed run whose execution context has expired, emit a safe deadline
|
||||
diagnostic and exit 1. Preserve existing classifications otherwise:
|
||||
success 0, operational/output error 1, completed invalid output 2. Do not
|
||||
reclassify a successful returned result merely because the timer expires
|
||||
after completion.
|
||||
5. Update CLI, subprocess integration, and operations docs with timeout
|
||||
layering. Provider Retry-After can exceed the ordinary backoff cap;
|
||||
per-attempt timeouts do not bound the whole invocation.
|
||||
**Status: Not started.**
|
||||
|
||||
Acceptance: CLI parsing preserves subsecond values and rejects negatives;
|
||||
omission/zero adds no deadline. A controlled waiting runner or HTTP boundary
|
||||
demonstrates cancellation and exit 1 without fragile elapsed-time assertions.
|
||||
Existing per-attempt timeout tests and generated-output behavior remain intact.
|
||||
Primary files: `internal/adapter/cli/run.go`, `inspect.go`,
|
||||
`internal/config/config.go`, and associated tests.
|
||||
|
||||
## Stage 5: Add HTTP Whole-Run Deadlines And Context Outcomes
|
||||
1. Route run, render, serve, inspect prompt, and inspect profile through the
|
||||
shared `Load` and `Settings` boundary. Preserve the explicit presence of
|
||||
`--config` in every parser. Centralize the small application-flag mapping
|
||||
rather than reproduce it in each command.
|
||||
2. Use `flag.FlagSet.Visit` (or the existing explicit-set helper) to create
|
||||
one `configkit.Override` per explicitly supplied application flag. Use
|
||||
`configkit.Keys`, built-in scalar values, and a safe `Flag` label containing
|
||||
only the flag name. Standard flag parsing's last occurrence supplies the
|
||||
single override; do not emit duplicate records or parser defaults.
|
||||
|
||||
**Status: Complete.**
|
||||
| Flag | Application path | Existing command scope after cutover |
|
||||
| --- | --- | --- |
|
||||
| `--prompt-dir` | `prompt_dir` | run, render, serve, inspect prompt |
|
||||
| `--profile-dir` | `profile_dir` | run, render, serve, inspect profile |
|
||||
| `--schema-dir` | `schema_dir` | run, render, serve |
|
||||
| `--addr` | `server.addr` | serve |
|
||||
| `--artifact-root` | `server.artifact_root` | serve |
|
||||
| `--max-request-bytes` | `server.max_request_bytes` | serve |
|
||||
| `--max-artifact-bytes` | `server.max_artifact_bytes` | serve |
|
||||
| `--max-response-bytes` | `server.max_response_bytes` | serve |
|
||||
|
||||
Selected interface: top-level `run_timeout_seconds`, an optional signed
|
||||
64-bit integer. Omission/null/zero adds no deadline; positive whole seconds
|
||||
create a child timeout; negatives are `400 invalid_request`. Reject values
|
||||
whose conversion would overflow time.Duration, not an arbitrary operational
|
||||
cap. JSON type or int64 representation errors remain `400 invalid_json`.
|
||||
Register the schema-directory flag on render as well as run/serve so the
|
||||
advertised application path override is consistently usable. This is not a
|
||||
general expansion of runtime/provider flags to other commands.
|
||||
3. Keep `--format`, `--out`, request `--transient-retries`, prompt/profile
|
||||
selection, model overrides, timeouts, inputs, variables, and reporting
|
||||
options outside the application configuration tree. In particular,
|
||||
`render --format` overrides that invocation's output format without
|
||||
rewriting `defaults.render_format`; omitted format inherits its resolved
|
||||
application value. Existing invocation-only parsing remains unchanged.
|
||||
4. Remove post-load directory trimming/cleaning/defaulting from argument
|
||||
finalizers, including unconditional schema-path cleaning. Keep only
|
||||
invocation-owned transformations such as output-file paths. Explicit
|
||||
empty profile/artifact-root overrides must clear file-provided values;
|
||||
explicit empty prompt directory must reach the existing execution
|
||||
prerequisite rather than revive the lower value.
|
||||
5. Keep existing command prerequisites: run/render/serve and prompt inspection
|
||||
need their effective prompt directory as currently required; profile
|
||||
inspection does not acquire a prompt-directory requirement. All commands
|
||||
still perform the common application schema validation. Do not introduce
|
||||
eager filesystem or credential validation for inspection.
|
||||
6. Add one CLI diagnostic path using `errors.As` and
|
||||
`configkit.WriteDiagnostics` for load errors. Carry successful load warnings
|
||||
to the command boundary and render them once on stderr before execution or
|
||||
output. Handle diagnostic writer errors as command failures. Do not flatten
|
||||
structured failures into raw parser text or print rejected values.
|
||||
Preserve existing process exit classifications; application/parse/load
|
||||
failures are exit 1, not generated-output validation exit 2.
|
||||
7. Remove the old YAML decoder, `strictInteger.UnmarshalYAML`, old on-disk
|
||||
types, `applyConfig`, `ApplyCLIOverrides`, and unused compatibility
|
||||
sentinels/helpers after migrating all callers/tests. Keep only one defaults
|
||||
factory. Remove the direct yaml.v3 import; module tidy may retain yaml.v3
|
||||
as an indirect Promptkit dependency.
|
||||
8. Update tests to the deliberate breaking semantics, not the legacy behavior.
|
||||
Protect explicit empty path clearing, scalar-null rejection, unknown fields,
|
||||
rejected empty/multi-document input, one representative YAML policy change,
|
||||
unchanged request-versus-engine retry precedence, absent optional
|
||||
credentials, and command-specific inspection prerequisites.
|
||||
9. Update configuration and CLI contracts, source/adapter internals, architecture
|
||||
ownership, and the contributor reading guide for the delivered cutover.
|
||||
Preserve existing application defaults, Promptkit definition support,
|
||||
HTTP containment, shared capacity, and report behavior. Adjust the current
|
||||
CLI claim that every command requires a prompt directory to match actual
|
||||
command-specific requirements.
|
||||
|
||||
1. Decode and validate before invocation. Derive the timeout from r.Context()
|
||||
immediately before Run and defer cancellation. Parent cancellation or an
|
||||
earlier parent deadline always wins. Body decoding and response writing are
|
||||
outside this engine-invocation deadline.
|
||||
2. For a failed Run, inspect the execution context before calling its deferred
|
||||
cancel. If that context is deadline-exceeded, return
|
||||
`504 deadline_exceeded`; if canceled, return `408 request_canceled`.
|
||||
Use fixed safe messages. These are deliberate context outcomes; do not
|
||||
infer them solely from a nested context error in an upstream generation
|
||||
error. A provider-attempt timeout while the execution context is live
|
||||
retains the existing generation-failure mapping.
|
||||
3. Otherwise use existing error mapping, including capacity and validation
|
||||
handling. A successful result remains successful even if cancellation
|
||||
races with subsequent response formatting. No response-delivery guarantee
|
||||
is possible after a client disconnect.
|
||||
4. Cover child/parent context propagation and overflow/negative input at the
|
||||
handler boundary. Add representative real-engine cancellation during a
|
||||
retry wait and a capacity wait; share useful fixtures with Stage 3 instead
|
||||
of duplicating the upstream scheduler suite. Synchronize entry into the
|
||||
wait; avoid exact backoff timing assertions.
|
||||
5. Update the API error table, request contract, and operations links.
|
||||
Acceptance: all existing commands use ConfigKit, no legacy parser or second
|
||||
application merge remains, relevant tests pass, and diagnostics and docs describe
|
||||
the native breaking behavior. Imports/overlays and new inspection commands are
|
||||
not yet advertised.
|
||||
|
||||
Acceptance: omission preserves the parent context, deadlines shorten rather
|
||||
than extend its lifetime, a live-context attempt timeout still maps to 502,
|
||||
and canceled waiters release capacity through Promptkit. The server has no
|
||||
extra timer queue or admission manager.
|
||||
## Stage 4: Enable Explicit Imports And Selected Configuration Overlays
|
||||
|
||||
## Stage 6: Implement A Shared Terminal Report Projection
|
||||
**Status: Not started.**
|
||||
|
||||
**Status: Complete.**
|
||||
Primary files: shared loader, CLI source-selection wiring, configuration and
|
||||
CLI tests, maintained configuration examples.
|
||||
|
||||
Create `internal/report` with a small Scriptorium-owned DTO and per-invocation
|
||||
collector. It depends only on Promptkit's supported root types and the standard
|
||||
library. It neither invokes the engine nor writes output.
|
||||
1. Enable `Options.Imports` for file-backed loads. Leave it false for
|
||||
`NoSource`, which rejects imports. Use ConfigKit's default canonical
|
||||
selected-file directory as the root; do not supply a wider filesystem root
|
||||
or add application traversal code.
|
||||
2. Wire `LoadOptions.Overlay` into `Options.Overlay`. Register
|
||||
`--config-overlay` on all five existing configuration-loading command
|
||||
modes. Treat the value exactly as an upstream overlay name: no trimming,
|
||||
case folding, extension fallback, path expansion, or local inheritance.
|
||||
Empty and exact lowercase default mean no active overlay under v0.4.0.
|
||||
A requested active overlay without a selected base source fails through
|
||||
ConfigKit; do not manufacture an empty base file or root.
|
||||
3. Use native `configkit.imports` directives for explicit paths/globs. The
|
||||
application enables the feature without requiring a second CLI opt-in;
|
||||
absent directives cause no discovery. Retain conflict-based base
|
||||
composition and native overlay replacement/null semantics.
|
||||
4. Preserve working-directory-relative application path values even when
|
||||
authored in fragments or overlays. Configuration-source containment is
|
||||
not the HTTP artifact policy and is not applied to prompt/schema paths.
|
||||
5. Add focused real-file tests for disjoint backend fragments, an import
|
||||
conflict, overlay replacement, explicit CLI precedence, missing selected
|
||||
overlay, and no-base overlay failure. Verify `--profile` still selects a
|
||||
Promptkit execution profile independently of `--config-overlay`.
|
||||
One real command workflow proves wiring; upstream owns exhaustive glob,
|
||||
symlink, cycle, and parser-limit tests.
|
||||
6. Add a small synthetic import/overlay example under `examples/`; preserve
|
||||
a simple single-file quickstart. Update config/CLI contracts and operations
|
||||
guidance for the new features, root selection, and relative-value policy.
|
||||
|
||||
1. Provide a collector callback compatible with ExecutionObserver. Ignore
|
||||
nonterminal events and project the execution-finished event into a value
|
||||
owned by that collector. Read it after synchronous Run returns. Retain no
|
||||
event history; no callback I/O, goroutines, queues, or global collector.
|
||||
Different invocations always use distinct collectors. No panic-recovery
|
||||
contract or synthetic terminal event is added.
|
||||
2. Use the same report object for CLI JSON files and the HTTP `execution`
|
||||
field. Explicitly map fields; do not marshal ExecutionEvent wholesale.
|
||||
The selected JSON shape is:
|
||||
Acceptance: supported commands load the same native composition pipeline;
|
||||
imports are explicit, overlays remain distinct from execution profiles, and no
|
||||
new scanning, remote acquisition, or path policy is introduced.
|
||||
|
||||
| Field | Representation and source |
|
||||
## Stage 5: Add Application Validation, Effective Output, And Source Inspection
|
||||
|
||||
**Status: Not started.**
|
||||
|
||||
Primary files: a focused `internal/adapter/cli/config.go`, shared config
|
||||
definition, CLI dispatch/help/tests, and canonical CLI/config documentation.
|
||||
|
||||
1. Apply the maintainer's selected inspection policy: no redaction for
|
||||
`extra_params`, including historical values. API-key values are the only
|
||||
relevant secrets, but this schema stores only `api_key_env` reference names.
|
||||
Keep those names visible and never resolve them in the ConfigKit load.
|
||||
Do not add secret tags to the reference name or parameter container, a
|
||||
catch-all secret predicate, sensitive-key guessing, or a selective masking
|
||||
interface. Actual credentials remain with Promptkit; raw API-key fields
|
||||
remain rejected. Use upstream inspection renderers without adding a second
|
||||
value-rendering path or a synthetic credential field.
|
||||
2. Add a top-level `config` command family using the existing standard-library
|
||||
flag style. The selected interfaces are:
|
||||
|
||||
| Command | Success stdout |
|
||||
| --- | --- |
|
||||
| `run_id`, `prompt_id`, `prompt_version`, `profile_id`, `backend_id`, `model` | Strings from terminal identity fields; omit unavailable empty strings. |
|
||||
| `outcome` | String matching a known upstream terminal outcome. |
|
||||
| `failure` | String matching a known upstream failure kind, including `none`. |
|
||||
| `status_code` | Provider status integer; omit when upstream reports zero/unavailable. |
|
||||
| `retries_exhausted` | Boolean, always present and independent of failure category. |
|
||||
| `attempts`, `transient_retries`, `repair_calls` | Unsigned terminal aggregate counts, always present. |
|
||||
| `duration_ms`, `capacity_wait_ms`, `generation_duration_ms`, `retry_delay_ms` | Integer milliseconds from the corresponding terminal durations, always present. |
|
||||
| `usage` | Object with the five token-count objects below, always present. |
|
||||
| `scriptorium config validate [flags]` | Empty; exit status communicates application validation success. |
|
||||
| `scriptorium config show [flags]` | `Result.WriteEffective` redacted YAML. |
|
||||
| `scriptorium config sources [flags]` | One newline-terminated JSON array of source records. |
|
||||
| `scriptorium config explain --path <pointer> [flags]` | Added in Stage 6. |
|
||||
|
||||
The usage keys are `prompt_tokens`, `completion_tokens`, `total_tokens`,
|
||||
`cached_tokens`, and `cache_write_tokens`. Each is an object containing
|
||||
`value` (integer), `known` (boolean), and `complete` (boolean), all always
|
||||
present. Copy all three upstream fields without substituting unknown
|
||||
values with known zero or complete totals.
|
||||
3. Map the v0.11 public outcome/failure constants explicitly; use `other` for
|
||||
an unrecognized value, not arbitrary new upstream text. Empty upstream
|
||||
no-failure values, if any, map to `none`. Durations are observed components,
|
||||
not a promise that their sum equals wall time.
|
||||
4. Copy terminal aggregates once. Do not sum attempt events, add terminal
|
||||
usage again, infer dispatch counts from retry budgets, or derive missing
|
||||
values from successful-result metadata. No terminal event means no report.
|
||||
5. Exclude inputs, outputs, session identifiers, headers, endpoint URLs,
|
||||
credentials, raw errors, and provider diagnostic text. Selected prompt,
|
||||
profile, backend, model, and run identifiers can still be sensitive.
|
||||
6. Test projection, omission of unavailable optional fields, known-zero versus
|
||||
unknown usage, incomplete totals, aggregate non-duplication, terminal
|
||||
absence, and separate collector isolation. Synthetic public events suit
|
||||
mapping tests; one real-engine terminal event proves wiring compatibility.
|
||||
7. Document the implemented package in internal overview/adapters. Do not
|
||||
advertise a CLI or HTTP opt-in until its stage lands.
|
||||
These are prospective interface specifications, not commands to execute
|
||||
before their stages are implemented.
|
||||
3. All configuration subcommands accept `--config`, `--config-overlay`,
|
||||
and the eight application override flags in Stage 3's table, regardless
|
||||
of which execution command normally uses them. They do not accept request,
|
||||
credential, backend-editing, output-format, or output-file flags. Shell
|
||||
redirection is sufficient for these inspection outputs. No positional
|
||||
arguments are accepted.
|
||||
4. Load exactly once through the common loader. Do not construct a Promptkit
|
||||
engine, restricted artifact reader, listener, or client. Missing prompt
|
||||
directories, unselected/absent credentials, and framework backend validity
|
||||
are not additional application-inspection gates. Invalid application
|
||||
types/ranges/format still fail. Validation describes application load
|
||||
validity, not end-to-end executability.
|
||||
5. For sources, map `Result.Sources()` to a small local DTO with JSON keys
|
||||
`id`, `label`, `layer` (always present), and `path` (omit when empty).
|
||||
Preserve upstream source order; emit `[]\n` for defaults-only input without
|
||||
overrides. Include synthetic CLI sources where present. Do not marshal
|
||||
opaque ConfigKit implementation objects. The standard JSON encoder escapes
|
||||
control characters; no custom terminal rendering is necessary.
|
||||
6. Diagnostics/warnings go only to stderr through the shared diagnostic helper.
|
||||
Successful operations exit 0; parse/load/output/diagnostic-write failures
|
||||
exit 1. No exit 2 meaning is introduced for configuration validation.
|
||||
Ordinary writer failures may leave partial output; do not promise atomic
|
||||
streams or buffering of an entire configuration.
|
||||
7. Test defaults-only show/sources, a composed load with CLI provenance,
|
||||
explicit empty clearing, normalized effective values, failed loads producing
|
||||
no successful stdout, and writer failure. Set a synthetic API-key environment
|
||||
variable: effective output must show its reference name, not its value.
|
||||
Repeat with the variable absent to establish that inspection does not require
|
||||
credentials. Assert that ordinary synthetic parameter values remain visible
|
||||
and runtime ExtraParams stay intact; no real environment or provider is used.
|
||||
8. Update CLI/config contracts, README navigation, subprocess integration,
|
||||
operations, and internal inventory to describe only these delivered modes.
|
||||
Retain the README's license/attribution ending.
|
||||
|
||||
Acceptance: a small bounded-by-one-event collector produces only the selected
|
||||
fields and preserves upstream meaning on both success and failure. It creates
|
||||
no new orchestration layer or durable state.
|
||||
Acceptance: validation, effective output, and source inspection use the same
|
||||
resolved configuration as execution, preserve stream separation, and honor the
|
||||
selected API-key boundary without masking or changing backend parameter data.
|
||||
|
||||
## Stage 7: Expose CLI Execution Report Files
|
||||
## Stage 6: Add Per-Path Explanation And Verify Credential Boundaries
|
||||
|
||||
**Status: Complete.**
|
||||
**Status: Not started.**
|
||||
|
||||
Selected interface: run-only `--execution-report <path>`, writing one
|
||||
newline-terminated JSON report to a separate file. A nonempty path opts in;
|
||||
an explicitly empty path is invalid. Paths use normal filesystem semantics;
|
||||
`-` has no special stdout/stderr meaning. No parent directories are created
|
||||
automatically. Existing files are replaced using ordinary output-file behavior;
|
||||
there is no atomicity, rollback, or crash-durability promise.
|
||||
Primary files: configuration command implementation/tests and CLI/config docs.
|
||||
|
||||
1. Add the flag and attach a fresh collector only when requested. No report
|
||||
destination means no observer and unchanged default streams.
|
||||
2. Keep generated content on stdout or --out. Reject a report/output collision
|
||||
before Run when it is detectable from equal cleaned absolute paths or
|
||||
os.SameFile for existing destinations. This prevents one selected output
|
||||
overwriting the other; it is not a filesystem sandbox or protection
|
||||
against concurrent path replacement.
|
||||
3. Do not create/truncate the report during parsing or engine setup. After
|
||||
Run, write it if a terminal event exists, even when Run failed or validation
|
||||
failed. Adapter failures before invocation leave the report path untouched.
|
||||
If no terminal event exists, leave the path untouched and give a concise
|
||||
diagnostic that no report was produced; do not fabricate an empty report
|
||||
or a RunResult. An existing report file may therefore be from an earlier run.
|
||||
4. On successful Run, attempt the normal artifact output first, then attempt
|
||||
report writing even if artifact output failed. On failed Run, report the
|
||||
safe execution error and still attempt report writing. Any artifact or
|
||||
report write failure yields exit 1; preserve the execution diagnostic if
|
||||
both execution and report writing fail. Otherwise retain exit 0 or 2 from
|
||||
the actual result. Absence of a terminal event alone does not override the
|
||||
engine result's exit classification. Already-written output is not undone.
|
||||
5. Reports describe engine execution, not later adapter output failures.
|
||||
Keep ordinary summary behavior when outputs succeed. Use standard JSON
|
||||
encoding and file writes consistent with existing CLI output permissions.
|
||||
6. Test success, validation failure, operational failure with terminal data,
|
||||
pre-invocation failure, report-write failure, output collision, and no-report
|
||||
defaults. Cover the consequential output-failure combinations without
|
||||
building an exhaustive filesystem-failure harness.
|
||||
7. Update CLI, subprocess, and operations contracts, including sensitive
|
||||
identifiers, caller-owned retention, and nontransactional output handling.
|
||||
Initially document the report fields in the CLI contract; Stage 8 moves
|
||||
that shared schema to the API contract once both adapters expose it.
|
||||
1. Implement `config explain` with the same selectors and override flags as
|
||||
Stage 5 plus a required, explicitly supplied `--path`. Parse using
|
||||
`configkit.ParsePath`; syntax is JSON Pointer, not dotted notation, a file
|
||||
path, or a URL fragment. Explicit `--path ""` selects the root; an omitted
|
||||
flag is an error. Preserve escaped and numeric mapping-key semantics.
|
||||
2. Use the successful shared result's `WriteExplanation`. Explain known
|
||||
effective and historical paths, including removed values. An unknown or
|
||||
malformed path exits 1 with no successful explanation output. Quote
|
||||
user-supplied paths safely if included in application diagnostics.
|
||||
Do not rebuild history, look up environment values, or reload sources.
|
||||
3. Keep the upstream human-readable renderer and its redaction semantics rather
|
||||
than inventing a JSON explanation schema or freezing the entire rendered
|
||||
prose as an application guarantee. Writer failures exit 1 and can leave
|
||||
partial output, as with effective output.
|
||||
4. Test a value's default/base/overlay/CLI history, recorded normalization,
|
||||
explicit equal-value writes, an escaped backend key, root selection, and
|
||||
an unknown path. Cover a displaced historical backend parameter value:
|
||||
it remains visible, including in a container/root explanation. Use distinct
|
||||
synthetic environment names and key values to verify reference names remain
|
||||
explainable but resolved API-key bytes are absent from all history/output.
|
||||
5. Update CLI/config docs and operations guidance with path syntax,
|
||||
historical-value meaning, representation differences, and sensitive
|
||||
identifiers/paths. Do not claim that redaction detects arbitrary secrets
|
||||
outside classified paths.
|
||||
|
||||
Acceptance: reporting never replaces or contaminates generated stdout;
|
||||
terminal failed invocations can yield reports; write failures are visible and
|
||||
do not erase the original run failure. Render and inspect remain unchanged.
|
||||
Acceptance: explain uses the existing snapshot and upstream semantics; its
|
||||
history matches the selected inputs and normalization, parameters remain
|
||||
visible, and explanation neither resolves nor discloses API-key values.
|
||||
|
||||
## Stage 8: Expose HTTP Execution Reports
|
||||
## Stage 7: Verify Integration, Finish Documentation, And Record Release Handoff
|
||||
|
||||
**Status: Complete.**
|
||||
**Status: Not started.**
|
||||
|
||||
Selected interface: optional top-level request boolean
|
||||
`include_execution_report`. Omission, null, or false leaves reporting off,
|
||||
consistent with the existing include_raw_output decoder behavior. True opts in.
|
||||
Both success and operational-error envelopes may then contain an optional
|
||||
top-level `execution` object using Stage 6's DTO.
|
||||
1. Review the feature roadmap against actual code. Confirm all five original
|
||||
command modes and all four new configuration subcommands use one definition
|
||||
and load pipeline. Remove any transitional loader, duplicated defaults,
|
||||
unused compatibility helpers, or post-snapshot configuration mutation.
|
||||
Do not add architecture machinery merely to assert a particular file layout.
|
||||
2. Exercise maintained single-file and import/overlay examples through the
|
||||
implemented configuration commands and a no-provider render workflow.
|
||||
Use focused backend integration evidence that dynamic integer parameters
|
||||
reach Promptkit correctly and missing optional credentials retain their
|
||||
established meaning. Use a local HTTP server only where provider serialization
|
||||
must be checked; never a real service or credential.
|
||||
3. Retain existing HTTP concurrency, containment, error mapping, retry,
|
||||
timeout, and report tests. Add a representative configuration-to-server
|
||||
wiring test only if existing coverage misses changed settings propagation;
|
||||
do not reproduce all upstream scheduling or HTTP cases.
|
||||
4. Run the development baseline on final code: `go test ./...`,
|
||||
`go test -race ./...`, `go vet ./...`, and
|
||||
`go build ./cmd/scriptorium`. Check formatting, affected examples/links,
|
||||
and `git diff --check`. Do not stage the built binary or scratch files.
|
||||
Compare any new standard license text byte-for-byte; preserve upstream
|
||||
whitespace rather than editing license wording to satisfy a style check.
|
||||
5. Ensure current configuration, CLI, subprocess, operations, internal docs,
|
||||
contributor routing, and architecture ownership agree. Testing policy
|
||||
should identify ConfigKit as the owner of exhaustive upstream loading tests,
|
||||
while Scriptorium owns application adoption tests. Change policy dates when
|
||||
revising policies, not base versions unless adopting new templates.
|
||||
Keep accepted ADR text and published release notes immutable.
|
||||
6. Record the release handoff below and mark only delivered stages complete.
|
||||
Update the feature roadmap's status, including the resolved inspection
|
||||
decision. Preserve deferred work. The next breaking release follows the
|
||||
release policy; version selection, candidate publication validation, release
|
||||
notes at that version, tag creation, and publishing are not this sprint.
|
||||
7. Commit the final coherent handoff and report validation evidence, remaining
|
||||
limitations, and any outstanding work without claiming source/binary
|
||||
publication or complete legal compliance.
|
||||
|
||||
1. Add the request flag and omitempty report pointers to success/error DTOs.
|
||||
Preserve existing artifact, validation, metadata, raw-output, and error
|
||||
fields. In particular, do not change metadata.usage to observed aggregate
|
||||
usage. The error envelope remains an error, not a partial success response.
|
||||
2. Allocate a collector per opted-in invocation and attach its callback.
|
||||
Populate execution only when a terminal observation exists. Adapter decode
|
||||
or validation failures before Run have no report. A failing invocation can
|
||||
still have terminal information even if no provider call was dispatched.
|
||||
3. Use Stage 5 context classifications and existing typed-error mappings.
|
||||
Preserve failure category and retries_exhausted separately in reports.
|
||||
Do not derive category labels from raw messages.
|
||||
4. Apply MaxResponseBytes to the complete serialized response including its
|
||||
report, for success and report-bearing error responses. On overflow return
|
||||
the existing compact `413 response_too_large` error without execution;
|
||||
do not recursively try to fit a report into that fallback. Preserve the
|
||||
existing compact-error fallback behavior rather than inventing a new
|
||||
minimum response limit.
|
||||
5. Test omitted-by-default shape, successful/invalid-output reporting,
|
||||
operational failures, no-terminal omission, and report-bearing response
|
||||
limits. Use a representative real-engine failure to ensure terminal data
|
||||
survives the adapter. Concurrent requests must return only their own
|
||||
identifiers/counters; exercise this under the race detector.
|
||||
6. Move the shared report-schema definition from the CLI contract to the API
|
||||
contract, removing the duplicate; CLI docs link to it for field meanings
|
||||
while owning file behavior.
|
||||
Update adapter internals and operations without duplicating the schema.
|
||||
Acceptance: the ConfigKit-native target is delivered with proportionate
|
||||
regression evidence and accurate documentation, and the workspace contains no
|
||||
temporary dependency substitutions or uncommitted implementation artifacts.
|
||||
|
||||
Acceptance: opt-in works for success and errors, reports participate in response
|
||||
limits, defaults remain compatible, and concurrent runs do not share collectors.
|
||||
Diagnostics include no disallowed content even when an upstream error contains it.
|
||||
### Release Handoff To Retain
|
||||
|
||||
## Stage 9: Add OSS Attribution And Release Links
|
||||
The upcoming release needs migration guidance for empty files versus `{}`,
|
||||
explicit empty values, nullable versus scalar fields, strict YAML/scalars,
|
||||
reserved keys and upstream size limits, and the invalid explicit-empty config
|
||||
path. Include imports/overlays, application path interpretation, CLI presence,
|
||||
inspection command scope, and the selected API-key-only secret policy. Explicit
|
||||
empty server address reaches standard HTTP server behavior instead of restoring
|
||||
Scriptorium's default; operators should supply their intended listen address.
|
||||
|
||||
**Status: Complete; distribution approach settled.**
|
||||
|
||||
Retain the existing static Linux binaries and source repositories. The selected
|
||||
approach is repository-hosted license texts and attribution, with a
|
||||
`LICENSE | ATTRIBUTION` footer in each new release document and hosted release
|
||||
page. This is the maintainer's selected arrangement, not a general assertion
|
||||
that repository co-location satisfies every license or distribution model.
|
||||
|
||||
1. Add verbatim GPLv3 and LGPLv3 texts under `docs/oss/licenses/`, using
|
||||
`GPL-3.0.txt` and `LGPL-3.0.txt`. Obtain the texts from the selected
|
||||
Promptkit release's COPYING and license text; retain the complete license
|
||||
text without folding Promptkit's project-specific preamble into the generic
|
||||
LGPL file. Keep Scriptorium's own LICENSE unchanged.
|
||||
2. Add `docs/oss/attribution.md` with a Promptkit heading and three
|
||||
attribution lines: `Copyright (C) 2026 Eric Rakestraw`;
|
||||
`SPDX-License-Identifier: LGPL-3.0-or-later`, linked to the local LGPL
|
||||
text; and a link to the Promptkit source repository. Link the corresponding
|
||||
v0.11.0 tag as the source destination for this adoption. Make the companion
|
||||
GPL text reachable from the document's general license-text navigation
|
||||
without expanding the three-line Promptkit entry into a legal essay.
|
||||
3. Preserve any additional notices required for other distributed components,
|
||||
including catalog dependencies, based on their actual selected licenses.
|
||||
The agreed Promptkit entry does not determine other components' terms.
|
||||
Keep attribution concise; do not add a dependency audit framework.
|
||||
4. Update the release policy and runbook to require the footer on future
|
||||
release notes, with absolute URLs targeting that Scriptorium release tag's
|
||||
root LICENSE and `docs/oss/attribution.md`. The visible text is
|
||||
`LICENSE | ATTRIBUTION`. Update the last-updated date of any edited
|
||||
policy, retaining its base template version. Do not edit published notes.
|
||||
5. Keep the checked-in release document as the source of the hosted release
|
||||
body. The existing workflow already uses it; no new asset packaging or
|
||||
separately maintained hosted notice is needed. During the next release
|
||||
preparation, add the footer with the selected version. Do not invent a
|
||||
release version or create a release solely to exercise this stage.
|
||||
6. Link the attribution document from README and assign ownership of
|
||||
attribution/license files in the documentation policy. Check local license
|
||||
links and the release-footer URL construction against the tagged-file
|
||||
layout and existing note-to-hosted-body flow. A not-yet-published tag need
|
||||
not resolve online before publication. Review the copied texts for fidelity.
|
||||
No live publication or network-dependent release gate is required.
|
||||
|
||||
The full Scriptorium and Promptkit sources, applicable permissions, and existing
|
||||
build materials are the basis for modification and recombination in this
|
||||
arrangement. Do not add a separate source/material archive, standalone license
|
||||
download assets, special modified-library rebuild instructions or verification
|
||||
exercise, dynamic linking, or a recurring legal-approval gate merely for this
|
||||
adoption. This does not excuse addressing an actual missing source/build
|
||||
material or a distinct dependency obligation if one is found.
|
||||
|
||||
Acceptance: attribution and both license texts exist and are linked; future
|
||||
release preparation specifies the version-specific footer; the existing
|
||||
workflow carries that footer to the hosted page. Binary packaging, checksum
|
||||
behavior, source-publication completion, and historical releases stay intact.
|
||||
The broader upstream policy template remains separate future work.
|
||||
|
||||
## Stage 10: Verify The Integrated End State And Prepare The Handoff
|
||||
|
||||
**Status: Complete to the pre-release boundary.**
|
||||
|
||||
1. Review the feature roadmap against implemented code and stage evidence.
|
||||
Confirm definition conformance, maintained profiles, retry inheritance,
|
||||
whole-run versus attempt timeouts, safe terminal reports, shared capacity,
|
||||
and distribution materials. Add only tests for a concrete uncovered risk;
|
||||
do not require every combination of these features.
|
||||
2. Run the repository's development validation on the final code:
|
||||
`go test ./...`, `go test -race ./...`, `go vet ./...`, and
|
||||
`go build ./cmd/scriptorium`. Check formatting and `git diff --check`.
|
||||
Validate affected offline examples and documentation links. Ensure no
|
||||
build output, local credentials, workspace, replacement, or generated
|
||||
packaging scratch data is staged.
|
||||
3. Check all current contracts and orientation links against v0.11 and delivered
|
||||
behavior. Preserve immutable accepted ADR content and historical release
|
||||
notes. The plan's interface tables cease to be the current contract once
|
||||
their canonical docs are implemented; retain this file as completion history.
|
||||
4. Record release-note handoff items here: removed profiles and migration
|
||||
choices, upstream retry default/cost implications and explicit zero,
|
||||
timeout layering and new context outcomes, opt-in reports and their limits,
|
||||
and the selected distribution approach. The next release should be a minor
|
||||
version under the release policy. Version selection and creation of the
|
||||
version-matched release note belong to release preparation, not an assumed
|
||||
version or publication in this implementation task.
|
||||
5. Mark stages and the feature roadmap complete only to the delivered boundary.
|
||||
If anything remains deferred or blocked, name it and do not claim full
|
||||
adoption/release readiness. Commit the final coherent documentation/validation
|
||||
changes and report remaining work.
|
||||
|
||||
Acceptance: the integrated implementation satisfies the feature roadmap, normal
|
||||
validation passes, and the release handoff is accurate. Actual source
|
||||
publication still follows the existing release policy and requires authorization.
|
||||
|
||||
### Release-Note Handoff
|
||||
|
||||
- This adoption merits a minor Scriptorium version under the release policy.
|
||||
Version selection, the matching release note, release-candidate validation,
|
||||
tag creation, and publication belong to release preparation and have not
|
||||
occurred here.
|
||||
- Promptkit v0.11.0 selects the `/v2` catalogs. The removed
|
||||
`deepseek-3-2`, `deepseek-4-flash`, and `deepseek-4-pro` profile IDs have no
|
||||
aliases; migrate to an available catalog profile, a maintained Promptkit
|
||||
name, or a complete application-owned profile.
|
||||
- Promptkit's built-in transient retry default is now two additional attempts.
|
||||
This can increase latency, provider calls, and cost; application or request
|
||||
`transient_retries: 0` selects a single attempt. Repair generations have
|
||||
separate transient budgets.
|
||||
- Generation timeouts apply per attempt. CLI `--run-timeout` and HTTP
|
||||
`run_timeout_seconds` opt into a whole-run deadline. A failed HTTP run whose
|
||||
execution context ended reports `504 deadline_exceeded` or
|
||||
`408 request_canceled`; live-context provider attempt failures retain 502.
|
||||
- CLI `--execution-report` files and HTTP `include_execution_report` expose
|
||||
opt-in terminal reports. They are omitted without a terminal observation,
|
||||
carry potentially sensitive identifiers, and do not replace result usage.
|
||||
HTTP report bytes count toward the response limit; CLI report writes are
|
||||
separate and nontransactional.
|
||||
- Distribution uses repository-hosted [attribution and dependency license
|
||||
texts](../oss/attribution.md). Each future release note needs the
|
||||
version-specific `LICENSE | ATTRIBUTION` footer described by the
|
||||
[release procedure](../release.md); no historical release was changed.
|
||||
Keep request retry overrides and Promptkit credential semantics distinct from
|
||||
application settings. Explain that configuration validation is not a provider
|
||||
probe or complete Promptkit definition validator, and that inspection output
|
||||
is not guaranteed reloadable. Include dependency/attribution changes and the
|
||||
existing LICENSE | ATTRIBUTION footer. Preserve the still-relevant Promptkit
|
||||
v0.11 release handoff identified above when preparing a combined next release.
|
||||
|
||||
## Open Questions
|
||||
|
||||
None currently. The previously open distribution question is resolved by
|
||||
Stage 9's repository-hosted attribution/license texts and release footer.
|
||||
The materials and future-release instructions are in place. Actual release
|
||||
preparation and publication remain outside this implementation plan.
|
||||
None remain for this scope. On 2026-09-26 the maintainer selected no redaction
|
||||
for `extra_params` and specified that the API key is the only secret requiring
|
||||
ConfigKit redaction. Stages 5–6 implement that decision through visible parameter
|
||||
values/reference names and the existing separation of actual credentials from
|
||||
application configuration. Do not reopen it or substitute blanket masking.
|
||||
|
||||
This does not claim automatic detection of credentials embedded in arbitrary
|
||||
strings or provider parameters. The application schema still rejects raw API-key
|
||||
fields, and Promptkit remains responsible for resolving named credential sources.
|
||||
|
||||
Reference in New Issue
Block a user