Document the Promptkit v0.9 adoption plan
This commit is contained in:
585
docs/roadmap/implementation.md
Normal file
585
docs/roadmap/implementation.md
Normal file
@@ -0,0 +1,585 @@
|
||||
# Promptkit v0.9.0 Adoption Implementation Plan
|
||||
|
||||
## Status And Authority
|
||||
|
||||
Planned. Implement the stages below in numeric order. The accepted feature
|
||||
scope and target state are defined by the
|
||||
[Promptkit v0.9.0 adoption roadmap](promptkit-v0.9.0-adoption.md), and the
|
||||
durable product boundary is defined by
|
||||
[ADR 0004](../adr/0004-definition-boundary.md). If this plan and either source
|
||||
conflict, the ADR governs product boundaries and the feature roadmap governs
|
||||
scope.
|
||||
|
||||
Follow the architecture, documentation, and testing policies under
|
||||
`docs/policy/` throughout the work. In particular:
|
||||
|
||||
- use only Promptkit's public root package and its released v0.9.0 tag;
|
||||
- do not add a workspace, module replacement, vendored Promptkit source,
|
||||
direct catalog import, or Scriptorium reimplementation of Promptkit parsing
|
||||
and validation;
|
||||
- keep default tests deterministic, offline, and free of real credentials or
|
||||
provider calls;
|
||||
- update each current-state contract in the same stage that implements its
|
||||
behavior; and
|
||||
- prefer a small number of boundary and integration tests over duplicating
|
||||
Promptkit's own framework test matrix.
|
||||
|
||||
At the start of every stage, inspect the working tree and preserve unrelated
|
||||
changes. At the end of every stage, run the focused tests named in that stage,
|
||||
`gofmt` changed Go files, and run `git diff --check`. Do not proceed while a
|
||||
stage's completion conditions are unmet.
|
||||
|
||||
## Stage 1: Adopt The Promptkit v0.9.0 Compatibility Baseline
|
||||
|
||||
Make the released dependency the build baseline before adding new application
|
||||
surface area.
|
||||
|
||||
### Implementation
|
||||
|
||||
1. Update `go.mod` to require
|
||||
`gitea.maximumdirect.net/eric/promptkit v0.9.0`, then run `go mod tidy` so
|
||||
`go.mod` and `go.sum` contain the module graph selected by that tag. Accept
|
||||
Promptkit's OpenRouter and Rakestrawhome catalog modules only as transitive
|
||||
dependencies; do not import them from Scriptorium.
|
||||
2. In the HTTP request DTO, change `model.reasoning_effort` from `string` to
|
||||
`*string`, and pass the pointer directly to
|
||||
`promptkit.ExecutionTargetOverride.ReasoningEffort`. This is required by the
|
||||
v0.9.0 API and preserves omitted versus explicit-empty input. Keep the JSON
|
||||
field itself a string when present.
|
||||
3. Compile every Promptkit public struct literal against v0.9.0. Keep literals
|
||||
keyed, do not restore the removed `RunRequest.Metadata` field, and do not
|
||||
add compatibility wrappers around Promptkit.
|
||||
4. Audit maintained prompt, profile, schema, configuration, and HTTP examples
|
||||
against v0.9.0. Correct only actual incompatibilities: endpoints must be
|
||||
absolute HTTP(S) URLs, roles must be `developer`, `system`, `user`, or
|
||||
`assistant`, positive repair budgets must be at most three and paired with
|
||||
`basic`, `json`, or `json_schema`, content and schema paths must remain
|
||||
within their Promptkit source rules, and extra parameters must be
|
||||
JSON-compatible and avoid reserved provider keys.
|
||||
5. Replace v0.1.0 links and current-version statements in current-state
|
||||
documentation with tagged v0.9.0 links. Do not rewrite historical version
|
||||
statements in accepted ADRs. Update the owning documentation to state the
|
||||
v0.9.0 credential and provider behavior: an optional missing environment
|
||||
credential may result in an unauthenticated provider request, while a
|
||||
profile that declares credentials required still fails as an invalid request
|
||||
when no source is supplied and with `ErrAPIKeyEnvMissing` when its selected
|
||||
environment source is empty; unset optional provider controls are omitted;
|
||||
and a positive repair budget can add provider calls, latency, token use, and
|
||||
cost.
|
||||
6. Preserve the architecture guard for forbidden replacements, workspaces,
|
||||
vendor trees, former facade packages, and Promptkit `internal` imports.
|
||||
Extend it only if the dependency graph introduces a new realistic bypass
|
||||
that the existing assertions do not cover.
|
||||
|
||||
### Verification
|
||||
|
||||
- Update HTTP mapping tests for the pointer-valued reasoning field, including
|
||||
separate omitted and explicit-empty cases, and update `docs/api.md` with the
|
||||
HTTP three-state behavior in this stage. Stage 4 adds the corresponding CLI
|
||||
control and session-ID support.
|
||||
- Run `go test ./...`, `go test -race ./...`, `go vet ./...`, and
|
||||
`go build ./cmd/scriptorium` against the tagged module with no local
|
||||
replacement.
|
||||
- Run the existing maintained render example and any example-validation tests.
|
||||
- Confirm that a search for `v0.1.0` finds only legitimate historical material,
|
||||
and that searches for `replace`, `go.work`, and Promptkit `/internal/`
|
||||
imports do not reveal a prohibited dependency path.
|
||||
|
||||
### Completion Conditions
|
||||
|
||||
The repository builds and all existing behavior tests pass on Promptkit v0.9.0;
|
||||
current documentation identifies v0.9.0 as the framework contract; no local or
|
||||
private Promptkit integration mechanism has been introduced.
|
||||
|
||||
## Stage 2: Remove Definition-Compatibility Restrictions In Run And Render
|
||||
|
||||
Allow the executable adapters to address every valid directory-backed prompt
|
||||
definition without imposing input requirements of their own.
|
||||
|
||||
### Implementation
|
||||
|
||||
1. Add `promptVersion string` to the shared run/render request configuration.
|
||||
Register `--prompt-version <version>` for both `run` and `render`, include it
|
||||
in usage text, and map it without local normalization to
|
||||
`promptkit.RunRequest.PromptVersion`.
|
||||
2. Make `--input` optional for `run` and `render`. Do not weaken
|
||||
`parseMappings` for callers that actually supplied mappings; instead, skip
|
||||
that parser when no input flag was supplied and pass a nil or empty input
|
||||
map to Promptkit. Promptkit remains responsible for declared-required and
|
||||
template-referenced input failures.
|
||||
3. In `POST /v1/runs`, remove the adapter check that rejects a nil or empty
|
||||
`inputs` object. Preserve strict JSON decoding and all validation of input
|
||||
references that are present. Continue mapping the already-supported
|
||||
`prompt_version` field exactly once into `RunRequest.PromptVersion`.
|
||||
4. Keep extra supplied inputs legal and retain the CLI file-reader and HTTP
|
||||
artifact containment and size policies.
|
||||
5. Update `docs/cli.md` and `docs/api.md` in this stage: prompt ID remains
|
||||
required, prompt version is optional, inputs are optional at the adapter
|
||||
boundary, and Promptkit decides whether the selected definition requires
|
||||
them. Update the shortest request examples only where necessary to avoid
|
||||
implying that all prompts need inputs.
|
||||
|
||||
### Verification
|
||||
|
||||
- Add CLI behavior tests proving that an omitted version selects a sole prompt
|
||||
definition, an explicit version selects the requested definition, and an
|
||||
omitted version fails clearly when a prompt ID has multiple versions.
|
||||
- Add CLI and HTTP adapter coverage for: a prompt with no declared inputs,
|
||||
omitted optional inputs, a missing declared-required input, and a missing
|
||||
template-referenced input. Use a capturing runner for pure request mapping
|
||||
and one real Promptkit engine integration fixture for the definition-driven
|
||||
failures; do not reproduce Promptkit's parser tests.
|
||||
- Retain tests for malformed supplied mappings, invalid HTTP input references,
|
||||
artifact containment, and request-size limits.
|
||||
- Run `go test ./internal/adapter/cli ./internal/adapter/http` and the maintained
|
||||
render workflow.
|
||||
|
||||
### Completion Conditions
|
||||
|
||||
Both executable interfaces can select a specific prompt version and can submit
|
||||
no inputs; only Promptkit rejects missing definition-required data.
|
||||
|
||||
## Stage 3: Add Engine-Scoped Custom Backend Configuration
|
||||
|
||||
Add the application-owned configuration needed for Promptkit profiles to select
|
||||
custom backend IDs and capacity policies.
|
||||
|
||||
### Configuration Contract
|
||||
|
||||
Use this exact strict-YAML shape:
|
||||
|
||||
```yaml
|
||||
backends:
|
||||
local-gpu:
|
||||
endpoint: http://localhost:11434/v1
|
||||
api_key_env: LOCAL_GPU_API_KEY
|
||||
extra_params:
|
||||
provider_option: enabled
|
||||
concurrency_limit: 2
|
||||
queue_capacity: 0
|
||||
```
|
||||
|
||||
The mapping key is the backend ID. `endpoint` is required by the effective
|
||||
contract. `api_key_env`, `extra_params`, `concurrency_limit`, and
|
||||
`queue_capacity` are optional. `concurrency_limit` uses zero as Promptkit's
|
||||
unlimited value. Represent `queue_capacity` as `*int` so omission uses
|
||||
Promptkit's default queue capacity and explicit zero disables queuing. There
|
||||
are no backend CLI overrides and no raw API-key field.
|
||||
|
||||
### Implementation
|
||||
|
||||
1. Add `Backends map[string]BackendConfig` to the on-disk `config.Config` and a
|
||||
`BackendConfig` with exactly the five YAML fields above. Add a resolved
|
||||
`BackendSettings` value containing `ID` plus the corresponding fields, and
|
||||
carry `[]BackendSettings` on `AppSettings`.
|
||||
2. During config application, sort the YAML map keys and build the resolved
|
||||
slice in that order. Preserve the `queue_capacity` pointer and the complete
|
||||
`extra_params` value. The deterministic order makes multi-error diagnostics
|
||||
and tests stable; it is not backend precedence.
|
||||
3. Do not duplicate Promptkit's backend validators in `internal/config`.
|
||||
Strict YAML decoding owns unknown-field and type errors. Promptkit
|
||||
`NewEngine` owns blank/reserved/duplicate IDs, endpoint and environment-name
|
||||
rules, reserved or invalid extra parameters, and capacity relationships.
|
||||
4. Refactor CLI engine construction around a small shared engine-settings value
|
||||
containing prompt, profile, and schema directories plus resolved backends.
|
||||
Convert every `BackendSettings` to `promptkit.Backend`, register it with
|
||||
`promptkit.WithBackend`, and use the same constructor for `run`, `render`,
|
||||
and `serve`. Wrap construction failures with application-configuration and
|
||||
engine-initialization context while preserving `errors.Is` identities.
|
||||
5. Retain one immutable backend registry per constructed engine. Do not create
|
||||
registries per HTTP request, and do not add direct imports of Promptkit's
|
||||
catalog modules; Promptkit's built-ins remain available automatically.
|
||||
6. Document `backends` in `docs/config.md`, including key identity,
|
||||
queue-capacity presence, validation ownership, optional environment
|
||||
credentials, and the raw-secret prohibition. Update
|
||||
`docs/internal/adapters.md` and `docs/internal/sources.md` to describe the
|
||||
implemented mapping and engine assembly. Add the exact shape to
|
||||
`examples/config.full.yml`; keep the minimal config minimal.
|
||||
|
||||
### Verification
|
||||
|
||||
- In `internal/config`, test strict decoding of all fields, unknown and raw
|
||||
secret-field rejection, deterministic resolved ordering, JSON-compatible
|
||||
nested `extra_params`, and omitted versus explicit-zero `queue_capacity`.
|
||||
- At the CLI/engine boundary, test a valid custom backend selected by a custom
|
||||
profile through `render`, and representative Promptkit validation failures:
|
||||
reserved backend ID, invalid endpoint, invalid capacity relationship, and a
|
||||
reserved `extra_params` key. Assert the stable error identity or a small
|
||||
diagnostic fragment, not Promptkit's full text.
|
||||
- Test that endpoint-only profiles and built-in profiles still work without a
|
||||
`backends` section.
|
||||
- Run `go test ./internal/config ./internal/adapter/cli` and render with the
|
||||
complete example without contacting a provider.
|
||||
|
||||
### Completion Conditions
|
||||
|
||||
Every command constructs its engine from the same resolved custom-backend
|
||||
configuration, Promptkit remains the validator, and profiles can select custom,
|
||||
built-in, or endpoint-only execution targets.
|
||||
|
||||
## Stage 4: Expose Session And Presence-Aware Reasoning Controls
|
||||
|
||||
Complete the request mapping for direct session identifiers and the v0.9.0
|
||||
reasoning override semantics.
|
||||
|
||||
### Implementation
|
||||
|
||||
1. Add `--session-id <id>` and `--reasoning-effort <value>` to both `run` and
|
||||
`render`, with no aliases. Map the session value to
|
||||
`RunRequest.SessionID`.
|
||||
2. Track flag presence for `--reasoning-effort`. Omission leaves
|
||||
`ExecutionTargetOverride.ReasoningEffort` nil, a nonblank value supplies a
|
||||
pointer to that value, and `--reasoning-effort=` supplies a pointer to the
|
||||
empty string. Include reasoning presence when deciding whether to allocate
|
||||
the enclosing execution override.
|
||||
3. Add optional `session_id` to the strict HTTP run-request DTO and map it to
|
||||
`RunRequest.SessionID` without application-level normalization. Keep
|
||||
`model.reasoning_effort` pointer-valued so omitted, nonblank, and empty JSON
|
||||
strings remain distinct.
|
||||
4. Add optional `session_id` to successful HTTP metadata and populate it from
|
||||
the effective `RunResult.SessionID`, not directly from the request. Use
|
||||
`omitempty`; absence means no effective session ID.
|
||||
5. Continue to reject unknown HTTP fields and do not add `api_key`, appended
|
||||
messages, or request-level output-contract fields.
|
||||
6. Update `docs/cli.md` and `docs/api.md` with the exact flag/field names,
|
||||
inheritance/replacement/clearing semantics, the non-secret nature and
|
||||
Promptkit length rules of session IDs, and effective-session response
|
||||
behavior.
|
||||
|
||||
### Verification
|
||||
|
||||
- CLI tests must distinguish an omitted reasoning flag, a nonblank replacement,
|
||||
and an explicit empty clear by observing the resulting prepared run through
|
||||
`render`. Add a mapping assertion that an explicit empty value still creates
|
||||
an execution override.
|
||||
- HTTP tests must capture and distinguish the same three reasoning states and
|
||||
continue to reject non-string values under the strict DTO contract. Treat
|
||||
JSON `null` like omission, matching ordinary pointer-field decoding; the
|
||||
documented explicit clear remains the empty JSON string.
|
||||
- CLI and HTTP tests must cover a direct session ID, a definition-rendered
|
||||
session ID when no direct value is supplied, no effective session ID, and an
|
||||
invalid overlong direct ID. Assert that successful HTTP metadata reports the
|
||||
engine's effective result.
|
||||
- Run `go test ./internal/adapter/cli ./internal/adapter/http ./internal/format`.
|
||||
|
||||
### Completion Conditions
|
||||
|
||||
CLI and HTTP callers can inherit, replace, or clear reasoning effort and can
|
||||
supply a direct session ID, with Promptkit retaining normalization and
|
||||
validation ownership.
|
||||
|
||||
## Stage 5: Present Backend Identity And Map Capacity Outcomes
|
||||
|
||||
Expose Promptkit's selected routing identity and make overload behavior a
|
||||
stable application contract.
|
||||
|
||||
### Implementation
|
||||
|
||||
1. In prepared-run text output, add `selected_backend_id` immediately after
|
||||
`selected_profile_id` when it is nonempty. Preserve direct JSON marshaling of
|
||||
`promptkit.PreparedRun`; v0.9.0 already supplies `selected_backend_id` and
|
||||
`effective_model_params.backend_id` JSON fields.
|
||||
2. In the CLI run summary, append `backend=<id>` when
|
||||
`RunResult.SelectedBackendID` is nonempty. Omit the token for endpoint-only
|
||||
profiles rather than deriving an ID from the endpoint.
|
||||
3. Add `selected_backend_id,omitempty` to HTTP metadata and
|
||||
`backend_id,omitempty` to `metadata.model_params`, mapping them from
|
||||
`RunResult.SelectedBackendID` and `ExecutionTarget.BackendID` respectively.
|
||||
Keep the two values independently mapped so an upstream inconsistency is not
|
||||
hidden by Scriptorium.
|
||||
4. In HTTP error mapping, classify `promptkit.ErrCapacityExceeded` before the
|
||||
general generation failure and return HTTP 503 with code
|
||||
`capacity_exceeded` and message `model backend capacity is exhausted`. Do
|
||||
not emit `Retry-After` and do not expose `CapacityError` details.
|
||||
5. In CLI `run`, classify the same identity as the existing runtime-error exit
|
||||
status 1 and print the stable safe diagnostic
|
||||
`run error: model backend capacity is exhausted`. Keep all other runtime
|
||||
error handling unchanged.
|
||||
6. Preserve `serveCommand`'s one-engine/one-handler construction outside the
|
||||
request path. Do not add per-request engine construction; that would split
|
||||
capacity state and violate ADR 0004.
|
||||
7. Update `docs/cli.md`, `docs/api.md`, and `docs/operations.md` with backend
|
||||
identity, endpoint-only omission, capacity status, lack of retry timing,
|
||||
shared per-engine admission, and the distinction between transient
|
||||
in-flight state and durable workflow state.
|
||||
|
||||
### Verification
|
||||
|
||||
- Formatter, CLI, and HTTP tests must cover a registered backend and an
|
||||
endpoint-only profile. Assert supplied backend IDs and empty/omitted identity
|
||||
rather than endpoint-derived guesses.
|
||||
- Add HTTP error-table coverage proving capacity is 503
|
||||
`capacity_exceeded`, provider failure remains 502 `llm_failed`, and neither
|
||||
response leaks wrapped provider or capacity diagnostics.
|
||||
- Add one offline concurrent HTTP integration test using a single real
|
||||
Promptkit engine, a custom backend with `concurrency_limit: 1` and explicit
|
||||
`queue_capacity: 0`, and a blocking fake LLM client. Hold the first request
|
||||
in generation, issue a second request, assert the second receives the 503
|
||||
contract, release the first, assert it succeeds, and assert observed peak
|
||||
generation concurrency is one. This test owns the Scriptorium shared-engine
|
||||
interaction; do not replicate Promptkit's FIFO or broader scheduler suite.
|
||||
- Run `go test -race ./internal/adapter/http ./internal/adapter/cli ./internal/format`.
|
||||
|
||||
### Completion Conditions
|
||||
|
||||
All presentations report actual backend identity when present, endpoint-only
|
||||
profiles remain identity-free, and one server instance enforces one engine's
|
||||
backend capacity across concurrent requests.
|
||||
|
||||
## Stage 6: Add Stable Prompt Inspection Output And CLI
|
||||
|
||||
Introduce reusable application-owned inspection presentation, then expose
|
||||
prompt inspection without model execution.
|
||||
|
||||
### Presentation Contract
|
||||
|
||||
1. Generalize the internal prepared-run format selector to an internal
|
||||
`OutputFormat` with `text` and `json` values, a text default, and one parser.
|
||||
Update config and prepared-run call sites to use it without changing
|
||||
existing prepared-run bytes or `defaults.render_format` behavior.
|
||||
2. Define a Scriptorium-owned prompt-inspection DTO; never marshal
|
||||
`promptkit.PromptInspection` directly. JSON contains these fields in this
|
||||
shape, including empty strings and an empty array where applicable:
|
||||
|
||||
```json
|
||||
{
|
||||
"prompt_id": "example",
|
||||
"prompt_version": "1",
|
||||
"prompt_hash": "opaque",
|
||||
"default_profile_id": "",
|
||||
"inputs": [
|
||||
{
|
||||
"name": "source",
|
||||
"required": true,
|
||||
"content_type": "text/plain",
|
||||
"description": "Input text"
|
||||
}
|
||||
],
|
||||
"output_contract": {
|
||||
"format": "text",
|
||||
"validation_mode": "none",
|
||||
"schema_path": "",
|
||||
"repair_attempts": 0
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
3. Preserve Promptkit's declared input order. Emit JSON with deterministic
|
||||
indentation and one trailing newline. Text output uses the same field order,
|
||||
renders `inputs: []` when empty, lists input records in declaration order,
|
||||
quotes free-form descriptions safely, includes empty declared values, and
|
||||
ends in one newline. The prompt hash is opaque and must not be interpreted.
|
||||
|
||||
### CLI Contract And Implementation
|
||||
|
||||
1. Add `inspect` to top-level dispatch and usage, with implementation in a new
|
||||
focused CLI file rather than further expanding `run.go`.
|
||||
2. Implement:
|
||||
|
||||
```text
|
||||
scriptorium inspect prompt --prompt ID [--prompt-version VERSION]
|
||||
[--config PATH] [--prompt-dir DIR] [--format text|json] [--out PATH]
|
||||
```
|
||||
|
||||
`--prompt` is required. All other flags are optional. Positional arguments
|
||||
and unknown inspection modes are errors. There are no deprecated aliases.
|
||||
3. Apply ordinary config discovery and CLI-over-file precedence. Require an
|
||||
effective prompt directory, pass the optional version directly to
|
||||
`Engine.InspectPrompt`, and construct the engine through the shared settings
|
||||
path so configured backends are validated consistently. Do not require a
|
||||
profile or schema directory for this operation.
|
||||
4. Default inspection format to text independently of
|
||||
`defaults.render_format`; `--format` is the only inspection format override.
|
||||
Reuse ordinary output-file handling. Write output only after successful
|
||||
inspection and formatting. Use exit 0 for success and exit 1 for parsing,
|
||||
configuration, load, formatting, or write failure.
|
||||
5. Update `docs/cli.md`, `docs/internal/adapters.md`, and the internal format
|
||||
component description. Make clear that inspection loads and normalizes a
|
||||
definition but does not resolve a profile, load a schema, render templates,
|
||||
reserve backend capacity, or call a model.
|
||||
|
||||
### Verification
|
||||
|
||||
- Formatter tests cover deterministic text and JSON, declared order, empty
|
||||
inputs/default profile/schema path, all nonempty fields, and nil inspection
|
||||
handling. Prefer structural JSON assertions plus a few stable text fragments
|
||||
over large snapshots.
|
||||
- CLI tests cover required flags, rejected positional arguments, config source
|
||||
selection, `--prompt-dir` precedence, sole and explicit multi-version
|
||||
selection, default and explicit formats, stdout versus `--out`, no partial
|
||||
output on failure, and Promptkit prompt-not-found/load identities.
|
||||
- Use a fake or deliberately failing model client to prove inspection performs
|
||||
no generation; do not assert Promptkit's internal call sequence.
|
||||
- Run `go test ./internal/format ./internal/adapter/cli`.
|
||||
|
||||
### Completion Conditions
|
||||
|
||||
`inspect prompt` produces stable Scriptorium text or JSON for every valid
|
||||
directory-backed v0.9.0 prompt definition without inputs or model access, and
|
||||
prepared-run presentation remains backward compatible.
|
||||
|
||||
## Stage 7: Add Stable Profile Inspection Output And CLI
|
||||
|
||||
Complete the inspection command family with inherited, built-in, custom, and
|
||||
endpoint-only profile support.
|
||||
|
||||
### Presentation Contract
|
||||
|
||||
1. Define a Scriptorium-owned profile-inspection DTO; never marshal
|
||||
`promptkit.ProfileInspection` directly. JSON has the following complete
|
||||
shape. Keep empty strings and zero numeric values because Promptkit reports
|
||||
unresolved optional controls as zero, and normalize absent `extra_params`
|
||||
to an empty object:
|
||||
|
||||
```json
|
||||
{
|
||||
"profile_id": "example",
|
||||
"effective_model_params": {
|
||||
"backend_id": "",
|
||||
"endpoint": "https://example.test/v1",
|
||||
"model": "model-name",
|
||||
"temperature": 0,
|
||||
"max_tokens": 0,
|
||||
"top_p": 0,
|
||||
"timeout_seconds": 0,
|
||||
"service_tier": "",
|
||||
"reasoning_effort": "",
|
||||
"api_key_env": "",
|
||||
"extra_params": {}
|
||||
},
|
||||
"api_key_required": false
|
||||
}
|
||||
```
|
||||
|
||||
2. Text output follows the same order, nests effective model parameters,
|
||||
encodes extra-parameter values as deterministic compact JSON with sorted
|
||||
keys, includes an empty `backend_id` for endpoint-only profiles, and ends in
|
||||
one newline. Never read or print an environment-variable value or direct
|
||||
API key.
|
||||
|
||||
### CLI Contract And Implementation
|
||||
|
||||
1. Implement the second mode:
|
||||
|
||||
```text
|
||||
scriptorium inspect profile --profile ID
|
||||
[--config PATH] [--profile-dir DIR] [--format text|json] [--out PATH]
|
||||
```
|
||||
|
||||
`--profile` is required. The common inspection parse, format, output, and
|
||||
exit rules from Stage 6 apply. No prompt, prompt directory, schema directory,
|
||||
or input is required, so built-in profiles are inspectable with only their
|
||||
ID.
|
||||
2. Resolve config and `--profile-dir` with normal precedence, construct the
|
||||
engine through the shared settings path so custom backends participate, and
|
||||
call `Engine.InspectProfile` exactly once.
|
||||
3. Update `docs/cli.md`, `docs/config.md`, and internal adapter/source docs.
|
||||
Explain that profile inspection resolves inheritance and backend defaults
|
||||
but does not read credentials, load a prompt, reserve capacity, or call a
|
||||
model. Document that zero provider-control values may mean unspecified,
|
||||
consistent with Promptkit v0.9.0.
|
||||
|
||||
### Verification
|
||||
|
||||
- Formatter tests cover deterministic text and JSON, registered and empty
|
||||
backend identity, optional environment-variable names, `api_key_required`,
|
||||
nested extra parameters, and the absence of credential values.
|
||||
- CLI tests cover built-in, inherited custom, custom-backend, and endpoint-only
|
||||
profiles; config and `--profile-dir` precedence; default and explicit format;
|
||||
stdout and `--out`; missing and invalid profiles; no partial output; and no
|
||||
requirement for `prompt_dir`.
|
||||
- Include a sentinel environment secret in a test and assert it appears in no
|
||||
inspection output or diagnostic while its variable name may appear.
|
||||
- Run `go test ./internal/format ./internal/adapter/cli ./internal/config`.
|
||||
|
||||
### Completion Conditions
|
||||
|
||||
`inspect profile` reports the complete safe effective profile view for every
|
||||
valid directory-backed v0.9.0 profile and configured backend combination
|
||||
without requiring a prompt or contacting a model.
|
||||
|
||||
## Stage 8: Complete Cross-Feature Conformance And Release Validation
|
||||
|
||||
Close the roadmap with representative definition-level integration coverage,
|
||||
copyable examples, documentation alignment, and full validation. Do not defer
|
||||
contract documentation from earlier stages to this stage; this is a final
|
||||
cross-check.
|
||||
|
||||
### Implementation And Documentation
|
||||
|
||||
1. Maintain one compact shared fixture set under
|
||||
`testdata/promptkit-v0.9/` with `prompts`, `profiles`, `schemas`, and any
|
||||
message content files needed by CLI and HTTP integration tests. Through a
|
||||
small set of composable definitions, cover:
|
||||
|
||||
- inline and file-backed messages;
|
||||
- `developer`, `system`, `user`, and `assistant` roles and message cache
|
||||
control;
|
||||
- required and optional inputs plus template-referenced values;
|
||||
- prompt versions, default profiles, and session-ID templates;
|
||||
- text and JSON output plus `none`, `basic`, `json`, and `json_schema`
|
||||
validation, a schema path, and a valid positive repair budget;
|
||||
- standalone and inherited profiles;
|
||||
- built-in, configured custom, and endpoint-only backends;
|
||||
- model controls, service tier, reasoning effort, timeout, optional
|
||||
credential environment names, and nested JSON-compatible extra
|
||||
parameters.
|
||||
|
||||
Exercise these fixtures through Scriptorium's `render` and inspection
|
||||
boundaries. Assert representative effective outcomes; do not independently
|
||||
decode YAML or exhaustively retest every Promptkit field combination.
|
||||
2. Add a copyable `examples/render-v0.9-features.sh` workflow that uses only
|
||||
`render` and `inspect` operations to demonstrate prompt-version selection,
|
||||
a direct session ID, reasoning replacement or clearing, a configured custom
|
||||
backend, and both inspection modes without provider access. Keep
|
||||
`examples/config.full.yml` complete and keep raw credentials out of every
|
||||
example.
|
||||
3. Review every canonical current-state owner named by the feature roadmap.
|
||||
Remove stale claims that inputs are always required, that every command
|
||||
needs `prompt_dir`, or that v0.1.0 semantics still apply. Ensure CLI, API,
|
||||
configuration, operations, internal adapter/source, development, and
|
||||
overview documents link instead of duplicating one another's contracts.
|
||||
Change architecture policy or internal component boundaries only if the
|
||||
implementation actually changed those durable responsibilities.
|
||||
4. Verify that all explicit non-goals remain absent: appended messages,
|
||||
retained prepared handles, HTTP inspection routes, alternative Promptkit
|
||||
source kinds, request output-contract replacement, public provider details,
|
||||
direct catalog imports, raw API keys, and durable workflow state.
|
||||
5. After every completion criterion below passes, change the feature roadmap's
|
||||
Status from `Planned` to `Complete`. Do not mark it complete for a partial
|
||||
implementation.
|
||||
|
||||
### Final Validation
|
||||
|
||||
Run all of the following from the repository root:
|
||||
|
||||
```bash
|
||||
gofmt -w $(git ls-files '*.go')
|
||||
go test ./...
|
||||
go test -race ./...
|
||||
go vet ./...
|
||||
go build ./cmd/scriptorium
|
||||
go run ./cmd/scriptorium render --config ./examples/config.yml --prompt generic.markdown_summary --input transcript=./examples/fixtures/transcript.md --input glossary=./examples/fixtures/glossary.yml
|
||||
bash ./examples/render-v0.9-features.sh
|
||||
git diff --check
|
||||
```
|
||||
|
||||
Also verify the architecture guard, strict example decoding, documentation
|
||||
links, executable permissions on shell examples, and absence of secrets. Review
|
||||
the suite for redundant upstream-semantic tests and retain only tests that
|
||||
protect Scriptorium mappings, presentation, configuration, concurrency
|
||||
integration, or complete workflows.
|
||||
|
||||
### Completion Conditions
|
||||
|
||||
Every roadmap validation criterion is satisfied on the tagged dependency; all
|
||||
current-state documentation and examples describe implemented behavior; the
|
||||
full application supports every Promptkit v0.9.0 feature expressible in its
|
||||
directory-backed prompt and profile definitions; and the ADR 0004 non-goals
|
||||
remain intact.
|
||||
|
||||
## Open Questions
|
||||
|
||||
None. The external syntax, application mappings, validation ownership,
|
||||
inspection wire shapes, concurrency behavior, testing boundaries, and stage
|
||||
order required for implementation are fixed above.
|
||||
Reference in New Issue
Block a user