210 lines
10 KiB
Markdown
210 lines
10 KiB
Markdown
# Operations Guide
|
|
|
|
## Scope And References
|
|
|
|
This runbook covers deployment, normal operation, capacity planning, and safe
|
|
recovery for Scriptorium. It does not redefine invocation syntax, configuration
|
|
fields, or HTTP wire behavior.
|
|
|
|
- [CLI reference](cli.md): commands, output destinations, and exit codes.
|
|
- [Configuration reference](config.md): application settings, source
|
|
locations, defaults, and credential mapping.
|
|
- [Promptkit framework formats](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.11.0/docs/formats.md):
|
|
prompt, profile, schema, execution-setting, and framework credential
|
|
contracts.
|
|
- [HTTP API reference](api.md): route, request/response schema, status codes,
|
|
limits, and HTTP artifact access.
|
|
- [Consumer integration overview](consumers/api.md): caller responsibilities.
|
|
|
|
## Operational Model And State
|
|
|
|
Scriptorium handles one prompt request for each CLI invocation or HTTP request.
|
|
It has no durable run store, archive, checkpoint, cache, or resume mechanism.
|
|
A failed or interrupted request is recovered by correcting its inputs,
|
|
configuration, or environment and submitting a new request.
|
|
|
|
Generated artifacts, rendered prompts, model output, and run metadata are
|
|
caller-owned data. Retention, encryption, backup, and deletion are deployment
|
|
responsibilities.
|
|
Optional [CLI execution reports](cli.md) also belong to the caller. They can
|
|
contain sensitive identifiers even though they omit prompt and response bodies.
|
|
Keep report paths separate from generated output and apply the same access and
|
|
retention controls. Output and report writes are not transactional; an older
|
|
report may remain when no terminal observation was produced.
|
|
HTTP callers may also request a [terminal execution report](api.md#execution-report).
|
|
It shares the same identifier sensitivity and should receive response access
|
|
and retention controls. An opted-in report increases the encoded response size;
|
|
an oversized success or report-bearing error becomes the compact 413 response
|
|
without a report.
|
|
|
|
## Deploy The Filesystem And Process
|
|
|
|
Provide the process with readable configured Promptkit prompt, profile, and
|
|
schema sources that follow the tagged framework formats. For an HTTP deployment
|
|
that accepts file artifacts, use a dedicated, narrow artifact directory rather
|
|
than a general-purpose or sensitive filesystem tree.
|
|
|
|
When upgrading existing definitions, review selected profile IDs.
|
|
`deepseek-3-2`, `deepseek-4-flash`, and `deepseek-4-pro` were removed from the
|
|
selected catalog. Choose an available catalog profile, one of Promptkit's
|
|
maintained names, or a complete application-owned profile. New DeepSeek names
|
|
are not aliases for the removed ones; see the
|
|
[v0.10 migration guide](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.10.0/docs/releases/v0.10.0.md).
|
|
|
|
Run Scriptorium under an identity that can:
|
|
|
|
- read only the prompt, profile, schema, and allowed input-artifact paths it
|
|
needs;
|
|
- read the required credential environment variables without writing them to
|
|
files or logs; and
|
|
- write only caller-selected output locations when CLI output files are used.
|
|
|
|
Do not make the HTTP artifact directory writable by untrusted users. The HTTP
|
|
artifact containment behavior is lexical and the operating system follows
|
|
symlinks; account for that when choosing ownership and mount boundaries. See
|
|
the [HTTP API reference](api.md) for the externally observable behavior.
|
|
|
|
## Supply Credentials And Protect Runtime Data
|
|
|
|
Set secret values in the process environment and configure only their
|
|
environment-variable names. Do not put raw keys in configuration, prompt or
|
|
profile files, process arguments, HTTP payloads, captured command lines, or
|
|
debug dumps.
|
|
|
|
Treat stdout, stderr, prepared-run output, generated artifacts, and HTTP
|
|
responses as potentially sensitive. Send service logs to a controlled collector
|
|
and apply the same retention and access rules as for model input and output.
|
|
|
|
## Run A Normal Workflow
|
|
|
|
Before changing production inputs, profiles, or schemas:
|
|
|
|
1. confirm the deployed configuration selects the intended sources and model
|
|
credentials;
|
|
2. use [`render`](cli.md) with the same request inputs and variables to confirm
|
|
preparation without a model call;
|
|
3. use [`run`](cli.md) for generation; and
|
|
4. retain or discard validation-failed output according to the caller's
|
|
policy.
|
|
|
|
The [maintained render script](../examples/render-markdown-summary.sh) is a
|
|
copyable preflight example. The CLI reference owns its complete invocation and
|
|
exit semantics.
|
|
|
|
## Expose The HTTP Service
|
|
|
|
The HTTP service has no built-in authentication or authorization. Place it on a
|
|
trusted network or behind an authenticated reverse proxy, API gateway, or
|
|
equivalent access control. Restrict who can reach it and who can read the
|
|
artifact root.
|
|
|
|
Use a service manager or supervisor appropriate to the deployment to manage
|
|
process lifetime, restart policy, log capture, and environment injection. The
|
|
[HTTP API reference](api.md) owns client request shapes, status behavior, and
|
|
artifact-access outcomes.
|
|
|
|
## Plan Capacity And Limits
|
|
|
|
Capacity is primarily determined by concurrent model calls, input and output
|
|
sizes, schema complexity, provider latency, and network behavior. Size limits
|
|
protect request bodies, HTTP file artifacts, and encoded responses; configure
|
|
them through the [configuration reference](config.md) and rely on the
|
|
[HTTP API reference](api.md) for their response effects.
|
|
|
|
Configured backend concurrency and queue capacity are enforced per constructed
|
|
Promptkit engine. A `serve` process constructs one engine for its handler, so
|
|
concurrent HTTP requests share that transient admission state. Scriptorium does
|
|
not retain workflow state: capacity is neither durable nor a queue of resumable
|
|
runs. When admission is exhausted, HTTP returns `503 capacity_exceeded` without
|
|
retry timing; callers choose any retry policy that is safe for another model
|
|
call.
|
|
|
|
Promptkit's built-in client now allows two additional attempts for eligible
|
|
transient provider statuses by default. Account for added calls, latency, and
|
|
cost when planning capacity. Each repair generation has its own transient
|
|
budget, separate from the structural repair count. Configure the engine budget
|
|
with [`transient_retries`](config.md); `run` and `render` accept a request
|
|
override described in the [CLI contract](cli.md), and HTTP callers can select
|
|
one through the [API contract](api.md). Omission inherits the engine
|
|
setting and then Promptkit's default, while explicit zero disables additional
|
|
attempts. Existing generation timeouts apply
|
|
per attempt, while the caller context covers the complete operation. Provider
|
|
`Retry-After` timing can exceed Promptkit's ordinary backoff cap; consult the
|
|
[outbound integration contract](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.11.0/docs/integrations/openai-compatible-chat.md)
|
|
for details.
|
|
|
|
CLI `run` can set a [whole-run deadline](cli.md) when total execution time must
|
|
be bounded. The existing generation timeout applies afresh to each attempt and
|
|
does not cap retry waits or the complete invocation. The whole-run interval
|
|
covers the engine call, including capacity waiting and repairs as supported by
|
|
context cancellation; it excludes parsing, engine setup, and output writes.
|
|
HTTP callers can use [`run_timeout_seconds`](api.md) to shorten their request
|
|
context for the same engine interval. The API distinguishes an expired
|
|
execution deadline from caller cancellation and from a provider-attempt
|
|
timeout. A canceled capacity waiter releases its Promptkit admission state.
|
|
|
|
Before increasing a limit:
|
|
|
|
1. measure representative input, generated-output, and optional raw-output
|
|
sizes;
|
|
2. confirm memory, network, and upstream-provider capacity;
|
|
3. retain an upstream request-size and authentication boundary; and
|
|
4. test the intended workload in a non-production environment.
|
|
|
|
For large local inputs, prefer a controlled file-artifact directory over
|
|
placing arbitrary paths on the service host. Avoid disabling a limit unless an
|
|
equivalent trusted control exists elsewhere.
|
|
|
|
## Diagnose And Recover
|
|
|
|
### Preparation Or Configuration Failure
|
|
|
|
Capture the CLI diagnostic or HTTP error response, then verify the selected
|
|
configuration, prompt ID, profile selection, source readability, and input
|
|
mapping. Use `render` with the same request when it is unclear whether failure
|
|
occurs before model execution. Consult the [CLI reference](cli.md), the
|
|
[configuration reference](config.md), and the [HTTP API reference](api.md) for
|
|
the exact interface contract.
|
|
|
|
### Credential Or Provider Failure
|
|
|
|
Confirm that the process environment contains the configured credential name
|
|
without printing the secret. Check endpoint reachability and provider health
|
|
from the process network. If preparation succeeds but generation fails, inspect
|
|
the selected model settings in prepared output and the service's controlled
|
|
logs. Correct the deployment or provider issue, then submit a new request.
|
|
|
|
### Artifact Or Permission Failure
|
|
|
|
Verify that the process can read the intended local input. For HTTP file
|
|
artifacts, verify the deployment's artifact root, ownership, path layout, and
|
|
file size. Do not widen filesystem permissions or the allowed root merely to
|
|
make an arbitrary path work; move or copy the required artifact into the
|
|
controlled location instead.
|
|
|
|
### Validation Failure
|
|
|
|
A generated-content validation failure is distinct from a runtime failure.
|
|
CLI `run` reports the validation result and error count in its success summary;
|
|
it does not print the individual validation messages. For HTTP, inspect the
|
|
validation object in the response according to the [HTTP API reference](api.md).
|
|
|
|
Use rendered input and generated output to determine whether prompt instructions,
|
|
the selected model, or the schema needs correction. If schema loading or
|
|
compilation itself fails, correct the source deployment or schema document
|
|
before rerunning.
|
|
|
|
### HTTP Limit Or Request Failure
|
|
|
|
Compare the request, artifact, or expected response size with the deployed
|
|
configuration, and validate the request against the [HTTP API reference](api.md).
|
|
Reduce the payload, use an appropriate controlled artifact source, omit
|
|
unneeded raw output, or adjust the deployment limit after capacity review.
|
|
|
|
## Cleanup And Reruns
|
|
|
|
Because no run state is retained, cleanup concerns caller-owned output files,
|
|
logs, and artifacts only. Remove or rotate them using the deployment's normal
|
|
retention policy. After a correction, rerun the request from the beginning;
|
|
there is no safe resume point.
|