Consolidate operational recovery guidance
This commit is contained in:
@@ -1,164 +1,157 @@
|
||||
# Operations Guide
|
||||
|
||||
## Scope
|
||||
## Scope And References
|
||||
|
||||
This guide covers operating the implemented CLI commands and HTTP service. It
|
||||
does not replace the [CLI reference](cli.md), [Configuration reference](config.md),
|
||||
or [HTTP API reference](api.md).
|
||||
This runbook covers deployment, normal operation, capacity planning, and safe
|
||||
recovery for Scriptorium. It does not redefine invocation syntax, configuration
|
||||
fields, or HTTP wire behavior.
|
||||
|
||||
## Operational Model
|
||||
- [CLI reference](cli.md): commands, output destinations, and exit codes.
|
||||
- [Configuration reference](config.md): configuration, prompt/profile/schema
|
||||
formats, defaults, and credentials.
|
||||
- [HTTP API reference](api.md): route, request/response schema, status codes,
|
||||
limits, and HTTP artifact access.
|
||||
- [Consumer integration overview](consumers/api.md): caller responsibilities.
|
||||
|
||||
Scriptorium executes one prompt request per CLI invocation or HTTP request.
|
||||
## Operational Model And State
|
||||
|
||||
Important boundaries:
|
||||
Scriptorium handles one prompt request for each CLI invocation or HTTP request.
|
||||
It has no durable run store, archive, checkpoint, cache, or resume mechanism.
|
||||
A failed or interrupted request is recovered by correcting its inputs,
|
||||
configuration, or environment and submitting a new request.
|
||||
|
||||
- No durable run state is stored.
|
||||
- No manifest, archive, checkpoint, or built-in backup workflow is written.
|
||||
- No built-in resume behavior exists.
|
||||
- Recovery is rerun-based: correct inputs, config, or environment, then run again.
|
||||
Generated artifacts, rendered prompts, model output, and run metadata are
|
||||
caller-owned data. Retention, encryption, backup, and deletion are deployment
|
||||
responsibilities.
|
||||
|
||||
## Filesystem Layout
|
||||
## Deploy The Filesystem And Process
|
||||
|
||||
Operational deployments usually provide:
|
||||
Provide the process with readable prompt, profile, and schema sources. Keep
|
||||
prompt templates adjacent to the prompt definitions that reference them. For an
|
||||
HTTP deployment that accepts file artifacts, use a dedicated, narrow artifact
|
||||
directory rather than a general-purpose or sensitive filesystem tree.
|
||||
|
||||
- `prompt_dir`: prompt definition YAML files and adjacent `content_file` templates.
|
||||
- `profile_dir`: optional custom profile YAML files.
|
||||
- `schema_dir`: optional JSON Schema files.
|
||||
- `server.artifact_root`: optional HTTP file-input root for `serve`.
|
||||
Run Scriptorium under an identity that can:
|
||||
|
||||
Keep these directories readable by the Scriptorium process. Keep
|
||||
`server.artifact_root` narrow and not writable by untrusted users.
|
||||
- read only the prompt, profile, schema, and allowed input-artifact paths it
|
||||
needs;
|
||||
- read the required credential environment variables without writing them to
|
||||
files or logs; and
|
||||
- write only caller-selected output locations when CLI output files are used.
|
||||
|
||||
## Normal CLI Workflow
|
||||
Do not make the HTTP artifact directory writable by untrusted users. The HTTP
|
||||
artifact containment behavior is lexical and the operating system follows
|
||||
symlinks; account for that when choosing ownership and mount boundaries. See
|
||||
the [HTTP API reference](api.md) for the externally observable behavior.
|
||||
|
||||
Use `render` before `run` when changing prompt/profile/input wiring:
|
||||
## Supply Credentials And Protect Runtime Data
|
||||
|
||||
```bash
|
||||
go run ./cmd/scriptorium render \
|
||||
--config ./examples/config.yml \
|
||||
--prompt generic.markdown_summary \
|
||||
--input transcript=./examples/fixtures/transcript.md \
|
||||
--input glossary=./examples/fixtures/glossary.yml \
|
||||
--format json
|
||||
```
|
||||
Set secret values in the process environment and configure only their
|
||||
environment-variable names. Do not put raw keys in configuration, prompt or
|
||||
profile files, process arguments, HTTP payloads, captured command lines, or
|
||||
debug dumps.
|
||||
|
||||
Use `run` for generation after preflight:
|
||||
Treat stdout, stderr, prepared-run output, generated artifacts, and HTTP
|
||||
responses as potentially sensitive. Send service logs to a controlled collector
|
||||
and apply the same retention and access rules as for model input and output.
|
||||
|
||||
```bash
|
||||
go run ./cmd/scriptorium run \
|
||||
--config ./examples/config.yml \
|
||||
--prompt generic.markdown_summary \
|
||||
--input transcript=./examples/fixtures/transcript.md \
|
||||
--input glossary=./examples/fixtures/glossary.yml \
|
||||
--out ./summary.md
|
||||
```
|
||||
## Run A Normal Workflow
|
||||
|
||||
Before production runs, confirm:
|
||||
Before changing production inputs, profiles, or schemas:
|
||||
|
||||
- the effective config path is the intended one;
|
||||
- prompt/profile/schema directories are readable;
|
||||
- input file paths exist and match prompt input names;
|
||||
- required API-key environment variables are set;
|
||||
- the selected model endpoint is reachable from the process environment.
|
||||
1. confirm the deployed configuration selects the intended sources and model
|
||||
credentials;
|
||||
2. use [`render`](cli.md) with the same request inputs and variables to confirm
|
||||
preparation without a model call;
|
||||
3. use [`run`](cli.md) for generation; and
|
||||
4. retain or discard validation-failed output according to the caller's
|
||||
policy.
|
||||
|
||||
## HTTP Service Operation
|
||||
The [maintained render script](../examples/render-markdown-summary.sh) is a
|
||||
copyable preflight example. The CLI reference owns its complete invocation and
|
||||
exit semantics.
|
||||
|
||||
Start the service with:
|
||||
## Expose The HTTP Service
|
||||
|
||||
```bash
|
||||
go run ./cmd/scriptorium serve --config ./examples/config.yml
|
||||
```
|
||||
The HTTP service has no built-in authentication or authorization. Place it on a
|
||||
trusted network or behind an authenticated reverse proxy, API gateway, or
|
||||
equivalent access control. Restrict who can reach it and who can read the
|
||||
artifact root.
|
||||
|
||||
The implemented HTTP route is `POST /v1/runs`; request and response fields are
|
||||
defined in the [HTTP API reference](api.md).
|
||||
Use a service manager or supervisor appropriate to the deployment to manage
|
||||
process lifetime, restart policy, log capture, and environment injection. The
|
||||
[HTTP API reference](api.md) owns client request shapes, status behavior, and
|
||||
artifact-access outcomes.
|
||||
|
||||
The maintained HTTP request-shape example is `examples/http-run.json`.
|
||||
## Plan Capacity And Limits
|
||||
|
||||
HTTP service notes:
|
||||
Capacity is primarily determined by concurrent model calls, input and output
|
||||
sizes, schema complexity, provider latency, and network behavior. Size limits
|
||||
protect request bodies, HTTP file artifacts, and encoded responses; configure
|
||||
them through the [configuration reference](config.md) and rely on the
|
||||
[HTTP API reference](api.md) for their response effects.
|
||||
|
||||
- Unknown JSON fields are rejected.
|
||||
- `inline` input references work without an artifact root.
|
||||
- `file` input references require `server.artifact_root` or `serve --artifact-root`.
|
||||
- Request bodies, HTTP file input artifacts, and encoded JSON responses are size-limited.
|
||||
- Validation content failures return `200 OK` with `validation.status: "failed"`.
|
||||
Before increasing a limit:
|
||||
|
||||
Security boundary:
|
||||
1. measure representative input, generated-output, and optional raw-output
|
||||
sizes;
|
||||
2. confirm memory, network, and upstream-provider capacity;
|
||||
3. retain an upstream request-size and authentication boundary; and
|
||||
4. test the intended workload in a non-production environment.
|
||||
|
||||
- `serve` has no built-in authentication or authorization.
|
||||
- Put it behind trusted controls such as a private network, authenticated reverse proxy, or API gateway.
|
||||
- Do not expose an artifact root containing unrelated sensitive files.
|
||||
- Symlinks inside the artifact root are followed by the operating system.
|
||||
For large local inputs, prefer a controlled file-artifact directory over
|
||||
placing arbitrary paths on the service host. Avoid disabling a limit unless an
|
||||
equivalent trusted control exists elsewhere.
|
||||
|
||||
## Secrets Handling
|
||||
## Diagnose And Recover
|
||||
|
||||
Raw API keys are not accepted in app config, profiles, CLI flags, or HTTP
|
||||
request bodies.
|
||||
### Preparation Or Configuration Failure
|
||||
|
||||
Use this pattern:
|
||||
Capture the CLI diagnostic or HTTP error response, then verify the selected
|
||||
configuration, prompt ID, profile selection, source readability, and input
|
||||
mapping. Use `render` with the same request when it is unclear whether failure
|
||||
occurs before model execution. Consult the [CLI reference](cli.md), the
|
||||
[configuration reference](config.md), and the [HTTP API reference](api.md) for
|
||||
the exact interface contract.
|
||||
|
||||
1. Set an environment variable containing the secret value.
|
||||
2. Store only the variable name in profile `api_key_env` or request override `api_key_env`.
|
||||
3. Scope the process environment to the minimum required variables.
|
||||
### Credential Or Provider Failure
|
||||
|
||||
## Output, Logs, And Exit Codes
|
||||
Confirm that the process environment contains the configured credential name
|
||||
without printing the secret. Check endpoint reachability and provider health
|
||||
from the process network. If preparation succeeds but generation fails, inspect
|
||||
the selected model settings in prepared output and the service's controlled
|
||||
logs. Correct the deployment or provider issue, then submit a new request.
|
||||
|
||||
`run`:
|
||||
### Artifact Or Permission Failure
|
||||
|
||||
- stdout: generated artifact body unless `--out` is used.
|
||||
- stderr: summary on success, errors on failure.
|
||||
- exit `2`: generation completed and output was written, but validation failed.
|
||||
Verify that the process can read the intended local input. For HTTP file
|
||||
artifacts, verify the deployment's artifact root, ownership, path layout, and
|
||||
file size. Do not widen filesystem permissions or the allowed root merely to
|
||||
make an arbitrary path work; move or copy the required artifact into the
|
||||
controlled location instead.
|
||||
|
||||
`render`:
|
||||
### Validation Failure
|
||||
|
||||
- stdout: prepared-run output unless `--out` is used.
|
||||
- stderr: errors.
|
||||
- exit `0` on success, `1` on failure.
|
||||
A generated-content validation failure is distinct from a runtime failure.
|
||||
CLI `run` reports the validation result and error count in its success summary;
|
||||
it does not print the individual validation messages. For HTTP, inspect the
|
||||
validation object in the response according to the [HTTP API reference](api.md).
|
||||
|
||||
`serve`:
|
||||
Use rendered input and generated output to determine whether prompt instructions,
|
||||
the selected model, or the schema needs correction. If schema loading or
|
||||
compilation itself fails, correct the source deployment or schema document
|
||||
before rerunning.
|
||||
|
||||
- stderr: startup and server errors.
|
||||
- HTTP response body: JSON success or error envelope.
|
||||
### HTTP Limit Or Request Failure
|
||||
|
||||
## Validation Behavior
|
||||
Compare the request, artifact, or expected response size with the deployed
|
||||
configuration, and validate the request against the [HTTP API reference](api.md).
|
||||
Reduce the payload, use an appropriate controlled artifact source, omit
|
||||
unneeded raw output, or adjust the deployment limit after capacity review.
|
||||
|
||||
Prompt `output.validation_mode` controls validation:
|
||||
## Cleanup And Reruns
|
||||
|
||||
- `none`: skipped.
|
||||
- `basic`: output body must not be empty.
|
||||
- `json`: output body must parse as JSON.
|
||||
- `json_schema`: output body must parse as JSON and satisfy the configured schema.
|
||||
|
||||
Runtime/schema failures are hard failures (`run` exit `1`, HTTP error).
|
||||
Generated-content validation failures are soft failures (`run` exit `2`, HTTP
|
||||
`200 OK` with failed validation status).
|
||||
|
||||
## Size Limits
|
||||
|
||||
Defaults are documented in [Configuration reference](config.md). Operationally:
|
||||
|
||||
- Keep default HTTP limits unless larger payloads are measured and expected.
|
||||
- Prefer `inline` HTTP inputs for small payloads.
|
||||
- Prefer `file` HTTP inputs for larger local artifacts under a controlled artifact root.
|
||||
- Increase `server.max_response_bytes` when generated artifacts or requested raw output are expected to be large.
|
||||
- Use `0` only when another trusted layer enforces size limits.
|
||||
|
||||
## Maintained Examples
|
||||
|
||||
- `examples/config.yml`
|
||||
- `examples/config.full.yml`
|
||||
- `examples/render-markdown-summary.sh`
|
||||
- `examples/http-run.json`
|
||||
|
||||
## Safe Recovery
|
||||
|
||||
For failed CLI commands or HTTP requests:
|
||||
|
||||
1. Capture stderr or the HTTP error `code` and `message`.
|
||||
2. Confirm config path and effective directory settings.
|
||||
3. Verify prompt ID, profile ID, schema path, and input mappings.
|
||||
4. Verify required API-key environment variables.
|
||||
5. Reproduce with `render --format json` when pre-LLM resolution is uncertain.
|
||||
6. Rerun after correction.
|
||||
|
||||
Because Scriptorium does not persist run state, rerun is the supported recovery
|
||||
path.
|
||||
Because no run state is retained, cleanup concerns caller-owned output files,
|
||||
logs, and artifacts only. Remove or rotate them using the deployment's normal
|
||||
retention policy. After a correction, rerun the request from the beginning;
|
||||
there is no safe resume point.
|
||||
|
||||
Reference in New Issue
Block a user