eric/scriptorium
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 6 Wiki Activity

Add operations and troubleshooting documentation

Browse Source
This commit is contained in:
Eric Rakestraw
2026-05-26 03:25:34 +00:00
parent b69ba96811
commit 4950a6bb14
3 changed files with 460 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

124
docs/operations.md Normal file
View File

@@ -0,0 +1,124 @@
# Operations Guide
## Scope
This document covers day-to-day operation of the CLI and HTTP service for currently implemented behavior.
For command syntax, see [CLI reference](cli.md). For file formats and defaults, see [Configuration reference](config.md).
## Operational Model
Scriptorium executes one request at a time per CLI invocation or HTTP request.
Important boundaries:
- No durable run state is stored.
- No built-in resume, checkpoint, archive, or backup workflow exists.
- Recovery is rerun-based: fix inputs/config, then rerun.
## Filesystem Layout And Config
Scriptorium depends on:
- prompt definition files (`prompt_dir`)
- execution profile files (`profile_dir`)
- optional JSON schemas (`schema_dir`)
Config discovery order when `--config` is omitted:
1. `/usr/local/etc/scriptorium/config.yml`
2. `/etc/scriptorium/config.yml`
If neither exists, built-in defaults are used. If `--config <path>` is provided, that file must exist and parse successfully.
Built-in defaults relevant to operations:
- `schema_dir: .`
- `server.addr: :8080`
- `defaults.render_format: text`
## Normal CLI Workflow
Use `render` first when you need to verify prompt resolution and runtime settings without calling a model.
Use `run` for generation.
Typical sequence:
1. Confirm prompt/profile directories resolve through config or flags.
2. Confirm required input files exist and map to prompt input names.
3. Confirm required API-key environment variables are set.
4. Run `render` for preflight when changing prompt/profile/input wiring.
5. Run `run` for actual generation.
## Secrets Handling
Raw API keys are not accepted in config files, profile files as `api_key`, CLI flags, or HTTP request bodies.
Operational pattern:
- Set environment variables that hold secret values.
- Set profile `api_key_env` (or runtime override `api_key_env`) to the environment variable name.
- Keep process environments scoped to only required variables.
## HTTP Service Operation
Start service with:
```bash
go run ./cmd/scriptorium serve --config ./examples/config.yml
```
Current inbound API behavior:
- Route: `POST /v1/runs`
- JSON request parsing rejects unknown fields.
- Validation content failures still return `200 OK` with `validation.status: "failed"`.
Security caveat:
- `serve` has no built-in authentication or authorization.
- Deploy only behind trusted controls (private network boundary, authenticated reverse proxy, API gateway, or equivalent).
## Output, Logs, And Exit Codes
`run` command:
- Generated artifact body goes to stdout by default.
- `--out` writes generated artifact to a file.
- Summary metadata line is written to stderr on success.
- Exit code `2` means generation completed but validation failed.
`render` command:
- Prepared-run output goes to stdout by default.
- `--out` writes prepared-run output to a file.
- Exit code is `0` on success and `1` on failure.
`serve` command:
- Startup and server errors are written to stderr.
## Validation Behavior In Operations
Validation modes (`none`, `basic`, `json`, `json_schema`) are defined by prompt output contract.
Operational interpretation:
- Validation runtime errors are hard failures (`run` exit `1`; HTTP error response).
- Validation content failures are soft failures (`run` exit `2`; HTTP `200` with failed status).
A failed validation run can still produce output. Decide whether to keep or discard that output in your surrounding workflow.
## Safe Recovery Steps
For failed runs or requests:
1. Capture stderr output or HTTP error code/message.
2. Confirm config path and directory settings.
3. Verify prompt/profile IDs and input mappings.
4. Verify API-key environment-variable presence when required.
5. Reproduce with `render --format json` when prompt/profile/input resolution is uncertain.
6. Rerun after correction.
Because Scriptorium does not persist run state, rerun is the canonical recovery path.

334
docs/troubleshooting.md Normal file
View File

@@ -0,0 +1,334 @@
# Troubleshooting
This guide lists recurring implemented failure modes and safe fixes.
For command syntax, see [CLI reference](cli.md). For configuration and file formats, see [Configuration reference](config.md). For operational behavior, see [Operations guide](operations.md).
## Missing Or Invalid Config File
Symptom:
- CLI errors such as `application config error: config file not found` or `invalid config YAML`.
Likely cause:
- `--config` points to a missing file.
- Config YAML has syntax errors or unknown fields.
Diagnostic step:
```bash
go run ./cmd/scriptorium render --config /path/to/config.yml --prompt generic.markdown_summary --input transcript=./examples/fixtures/transcript.md --input glossary=./examples/fixtures/glossary.yml
```
Safe fix:
- Correct file path.
- Remove unknown fields.
- Fix YAML syntax.
- Keep secrets out of config.
Relevant links:
- [Configuration reference](config.md)
- [CLI reference](cli.md)
## Missing Prompt/Profile Directory Settings
Symptom:
- CLI parse errors saying prompt directory or profile directory is required.
Likely cause:
- Neither CLI flags nor config provide effective `prompt_dir` / `profile_dir`.
Diagnostic step:
- Run the failing command with explicit `--prompt-dir` and `--profile-dir` once to verify.
Safe fix:
- Set `prompt_dir` and `profile_dir` in config, or always pass both flags.
Relevant links:
- [Configuration reference](config.md)
- [CLI reference](cli.md)
## Unknown Or Unsupported Flags
Symptom:
- CLI parse error for an unknown flag.
Likely cause:
- Typo or command mismatch (for example, `serve` with runtime model override flags).
Diagnostic step:
- Compare command against the command-specific flag list.
Safe fix:
- Remove unsupported flags.
- Use `run`/`render` for runtime model overrides.
Relevant links:
- [CLI reference](cli.md)
## Prompt Definition Load Failures
Symptom:
- CLI run/render error from prompt loading.
- HTTP `404 prompt_not_found` or `400 prompt_load_failed`.
Likely cause:
- Prompt ID not found.
- Invalid prompt YAML.
- Invalid prompt contract (for example bad validation mode, message content/content_file rule violation, missing schema path for `json_schema`).
Diagnostic step:
```bash
go run ./cmd/scriptorium render --config ./examples/config.yml --prompt <prompt-id> --input transcript=./examples/fixtures/transcript.md --input glossary=./examples/fixtures/glossary.yml --format json
```
Safe fix:
- Correct prompt ID.
- Fix prompt YAML and contract fields.
- Ensure referenced `content_file` paths exist.
Relevant links:
- [Configuration reference](config.md)
- [CLI reference](cli.md)
## Profile Definition Load Failures
Symptom:
- CLI run/render error from profile loading.
- HTTP `404 profile_not_found` or `400 profile_load_failed`.
Likely cause:
- Profile ID missing/not found.
- Invalid profile YAML.
- Invalid profile values.
- Raw `api_key` field present (rejected).
Diagnostic step:
```bash
go run ./cmd/scriptorium render --config ./examples/config.yml --prompt generic.markdown_summary --profile <profile-id> --input transcript=./examples/fixtures/transcript.md --input glossary=./examples/fixtures/glossary.yml
```
Safe fix:
- Correct profile ID.
- Fix profile YAML and value ranges.
- Replace `api_key` with `api_key_env`.
Relevant links:
- [Configuration reference](config.md)
- [CLI reference](cli.md)
## Input Artifact Read Failures
Symptom:
- CLI run/render error reading input artifacts.
- HTTP `400 artifact_read_failed`.
Likely cause:
- File path in input mapping does not exist or is unreadable.
- Unsupported artifact reference type in HTTP request.
Diagnostic step:
- Verify every mapped file path exists and is readable by the process.
- For HTTP, verify each input uses supported `type` values.
Safe fix:
- Correct file paths and permissions.
- Use supported input types (`file`, `inline`).
Relevant links:
- [CLI reference](cli.md)
- [Configuration reference](config.md)
## Prompt Template Render Failures
Symptom:
- CLI run/render error from prompt rendering.
- HTTP `400 prompt_render_failed`.
Likely cause:
- Template references missing input names.
- Template syntax or data reference issues.
Diagnostic step:
- Run `render --format json` with the same prompt, inputs, vars, and profile selection.
Safe fix:
- Align template `{{input "name"}}` references with actual input mappings.
- Fix template syntax and variable names.
Relevant links:
- [CLI reference](cli.md)
- [Configuration reference](config.md)
## Missing API-Key Environment Variable
Symptom:
- CLI run/render invalid request error about missing API-key environment variable.
- HTTP `400 api_key_env_missing`.
Likely cause:
- Selected profile or override sets `api_key_env`, but that environment variable is unset/empty.
Diagnostic step:
```bash
printenv SCRIPTORIUM_API_KEY
```
Safe fix:
- Set the required environment variable before invoking CLI/service.
- Or use a profile that does not require API key auth for the target endpoint.
Relevant links:
- [Configuration reference](config.md)
- [Operations guide](operations.md)
## LLM Request Failures
Symptom:
- CLI `run` fails with LLM generation errors.
- HTTP returns `502 llm_failed`.
Likely cause:
- Endpoint unreachable.
- Non-2xx response from provider.
- Timeout.
- Malformed provider response.
Diagnostic step:
- Confirm endpoint URL and model in selected profile/overrides.
- Retry with `render` first to confirm pre-LLM preparation works.
- Check provider/network logs for non-2xx responses and timeouts.
Safe fix:
- Correct endpoint/model settings.
- Adjust timeout if needed.
- Resolve provider-side or network issues.
Relevant links:
- [CLI reference](cli.md)
- [Configuration reference](config.md)
- [Operations guide](operations.md)
## Validation Status Failed (`run` Exit 2 Or HTTP 200 With Failed Status)
Symptom:
- CLI exits with code `2`.
- HTTP returns `200`, but `validation.status` is `failed`.
Likely cause:
- Generated output failed `basic`, `json`, or `json_schema` content validation.
Diagnostic step:
- Inspect validation mode and validation errors in CLI summary/HTTP response.
Safe fix:
- Refine prompt constraints.
- Tighten schema or adjust model/profile settings.
- Rerun after correction.
Relevant links:
- [Configuration reference](config.md)
- [Operations guide](operations.md)
## Validation Runtime Failure
Symptom:
- CLI `run` fails with validation runtime error.
- HTTP returns `500 validation_runtime_failed`.
Likely cause:
- `json_schema` schema file missing/inaccessible.
- Invalid schema JSON document.
Diagnostic step:
- Verify `schema_dir` and `output.schema_path` resolution.
- Check schema file readability and valid JSON syntax.
Safe fix:
- Correct schema path.
- Fix schema JSON content.
- Rerun.
Relevant links:
- [Configuration reference](config.md)
- [Operations guide](operations.md)
## HTTP Request Parsing/Contract Errors
Symptom:
- HTTP `400 invalid_json` or `400 invalid_request`.
Likely cause:
- Malformed JSON body.
- Unknown JSON fields.
- Missing required `prompt_id` or `inputs`.
Diagnostic step:
- Revalidate request JSON.
- Confirm required request fields are present.
Safe fix:
- Send valid JSON with only supported fields.
- Ensure `prompt_id` and at least one input mapping are included.
Relevant links:
- [Operations guide](operations.md)
- [CLI reference](cli.md)