From 4950a6bb14c1a8151a99ac5d37100adc41f5b275 Mon Sep 17 00:00:00 2001
From: Eric Rakestraw <eric@rakestraw.us>
Date: Tue, 26 May 2026 03:25:34 +0000
Subject: [PATCH] Add operations and troubleshooting documentation

---
 README.md               |   2 +
 docs/operations.md      | 124 +++++++++++++++
 docs/troubleshooting.md | 334 ++++++++++++++++++++++++++++++++++++++++
 3 files changed, 460 insertions(+)
 create mode 100644 docs/operations.md
 create mode 100644 docs/troubleshooting.md

diff --git a/README.md b/README.md
index 1113e1f..c6064c0 100644
--- a/README.md
+++ b/README.md
@@ -23,6 +23,8 @@ This command renders the prepared prompt and effective runtime settings without
 
 - [CLI reference](docs/cli.md)
 - [Configuration reference](docs/config.md)
+- [Operations guide](docs/operations.md)
+- [Troubleshooting](docs/troubleshooting.md)
 - [Narratio subprocess integration](docs/integrations/narratio.md)
 - [Architecture policy](docs/policy/architecture.md)
 - [Documentation roadmap](docs/roadmap/documentation.md)
diff --git a/docs/operations.md b/docs/operations.md
new file mode 100644
index 0000000..6d9290c
--- /dev/null
+++ b/docs/operations.md
@@ -0,0 +1,124 @@
+# Operations Guide
+
+## Scope
+
+This document covers day-to-day operation of the CLI and HTTP service for currently implemented behavior.
+
+For command syntax, see [CLI reference](cli.md). For file formats and defaults, see [Configuration reference](config.md).
+
+## Operational Model
+
+Scriptorium executes one request at a time per CLI invocation or HTTP request.
+
+Important boundaries:
+
+- No durable run state is stored.
+- No built-in resume, checkpoint, archive, or backup workflow exists.
+- Recovery is rerun-based: fix inputs/config, then rerun.
+
+## Filesystem Layout And Config
+
+Scriptorium depends on:
+
+- prompt definition files (`prompt_dir`)
+- execution profile files (`profile_dir`)
+- optional JSON schemas (`schema_dir`)
+
+Config discovery order when `--config` is omitted:
+
+1. `/usr/local/etc/scriptorium/config.yml`
+2. `/etc/scriptorium/config.yml`
+
+If neither exists, built-in defaults are used. If `--config <path>` is provided, that file must exist and parse successfully.
+
+Built-in defaults relevant to operations:
+
+- `schema_dir: .`
+- `server.addr: :8080`
+- `defaults.render_format: text`
+
+## Normal CLI Workflow
+
+Use `render` first when you need to verify prompt resolution and runtime settings without calling a model.
+
+Use `run` for generation.
+
+Typical sequence:
+
+1. Confirm prompt/profile directories resolve through config or flags.
+2. Confirm required input files exist and map to prompt input names.
+3. Confirm required API-key environment variables are set.
+4. Run `render` for preflight when changing prompt/profile/input wiring.
+5. Run `run` for actual generation.
+
+## Secrets Handling
+
+Raw API keys are not accepted in config files, profile files as `api_key`, CLI flags, or HTTP request bodies.
+
+Operational pattern:
+
+- Set environment variables that hold secret values.
+- Set profile `api_key_env` (or runtime override `api_key_env`) to the environment variable name.
+- Keep process environments scoped to only required variables.
+
+## HTTP Service Operation
+
+Start service with:
+
+```bash
+go run ./cmd/scriptorium serve --config ./examples/config.yml
+```
+
+Current inbound API behavior:
+
+- Route: `POST /v1/runs`
+- JSON request parsing rejects unknown fields.
+- Validation content failures still return `200 OK` with `validation.status: "failed"`.
+
+Security caveat:
+
+- `serve` has no built-in authentication or authorization.
+- Deploy only behind trusted controls (private network boundary, authenticated reverse proxy, API gateway, or equivalent).
+
+## Output, Logs, And Exit Codes
+
+`run` command:
+
+- Generated artifact body goes to stdout by default.
+- `--out` writes generated artifact to a file.
+- Summary metadata line is written to stderr on success.
+- Exit code `2` means generation completed but validation failed.
+
+`render` command:
+
+- Prepared-run output goes to stdout by default.
+- `--out` writes prepared-run output to a file.
+- Exit code is `0` on success and `1` on failure.
+
+`serve` command:
+
+- Startup and server errors are written to stderr.
+
+## Validation Behavior In Operations
+
+Validation modes (`none`, `basic`, `json`, `json_schema`) are defined by prompt output contract.
+
+Operational interpretation:
+
+- Validation runtime errors are hard failures (`run` exit `1`; HTTP error response).
+- Validation content failures are soft failures (`run` exit `2`; HTTP `200` with failed status).
+
+A failed validation run can still produce output. Decide whether to keep or discard that output in your surrounding workflow.
+
+## Safe Recovery Steps
+
+For failed runs or requests:
+
+1. Capture stderr output or HTTP error code/message.
+2. Confirm config path and directory settings.
+3. Verify prompt/profile IDs and input mappings.
+4. Verify API-key environment-variable presence when required.
+5. Reproduce with `render --format json` when prompt/profile/input resolution is uncertain.
+6. Rerun after correction.
+
+Because Scriptorium does not persist run state, rerun is the canonical recovery path.
diff --git a/docs/troubleshooting.md b/docs/troubleshooting.md
new file mode 100644
index 0000000..4e3e898
--- /dev/null
+++ b/docs/troubleshooting.md
@@ -0,0 +1,334 @@
+# Troubleshooting
+
+This guide lists recurring implemented failure modes and safe fixes.
+
+For command syntax, see [CLI reference](cli.md). For configuration and file formats, see [Configuration reference](config.md). For operational behavior, see [Operations guide](operations.md).
+
+## Missing Or Invalid Config File
+
+Symptom:
+
+- CLI errors such as `application config error: config file not found` or `invalid config YAML`.
+
+Likely cause:
+
+- `--config` points to a missing file.
+- Config YAML has syntax errors or unknown fields.
+
+Diagnostic step:
+
+```bash
+go run ./cmd/scriptorium render --config /path/to/config.yml --prompt generic.markdown_summary --input transcript=./examples/fixtures/transcript.md --input glossary=./examples/fixtures/glossary.yml
+```
+
+Safe fix:
+
+- Correct file path.
+- Remove unknown fields.
+- Fix YAML syntax.
+- Keep secrets out of config.
+
+Relevant links:
+
+- [Configuration reference](config.md)
+- [CLI reference](cli.md)
+
+## Missing Prompt/Profile Directory Settings
+
+Symptom:
+
+- CLI parse errors saying prompt directory or profile directory is required.
+
+Likely cause:
+
+- Neither CLI flags nor config provide effective `prompt_dir` / `profile_dir`.
+
+Diagnostic step:
+
+- Run the failing command with explicit `--prompt-dir` and `--profile-dir` once to verify.
+
+Safe fix:
+
+- Set `prompt_dir` and `profile_dir` in config, or always pass both flags.
+
+Relevant links:
+
+- [Configuration reference](config.md)
+- [CLI reference](cli.md)
+
+## Unknown Or Unsupported Flags
+
+Symptom:
+
+- CLI parse error for an unknown flag.
+
+Likely cause:
+
+- Typo or command mismatch (for example, `serve` with runtime model override flags).
+
+Diagnostic step:
+
+- Compare command against the command-specific flag list.
+
+Safe fix:
+
+- Remove unsupported flags.
+- Use `run`/`render` for runtime model overrides.
+
+Relevant links:
+
+- [CLI reference](cli.md)
+
+## Prompt Definition Load Failures
+
+Symptom:
+
+- CLI run/render error from prompt loading.
+- HTTP `404 prompt_not_found` or `400 prompt_load_failed`.
+
+Likely cause:
+
+- Prompt ID not found.
+- Invalid prompt YAML.
+- Invalid prompt contract (for example bad validation mode, message content/content_file rule violation, missing schema path for `json_schema`).
+
+Diagnostic step:
+
+```bash
+go run ./cmd/scriptorium render --config ./examples/config.yml --prompt <prompt-id> --input transcript=./examples/fixtures/transcript.md --input glossary=./examples/fixtures/glossary.yml --format json
+```
+
+Safe fix:
+
+- Correct prompt ID.
+- Fix prompt YAML and contract fields.
+- Ensure referenced `content_file` paths exist.
+
+Relevant links:
+
+- [Configuration reference](config.md)
+- [CLI reference](cli.md)
+
+## Profile Definition Load Failures
+
+Symptom:
+
+- CLI run/render error from profile loading.
+- HTTP `404 profile_not_found` or `400 profile_load_failed`.
+
+Likely cause:
+
+- Profile ID missing/not found.
+- Invalid profile YAML.
+- Invalid profile values.
+- Raw `api_key` field present (rejected).
+
+Diagnostic step:
+
+```bash
+go run ./cmd/scriptorium render --config ./examples/config.yml --prompt generic.markdown_summary --profile <profile-id> --input transcript=./examples/fixtures/transcript.md --input glossary=./examples/fixtures/glossary.yml
+```
+
+Safe fix:
+
+- Correct profile ID.
+- Fix profile YAML and value ranges.
+- Replace `api_key` with `api_key_env`.
+
+Relevant links:
+
+- [Configuration reference](config.md)
+- [CLI reference](cli.md)
+
+## Input Artifact Read Failures
+
+Symptom:
+
+- CLI run/render error reading input artifacts.
+- HTTP `400 artifact_read_failed`.
+
+Likely cause:
+
+- File path in input mapping does not exist or is unreadable.
+- Unsupported artifact reference type in HTTP request.
+
+Diagnostic step:
+
+- Verify every mapped file path exists and is readable by the process.
+- For HTTP, verify each input uses supported `type` values.
+
+Safe fix:
+
+- Correct file paths and permissions.
+- Use supported input types (`file`, `inline`).
+
+Relevant links:
+
+- [CLI reference](cli.md)
+- [Configuration reference](config.md)
+
+## Prompt Template Render Failures
+
+Symptom:
+
+- CLI run/render error from prompt rendering.
+- HTTP `400 prompt_render_failed`.
+
+Likely cause:
+
+- Template references missing input names.
+- Template syntax or data reference issues.
+
+Diagnostic step:
+
+- Run `render --format json` with the same prompt, inputs, vars, and profile selection.
+
+Safe fix:
+
+- Align template `{{input "name"}}` references with actual input mappings.
+- Fix template syntax and variable names.
+
+Relevant links:
+
+- [CLI reference](cli.md)
+- [Configuration reference](config.md)
+
+## Missing API-Key Environment Variable
+
+Symptom:
+
+- CLI run/render invalid request error about missing API-key environment variable.
+- HTTP `400 api_key_env_missing`.
+
+Likely cause:
+
+- Selected profile or override sets `api_key_env`, but that environment variable is unset/empty.
+
+Diagnostic step:
+
+```bash
+printenv SCRIPTORIUM_API_KEY
+```
+
+Safe fix:
+
+- Set the required environment variable before invoking CLI/service.
+- Or use a profile that does not require API key auth for the target endpoint.
+
+Relevant links:
+
+- [Configuration reference](config.md)
+- [Operations guide](operations.md)
+
+## LLM Request Failures
+
+Symptom:
+
+- CLI `run` fails with LLM generation errors.
+- HTTP returns `502 llm_failed`.
+
+Likely cause:
+
+- Endpoint unreachable.
+- Non-2xx response from provider.
+- Timeout.
+- Malformed provider response.
+
+Diagnostic step:
+
+- Confirm endpoint URL and model in selected profile/overrides.
+- Retry with `render` first to confirm pre-LLM preparation works.
+- Check provider/network logs for non-2xx responses and timeouts.
+
+Safe fix:
+
+- Correct endpoint/model settings.
+- Adjust timeout if needed.
+- Resolve provider-side or network issues.
+
+Relevant links:
+
+- [CLI reference](cli.md)
+- [Configuration reference](config.md)
+- [Operations guide](operations.md)
+
+## Validation Status Failed (`run` Exit 2 Or HTTP 200 With Failed Status)
+
+Symptom:
+
+- CLI exits with code `2`.
+- HTTP returns `200`, but `validation.status` is `failed`.
+
+Likely cause:
+
+- Generated output failed `basic`, `json`, or `json_schema` content validation.
+
+Diagnostic step:
+
+- Inspect validation mode and validation errors in CLI summary/HTTP response.
+
+Safe fix:
+
+- Refine prompt constraints.
+- Tighten schema or adjust model/profile settings.
+- Rerun after correction.
+
+Relevant links:
+
+- [Configuration reference](config.md)
+- [Operations guide](operations.md)
+
+## Validation Runtime Failure
+
+Symptom:
+
+- CLI `run` fails with validation runtime error.
+- HTTP returns `500 validation_runtime_failed`.
+
+Likely cause:
+
+- `json_schema` schema file missing/inaccessible.
+- Invalid schema JSON document.
+
+Diagnostic step:
+
+- Verify `schema_dir` and `output.schema_path` resolution.
+- Check schema file readability and valid JSON syntax.
+
+Safe fix:
+
+- Correct schema path.
+- Fix schema JSON content.
+- Rerun.
+
+Relevant links:
+
+- [Configuration reference](config.md)
+- [Operations guide](operations.md)
+
+## HTTP Request Parsing/Contract Errors
+
+Symptom:
+
+- HTTP `400 invalid_json` or `400 invalid_request`.
+
+Likely cause:
+
+- Malformed JSON body.
+- Unknown JSON fields.
+- Missing required `prompt_id` or `inputs`.
+
+Diagnostic step:
+
+- Revalidate request JSON.
+- Confirm required request fields are present.
+
+Safe fix:
+
+- Send valid JSON with only supported fields.
+- Ensure `prompt_id` and at least one input mapping are included.
+
+Relevant links:
+
+- [Operations guide](operations.md)
+- [CLI reference](cli.md)