From 2b9658fb017a639115eedfe9bddb44289ebeb12d Mon Sep 17 00:00:00 2001 From: Eric Rakestraw Date: Tue, 5 May 2026 11:16:33 -0500 Subject: [PATCH] Update README and architecture documentation to reflect prompt/profile separation --- README.md | 366 ++++++++++++++++++++++-------------------------- architecture.md | 259 ++++++++++------------------------ 2 files changed, 237 insertions(+), 388 deletions(-) diff --git a/README.md b/README.md index f3ee700..ee93d22 100644 --- a/README.md +++ b/README.md @@ -1,134 +1,159 @@ # scriptorium -Scriptorium is a generic prompt-definition execution engine written in Go. +Scriptorium is a generic prompt execution engine. -Given named input artifacts and a prompt definition, Scriptorium: +It takes: +- a prompt definition +- a selected or default execution profile +- named input artifacts +- template variables +- optional runtime overrides -1. Loads the prompt definition. -2. Resolves input artifact references. -3. Renders prompt messages from templates. -4. Calls an OpenAI-compatible LLM endpoint. -5. Validates output if configured. -6. Optionally performs bounded structured-output repair. -7. Returns a generated artifact plus run metadata. +It returns: +- generated artifact +- validation result +- metadata -## Where Scriptorium Fits +## Prompt vs Profile -Scriptorium is not an orchestrator. +Scriptorium separates **what** to do (Prompt) from **how** to do it (Profile). -In the D&D workflow: +### Prompt Definition +Defines the task logic and output contract. +- Task description and version. +- Message templates (system, user, etc.). +- Required and optional input artifacts. +- Output format and validation rules. +- Repair settings for structured output. +- Optional `default_profile` for convenience. -- Narratio orchestrates the full pipeline. -- WhisperX transcribes audio. -- Seriatim merges transcripts. -- Audita polishes transcripts. -- Scriptorium generates final artifacts from prepared inputs. +### Execution Profile +Defines the runtime environment and model settings. +- LLM endpoint (URL). +- Model name. +- Generation parameters: `temperature`, `max_tokens`, `top_p`. +- Runtime settings: `timeout`, `reasoning_effort`. +- API key source via `api_key_env`. -D&D-specific behavior belongs in profiles, schemas, fixtures, and caller inputs, not in core Go logic. +Callers can explicitly provide a `profile_id` to override the prompt's `default_profile`. -## Core Concepts +## Precedence -- Prompt definition: YAML config for templates, inputs, output format, and validation behavior. -- Execution profile: conceptual runtime config (endpoint/model/timeouts/auth source). In this transition, execution settings are supplied as run-time overrides. -- Named inputs: logical names (for example `transcript`, `glossary`) mapped to artifact references. -- Artifact refs: currently `file` and `inline` are supported. -- Template variables: key/value vars passed at run time and referenced as `{{.var_name}}`. -- Execution target: endpoint/model plus generation/runtime parameters (`temperature`, `max_tokens`, `top_p`, `timeout_seconds`, `reasoning_effort`, `api_key_env`). -- Output format: `text`, `markdown`, or `json`. -- Validation mode: `none`, `basic`, `json`, `json_schema`. -- Repair attempts: bounded retries for structured modes (`json`, `json_schema`) when output validation fails. +When resolving runtime settings, Scriptorium follows this precedence model (highest to lowest): -## Build and Test +1. **Runtime Overrides**: Provided via CLI flags or HTTP request `model` object. +2. **Execution Profile**: Settings defined in the selected profile. +3. **Application Defaults**: Built-in fallback values. -```bash -go build -o scriptorium ./cmd/scriptorium -go test ./... -``` +### Profile Selection Logic +The engine determines which profile to use in this order: +1. Explicit `profile_id` (via `--profile` or HTTP request). +2. The `default_profile` named in the Prompt Definition. +3. Error: If neither is provided and no default exists. + +## API Key Policy + +To ensure security, Scriptorium does not support raw API keys in configuration files, CLI arguments, or HTTP requests. + +- **`api_key_env`**: Profiles and overrides specify the name of an environment variable (e.g., `SCRIPTORIUM_API_KEY`). +- **Runtime Resolution**: The value of the environment variable is read directly from the process environment at runtime. +- **Zero Leakage**: API key values are never included in metadata, logs, or response bodies. ## CLI Usage ### `scriptorium run` -Required flags: +Runs a single prompt execution. -- `--prompt-dir` -- `--profile-dir` -- `--prompt` -- `--input` (repeatable `name=path`) +**Required Flags:** +- `--prompt-dir`: Directory containing prompt YAML files. +- `--profile-dir`: Directory containing profile YAML files. +- `--prompt`: The prompt ID to execute. +- `--input`: Input mapping `name=path` (repeatable). -Common optional flags: +**Optional Flags:** +- `--profile`: Override the prompt's default profile. +- `--var`: Template variable `name=value` (repeatable). +- `--out`: Write output to a file instead of stdout. +- `--llm-base-url`: Override endpoint. +- `--model`: Override model name. +- `--api-key-env`: Override API key environment variable name. +- `--temperature`: Override temperature. +- `--max-tokens`: Override max tokens. +- `--top-p`: Override top_p. +- `--timeout`: Override request timeout (e.g., `30s`, `1m`). -- `--profile` (execution profile selector; falls back to prompt `default_profile`) -- `--var` (repeatable `name=value`) -- `--out` -- `--llm-base-url` -- `--model` -- `--api-key-env` -- `--temperature` -- `--max-tokens` -- `--schema-dir` -- `--timeout` - -Current transitional behavior: execution-profile loading is not implemented yet, so run-time execution settings must be supplied via overrides. In practice, provide at least endpoint and model (`--llm-base-url` and `--model`). - -Example: +**Examples:** +Using the prompt's `default_profile`: ```bash -export SCRIPTORIUM_API_KEY="your-key" - -go run ./cmd/scriptorium run \ +export SCRIPTORIUM_API_KEY="sk-..." +scriptorium run \ --prompt-dir ./prompts \ --profile-dir ./profiles \ --prompt generic.markdown_summary \ - --profile local-fast \ - --input transcript=./examples/fixtures/transcript.md \ - --input glossary=./examples/fixtures/glossary.yml \ - --llm-base-url http://localhost:8000/v1 \ - --model gpt-4o-mini \ - --api-key-env SCRIPTORIUM_API_KEY \ - --out ./out.md + --input transcript=./examples/fixtures/transcript.md ``` -Output behavior: +Overriding the profile: +```bash +scriptorium run \ + --prompt-dir ./prompts \ + --profile-dir ./profiles \ + --prompt generic.markdown_summary \ + --profile local-quality \ + --input transcript=./examples/fixtures/transcript.md +``` -- Artifact content goes to stdout unless `--out` is set. -- Summaries and errors are written to stderr. -- Exit code `2` means the run succeeded but validation status is `failed`. +Overriding model and runtime values: +```bash +scriptorium run \ + --prompt-dir ./prompts \ + --profile-dir ./profiles \ + --prompt generic.markdown_summary \ + --model gpt-4o \ + --temperature 0.7 \ + --input transcript=./examples/fixtures/transcript.md +``` + +Using a local OpenAI-compatible vLLM endpoint: +```bash +scriptorium run \ + --prompt-dir ./prompts \ + --profile-dir ./profiles \ + --prompt generic.markdown_summary \ + --llm-base-url http://localhost:8000/v1 \ + --model meta-llama-3-8b \ + --input transcript=./examples/fixtures/transcript.md +``` ### `scriptorium serve` -Starts HTTP API. +Starts the HTTP API. -Required flags: +**Required Flags:** +- `--prompt-dir`: Directory containing prompt YAML files. +- `--profile-dir`: Directory containing profile YAML files. -- `--prompt-dir` -- `--profile-dir` - -Common optional flags: - -- `--addr` (default `:8080`) -- `--schema-dir` (default `.`) -- `--model` -- `--timeout` (default `10m`) +**Optional Flags:** +- `--addr`: Listen address (default `:8080`). +- `--schema-dir`: Base directory for validation schemas. +- `--model`: Default model override. +- `--timeout`: Default request timeout. ## HTTP API -Endpoint: +### `POST /v1/runs` -- `POST /v1/runs` - -No built-in authentication is provided by the server itself. Deploy behind a trusted boundary or gateway. - -Request example: +Executes a prompt. No built-in authentication is provided; deploy behind a trusted gateway. +**Request Body:** ```json { "prompt_id": "generic.structured_events", - "prompt_version": "1.0.0", - "profile_id": "local-default", + "profile_id": "local-quality", "inputs": { - "transcript": {"type": "file", "uri": "./examples/fixtures/transcript.md"}, - "glossary": {"type": "file", "uri": "./examples/fixtures/glossary.yml"} + "transcript": {"type": "file", "uri": "./examples/fixtures/transcript.md"} }, "vars": { "session_date": "2026-05-04" @@ -136,149 +161,86 @@ Request example: "model": { "endpoint": "http://localhost:8000/v1", "model": "gpt-4o-mini", - "temperature": 0.0, - "max_tokens": 600, - "top_p": 1.0, - "timeout_seconds": 120, - "api_key_env": "SCRIPTORIUM_API_KEY" + "temperature": 0.0 } } ``` -Response shape: +**Response:** +Returns a `200 OK` with the generated artifact, validation results, and metadata including the `prompt_id` and the `selected_profile_id`. -```json -{ - "artifact": { - "name": "output", - "content_type": "application/json", - "body": "{...}", - "uri": "", - "size": 123, - "hash": "..." - }, - "validation": { - "status": "passed", - "mode": "json_schema", - "errors": [], - "schema_path": "structured_events.schema.json", - "repair_attempts": 0, - "is_valid": true - }, - "metadata": { - "run_id": "xxxxxxxx-xxxx-4xxx-8xxx-xxxxxxxxxxxx", - "prompt_id": "generic.structured_events", - "prompt_version": "1.0.0", - "prompt_hash": "...", - "rendered_prompt_hash": "...", - "selected_profile_id": "local-default", - "model_name": "gpt-4o-mini", - "endpoint": "http://localhost:8000/v1", - "model_params": { - "endpoint": "http://localhost:8000/v1", - "model": "gpt-4o-mini", - "temperature": 0, - "max_tokens": 600, - "top_p": 1, - "timeout_seconds": 120, - "api_key_env": "SCRIPTORIUM_API_KEY" - }, - "input_hashes": {"transcript": "...", "glossary": "..."}, - "usage": {"prompt_tokens": 10, "completion_tokens": 20, "total_tokens": 30}, - "start_time": "...", - "end_time": "...", - "duration_ms": 1523, - "validation_mode": "json_schema", - "validation_status": "passed", - "repair_attempts_used": 0 - }, - "raw_model_output": "{...}" -} -``` - -Validation content failures return `200` with `validation.status = "failed"` and preserve `raw_model_output`. - -Error response shape: - -```json -{ - "error": { - "code": "artifact_read_failed", - "message": "failed to read input artifact" - } -} -``` +**Validation Failures:** +If the model output fails validation (e.g., invalid JSON), the API returns `200 OK` with `validation.status = "failed"`. The original `raw_model_output` is preserved in the response to allow debugging. ## Prompt Definition Authoring -### Minimal Markdown prompt definition - -```yaml -id: generic.markdown_summary -version: "1.0.0" -default_profile: local-default -inputs: - - name: transcript - required: true -templates: - - role: system - content: "You are a concise assistant." - - role: user - content: | - Summarize: - {{input "transcript"}} -output_format: markdown -validation: - validation_mode: basic -``` - -### Structured JSON prompt definition with schema validation +Prompts are defined in YAML. +### Canonical Shape ```yaml id: generic.structured_events version: "1.0.0" -default_profile: local-default +description: "Extracts structured events from a transcript" +default_profile: local-quality + inputs: - name: transcript required: true + description: "The raw session transcript" + - name: glossary + required: false + templates: - role: system - content: "Return only JSON." + content: "You are a helpful assistant." - role: user - content: | - Extract events from: - {{input "transcript"}} + content_file: messages/extract_events.tmpl + output_format: json validation: - format: json validation_mode: json_schema schema_path: structured_events.schema.json - repair_attempts: 1 + repair_attempts: 2 ``` -`repair_attempts` is strictly bounded and only applies to structured validation modes. +**Key Features:** +- **Inline vs File**: Use `content` for short prompts or `content_file` for larger templates. +- **Inputs**: Mark inputs as `required` to ensure the runner fails early if they are missing. +- **Validation**: Support `none`, `basic`, `json`, and `json_schema`. +- **Repair**: `repair_attempts` enables bounded retries to fix structured output. -## Validation Modes +## Execution Profile Authoring -- `none`: skipped validation result. -- `basic`: fails for empty/whitespace output. -- `json`: output must parse as JSON. -- `json_schema`: output must parse as JSON and satisfy configured schema. +Profiles are defined in YAML. -Validation content failures are returned in the structured result; raw model output is preserved. +### Canonical Shape +```yaml +id: local-quality +endpoint: http://localhost:8000/v1 +model: gpt-4o +temperature: 0.0 +max_tokens: 4096 +top_p: 1.0 +timeout_seconds: 300 +reasoning_effort: high +api_key_env: SCRIPTORIUM_API_KEY +``` + +**Constraints:** +- **No Raw Keys**: Do not include actual API keys. Only specify the environment variable name in `api_key_env`. +- **Local Profiles**: For local endpoints that don't require auth, `api_key_env` can be omitted. ## Examples -- Prompt definitions: `prompts/` -- Execution profiles: `profiles/` -- Schemas: `schemas/` -- Fixtures: `examples/fixtures/` -- Local experimentation: `local-test/` +- **Prompt Definitions**: `prompts/` +- **Execution Profiles**: `profiles/` +- **Schemas**: `schemas/` +- **Fixtures**: `examples/fixtures/` +- **Local Experimentation**: `local-test/` -## Development Notes +## Build and Test -- Core follows ports-and-adapters and remains domain-generic. -- Domain/usecase packages do not depend on HTTP/CLI wire DTOs. -- To add a new LLM adapter: implement `internal/llm.Client`. -- To add a new artifact reader: extend `internal/artifact.Reader` routing. -- To add a new validation mode: extend `internal/validate` and preserve run semantics. +```bash +go build -o scriptorium ./cmd/scriptorium +go test ./... +``` diff --git a/architecture.md b/architecture.md index d2249be..655cfbd 100644 --- a/architecture.md +++ b/architecture.md @@ -9,7 +9,6 @@ It accepts named input artifacts, renders prompt templates, calls an LLM, valida Scriptorium is not an orchestrator. It must not own transcription, transcript merge/polish steps, notifications, or cross-step workflow control. For the motivating D&D workflow: - - Narratio orchestrates. - WhisperX transcribes. - Seriatim merges transcripts. @@ -20,207 +19,95 @@ Core Go code remains generic. ## 2. Current Architecture -Current implementation structure: +Scriptorium uses a ports-and-adapters architecture to decouple the core execution logic from external dependencies. -- `cmd/scriptorium`: binary entrypoint. -- `internal/domain`: core domain contracts. -- `internal/usecase`: `Runner` run flow, validation integration, bounded repair coordination. -- `internal/profile`: transitional filesystem prompt-definition repository (package rename deferred). -- `internal/artifact`: input artifact resolution (`inline`, `file`). -- `internal/prompt`: template rendering. -- `internal/llm`: provider-neutral client interface + OpenAI-compatible HTTP adapter. -- `internal/validate`: validation implementation (`none/basic/json/json_schema`). -- `internal/adapter/cli`: CLI adapter. -- `internal/adapter/http`: HTTP adapter (`POST /v1/runs`). +### Package Responsibilities +- `cmd/scriptorium`: Binary entrypoint for CLI and HTTP server. +- `internal/domain`: Core domain contracts, including `PromptDefinition`, `ExecutionProfile`, and `RunResult`. +- `internal/usecase`: `Runner` orchestration, including the logic for profile selection, runtime override resolution, and bounded repair. +- `internal/promptdef`: Repository for loading and validating Prompt Definitions from the filesystem. +- `internal/profile`: Repository for loading Execution Profiles from the filesystem. +- `internal/artifact`: Input artifact resolution (`inline`, `file`). +- `internal/prompt`: Template rendering via Go templates. +- `internal/llm`: Provider-neutral client interface and OpenAI-compatible HTTP adapter. +- `internal/validate`: Output validation implementation (`none/basic/json/json_schema`). +- `internal/adapter/cli`: CLI flag parsing and output handling. +- `internal/adapter/http`: HTTP request/response mapping. ## 3. Run Data Flow -`Runner.Run(ctx, RunRequest)` currently executes: +The `Runner.Run` flow executes the following steps: -1. Validate request (`prompt_id` required). -2. Load `PromptDefinition` by ID/version. -3. Determine selected profile ID (`request.profile_id` or prompt `default_profile`). -4. Resolve effective execution target from request override (execution-profile loading is deferred in this phase). -5. Resolve named input artifact refs. -6. Render prompt messages. -7. Hash prompt definition and rendered prompt. -8. Call LLM client with `GenerateRequest`. -9. Build output artifact. -10. Validate output. -11. If structured validation failed and repair is enabled, run bounded repair attempts and re-validate. -12. Return `RunResult` with artifact, raw output, validation result, metadata. +1. **Load Prompt Definition**: Retrieve the `PromptDefinition` by ID from the prompt repository. +2. **Select Profile**: Determine the `profile_id` using the precedence: + - Explicit `profile_id` in `RunRequest`. + - `default_profile` specified in the `PromptDefinition`. + - Error if neither is available. +3. **Load Execution Profile**: Retrieve the `ExecutionProfile` from the profile repository. +4. **Resolve Runtime Overrides**: Merge settings based on precedence (Highest to Lowest): + - Runtime overrides (CLI flags or HTTP `model` object). + - Execution Profile settings. + - Built-in application defaults. +5. **Resolve Artifacts**: Load all named input artifacts defined in the request. +6. **Render Prompt**: Apply template variables and input artifacts to the prompt templates. +7. **Call LLM**: Execute the generation request using the resolved `ExecutionTarget`. +8. **Validate/Repair**: + - Validate the model output against the output contract. + - If structured validation fails and `repair_attempts > 0`, perform bounded repair and re-validate. +9. **Return Result**: Produce a `RunResult` containing the final artifact, metadata, and validation status. -Validation content failures are returned as successful runs with `validation.status=failed`. +## 4. Domain Model -## 4. Package Responsibilities +Key domain types: +- `PromptDefinition`: Defines the "what" (templates, inputs, validation contract, and an optional `default_profile`). +- `ExecutionProfile`: Defines the "how" (endpoint, model, generation parameters, and `api_key_env`). +- `RunRequest`: The intent to execute a prompt, including `prompt_id`, optional `profile_id`, inputs, variables, and optional runtime overrides. +- `RunResult`: The outcome of a run, including the generated `Artifact`, `ValidationResult`, and auditing `RunMetadata`. +- `RunMetadata`: Detailed tracing info: `prompt_id`, `selected_profile_id`, model params, usage tokens, and hashes. -- `domain` - - Owns core nouns/contracts. - - Must not depend on adapters/provider SDK types. +## 5. Interfaces and Adapters -- `usecase` - - Owns single-run orchestration across ports. - - Owns bounded repair control flow. - - Must not own transport/wire concerns. +### Primary Ports +- `promptdef.Repository`: Lookup for prompt definitions. +- `profile.Repository`: Lookup for execution profiles. +- `artifact.Reader`: Loading of artifact content. +- `prompt.Renderer`: Template rendering. +- `llm.Client`: Model generation. +- `validate.Validator`: Output validation. -- `profile` (transitional) - - Currently loads prompt definitions from YAML. - - Package naming split (`prompt definition repo` vs `execution profile repo`) is deferred follow-up. +### Current Adapters +- **Repositories**: Filesystem YAML loaders for both prompts and profiles. +- **Artifact Reader**: Composite reader supporting `file` and `inline`. +- **Prompt Renderer**: Go templates with a custom `input` helper. +- **LLM Client**: OpenAI-compatible `/chat/completions` over HTTP. +- **Validator**: Standard validator supporting `none`, `basic`, `json`, and `json_schema`. -- `artifact` - - Loads artifacts from refs and normalizes payload metadata. - -- `prompt` - - Renders templates and enforces required inputs. - -- `llm` - - Defines generation client contract and protocol adapters. - -- `validate` - - Owns output validation semantics and schema validation. - -- `adapter/http`, `adapter/cli` - - Own request/response/flag mapping only. - - Delegate business flow to `usecase.Runner`. - -## 5. Domain Model (Current) - -Key types: - -- `PromptDefinition` - - `id`, `version`, `default_profile`, `inputs`, `templates`, `output_format`, `validation`. -- `ExecutionProfile` - - Execution/runtime settings shape (`endpoint`, `model`, timeouts, `api_key_env`, etc.). - - Loading/persistence is deferred in this pass. -- `ExecutionTarget` - - Effective execution settings for a run. -- `RunRequest` - - `prompt_id`, `prompt_version`, optional `profile_id`, `inputs`, `vars`, optional `execution` override, optional validation override. -- `RunResult` - - Output artifact, validation, raw output, prompt/profile/model metadata, hashes, timing, usage. -- `ArtifactRef` / `Artifact` - - Input reference and loaded content contracts. -- `RenderedPrompt` / `RenderedMessage` - - Provider-neutral rendered prompt. -- `GenerateRequest` / `GenerateResponse` - - Provider-neutral model I/O. - -## 6. Interfaces and Adapters - -Primary ports: - -- `profile.Repository` (transitional prompt-definition lookup) -- `artifact.Reader` -- `prompt.Renderer` -- `llm.Client` -- `validate.Validator` -- `usecase.OutputRepairer` (usecase-local) - -Current adapters: - -- Prompt definition repository: filesystem YAML loader. -- Artifact readers: `file`, `inline` via composite reader. -- Prompt renderer: Go templates with `input` helper. -- LLM adapter: OpenAI-compatible `/chat/completions` over `net/http`. -- Validator: standard validator (`none/basic/json/json_schema`). -- CLI/HTTP adapters. - -## 7. Validation and Repair Model - -Validation modes: - -- `none` -- `basic` -- `json` -- `json_schema` - -Repair behavior: - -- Applies only to structured modes (`json`, `json_schema`). -- Triggered only on failed validation and only when `repair_attempts > 0`. -- Strictly bounded by `repair_attempts`. -- Uses a narrow repair prompt asking for corrected JSON only. -- Runtime validator/repair errors are run errors. - -## 8. Public Contracts +## 6. Public Contracts ### CLI +- `run`: Executes a prompt. Uses flags like `--prompt`, `--profile`, `--input`, and various runtime overrides (e.g., `--model`, `--temperature`). +- `serve`: Starts the HTTP API. -Commands: +### HTTP API +- `POST /v1/runs`: Accepts `RunRequest` JSON and returns `RunResponse` JSON. No built-in auth. -- `scriptorium run` -- `scriptorium serve` +### YAML Shapes +- **Prompt YAML**: Includes `id`, `version`, `default_profile`, `inputs`, `templates`, and `validation`. +- **Profile YAML**: Includes `id`, `endpoint`, `model`, generation params, and `api_key_env`. -`run` flags: +## 7. Guardrails -- Required: `--profile-dir`, `--prompt-id`, `--input`. -- Optional: `--profile-id`, `--var`, `--out`, `--llm-base-url`, `--model`, `--api-key-env`, `--temperature`, `--max-tokens`, `--schema-dir`, `--timeout`. +- **Separation of Concerns**: Prompt content must not belong in execution profiles; model/API settings must not belong in prompt definitions. +- **Security**: Raw API keys are unsupported in all configuration and transport layers. Only `api_key_env` is used. +- **Path Resolution**: `content_file` paths in prompt definitions resolve relative to the prompt YAML file. +- **Integrity**: No silent prompt truncation or omission of content. +- **Reliability**: Repair loops are strictly bounded by `repair_attempts`. -Current transitional runtime behavior: +## 8. Extension Points -- Prompt definitions may provide `default_profile` selection. -- Execution-profile loading is deferred; execution settings must currently be supplied via run-time overrides. - -### HTTP - -- Endpoint: `POST /v1/runs`. -- Request maps to `RunRequest` with `prompt_id` (required), `inputs`, optional `profile_id`, `vars`, optional execution override (`model` object). -- Response includes `artifact`, `validation`, `metadata`, `raw_model_output`. -- Validation content failures return `200` with failed validation status. -- Error response shape: `{ "error": { "code": "...", "message": "..." } }`. - -### Prompt Definition YAML - -Current prompt-definition fields: - -- `id`, `version`, optional `default_profile`, optional `description` -- `inputs[]` with `name`, `required`, optional `content_type`, optional `description` -- `templates[]` with `role` and either `content` or `content_file` -- `output_format` -- `validation` (`format`, `validation_mode`, `schema_path`, `repair_attempts`) - -Strict YAML decoding (`KnownFields`) is enabled. - -### API Key Policy - -- Raw API keys are not accepted in YAML, CLI flags, HTTP body, or domain metadata. -- Auth is configured only by env var reference (`api_key_env`), resolved at request time by the LLM adapter. - -## 9. Extension Points (Future Work) - -Planned next extensions should reuse current boundaries: - -- Execution-profile repository/loader implementation. -- Split transitional `internal/profile` into clearer prompt-definition/profile repositories. -- S3 artifact refs. -- Token budgeting/policy layer. -- Streaming generation. -- Batch run use case. -- Additional provider adapters. -- Additional validation modes. - -## 10. Architectural Guardrails - -- No D&D-specific logic in core Go packages. -- No orchestration creep into Scriptorium. -- No unbounded repair loops. -- No silent content truncation/omission. -- Do not log full prompts/artifacts by default. -- Keep provider-specific wire/SDK details out of domain types. -- Keep adapters thin. - -## 11. Testing Strategy - -Protect these behaviors with focused tests: - -- Prompt-definition loading/validation errors. -- Artifact loading/hash/content-type behavior. -- Prompt rendering required-input and template error paths. -- LLM adapter request/response/auth/error/timeout behavior. -- Runner success/failure/metadata behavior. -- Validation failure raw-output preservation. -- Bounded repair behavior. -- HTTP mapping and error mapping. -- CLI parsing and output stream separation. - -Prefer small unit tests and minimal integration-style tests with fake LLMs. +Future work should remain grounded in the current architecture: +- **Artifacts**: Add S3 artifact references via a new `artifact.Reader`. +- **LLM**: Implement additional provider adapters (e.g., Anthropic, Google). +- **Execution**: Add token budgeting, streaming generation, and batch execution capabilities. +- **Repositories**: Implement database-backed repositories for prompts and profiles. +- **Profiles**: Support more granular profile versioning and environment-specific profiles.