119 lines
6.4 KiB
Markdown
119 lines
6.4 KiB
Markdown
# Scriptorium Architecture
|
|
|
|
## 1. Purpose and Non-Goals
|
|
|
|
Scriptorium is a prompt-definition execution engine.
|
|
|
|
It accepts named input artifacts, renders prompt templates, calls an LLM, validates output, optionally performs bounded structured-output repair, and returns an artifact with metadata.
|
|
|
|
Scriptorium is not an orchestrator. It must not own transcription, transcript merge/polish steps, notifications, or cross-step workflow control.
|
|
|
|
For the motivating D&D workflow:
|
|
- Narratio orchestrates.
|
|
- WhisperX transcribes.
|
|
- Seriatim merges transcripts.
|
|
- Audita polishes transcripts.
|
|
- Scriptorium generates final artifacts from prepared inputs.
|
|
|
|
Core Go code remains generic.
|
|
|
|
## 2. Current Architecture
|
|
|
|
Scriptorium uses a ports-and-adapters architecture to decouple the core execution logic from external dependencies.
|
|
|
|
### Package Responsibilities
|
|
- `cmd/scriptorium`: Binary entrypoint for CLI and HTTP server.
|
|
- `internal/domain`: Core domain contracts, including `PromptDefinition`, `ExecutionProfile`, and `RunResult`.
|
|
- `internal/usecase`: `Runner` orchestration, including the logic for profile selection, runtime override resolution, and bounded repair.
|
|
- `internal/promptdef`: Repository for loading and validating Prompt Definitions from the filesystem.
|
|
- `internal/profile`: Repository for loading Execution Profiles from the filesystem.
|
|
- `internal/artifact`: Input artifact resolution (`inline`, `file`).
|
|
- `internal/prompt`: Template rendering via Go templates.
|
|
- `internal/llm`: Provider-neutral client interface and OpenAI-compatible HTTP adapter.
|
|
- `internal/validate`: Output validation implementation (`none/basic/json/json_schema`).
|
|
- `internal/adapter/cli`: CLI flag parsing and output handling.
|
|
- `internal/adapter/http`: HTTP request/response mapping.
|
|
|
|
## 3. Run Data Flow
|
|
|
|
The `Runner.Run` flow executes the following steps:
|
|
|
|
1. **Load Prompt Definition**: Retrieve the `PromptDefinition` by ID from the prompt repository.
|
|
2. **Select Profile**: Determine the `profile_id` using the precedence:
|
|
- Explicit `profile_id` in `RunRequest`.
|
|
- `default_profile` specified in the `PromptDefinition`.
|
|
- Error if neither is available.
|
|
3. **Load Execution Profile**: Retrieve the `ExecutionProfile` from the profile repository.
|
|
4. **Resolve Runtime Overrides**: Merge settings based on precedence (Highest to Lowest):
|
|
- Runtime overrides (CLI flags or HTTP `model` object).
|
|
- Execution Profile settings.
|
|
- Built-in application defaults.
|
|
5. **Resolve Artifacts**: Load all named input artifacts defined in the request.
|
|
6. **Render Prompt**: Apply template variables and input artifacts to the prompt templates.
|
|
7. **Call LLM**: Execute the generation request using the resolved execution target (effective runtime settings).
|
|
8. **Validate/Repair**:
|
|
- Validate the model output against the output contract.
|
|
- If structured validation fails and `repair_attempts > 0`, perform bounded repair and re-validate.
|
|
9. **Return Result**: Produce a `RunResult` containing the final artifact, metadata, and validation status.
|
|
|
|
## 4. Domain Model
|
|
|
|
Key domain types:
|
|
- `PromptDefinition`: Defines the "what" (templates, inputs, validation contract, and an optional `default_profile`).
|
|
- `ExecutionProfile`: Defines the "how" (endpoint, model, generation parameters, and `api_key_env`).
|
|
- `RunRequest`: The intent to execute a prompt, including `prompt_id`, optional `profile_id`, inputs, variables, and optional runtime overrides.
|
|
- `RunResult`: The outcome of a run, including the generated `Artifact`, `ValidationResult`, and auditing `RunMetadata`.
|
|
- `RunMetadata`: Detailed tracing info: `prompt_id`, `selected_profile_id`, model params, usage tokens, and hashes.
|
|
|
|
## 5. Interfaces and Adapters
|
|
|
|
### Primary Ports
|
|
- `promptdef.Repository`: Lookup for prompt definitions.
|
|
- `profile.Repository`: Lookup for execution profiles.
|
|
- `artifact.Reader`: Loading of artifact content.
|
|
- `prompt.Renderer`: Template rendering.
|
|
- `llm.Client`: Model generation.
|
|
- `validate.Validator`: Output validation.
|
|
|
|
### Current Adapters
|
|
- **Repositories**: Filesystem YAML loaders for both prompts and profiles.
|
|
- **Artifact Reader**: Composite reader supporting `file` and `inline`.
|
|
- **Prompt Renderer**: Go templates with a custom `input` helper.
|
|
- **LLM Client**: OpenAI-compatible `/chat/completions` over HTTP.
|
|
- **Validator**: Standard validator supporting `none`, `basic`, `json`, and `json_schema`.
|
|
|
|
## 6. Public Contracts
|
|
|
|
### CLI
|
|
- `run`: Executes a prompt. Uses flags like `--prompt`, `--profile`, `--input`, and various runtime overrides (e.g., `--model`, `--temperature`).
|
|
- `serve`: Starts the HTTP API using infrastructure-only flags (`--addr`, `--prompt-dir`, `--profile-dir`, `--schema-dir`). It does not introduce a server-level model/runtime precedence layer.
|
|
|
|
### HTTP API
|
|
- `POST /v1/runs`: Accepts `RunRequest` JSON and returns `RunResponse` JSON. No built-in auth.
|
|
- Request may include runtime overrides under `model` and an `include_raw_output` boolean.
|
|
- `raw_model_output` is exposed only when explicitly requested with `include_raw_output=true`.
|
|
|
|
### YAML Shapes
|
|
- **Prompt YAML**: Includes `id`, `version`, optional `default_profile`, `inputs`, `messages`, and `output`.
|
|
- Inputs support `name`, `required`, optional `content_type`, and `description`.
|
|
- Messages require `role` and exactly one of `content` or `content_file`.
|
|
- `content_file` resolves relative to the prompt YAML location.
|
|
- **Profile YAML**: Includes `id`, `endpoint`, `model`, generation params, and `api_key_env`.
|
|
|
|
## 7. Guardrails
|
|
|
|
- **Separation of Concerns**: Prompt content must not belong in execution profiles; model/API settings must not belong in prompt definitions.
|
|
- **Security**: Raw API keys are unsupported in all configuration and transport layers. Only `api_key_env` is used.
|
|
- **Path Resolution**: `content_file` paths in prompt definitions resolve relative to the prompt YAML file.
|
|
- **Integrity**: No silent prompt truncation or omission of content.
|
|
- **Reliability**: Repair loops are strictly bounded by `repair_attempts`.
|
|
|
|
## 8. Extension Points
|
|
|
|
Future work should remain grounded in the current architecture:
|
|
- **Artifacts**: Add S3 artifact references via a new `artifact.Reader`.
|
|
- **LLM**: Implement additional provider adapters (e.g., Anthropic, Google).
|
|
- **Execution**: Add token budgeting, streaming generation, and batch execution capabilities.
|
|
- **Repositories**: Implement database-backed repositories for prompts and profiles.
|
|
- **Profiles**: Support more granular profile versioning and environment-specific profiles.
|