scriptorium/architecture.md

Scriptorium Architecture

1. Purpose and Non-Goals

Scriptorium is a prompt-definition execution engine.

It accepts named input artifacts, renders prompt templates, calls an LLM, validates output, optionally performs bounded structured-output repair, and returns an artifact with metadata.

Scriptorium is not an orchestrator. It must not own transcription, transcript merge/polish steps, notifications, or cross-step workflow control.

For the motivating D&D workflow:

• Narratio orchestrates.
• WhisperX transcribes.
• Seriatim merges transcripts.
• Audita polishes transcripts.
• Scriptorium generates final artifacts from prepared inputs.

Core Go code remains generic.

2. Current Architecture

Current implementation structure:

• cmd/scriptorium: binary entrypoint.
• internal/domain: core domain contracts.
• internal/usecase: Runner run flow, validation integration, bounded repair coordination.
• internal/profile: transitional filesystem prompt-definition repository (package rename deferred).
• internal/artifact: input artifact resolution (inline, file).
• internal/prompt: template rendering.
• internal/llm: provider-neutral client interface + OpenAI-compatible HTTP adapter.
• internal/validate: validation implementation (none/basic/json/json_schema).
• internal/adapter/cli: CLI adapter.
• internal/adapter/http: HTTP adapter (POST /v1/runs).

3. Run Data Flow

Runner.Run(ctx, RunRequest) currently executes:

1. Validate request (prompt_id required).
2. Load PromptDefinition by ID/version.
3. Determine selected profile ID (request.profile_id or prompt default_profile).
4. Resolve effective execution target from request override (execution-profile loading is deferred in this phase).
5. Resolve named input artifact refs.
6. Render prompt messages.
7. Hash prompt definition and rendered prompt.
8. Call LLM client with GenerateRequest.
9. Build output artifact.
10. Validate output.
11. If structured validation failed and repair is enabled, run bounded repair attempts and re-validate.
12. Return RunResult with artifact, raw output, validation result, metadata.

Validation content failures are returned as successful runs with validation.status=failed.

4. Package Responsibilities

• domain

  • Owns core nouns/contracts.
  • Must not depend on adapters/provider SDK types.

• usecase

  • Owns single-run orchestration across ports.
  • Owns bounded repair control flow.
  • Must not own transport/wire concerns.

• profile (transitional)

  • Currently loads prompt definitions from YAML.
  • Package naming split (prompt definition repo vs execution profile repo) is deferred follow-up.

• artifact

  • Loads artifacts from refs and normalizes payload metadata.

• prompt

  • Renders templates and enforces required inputs.

• llm

  • Defines generation client contract and protocol adapters.

• validate

  • Owns output validation semantics and schema validation.

• adapter/http, adapter/cli

  • Own request/response/flag mapping only.
  • Delegate business flow to usecase.Runner.

5. Domain Model (Current)

Key types:

• PromptDefinition
  • id, version, default_profile, inputs, templates, output_format, validation.
• ExecutionProfile
  • Execution/runtime settings shape (endpoint, model, timeouts, api_key_env, etc.).
  • Loading/persistence is deferred in this pass.
• ExecutionTarget
  • Effective execution settings for a run.
• RunRequest
  • prompt_id, prompt_version, optional profile_id, inputs, vars, optional execution override, optional validation override.
• RunResult
  • Output artifact, validation, raw output, prompt/profile/model metadata, hashes, timing, usage.
• ArtifactRef / Artifact
  • Input reference and loaded content contracts.
• RenderedPrompt / RenderedMessage
  • Provider-neutral rendered prompt.
• GenerateRequest / GenerateResponse
  • Provider-neutral model I/O.

6. Interfaces and Adapters

Primary ports:

• profile.Repository (transitional prompt-definition lookup)
• artifact.Reader
• prompt.Renderer
• llm.Client
• validate.Validator
• usecase.OutputRepairer (usecase-local)

Current adapters:

• Prompt definition repository: filesystem YAML loader.
• Artifact readers: file, inline via composite reader.
• Prompt renderer: Go templates with input helper.
• LLM adapter: OpenAI-compatible /chat/completions over net/http.
• Validator: standard validator (none/basic/json/json_schema).
• CLI/HTTP adapters.

7. Validation and Repair Model

Validation modes:

• none
• basic
• json
• json_schema

Repair behavior:

• Applies only to structured modes (json, json_schema).
• Triggered only on failed validation and only when repair_attempts > 0.
• Strictly bounded by repair_attempts.
• Uses a narrow repair prompt asking for corrected JSON only.
• Runtime validator/repair errors are run errors.

8. Public Contracts

CLI

Commands:

• scriptorium run
• scriptorium serve

run flags:

• Required: --profile-dir, --prompt-id, --input.
• Optional: --profile-id, --var, --out, --llm-base-url, --model, --api-key-env, --temperature, --max-tokens, --schema-dir, --timeout.

Current transitional runtime behavior:

• Prompt definitions may provide default_profile selection.
• Execution-profile loading is deferred; execution settings must currently be supplied via run-time overrides.

HTTP

• Endpoint: POST /v1/runs.
• Request maps to RunRequest with prompt_id (required), inputs, optional profile_id, vars, optional execution override (model object).
• Response includes artifact, validation, metadata, raw_model_output.
• Validation content failures return 200 with failed validation status.
• Error response shape: { "error": { "code": "...", "message": "..." } }.

Prompt Definition YAML

Current prompt-definition fields:

• id, version, optional default_profile, optional description
• inputs[] with name, required, optional content_type, optional description
• templates[] with role and either content or content_file
• output_format
• validation (format, validation_mode, schema_path, repair_attempts)

Strict YAML decoding (KnownFields) is enabled.

API Key Policy

• Raw API keys are not accepted in YAML, CLI flags, HTTP body, or domain metadata.
• Auth is configured only by env var reference (api_key_env), resolved at request time by the LLM adapter.

9. Extension Points (Future Work)

Planned next extensions should reuse current boundaries:

• Execution-profile repository/loader implementation.
• Split transitional internal/profile into clearer prompt-definition/profile repositories.
• S3 artifact refs.
• Token budgeting/policy layer.
• Streaming generation.
• Batch run use case.
• Additional provider adapters.
• Additional validation modes.

10. Architectural Guardrails

• No D&D-specific logic in core Go packages.
• No orchestration creep into Scriptorium.
• No unbounded repair loops.
• No silent content truncation/omission.
• Do not log full prompts/artifacts by default.
• Keep provider-specific wire/SDK details out of domain types.
• Keep adapters thin.

11. Testing Strategy

Protect these behaviors with focused tests:

• Prompt-definition loading/validation errors.
• Artifact loading/hash/content-type behavior.
• Prompt rendering required-input and template error paths.
• LLM adapter request/response/auth/error/timeout behavior.
• Runner success/failure/metadata behavior.
• Validation failure raw-output preservation.
• Bounded repair behavior.
• HTTP mapping and error mapping.
• CLI parsing and output stream separation.

Prefer small unit tests and minimal integration-style tests with fake LLMs.