scriptorium/docs/roadmap/implementation.md

Library API Implementation Plan

This plan implements the target state in docs/roadmap/library.md.

Audience: LLM coding agents implementing the feature in order. Follow docs/policy/architecture.md, docs/policy/development.md, and docs/policy/documentation.md before changing code.

Constraints

• Add a public root package named scriptorium; keep existing internal/* packages internal.
• Define public facade types and convert to/from internal domain types. Do not alias internal domain types as the public API.
• Do not rewire CLI or HTTP through the public facade in this implementation.
• Preserve current CLI, HTTP, prompt/profile loading, validation, secret-handling, and outbound LLM behavior.
• Do not add dependencies.
• Keep each stage passing go test ./... before moving to the next stage.

Stage 1: Public Types, Engine Construction, And Prepare

Goal: make prompt preparation usable from an imported root package without calling an LLM.

Public Package

Create Go files at the module root using:

package scriptorium

Expose:

type Engine struct { /* unexported fields */ }

type Config struct {
	PromptDir  string
	ProfileDir string
	SchemaDir  string
	Timeout    time.Duration
	HTTPClient *http.Client
}

type Option func(*engineOptions) error

func NewEngine(cfg Config, opts ...Option) (*Engine, error)
func (e *Engine) Prepare(ctx context.Context, req RunRequest) (*PreparedRun, error)

Construction rules:

• PromptDir and ProfileDir are required.
• SchemaDir defaults to the same built-in default used by app config.
• Timeout, when non-zero, configures the default OpenAI-compatible client timeout.
• HTTPClient, when non-nil, is used by the default OpenAI-compatible client.
• NewEngine wires the same internal components used by CLI/HTTP: filesystem prompt/profile repositories, composite artifact reader, Go template renderer, standard validator, and OpenAI-compatible LLM client.
• Return public ErrInvalidConfig for invalid engine configuration.

Public Types

Define public facade types with exported fields:

• RunRequest
• PreparedRun
• ArtifactRef
• Artifact
• ExecutionTarget
• ExecutionTargetOverride
• ExecutionTargetPresence
• OutputContract
• ValidationResult
• TokenUsage
• RenderedMessage
• CacheControl
• StructuredOutputSpec

Use the same enum string values as internal domain types for formats, validation modes, validation statuses, artifact ref types, cache-control type, and structured-output type.

Required request shape:

type RunRequest struct {
	PromptID      string
	PromptVersion string
	ProfileID     string
	Inputs        map[string]ArtifactRef
	Vars          map[string]string
	Execution     *ExecutionTargetOverride
	Validation    *OutputContract
	Metadata      map[string]string
}

ExecutionTargetOverride must preserve numeric override presence using pointer fields:

Temperature    *float64
MaxTokens      *int
TopP           *float64
TimeoutSeconds *int

PreparedRun should include the same user-observable fields as internal domain.PreparedRun, but should not expose internal-only target presence metadata.

Input Helpers

Expose:

func File(path string) ArtifactRef
func Inline(body string) ArtifactRef
func InlineWithURI(uri string, body string) ArtifactRef

Mapping:

• File(path) maps to artifact type file with URI: path.
• Inline(body) maps to artifact type inline with Body: body.
• InlineWithURI(uri, body) maps to artifact type inline with both fields set.

Conversion Layer

Implement unexported conversion helpers in the public package:

• public run request to internal domain.RunRequest
• internal domain.PreparedRun to public PreparedRun
• internal artifacts/messages/contracts/validation/usage/structured-output to public equivalents
• public execution override to internal domain.ExecutionTargetOverride

Conversions must deep-copy maps and slices that cross the public/internal boundary.

Tests

Add root package tests.

Required tests:

• NewEngine rejects missing PromptDir.
• NewEngine rejects missing ProfileDir.
• Prepare works with examples/config.yml directories when passed directly through Config.
• Prepare works with File input refs.
• Prepare works with Inline input refs.
• Prepare output does not expose raw API-key values or internal target presence metadata in JSON.
• Explicit zero execution overrides survive into prepared effective settings.

Verification

Run:

go test ./...

Stage 2: Run, LLM Injection, And Public Errors

Goal: make full execution usable and testable without real provider credentials.

Public Run Method

Expose:

func (e *Engine) Run(ctx context.Context, req RunRequest) (*RunResult, error)

RunResult should expose:

• run ID
• artifact
• raw output
• validation result
• prompt/profile/model metadata
• effective model params
• input hashes
• token/cache usage
• start/end/duration timing

Do not expose raw API-key values.

Public LLM Injection

Expose:

type LLMClient interface {
	Generate(context.Context, GenerateRequest) (*GenerateResponse, error)
}

func WithLLMClient(client LLMClient) Option

Public GenerateRequest must include:

• rendered prompt
• effective execution target
• execution target presence
• structured-output spec

Public GenerateResponse must include:

• content
• token usage

Implementation rule:

• WithLLMClient wraps the public client in an unexported adapter that satisfies internal/llm.Client.
• The adapter converts internal generate requests to public generate requests and converts public generate responses back to internal responses.
• A nil client passed to WithLLMClient returns ErrInvalidConfig.

Default behavior:

• If no custom LLM client is supplied, NewEngine uses internal/llm.NewOpenAICompatibleClient.
• Config.Timeout and Config.HTTPClient apply only to the default OpenAI-compatible client.

Public Errors

Define public sentinel errors:

• ErrInvalidConfig
• ErrInvalidRequest
• ErrPromptNotFound
• ErrProfileNotFound
• ErrPromptLoad
• ErrProfileLoad
• ErrArtifactLoad
• ErrPromptRender
• ErrLLMGenerate
• ErrValidation

Public methods must map internal errors to public sentinels while preserving wrapped context. Callers must be able to use errors.Is.

Mapping rules:

• missing/invalid public engine config -> ErrInvalidConfig
• internal usecase.ErrInvalidRequest -> ErrInvalidRequest
• internal prompt not found -> ErrPromptNotFound
• internal profile not found -> ErrProfileNotFound
• internal prompt load errors -> ErrPromptLoad
• internal profile load errors -> ErrProfileLoad
• internal artifact load errors -> ErrArtifactLoad
• internal prompt render errors -> ErrPromptRender
• internal LLM generate errors -> ErrLLMGenerate
• internal validation runtime errors -> ErrValidation

Do not expose internal sentinel values as public API.

Tests

Required tests:

• Run succeeds with WithLLMClient fake and returns typed artifact, raw output, validation, metadata, and usage.
• Run passes rendered prompt, effective execution target, and target presence to the injected LLM client.
• Run validation failure returns a successful result with failed validation, not an error.
• public errors support errors.Is for invalid request, prompt not found, profile not found, artifact load, render failure, LLM failure, and validation runtime failure.
• nil WithLLMClient(nil) returns ErrInvalidConfig.
• default OpenAI-compatible client can still be constructed without real provider credentials.

Verification

Run:

go test ./...

Stage 3: Public Documentation And Consumer Examples

Goal: document implemented library behavior in canonical public-consumer docs.

Docs

After Stages 1 and 2 are implemented, update:

• README.md: add a short link to library usage without turning the README into a manual.
• docs/internal/adapters.md: list the public library facade as an implemented adapter surface.
• docs/consumers/api.md: describe the public consumer API at a high level.
• docs/consumers/pkg-scriptorium.md: document the root package usage, types, errors, and examples.

Create docs/consumers/ if it does not exist.

Do not document unimplemented future library features outside docs/roadmap/.

Examples

Add copyable library examples only if they can be tested without real credentials.

Recommended example:

• examples/go-library/prepare/main.go or equivalent prepare-only example using examples/ prompt/profile/fixture assets.

If adding a run example, it must use an injected fake LLM client and must not require provider credentials.

Tests

Required tests:

• doc/example smoke coverage for any added Go example using go test or go test ./....
• existing CLI/HTTP tests continue to pass unchanged.

Verification

Run:

go test ./...
go run ./cmd/scriptorium render \
  --config ./examples/config.yml \
  --prompt generic.markdown_summary \
  --input transcript=./examples/fixtures/transcript.md \
  --input glossary=./examples/fixtures/glossary.yml \
  --format json

Final Checks

Before considering the feature complete:

1. Confirm the root package can be imported as gitea.maximumdirect.net/eric/scriptorium.
2. Confirm public package tests do not require real provider credentials.
3. Confirm go test ./... passes.
4. Confirm the render smoke command passes.
5. Confirm non-roadmap docs describe only implemented behavior.
6. Confirm no public result or rendered/prepared output exposes raw API-key values.
7. Confirm git diff does not include unrelated CLI/HTTP behavior changes.