eric/scriptorium
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 6 Wiki Activity

Align internal documentation with architecture

Browse Source
This commit is contained in:
Eric Rakestraw
2026-07-05 03:13:50 +00:00
parent 07ac7e54c5
commit 41083de46a
5 changed files with 362 additions and 254 deletions
Download Patch File Download Diff File Expand all files Collapse all files

247
docs/internal/adapters.md
View File

@@ -1,163 +1,81 @@
# Adapter And Repository Internals
# Adapter Internals
## Purpose
This document describes implemented adapter/repository boundaries and their current behavior.
Adapters translate external interfaces into domain requests and translate domain results back out. They wire dependencies, apply app config, and own IO concerns, but they do not make runner decisions.
Source-loading behavior belongs in `docs/internal/sources.md`. User-facing CLI, HTTP, and package contracts belong in `docs/cli.md`, `docs/api.md`, and `docs/consumers/pkg-scriptorium.md`.
## Adapter Map
- `internal/adapter/cli`: CLI command parsing, app wiring, stdout/stderr handling, exit codes.
- `internal/adapter/http`: HTTP request/response mapping for `POST /v1/runs`.
- root package `scriptorium`: public Go library facade for preparing and running prompt requests.
- `internal/promptdef`: filesystem and `fs.FS` prompt-definition repositories.
- `internal/profile`: filesystem, `fs.FS`, and overlay execution-profile repositories.
- `internal/filecatalog`: shared YAML discovery, display-path, and `fs.FS` source-root resolution helpers.
- `internal/profile/builtin`: embedded built-in execution-profile repository.
- `internal/artifact`: input artifact reader.
- `internal/prompt`: Go-template renderer.
- `internal/llm`: OpenAI-compatible LLM client implementation.
- `internal/validate`: filesystem and `fs.FS` output validators.
- `internal/format`: prepared-run formatters for `render` output.
- `cmd/scriptorium`: process entrypoint.
- `internal/adapter/cli`: command parsing, config handoff, runner construction, stdout/stderr, exit codes.
- `internal/adapter/http`: `POST /v1/runs` request/response mapping and HTTP error/status mapping.
- root package `scriptorium`: public Go facade over internal runner types and dependencies.
Supporting implementation packages used during adapter wiring:
- `internal/config`
- `internal/defaults`
- `internal/format`
- `internal/llm`
- `internal/prompt`
## Inputs And Outputs
CLI adapter:
- Input: process args, filesystem config/assets, environment.
- Output: exit code, stdout artifact/prepared output, stderr summaries/errors.
- `run` summaries include cache usage counters only when either parsed cache counter is non-zero.
- Input: process args, optional config file, filesystem sources, environment variables.
- Output: process exit code, stdout artifact/prepared output, stderr summaries and errors.
HTTP adapter:
- Input: JSON request body (`runRequestDTO`).
- Output: JSON success/error body with mapped status codes.
- Success metadata includes token usage plus cache usage counters.
- Input: HTTP request method/path/headers/body for `POST /v1/runs`.
- Output: JSON success or error body with mapped status code.
Public library facade:
Public Go facade:
- Input: typed `scriptorium.RunRequest` values.
- Input: typed `scriptorium.Config`, `Option`, and `RunRequest` values.
- Output: typed `PreparedRun` and `RunResult` values plus public sentinel errors.
- Custom LLM behavior is injected with `WithLLMClient`; otherwise the default OpenAI-compatible client is used.
- `RunRequest.APIKey` is a request-scoped Go value only; it is converted into internal execution state for LLM generation and stripped from public result types.
- Prompt, profile, and schema source options can use directories, single files, or `fs.FS` roots. Explicit source options override the matching `Config` directory field.
- Public types are facade types converted at the package boundary; internal domain types remain internal.
Prompt/profile repositories:
- Input: prompt/profile YAML files under configured directories or `fs.FS` roots.
- Output: normalized domain definitions/profiles or typed errors.
- Shared YAML catalog helpers provide recursive discovery, extension filtering, deterministic ordering, file stems, `fs.FS` display paths, and source-root containment checks.
- Single-file public sources are represented as `fs.FS` roots containing one YAML file; lookup still uses YAML `id` values.
Profile repository composition:
- Built-in profiles are embedded and loaded through the same profile validation rules as filesystem profiles.
- When no custom profile directory is configured, the runner receives the built-in profile repository.
- When a custom profile directory/file/`fs.FS` source is configured, the runner receives an overlay repository with custom profiles as primary and built-ins as fallback.
- Overlay lookup falls back only after custom profile-not-found errors; custom load/validation/raw-key errors are returned directly.
Artifact reader:
- Input: `domain.ArtifactRef`.
- Output: loaded `domain.Artifact`.
LLM adapter:
- Input: `domain.GenerateRequest`.
- Output: `domain.GenerateResponse`.
- Direct API-key values are preferred when present; otherwise `api_key_env` is resolved from the process environment.
Validator:
- Input: artifact body + output contract.
- Output: validation result or runtime validation error.
- Schema documents may be loaded from a directory, single file, or `fs.FS` root in the public package. CLI and HTTP continue to use directory-backed schema loading.
## Boundaries
- Adapters convert external representations to domain requests and back.
- Use-case decisions remain in `internal/usecase`.
- External dependency details stay scoped to adapter packages.
- Adapters convert external shapes to `domain.RunRequest` and back.
- Runner orchestration remains in `internal/usecase`.
- Prompt/profile/schema/artifact source rules remain in repository, validator, and artifact packages.
- LLM provider request serialization remains in `internal/llm`.
- Public package types are facade types; internal domain types do not leak across the package boundary.
## Config Fields Used
Primary app settings consumed by adapters:
Adapter app settings:
- `prompt_dir`
- `profile_dir` (optional custom profile source)
- `profile_dir`
- `schema_dir`
- `server.addr`
- `server.artifact_root` (HTTP `serve` file input root)
- `server.artifact_root`
- `server.max_request_bytes`
- `server.max_artifact_bytes`
- `server.max_response_bytes`
- `defaults.render_format`
Execution profile/request settings used through runner:
Execution request/profile settings passed through the runner:
- `endpoint`, `model`, `temperature`, `max_tokens`, `top_p`, `timeout_seconds`, `service_tier`, `api_key_env`, `reasoning_effort`, `extra_params`
- CLI and HTTP request adapters preserve caller intent for numeric runtime overrides. Omitted values remain absent; explicit zero values are mapped as explicit overrides.
- HTTP `extra_params` accepts JSON-compatible values and maps them to domain request overrides without provider-specific adapter logic.
- `endpoint`
- `model`
- `temperature`
- `max_tokens`
- `top_p`
- `timeout_seconds`
- `service_tier`
- `api_key_env`
- `reasoning_effort`
- `extra_params`
## External Dependencies
CLI and HTTP preserve numeric override presence so omitted values and explicit zero values remain distinct.
- YAML decoding: `gopkg.in/yaml.v3` (strict known-fields mode in config/prompt/profile loaders).
- JSON Schema validation: `github.com/santhosh-tekuri/jsonschema/v6`.
- HTTP client/server: Go standard library.
## Failure Behavior
Strict decoding and input checks:
- config/prompt/profile loaders reject unknown YAML fields.
- prompt/profile repositories scan nested subdirectories recursively.
- prompt/profile lookup uses YAML `id` values; subdirectory paths are organizational only.
- prompt `content_file` paths resolve relative to the prompt YAML file within the same source.
- `fs.FS` prompt `content_file` paths and schema paths must remain inside the configured source root; absolute paths and relative traversal outside the root are rejected.
- duplicate prompt/profile IDs are invalid and fail instead of using first-match behavior.
- duplicate profile IDs across custom and built-in sources are allowed; the custom source overrides the built-in profile.
- HTTP DTO decoder rejects unknown JSON fields.
- raw API key payload fields are rejected by strict decoding in profile/http paths.
Artifact refs:
- Supported reference types: `inline`, `file`.
- Unsupported types return `ErrUnsupportedRefType`.
- CLI `run` and `render` use direct filesystem file reads for `file` references.
- HTTP `serve` uses a restricted artifact reader: `inline` references work without a root, while `file` references require `server.artifact_root` or `--artifact-root` and must pass lexical containment checks against that root.
- HTTP `serve` applies request-body, file-artifact, and encoded-response size limits. CLI `run` and `render` do not use these HTTP limits.
- HTTP file paths are resolved with clean absolute paths and lexical containment checks, not string-prefix checks.
- Symlinks inside the root are followed by the operating system, including symlinks that point outside the root; the configured root must not be writable by untrusted users.
LLM adapter:
- endpoint appends `/chat/completions`.
- rendered messages without cache control serialize with string `content`.
- rendered messages with cache control serialize as one text content block with `cache_control`.
- non-empty `reasoning_effort` serializes as a top-level provider request field.
- `extra_params` flatten into provider-specific top-level JSON request fields.
- reserved `extra_params` keys are rejected before the provider call: `model`, `session_id`, `messages`, `temperature`, `max_tokens`, `top_p`, `service_tier`, `reasoning_effort`, and `response_format`.
- empty `extra_params` keys and values that cannot be JSON-encoded are rejected before the provider call.
- compatible cache usage response fields are parsed into domain token usage.
- non-2xx responses map to request failure errors.
- malformed responses (including missing/empty first choice content) are errors.
- direct API-key values are never serialized in provider request bodies.
Validator:
- `basic`, `json`, `json_schema` content failures return `ValidationFailed` results.
- schema load/compile/path failures are runtime errors.
- directory-backed schema lookup uses explicit `schema_path` values relative to `schema_dir`; it does not recursively search by basename.
- `fs.FS` schema lookup uses explicit `schema_path` values inside the configured source root. Single-file public schema sources match by base name.
HTTP error mapping:
- maps domain/use-case errors to stable HTTP code + error code/message.
- maps request, artifact, and response size failures to `413` errors.
- distinguishes missing profile selection and missing `api_key_env` variable using stable use-case sentinel errors.
- avoids returning internal wrapped-cause details in response payload.
## CLI Adapter Semantics
## CLI Adapter
Implemented commands:
@@ -165,30 +83,73 @@ Implemented commands:
- `render`
- `serve`
Behavior highlights:
Behavior:
- `run` exit `2` indicates validation failed after generation.
- `render` does not call the LLM.
- `serve` exposes HTTP handler only; no built-in auth.
- `render` supports `--format text|json`; `render` does not expose `--schema-dir`.
- deprecated aliases `--prompt-id` and `--profile-id` are still accepted.
- `run` constructs a runner with direct filesystem artifact reading and calls `Runner.Run`.
- `render` constructs a runner and calls `Runner.Prepare`; it does not call the LLM.
- `serve` constructs a restricted artifact reader and HTTP handler, then starts an unauthenticated HTTP server.
- `run` exits `2` when generation succeeds but validation fails.
- parse, runtime, and output-write errors exit `1`.
- deprecated `--prompt-id` and `--profile-id` aliases are accepted.
## Tests To Inspect Before Changing
## HTTP Adapter
Behavior:
- Accepts only `POST /v1/runs`.
- Decodes JSON strictly and rejects unknown fields and trailing JSON tokens.
- Rejects empty `prompt_id` and empty `inputs` before calling the runner.
- Does not accept raw API key values in the request body.
- Returns validation failures as `200` responses with failed validation details.
- Maps request-body, artifact, and encoded-response size failures to `413`.
- Maps domain and repository errors to stable error codes without returning wrapped internal cause text.
The HTTP adapter has no built-in authentication or authorization. Deployment controls must be provided outside the process.
## Public Go Facade
Behavior:
- `NewEngine` wires the same default runner components as CLI/HTTP unless options override them.
- Prompt, profile, and schema sources may come from directories, single files, or `fs.FS` roots.
- `WithProfiles` adds in-memory profiles ahead of file-backed and built-in profiles.
- `WithLLMClient` injects custom model behavior.
- `RunRequest.APIKey` is request-scoped and direct; it is used only for generation and is stripped from public results.
- internal errors are mapped to public sentinels in `errors.go`.
## Failure Behavior
Adapters should:
- keep external error payloads concise and stable.
- avoid leaking raw secret values.
- use sentinels and typed errors for mapping.
- preserve strict external input decoding.
- keep validation content failures distinct from runtime errors.
CLI writes human-readable summaries to stderr. HTTP writes JSON error envelopes. The public Go facade returns typed errors.
## State And Manifests
Adapters do not add durable run state.
- No adapter writes run manifests.
- No adapter implements checkpoint, skip, or resume behavior.
- CLI output files are caller-selected artifacts, not internal state.
## Tests To Inspect
- `internal/adapter/cli/run_test.go`
- `internal/adapter/http/handler_test.go`
- `internal/promptdef/repository_test.go`
- `internal/profile/repository_test.go`
- `internal/artifact/reader_test.go`
- `internal/prompt/renderer_test.go`
- `internal/llm/openai_compatible_client_test.go`
- `internal/validate/standard_validator_test.go`
- `engine_test.go`
- `internal/format/prepared_run_test.go`
- `internal/llm/openai_compatible_client_test.go`
## Architectural Invariants
- Adapter packages do not own runner decision logic.
- External request/response strictness is part of contract stability.
- Prepared-render output never includes resolved API key values.
- Outbound OpenAI-compatible request includes currently serialized first-class fields (`model`, optional `session_id`, `messages`, optional `temperature`, `max_tokens`, `top_p`, optional `service_tier`, optional `reasoning_effort`, optional `response_format`) plus validated `extra_params` flattened as provider-specific top-level fields.
- Outbound cache control is message-level only; no top-level cache-control field is serialized.
- Adapter packages stay thin and translation-focused.
- App config is resolved before dependency construction.
- External input strictness is part of contract stability.
- CLI and HTTP construct runners without a repairer.
- HTTP endpoint details remain canonical in `docs/api.md`.
- Public Go package details remain canonical in `docs/consumers/pkg-scriptorium.md`.