From 359e910572573fc1dbec62f14f7f71af5ec23698 Mon Sep 17 00:00:00 2001 From: Eric Rakestraw Date: Tue, 26 May 2026 03:28:28 +0000 Subject: [PATCH] Add development policy and internal architecture docs --- docs/internal/adapters.md | 139 ++++++++++++++++++++++++++++++++++ docs/internal/runner.md | 146 ++++++++++++++++++++++++++++++++++++ docs/policy/architecture.md | 135 ++++++++++++++++++--------------- docs/policy/development.md | 103 +++++++++++++++++++++++++ 4 files changed, 463 insertions(+), 60 deletions(-) create mode 100644 docs/internal/adapters.md create mode 100644 docs/internal/runner.md create mode 100644 docs/policy/development.md diff --git a/docs/internal/adapters.md b/docs/internal/adapters.md new file mode 100644 index 0000000..b574b8e --- /dev/null +++ b/docs/internal/adapters.md @@ -0,0 +1,139 @@ +# Adapter And Repository Internals + +## Purpose + +This document describes implemented adapter/repository boundaries and their current behavior. + +## Adapter Map + +- `internal/adapter/cli`: CLI command parsing, app wiring, stdout/stderr handling, exit codes. +- `internal/adapter/http`: HTTP request/response mapping for `POST /v1/runs`. +- `internal/promptdef`: filesystem prompt-definition repository. +- `internal/profile`: filesystem execution-profile repository. +- `internal/artifact`: input artifact reader. +- `internal/prompt`: Go-template renderer. +- `internal/llm`: OpenAI-compatible LLM client implementation. +- `internal/validate`: output validator. +- `internal/format`: prepared-run formatters for `render` output. + +## Inputs And Outputs + +CLI adapter: + +- Input: process args, filesystem config/assets, environment. +- Output: exit code, stdout artifact/prepared output, stderr summaries/errors. + +HTTP adapter: + +- Input: JSON request body (`runRequestDTO`). +- Output: JSON success/error body with mapped status codes. + +Filesystem repositories: + +- Input: prompt/profile YAML files. +- Output: normalized domain definitions/profiles or typed errors. + +Artifact reader: + +- Input: `domain.ArtifactRef`. +- Output: loaded `domain.Artifact`. + +LLM adapter: + +- Input: `domain.GenerateRequest`. +- Output: `domain.GenerateResponse`. + +Validator: + +- Input: artifact body + output contract. +- Output: validation result or runtime validation error. + +## Boundaries + +- Adapters convert external representations to domain requests and back. +- Use-case decisions remain in `internal/usecase`. +- External dependency details stay scoped to adapter packages. + +## Config Fields Used + +Primary app settings consumed by adapters: + +- `prompt_dir` +- `profile_dir` +- `schema_dir` +- `server.addr` +- `defaults.render_format` + +Execution profile/request settings used through runner: + +- `endpoint`, `model`, `temperature`, `max_tokens`, `top_p`, `timeout_seconds`, `api_key_env`, `reasoning_effort`, `extra_params` + +## External Dependencies + +- YAML decoding: `gopkg.in/yaml.v3` (strict known-fields mode in config/prompt/profile loaders). +- JSON Schema validation: `github.com/santhosh-tekuri/jsonschema/v6`. +- HTTP client/server: Go standard library. + +## Failure Behavior + +Strict decoding and input checks: + +- config/prompt/profile loaders reject unknown YAML fields. +- HTTP DTO decoder rejects unknown JSON fields. +- raw API key payload fields are rejected by strict decoding in profile/http paths. + +Artifact refs: + +- Supported reference types: `inline`, `file`. +- Unsupported types return `ErrUnsupportedRefType`. + +LLM adapter: + +- endpoint appends `/chat/completions`. +- non-2xx responses map to request failure errors. +- malformed responses (including missing/empty first choice content) are errors. + +Validator: + +- `basic`, `json`, `json_schema` content failures return `ValidationFailed` results. +- schema load/compile/path failures are runtime errors. + +HTTP error mapping: + +- maps domain/use-case errors to stable HTTP code + error code/message. +- avoids returning internal wrapped-cause details in response payload. + +## CLI Adapter Semantics + +Implemented commands: + +- `run` +- `render` +- `serve` + +Behavior highlights: + +- `run` exit `2` indicates validation failed after generation. +- `render` does not call the LLM. +- `serve` exposes HTTP handler only; no built-in auth. +- `render` supports `--format text|json`; `render` does not expose `--schema-dir`. +- deprecated aliases `--prompt-id` and `--profile-id` are still accepted. + +## Tests To Inspect Before Changing + +- `internal/adapter/cli/run_test.go` +- `internal/adapter/http/handler_test.go` +- `internal/promptdef/repository_test.go` +- `internal/profile/repository_test.go` +- `internal/artifact/reader_test.go` +- `internal/prompt/renderer_test.go` +- `internal/llm/openai_compatible_client_test.go` +- `internal/validate/standard_validator_test.go` +- `internal/format/prepared_run_test.go` + +## Architectural Invariants + +- Adapter packages do not own runner decision logic. +- External request/response strictness is part of contract stability. +- Prepared-render output never includes resolved API key values. +- Outbound OpenAI-compatible request includes only currently serialized fields (`model`, `messages`, optional `temperature`, `max_tokens`, `top_p`, optional `response_format`). diff --git a/docs/internal/runner.md b/docs/internal/runner.md new file mode 100644 index 0000000..552e3b7 --- /dev/null +++ b/docs/internal/runner.md @@ -0,0 +1,146 @@ +# Runner Internals + +## Purpose + +`internal/usecase.Runner` is the core use case orchestrator for prompt preparation and execution. + +It owns request validation, prompt/profile resolution, runtime-parameter merge, artifact loading, prompt rendering, structured-output setup, LLM invocation, output validation, and result metadata. + +## Inputs And Outputs + +Primary input type: + +- `domain.RunRequest` + +Primary output types: + +- `domain.PreparedRun` from `Prepare` +- `domain.RunResult` from `Run` + +LLM boundary types: + +- `domain.GenerateRequest` +- `domain.GenerateResponse` + +## Boundaries + +`Runner` coordinates the following interfaces: + +- `promptdef.Repository` +- `profile.Repository` +- `artifact.Reader` +- `prompt.Renderer` +- `llm.Client` +- `validate.Validator` +- optional `usecase.OutputRepairer` + +Transport concerns (CLI flags, HTTP DTO parsing, status-code mapping) stay outside runner. + +## Config Fields Used + +`Runner` does not read app config files directly. + +It receives fully constructed repositories/readers/validators from adapters. Effective behavior depends on adapter wiring, including: + +- prompt/profile directories +- schema base directory +- selected profile/runtime overrides in request + +## External Adapters Used + +`Runner` works with adapter implementations via interfaces. Current wiring from CLI/HTTP uses: + +- filesystem prompt/profile repositories +- composite artifact reader +- Go-template prompt renderer +- OpenAI-compatible LLM client +- standard validator + +## State And Resume Behavior + +`Runner` is stateless across requests. + +- No durable run-state storage. +- No built-in resume/skip checkpoints. +- Each `Run`/`Prepare` executes from request inputs and current repositories. + +## Failure Behavior + +Key error classes surfaced from `Runner`: + +- `ErrInvalidRequest`: invalid prompt/profile/request/runtime/API-key-env prerequisites. +- `ErrProfileLoad`: prompt or profile load failures. +- `ErrArtifactLoad`: artifact read failures. +- `ErrPromptRender`: template render failures. +- `ErrLLMGenerate`: model request failures. +- `ErrValidation`: validation runtime failures (including schema load/compile failures). + +Validation content failures are not run errors: + +- `Run` can succeed with `Validation.Status == failed`. +- CLI maps this to exit code `2`. +- HTTP returns `200` with failed validation details. + +## Prepare Flow + +`Prepare` performs: + +1. validate request basics (prompt ID present). +2. load prompt definition by ID/version. +3. select profile ID: + - explicit request profile ID + - prompt `default_profile` + - otherwise request error +4. load execution profile. +5. merge effective runtime target: + - built-in execution defaults + - selected profile values + - request overrides +6. verify required `api_key_env` environment variable (name only; value is not returned). +7. resolve output contract and structured-output schema payload when `json_schema` mode is active. +8. read input artifacts. +9. render prompt messages. +10. compute prompt/input/render hashes and return `PreparedRun`. + +`Prepare` does not call the LLM. + +## Run Flow + +`Run` performs: + +1. generate run ID. +2. call `Prepare`. +3. call LLM with prepared messages/effective target/structured-output spec. +4. build output artifact content type from output format. +5. validate output. +6. optionally attempt bounded repair when repairer is injected and contract allows it. +7. return `RunResult` with artifact, raw output, validation, hashes, profile/model metadata, usage, and timestamps. + +## Repair Hook Boundary + +Repair attempts occur only when all are true: + +- repairer is injected +- `repair_attempts > 0` +- validation status is `failed` +- validation mode is `json` or `json_schema` + +Current production wiring boundary: + +- CLI and HTTP adapters call `usecase.NewRunner(...)` (no repairer argument). +- Therefore normal CLI/HTTP execution does not perform repair attempts today. + +## Tests To Inspect Before Changing + +- `internal/usecase/runner_test.go` +- `internal/usecase/integration_test.go` +- `internal/adapter/cli/run_test.go` +- `internal/adapter/http/handler_test.go` + +## Architectural Invariants + +- `Run` reuses `Prepare`; prepare logic is not duplicated. +- Effective API-key environment-variable name may appear; resolved secret value must not. +- Structured-output schema document must load before LLM call for `json_schema` mode. +- Repair loops are bounded by `repair_attempts` and repairer presence. +- Runner stays transport-agnostic. diff --git a/docs/policy/architecture.md b/docs/policy/architecture.md index 2f5c672..c7af4bd 100644 --- a/docs/policy/architecture.md +++ b/docs/policy/architecture.md @@ -1,97 +1,112 @@ # Architecture -This document defines the development principles for this Go project. It is inward-facing: developers and LLM coding agents should use it to preserve the project’s shape, boundaries, and invariants as the code evolves. +This document is the development architecture policy for Scriptorium. + +It is for developers and LLM coding agents. User-facing behavior belongs in `README.md` and the docs under `docs/` that target operators/users. ## Project Shape -Default to a small, explicit, dependency-light Go application. Keep the design modular enough to test and change safely, but do not add abstraction unless it protects a real boundary or enables a real extension point. +Scriptorium is a narrow prompt-execution application with three entry paths: -Business/domain logic should live outside CLI, transport, and external-adapter packages. +- CLI `run` +- CLI `render` +- HTTP `POST /v1/runs` through `serve` -## Dependency Policy +Domain behavior is centralized in `internal/usecase` and `internal/domain`. -Prefer the Go standard library where practical. +## Core Principles -Use external dependencies only when justified by correctness, security, interoperability, or substantial complexity reduction. Good reasons include complex security-sensitive behavior, such as HTML sanitization, or widely used de facto standards, such as YAML parsing. +- Keep orchestration narrow: Scriptorium executes one prompt request; it is not a multi-step workflow engine. +- Keep adapter logic thin: adapters map external shapes to domain requests/results and should not hold domain decisions. +- Keep boundaries explicit: repositories/loaders/renderers/validators/LLM client stay behind package interfaces. +- Keep config strict: YAML/JSON decoding for external inputs should reject unknown fields. +- Keep secrets out of payloads: raw API key values must not be accepted or emitted. -Avoid dependencies for small conveniences. Do not let external dependency types leak across internal package boundaries unless the dependency is itself the explicit public contract of that package. +## Package Boundaries -## Package Layout +Current package map: -Use this layout unless the project has a documented reason to differ: +- `cmd/scriptorium`: process entrypoint. +- `internal/adapter/cli`: command parsing, app wiring for CLI commands, output behavior. +- `internal/adapter/http`: HTTP DTO mapping and error/status mapping. +- `internal/config`: application settings loading and CLI override precedence. +- `internal/defaults`: compile-time default constants. +- `internal/domain`: core request/result and contract types. +- `internal/usecase`: `Runner` prepare/run orchestration and repair-hook boundary. +- `internal/promptdef`: filesystem prompt-definition repository. +- `internal/profile`: filesystem execution-profile repository. +- `internal/artifact`: artifact reference readers. +- `internal/prompt`: template renderer. +- `internal/llm`: provider-neutral LLM client interface and OpenAI-compatible implementation. +- `internal/validate`: validator interfaces and standard implementation. +- `internal/format`: prepared-run output formatting. -- `internal/app`: application orchestration and top-level use cases. -- `internal/cli`: CLI command definitions, flags, argument parsing, and command wiring. -- `internal/config`: configuration structs, defaults, loading, precedence, and validation. -- `internal/adapters/`: adapters for external CLIs, APIs, databases, object stores, or libraries. -- `internal/api`: HTTP API handlers and request/response types, when the application exposes an HTTP API. -- `internal/transport/http`: HTTP client code, when the application calls HTTP services. +Detailed component behavior is documented in: -Package-private implementation constants may live near the package that owns them, preferably in `constants.go` when useful. +- `docs/internal/runner.md` +- `docs/internal/adapters.md` -## Configuration +## Configuration And Precedence -Centralize configuration loading, processing, precedence, defaults, and validation in `internal/config`. +Application settings are resolved as: -The goal is to make configuration discoverable and avoid implicit or hidden operational values. User-visible defaults and cross-package operational defaults should be defined in `internal/config/defaults.go`. +1. built-in defaults +2. config file values +3. CLI overrides -Unless documented otherwise, precedence is: +`config.yml` is for application wiring (directories, server address, render default format), not prompt/profile runtime execution settings. -1. CLI flags -2. environment variables -3. configuration file -4. built-in defaults +Profile selection and runtime model resolution remain use-case concerns. -Prefer YAML configuration unless the project has a strong reason to use another format. Config files should be discovered at `/usr/local/etc//config.yml`, with a CLI override via `--config`. +## State And Persistence Policy -Configuration files should not contain raw secrets unless the application is explicitly designed for that. Prefer environment variables or secret files for secrets. +Scriptorium has no durable run-state store. -## Adapters and External Integrations +- No built-in resume/checkpoint/archive behavior. +- Recovery model is rerun after correcting inputs/config/environment. -Use a hexagonal architecture style for external integrations. +## External Integration Policy -External adapters belong under `internal/adapters/`. If an adapter uses an external dependency, that dependency’s interface must not leak outside the adapter package. Other packages should interact only with the adapter’s API, so the dependency can be swapped, upgraded, or removed without touching unrelated code. +Current external contracts: -Adapters should be thin. Domain decisions belong in application/domain packages, not inside adapter glue. +- inbound HTTP contract: `POST /v1/runs` +- outbound model contract: OpenAI-compatible chat completions subset +- subprocess contract for integrators: CLI `run`/`render` -## Modules, Stages, and Registries +Integration docs belong under `docs/integrations/`. -When the application has stages or modules, each major stage/module should live in its own package and have an explicit input/output contract. +## Error Handling And Logging -The orchestrator should be able to compose, skip, resume, or run individual stages/modules when their prerequisites are satisfied. Ordering should be explicit: use a default sequence, dependency graph, or documented orchestration rule. +- Wrap errors with domain/operation context. +- Map domain errors to adapter-appropriate statuses/codes without leaking sensitive internals. +- Keep stderr summaries concise for CLI success/error paths. +- Never emit raw secret values. -If users can select modules, stages, validators, renderers, or adapters, selection should go through a registry or equivalent mechanism rather than scattered conditionals. +## Testing Expectations -## Embedded Assets +- Core runner behavior should be covered with isolated unit tests and fixture-based integration tests. +- Adapter behavior should be tested for parse/mapping/error semantics. +- Config parsing, prompt/profile loading, validator behavior, and LLM client error handling should remain covered by package tests. +- Repository-level docs/examples that claim runnable behavior should be validated by tests or smoke commands. -Store embedded JSON schemas, Markdown prompts, templates, and similar assets as separate files, not inline string literals, unless there is a strong reason otherwise. +## Documentation Expectations -## Errors and Logging +- Document implemented behavior only outside `docs/roadmap/`. +- Keep canonical reference locations stable (`docs/cli.md`, `docs/config.md`, `docs/operations.md`, `docs/troubleshooting.md`, `docs/internal/`). +- Update docs in the same change when architecture-relevant behavior changes. -Errors should be actionable and preserve context. Wrap errors with operation and path/resource context. CLI code should convert internal errors into concise user-facing messages. +## Architectural Invariants -Errors and logs must not expose secrets. +- `Runner.Run` reuses `Runner.Prepare` flow. +- CLI and HTTP currently instantiate `Runner` without a repairer. +- Artifact reading supports `inline` and `file` references. +- Unknown input fields in config/prompt/profile/http JSON should be rejected by strict decoding. +- Raw API key values must not be accepted through config/HTTP payloads. -Use structured logging where practical. Logs should describe operations, paths, external calls, retries, and failure causes, but should not include large user data by default. +## Non-Goals -## Context, Timeouts, and Cancellation +- Do not move orchestration responsibilities from external callers into Scriptorium. +- Do not add adapter-specific business logic in `internal/adapter/*` packages. +- Do not bypass repository/renderer/validator/LLM boundaries by introducing cross-package coupling. -Long-running operations should accept `context.Context`. External calls, subprocesses, HTTP requests, storage operations, and multi-stage workflows should respect cancellation and timeouts. - -## State, Files, and Safety - -If the application writes durable state, writes should be atomic where practical. Multi-step workflows should preserve enough state to support inspection, retry, or resume after failure. - -Code that deletes, moves, or overwrites files must use narrow, explicit paths. Avoid broad parent-directory operations. Cleanup that can cause data loss must be opt-in. - -## Testing - -Core logic should be testable without real external services. Use fakes, fixtures, or local test doubles for adapters where practical. - -Config examples should be load-tested. Important CLI workflows should have parser or command tests. Stage/module contracts should have focused tests that do not require running the full application unless end-to-end coverage is intentional. - -## Documentation - -Documentation should follow the project documentation policy. Keep user docs focused on implemented behavior. Put future, planned, or aspirational work only under `docs/roadmap/`. - -When changing architecture, config, CLI behavior, adapters, or stage/module contracts, update the relevant docs and examples in the same change. +Work that is not implemented belongs in `docs/roadmap/`. diff --git a/docs/policy/development.md b/docs/policy/development.md new file mode 100644 index 0000000..512ea76 --- /dev/null +++ b/docs/policy/development.md @@ -0,0 +1,103 @@ +# Development Guide + +This document defines contributor workflow for Scriptorium. + +## Repository Layout + +- `cmd/scriptorium`: application entrypoint. +- `internal/domain`: core contracts. +- `internal/usecase`: runner orchestration. +- `internal/adapter/cli`: CLI adapter. +- `internal/adapter/http`: HTTP adapter. +- `internal/config`: application settings loading and precedence. +- `internal/defaults`: default constants. +- `internal/promptdef`: prompt-definition repository. +- `internal/profile`: execution-profile repository. +- `internal/artifact`: artifact readers. +- `internal/prompt`: prompt rendering. +- `internal/llm`: LLM client interface and OpenAI-compatible implementation. +- `internal/validate`: validation interfaces and implementation. +- `internal/format`: prepared-run formatting. +- `docs/`: canonical documentation. +- `examples/`: copyable maintained examples and fixtures. + +## Common Commands + +Build: + +```bash +go build ./cmd/scriptorium +``` + +Test: + +```bash +go test ./... +``` + +Targeted test runs commonly used during changes: + +```bash +go test ./internal/adapter/cli ./internal/adapter/http ./internal/usecase +``` + +## Coding Conventions + +- Prefer small interfaces at package boundaries. +- Keep adapter packages focused on translation and IO concerns. +- Keep domain/use-case logic outside adapters. +- Wrap errors with operation context. +- Use strict decoding for user-provided YAML/JSON where applicable. +- Avoid introducing dependencies unless they materially reduce risk/complexity. + +## Dependency Policy + +- Prefer standard library unless an external library is clearly justified. +- Current non-stdlib dependencies are intentionally small: + - `gopkg.in/yaml.v3` for YAML decoding. + - `github.com/santhosh-tekuri/jsonschema/v6` for JSON Schema validation. +- Do not leak dependency-specific types across unrelated package boundaries. + +## How To Add App Config Fields + +1. Add fields in `internal/config/config.go` (`Config`, `AppSettings`, and/or `CLIOverrides` as needed). +2. Apply defaults in `BuiltInDefaults()` when required. +3. Parse and validate in `applyConfig` / `ApplyCLIOverrides`. +4. Wire the field through the consuming adapter(s). +5. Add/update config tests in `internal/config/config_test.go`. +6. Update canonical docs (`docs/config.md`, and other affected docs). + +## How To Add CLI Flags + +1. Add flags in `internal/adapter/cli/run.go` for the relevant command. +2. Ensure precedence behavior remains consistent with app config rules. +3. Keep `run`, `render`, and `serve` flag surfaces intentionally scoped. +4. Add/update parser and command tests in `internal/adapter/cli/run_test.go`. +5. Update `docs/cli.md` and any related docs/examples. + +## How To Add Adapters Or Adapter Capabilities + +1. Define or reuse the appropriate interface boundary in domain/use-case packages. +2. Implement adapter code under `internal/adapter/` (or relevant boundary package). +3. Keep business decisions in `internal/usecase`. +4. Add focused adapter tests for mapping, parse, and error behavior. +5. Document the new/changed boundary in `docs/internal/adapters.md`. +6. If external contract changes, update `docs/integrations/` in the same change. + +## How To Update Prompt/Profile/Schema Assets + +1. Keep prompt/profile/schema files valid under strict loaders. +2. Keep examples secret-free. +3. Re-run tests that cover prompt/profile/validation behavior. +4. Update `docs/config.md` and any docs that reference changed contracts. + +## Documentation Update Expectations + +When behavior changes: + +1. Update canonical doc locations, not duplicate files. +2. Keep non-roadmap docs limited to implemented behavior. +3. Update links after file moves/renames. +4. Re-run relevant tests and smoke commands. + +Docs work is complete only when code/tests/examples/docs agree.