eric/scriptorium
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 6 Wiki Activity

Add development policy and internal architecture docs

Browse Source
This commit is contained in:
Eric Rakestraw
2026-05-26 03:28:28 +00:00
parent 4950a6bb14
commit 359e910572
4 changed files with 463 additions and 60 deletions
Download Patch File Download Diff File Expand all files Collapse all files

135
docs/policy/architecture.md
View File

@@ -1,97 +1,112 @@
# Architecture
This document defines the development principles for this Go project. It is inward-facing: developers and LLM coding agents should use it to preserve the project’s shape, boundaries, and invariants as the code evolves.
This document is the development architecture policy for Scriptorium.
It is for developers and LLM coding agents. User-facing behavior belongs in `README.md` and the docs under `docs/` that target operators/users.
## Project Shape
Default to a small, explicit, dependency-light Go application. Keep the design modular enough to test and change safely, but do not add abstraction unless it protects a real boundary or enables a real extension point.
Scriptorium is a narrow prompt-execution application with three entry paths:
Business/domain logic should live outside CLI, transport, and external-adapter packages.
- CLI `run`
- CLI `render`
- HTTP `POST /v1/runs` through `serve`
## Dependency Policy
Domain behavior is centralized in `internal/usecase` and `internal/domain`.
Prefer the Go standard library where practical.
## Core Principles
Use external dependencies only when justified by correctness, security, interoperability, or substantial complexity reduction. Good reasons include complex security-sensitive behavior, such as HTML sanitization, or widely used de facto standards, such as YAML parsing.
- Keep orchestration narrow: Scriptorium executes one prompt request; it is not a multi-step workflow engine.
- Keep adapter logic thin: adapters map external shapes to domain requests/results and should not hold domain decisions.
- Keep boundaries explicit: repositories/loaders/renderers/validators/LLM client stay behind package interfaces.
- Keep config strict: YAML/JSON decoding for external inputs should reject unknown fields.
- Keep secrets out of payloads: raw API key values must not be accepted or emitted.
Avoid dependencies for small conveniences. Do not let external dependency types leak across internal package boundaries unless the dependency is itself the explicit public contract of that package.
## Package Boundaries
## Package Layout
Current package map:
Use this layout unless the project has a documented reason to differ:
- `cmd/scriptorium`: process entrypoint.
- `internal/adapter/cli`: command parsing, app wiring for CLI commands, output behavior.
- `internal/adapter/http`: HTTP DTO mapping and error/status mapping.
- `internal/config`: application settings loading and CLI override precedence.
- `internal/defaults`: compile-time default constants.
- `internal/domain`: core request/result and contract types.
- `internal/usecase`: `Runner` prepare/run orchestration and repair-hook boundary.
- `internal/promptdef`: filesystem prompt-definition repository.
- `internal/profile`: filesystem execution-profile repository.
- `internal/artifact`: artifact reference readers.
- `internal/prompt`: template renderer.
- `internal/llm`: provider-neutral LLM client interface and OpenAI-compatible implementation.
- `internal/validate`: validator interfaces and standard implementation.
- `internal/format`: prepared-run output formatting.
- `internal/app`: application orchestration and top-level use cases.
- `internal/cli`: CLI command definitions, flags, argument parsing, and command wiring.
- `internal/config`: configuration structs, defaults, loading, precedence, and validation.
- `internal/adapters/<name>`: adapters for external CLIs, APIs, databases, object stores, or libraries.
- `internal/api`: HTTP API handlers and request/response types, when the application exposes an HTTP API.
- `internal/transport/http`: HTTP client code, when the application calls HTTP services.
Detailed component behavior is documented in:
Package-private implementation constants may live near the package that owns them, preferably in `constants.go` when useful.
- `docs/internal/runner.md`
- `docs/internal/adapters.md`
## Configuration
## Configuration And Precedence
Centralize configuration loading, processing, precedence, defaults, and validation in `internal/config`.
Application settings are resolved as:
The goal is to make configuration discoverable and avoid implicit or hidden operational values. User-visible defaults and cross-package operational defaults should be defined in `internal/config/defaults.go`.
1. built-in defaults
2. config file values
3. CLI overrides
Unless documented otherwise, precedence is:
`config.yml` is for application wiring (directories, server address, render default format), not prompt/profile runtime execution settings.
1. CLI flags
2. environment variables
3. configuration file
4. built-in defaults
Profile selection and runtime model resolution remain use-case concerns.
Prefer YAML configuration unless the project has a strong reason to use another format. Config files should be discovered at `/usr/local/etc/<app_name>/config.yml`, with a CLI override via `--config`.
## State And Persistence Policy
Configuration files should not contain raw secrets unless the application is explicitly designed for that. Prefer environment variables or secret files for secrets.
Scriptorium has no durable run-state store.
## Adapters and External Integrations
- No built-in resume/checkpoint/archive behavior.
- Recovery model is rerun after correcting inputs/config/environment.
Use a hexagonal architecture style for external integrations.
## External Integration Policy
External adapters belong under `internal/adapters/<name>`. If an adapter uses an external dependency, that dependency’s interface must not leak outside the adapter package. Other packages should interact only with the adapter’s API, so the dependency can be swapped, upgraded, or removed without touching unrelated code.
Current external contracts:
Adapters should be thin. Domain decisions belong in application/domain packages, not inside adapter glue.
- inbound HTTP contract: `POST /v1/runs`
- outbound model contract: OpenAI-compatible chat completions subset
- subprocess contract for integrators: CLI `run`/`render`
## Modules, Stages, and Registries
Integration docs belong under `docs/integrations/`.
When the application has stages or modules, each major stage/module should live in its own package and have an explicit input/output contract.
## Error Handling And Logging
The orchestrator should be able to compose, skip, resume, or run individual stages/modules when their prerequisites are satisfied. Ordering should be explicit: use a default sequence, dependency graph, or documented orchestration rule.
- Wrap errors with domain/operation context.
- Map domain errors to adapter-appropriate statuses/codes without leaking sensitive internals.
- Keep stderr summaries concise for CLI success/error paths.
- Never emit raw secret values.
If users can select modules, stages, validators, renderers, or adapters, selection should go through a registry or equivalent mechanism rather than scattered conditionals.
## Testing Expectations
## Embedded Assets
- Core runner behavior should be covered with isolated unit tests and fixture-based integration tests.
- Adapter behavior should be tested for parse/mapping/error semantics.
- Config parsing, prompt/profile loading, validator behavior, and LLM client error handling should remain covered by package tests.
- Repository-level docs/examples that claim runnable behavior should be validated by tests or smoke commands.
Store embedded JSON schemas, Markdown prompts, templates, and similar assets as separate files, not inline string literals, unless there is a strong reason otherwise.
## Documentation Expectations
## Errors and Logging
- Document implemented behavior only outside `docs/roadmap/`.
- Keep canonical reference locations stable (`docs/cli.md`, `docs/config.md`, `docs/operations.md`, `docs/troubleshooting.md`, `docs/internal/`).
- Update docs in the same change when architecture-relevant behavior changes.
Errors should be actionable and preserve context. Wrap errors with operation and path/resource context. CLI code should convert internal errors into concise user-facing messages.
## Architectural Invariants
Errors and logs must not expose secrets.
- `Runner.Run` reuses `Runner.Prepare` flow.
- CLI and HTTP currently instantiate `Runner` without a repairer.
- Artifact reading supports `inline` and `file` references.
- Unknown input fields in config/prompt/profile/http JSON should be rejected by strict decoding.
- Raw API key values must not be accepted through config/HTTP payloads.
Use structured logging where practical. Logs should describe operations, paths, external calls, retries, and failure causes, but should not include large user data by default.
## Non-Goals
## Context, Timeouts, and Cancellation
- Do not move orchestration responsibilities from external callers into Scriptorium.
- Do not add adapter-specific business logic in `internal/adapter/*` packages.
- Do not bypass repository/renderer/validator/LLM boundaries by introducing cross-package coupling.
Long-running operations should accept `context.Context`. External calls, subprocesses, HTTP requests, storage operations, and multi-stage workflows should respect cancellation and timeouts.
## State, Files, and Safety
If the application writes durable state, writes should be atomic where practical. Multi-step workflows should preserve enough state to support inspection, retry, or resume after failure.
Code that deletes, moves, or overwrites files must use narrow, explicit paths. Avoid broad parent-directory operations. Cleanup that can cause data loss must be opt-in.
## Testing
Core logic should be testable without real external services. Use fakes, fixtures, or local test doubles for adapters where practical.
Config examples should be load-tested. Important CLI workflows should have parser or command tests. Stage/module contracts should have focused tests that do not require running the full application unless end-to-end coverage is intentional.
## Documentation
Documentation should follow the project documentation policy. Keep user docs focused on implemented behavior. Put future, planned, or aspirational work only under `docs/roadmap/`.
When changing architecture, config, CLI behavior, adapters, or stage/module contracts, update the relevant docs and examples in the same change.
Work that is not implemented belongs in `docs/roadmap/`.

103
docs/policy/development.md Normal file
View File

@@ -0,0 +1,103 @@
# Development Guide
This document defines contributor workflow for Scriptorium.
## Repository Layout
- `cmd/scriptorium`: application entrypoint.
- `internal/domain`: core contracts.
- `internal/usecase`: runner orchestration.
- `internal/adapter/cli`: CLI adapter.
- `internal/adapter/http`: HTTP adapter.
- `internal/config`: application settings loading and precedence.
- `internal/defaults`: default constants.
- `internal/promptdef`: prompt-definition repository.
- `internal/profile`: execution-profile repository.
- `internal/artifact`: artifact readers.
- `internal/prompt`: prompt rendering.
- `internal/llm`: LLM client interface and OpenAI-compatible implementation.
- `internal/validate`: validation interfaces and implementation.
- `internal/format`: prepared-run formatting.
- `docs/`: canonical documentation.
- `examples/`: copyable maintained examples and fixtures.
## Common Commands
Build:
```bash
go build ./cmd/scriptorium
```
Test:
```bash
go test ./...
```
Targeted test runs commonly used during changes:
```bash
go test ./internal/adapter/cli ./internal/adapter/http ./internal/usecase
```
## Coding Conventions
- Prefer small interfaces at package boundaries.
- Keep adapter packages focused on translation and IO concerns.
- Keep domain/use-case logic outside adapters.
- Wrap errors with operation context.
- Use strict decoding for user-provided YAML/JSON where applicable.
- Avoid introducing dependencies unless they materially reduce risk/complexity.
## Dependency Policy
- Prefer standard library unless an external library is clearly justified.
- Current non-stdlib dependencies are intentionally small:
- `gopkg.in/yaml.v3` for YAML decoding.
- `github.com/santhosh-tekuri/jsonschema/v6` for JSON Schema validation.
- Do not leak dependency-specific types across unrelated package boundaries.
## How To Add App Config Fields
1. Add fields in `internal/config/config.go` (`Config`, `AppSettings`, and/or `CLIOverrides` as needed).
2. Apply defaults in `BuiltInDefaults()` when required.
3. Parse and validate in `applyConfig` / `ApplyCLIOverrides`.
4. Wire the field through the consuming adapter(s).
5. Add/update config tests in `internal/config/config_test.go`.
6. Update canonical docs (`docs/config.md`, and other affected docs).
## How To Add CLI Flags
1. Add flags in `internal/adapter/cli/run.go` for the relevant command.
2. Ensure precedence behavior remains consistent with app config rules.
3. Keep `run`, `render`, and `serve` flag surfaces intentionally scoped.
4. Add/update parser and command tests in `internal/adapter/cli/run_test.go`.
5. Update `docs/cli.md` and any related docs/examples.
## How To Add Adapters Or Adapter Capabilities
1. Define or reuse the appropriate interface boundary in domain/use-case packages.
2. Implement adapter code under `internal/adapter/<name>` (or relevant boundary package).
3. Keep business decisions in `internal/usecase`.
4. Add focused adapter tests for mapping, parse, and error behavior.
5. Document the new/changed boundary in `docs/internal/adapters.md`.
6. If external contract changes, update `docs/integrations/` in the same change.
## How To Update Prompt/Profile/Schema Assets
1. Keep prompt/profile/schema files valid under strict loaders.
2. Keep examples secret-free.
3. Re-run tests that cover prompt/profile/validation behavior.
4. Update `docs/config.md` and any docs that reference changed contracts.
## Documentation Update Expectations
When behavior changes:
1. Update canonical doc locations, not duplicate files.
2. Keep non-roadmap docs limited to implemented behavior.
3. Update links after file moves/renames.
4. Re-run relevant tests and smoke commands.
Docs work is complete only when code/tests/examples/docs agree.