From 6d1fb66dd7873f0353570899ef36d492a0ffdada Mon Sep 17 00:00:00 2001 From: Eric Rakestraw Date: Sun, 26 Jul 2026 14:06:55 +0000 Subject: [PATCH] Establish canonical developer documentation structure --- ...adopt-canonical-documentation-ownership.md | 49 ++++++++++ docs/development.md | 15 +-- docs/internal/adapters.md | 31 ++++++ docs/internal/overview.md | 48 ++++++++++ docs/internal/sources.md | 12 +++ docs/policy/architecture.md | 96 ++++++------------- 6 files changed, 176 insertions(+), 75 deletions(-) create mode 100644 docs/adr/0001-adopt-canonical-documentation-ownership.md create mode 100644 docs/internal/overview.md diff --git a/docs/adr/0001-adopt-canonical-documentation-ownership.md b/docs/adr/0001-adopt-canonical-documentation-ownership.md new file mode 100644 index 0000000..e2a11e4 --- /dev/null +++ b/docs/adr/0001-adopt-canonical-documentation-ownership.md @@ -0,0 +1,49 @@ +# ADR 0001: Adopt Canonical Documentation Ownership + +## Status + +Accepted + +## Date + +2026-07-26 + +## Context + +Scriptorium's documentation grew alongside its CLI, HTTP, public Go, and +integration interfaces. As a result, several documents repeated mutable +contracts such as flags, configuration fields, and status behavior. Those +parallel definitions made it unclear which document to update when behavior +changed and increased the risk of documentation drift. + +## Decision + +Assign each documentation topic one canonical owner, as defined in +[`docs/policy/documentation.md`](../policy/documentation.md). Non-owning +documents may provide short orientation and links, but do not redefine volatile +contracts. Current behavior is documented outside `docs/roadmap/`; roadmaps own +future work, sequencing, and implementation status. + +## Alternatives Considered + +- Keep broad reference material in several audience-specific documents. This + would preserve local convenience but leave conflicting contract definitions + likely. +- Consolidate all documentation into one reference. This would reduce duplicate + text but would not serve the distinct needs of users, operators, consumers, + and contributors. + +## Rationale + +Canonical ownership retains audience-specific guidance while making the source +of truth for each contract discoverable. It also makes documentation changes +reviewable alongside the implementation change that requires them. + +## Consequences + +- Changes to behavior must update the canonical owner in the same change. +- Cross-cutting documentation links to the owner instead of copying its + details. +- Documentation restructuring follows the implementation sequence in + [`docs/roadmap/documentation.md`](../roadmap/documentation.md); the roadmap, + not this ADR, records completion status. diff --git a/docs/development.md b/docs/development.md index 7e96c3c..9f82204 100644 --- a/docs/development.md +++ b/docs/development.md @@ -18,6 +18,8 @@ Start with: - [Architecture policy](policy/architecture.md) for system boundaries, invariants, and non-goals; +- [Internal component overview](internal/overview.md) for the current package + and component map; - [Documentation policy](policy/documentation.md) before changing documentation; - [Testing policy](policy/testing.md) before adding, rewriting, or deleting @@ -27,17 +29,18 @@ Start with: | Task | Read before changing | | --- | --- | -| Public Go package or engine behavior | [Go package consumer contract](consumers/pkg-scriptorium.md), [runner internals](internal/runner.md), [adapter internals](internal/adapters.md), and [source internals](internal/sources.md) | -| CLI commands, flags, output, or exit behavior | [CLI contract](cli.md) and [adapter internals](internal/adapters.md) | -| HTTP routes, DTOs, limits, or status mapping | [HTTP API contract](api.md), [adapter internals](internal/adapters.md), and [source internals](internal/sources.md) | -| Application configuration | [Configuration contract](config.md), [adapter internals](internal/adapters.md), and [source internals](internal/sources.md) | -| Prompt, profile, schema, or artifact loading | [Configuration contract](config.md) and [source internals](internal/sources.md) | +| Repository orientation or component responsibility | [Internal component overview](internal/overview.md) and [architecture policy](policy/architecture.md) | +| Public Go package or engine behavior | [Go package consumer contract](consumers/pkg-scriptorium.md), [internal component overview](internal/overview.md), [runner internals](internal/runner.md), [adapter internals](internal/adapters.md), and [source internals](internal/sources.md) | +| CLI commands, flags, output, or exit behavior | [CLI contract](cli.md), [internal component overview](internal/overview.md), and [adapter internals](internal/adapters.md) | +| HTTP routes, DTOs, limits, or status mapping | [HTTP API contract](api.md), [internal component overview](internal/overview.md), [adapter internals](internal/adapters.md), and [source internals](internal/sources.md) | +| Application configuration | [Configuration contract](config.md), [internal component overview](internal/overview.md), [adapter internals](internal/adapters.md), and [source internals](internal/sources.md) | +| Prompt, profile, schema, or artifact loading | [Configuration contract](config.md), [internal component overview](internal/overview.md), and [source internals](internal/sources.md) | | Runner orchestration, rendering, validation, or repair | [Runner internals](internal/runner.md) and [source internals](internal/sources.md) | | OpenAI-compatible request or response behavior | [OpenAI-compatible integration](integrations/openai-compatible-chat.md), [runner internals](internal/runner.md), and [adapter internals](internal/adapters.md) | | Subprocess behavior | [Subprocess integration](integrations/subprocess.md) and [CLI contract](cli.md) | | Runtime operation, recovery, or troubleshooting | [Operations](operations.md) and [troubleshooting](troubleshooting.md) | | Examples or copyable assets | The owning contract for the demonstrated behavior and the related files under `examples/` | -| Architecture decisions or future work | The [documentation policy](policy/documentation.md), relevant accepted ADRs under `adr/`, and relevant roadmap documents under `roadmap/` | +| Architecture decisions or future work | The [documentation policy](policy/documentation.md), relevant accepted ADRs such as [ADR 0001](adr/0001-adopt-canonical-documentation-ownership.md), and relevant roadmap documents under `roadmap/` | For cross-cutting changes, follow every applicable row. Internal component documents own detailed subsystem change recipes. diff --git a/docs/internal/adapters.md b/docs/internal/adapters.md index 9d3ab67..09d1a4b 100644 --- a/docs/internal/adapters.md +++ b/docs/internal/adapters.md @@ -145,6 +145,37 @@ Adapters do not add durable run state. - `internal/format/prepared_run_test.go` - `internal/llm/openai_compatible_client_test.go` +## Change Recipes + +### Application Configuration Fields + +1. Add the field to the relevant `internal/config` shape and default handling. +2. Parse and validate it, then preserve configuration and CLI-override + precedence while wiring it through its consuming adapter. +3. Add focused configuration and adapter tests for parsing, mapping, and + effective behavior. +4. Update the [configuration contract](../config.md) and any external contract + affected by the new behavior. + +### CLI Flags + +1. Add the flag to the relevant command in `internal/adapter/cli/run.go`. +2. Keep command scope and application-configuration precedence intentional. +3. Add or update parser and command tests in + `internal/adapter/cli/run_test.go`. +4. Update the [CLI contract](../cli.md) and any maintained examples affected by + the invocation. + +### Adapter Capabilities + +1. Define or reuse the appropriate domain or use-case interface boundary. +2. Implement translation and IO behavior in the adapter without moving + use-case decisions out of `internal/usecase`. +3. Add focused mapping, parsing, and error-behavior tests. +4. Update this document and the affected canonical public or integration + contract. Update [source internals](sources.md) when source-loading behavior + changes. + ## Architectural Invariants - Adapter packages stay thin and translation-focused. diff --git a/docs/internal/overview.md b/docs/internal/overview.md new file mode 100644 index 0000000..0dd890b --- /dev/null +++ b/docs/internal/overview.md @@ -0,0 +1,48 @@ +# Internal Component Overview + +## Purpose + +This is the inventory of Scriptorium's implemented components for contributors. +The [architecture policy](../policy/architecture.md) owns normative boundaries +and invariants; public behavior belongs in the linked contracts. + +## Public And Command Entrypoints + +| Component | Implemented responsibility | References | +| --- | --- | --- | +| Root package `scriptorium` | Public Go facade that constructs the engine, exposes request/result types and options, and maps internal errors. | [Go package contract](../consumers/pkg-scriptorium.md), [adapter internals](adapters.md) | +| `cmd/scriptorium` | Process entrypoint that delegates command execution to the CLI adapter. | [CLI contract](../cli.md), [adapter internals](adapters.md) | + +## Adapters, Domain, And Use Case + +| Component | Implemented responsibility | References | +| --- | --- | --- | +| `internal/adapter/cli` | Parses CLI commands, applies application wiring, and handles process input and output. | [CLI contract](../cli.md), [adapter internals](adapters.md) | +| `internal/adapter/http` | Maps HTTP requests and responses to domain operations and maps public errors. | [HTTP API contract](../api.md), [adapter internals](adapters.md) | +| `internal/domain` | Defines core request, result, output-contract, and LLM-boundary types. | [runner internals](runner.md) | +| `internal/usecase` | Implements `Runner` preparation, execution, validation coordination, and the repairer boundary. | [runner internals](runner.md) | + +## Configuration And Sources + +| Component | Implemented responsibility | References | +| --- | --- | --- | +| `internal/config` | Loads application settings, applies defaults, and applies CLI overrides. | [configuration contract](../config.md), [adapter internals](adapters.md) | +| `internal/defaults` | Holds compile-time default values used when application settings are resolved. | [configuration contract](../config.md) | +| `internal/promptdef` | Loads prompt definitions from filesystem and `fs.FS` sources. | [configuration contract](../config.md), [source internals](sources.md) | +| `internal/profile` | Loads filesystem and `fs.FS` execution profiles and combines profile repositories. | [configuration contract](../config.md), [source internals](sources.md) | +| `internal/profile/builtin` | Provides embedded built-in execution profiles as a repository. | [configuration contract](../config.md), [source internals](sources.md) | +| `internal/filecatalog` | Provides shared YAML discovery and source-root helpers. | [source internals](sources.md) | +| `internal/artifact` | Reads inline and file-backed input artifacts. | [configuration contract](../config.md), [HTTP API contract](../api.md), [source internals](sources.md) | +| `internal/prompt` | Renders prompt templates into messages. | [runner internals](runner.md) | + +## Formatting, Validation, And Model Access + +| Component | Implemented responsibility | References | +| --- | --- | --- | +| `internal/format` | Formats prepared-run information for CLI output. | [CLI contract](../cli.md), [adapter internals](adapters.md) | +| `internal/validate` | Defines validation interfaces and provides standard filesystem and `fs.FS` schema validation. | [configuration contract](../config.md), [source internals](sources.md), [runner internals](runner.md) | +| `internal/llm` | Defines the provider-neutral LLM client boundary and its OpenAI-compatible implementation. | [OpenAI-compatible integration](../integrations/openai-compatible-chat.md), [runner internals](runner.md) | + +Focused internal documents describe the components that have detailed +orchestration, adapter, or source behavior. Package tests live alongside the +implementation and are identified in those focused documents where relevant. diff --git a/docs/internal/sources.md b/docs/internal/sources.md index 567492b..2875c6a 100644 --- a/docs/internal/sources.md +++ b/docs/internal/sources.md @@ -147,6 +147,18 @@ Source packages do not persist run state. - `internal/usecase/integration_test.go` - `engine_test.go` +## Change Recipe + +When updating prompt, profile, schema, or built-in-profile assets: + +1. Keep files valid for their strict loader and source boundary. +2. Keep maintained examples and fixtures secret-free. +3. Run focused prompt, profile, schema, or validation tests for the changed + source. +4. Update the [configuration contract](../config.md) and every affected + external contract; update this document when loading or precedence mechanics + change. + ## Architectural Invariants - Prompt/profile identity comes from YAML `id`. diff --git a/docs/policy/architecture.md b/docs/policy/architecture.md index 4a8ccbf..baabeaf 100644 --- a/docs/policy/architecture.md +++ b/docs/policy/architecture.md @@ -4,14 +4,12 @@ This document is the development architecture policy for Scriptorium. It is for developers and LLM coding agents. User-facing behavior belongs in `README.md` and the docs under `docs/` that target operators/users. -## Project Shape +## System Shape -Scriptorium is a narrow prompt-execution application with three entry paths: - -- CLI `run` -- CLI `render` -- HTTP `POST /v1/runs` through `serve` -- public Go package `gitea.maximumdirect.net/eric/scriptorium` +Scriptorium is a narrow prompt-execution application with three executable +entry paths: CLI `run`, CLI `render`, and the HTTP service started by `serve`. +It also provides a public Go package for in-process use. Its current component +inventory is maintained in the [internal overview](../internal/overview.md). Domain behavior is centralized in `internal/usecase` and `internal/domain`. @@ -23,45 +21,17 @@ Domain behavior is centralized in `internal/usecase` and `internal/domain`. - Keep config strict: YAML/JSON decoding for external inputs should reject unknown fields. - Keep secrets out of payloads: raw API key values must not be accepted or emitted. -## Package Boundaries +## Dependency Direction -Current package map: - -- root package `scriptorium`: public Go facade over engine construction, source options, request/result types, and error mapping. -- `cmd/scriptorium`: process entrypoint. -- `internal/adapter/cli`: command parsing, app wiring for CLI commands, output behavior. -- `internal/adapter/http`: HTTP DTO mapping and error/status mapping. -- `internal/config`: application settings loading and CLI override precedence. -- `internal/defaults`: compile-time default constants. -- `internal/domain`: core request/result and contract types. -- `internal/usecase`: `Runner` prepare/run orchestration and repair-hook boundary. -- `internal/promptdef`: filesystem prompt-definition repository. -- `internal/profile`: filesystem, `fs.FS`, and overlay execution-profile repositories. -- `internal/profile/builtin`: embedded built-in execution profiles. -- `internal/filecatalog`: shared YAML discovery and `fs.FS` source helpers. -- `internal/artifact`: artifact reference readers. -- `internal/prompt`: template renderer. -- `internal/llm`: provider-neutral LLM client interface and OpenAI-compatible implementation. -- `internal/validate`: validator interfaces and standard implementation. -- `internal/format`: prepared-run output formatting. - -Detailed component behavior is documented in: - -- `docs/internal/runner.md` -- `docs/internal/adapters.md` -- `docs/internal/sources.md` - -## Configuration And Precedence - -Application settings are resolved as: - -1. built-in defaults -2. config file values -3. CLI overrides - -`config.yml` is for application wiring (directories, server address, render default format), not prompt/profile runtime execution settings. - -Profile selection and runtime model resolution remain use-case concerns. +- Adapters translate external shapes and IO concerns; they do not make + use-case decisions. +- Use-case and domain code depend on explicit repository, renderer, validator, + and LLM interfaces rather than adapter implementations. +- Source, rendering, validation, and LLM implementations remain behind their + package boundaries. +- Dependency-specific types must not leak across unrelated package boundaries. +- Prefer the standard library; add an external dependency only when it + materially reduces risk or complexity. ## State And Persistence Policy @@ -70,44 +40,32 @@ Scriptorium has no durable run-state store. - No built-in resume/checkpoint/archive behavior. - Recovery model is rerun after correcting inputs/config/environment. -## External Integration Policy +## Contract Ownership -Current external contracts: - -- inbound HTTP contract: `POST /v1/runs`, documented canonically in `docs/api.md` -- outbound model contract: OpenAI-compatible chat completions subset -- subprocess contract for integrators: CLI `run`/`render` -- public Go package contract: `docs/consumers/pkg-scriptorium.md` - -Integration docs belong under `docs/integrations/`. +The [CLI](../cli.md), [configuration](../config.md), [HTTP API](../api.md), +[public Go package](../consumers/pkg-scriptorium.md), and +[integration](../integrations/) documents own their respective external +contracts. This policy keeps only the architectural boundaries that govern +their implementation. ## Error Handling And Logging - Wrap errors with domain/operation context. - Map domain errors to adapter-appropriate statuses/codes without leaking sensitive internals. -- Keep stderr summaries concise for CLI success/error paths. - Never emit raw secret values. -## Testing Expectations +## Testing And Documentation -- Core runner behavior should be covered with isolated unit tests and fixture-based integration tests. -- Adapter behavior should be tested for parse/mapping/error semantics. -- Config parsing, prompt/profile loading, validator behavior, and LLM client error handling should remain covered by package tests. -- Repository-level docs/examples that claim runnable behavior should be validated by tests or smoke commands. - -## Documentation Expectations - -- Document implemented behavior only outside `docs/roadmap/`. -- Keep canonical reference locations stable (`docs/cli.md`, `docs/config.md`, `docs/operations.md`, `docs/troubleshooting.md`, `docs/internal/`). -- Update docs in the same change when architecture-relevant behavior changes. +Testing philosophy and change-validation expectations are defined by the +[testing policy](testing.md). Documentation ownership and maintenance rules are +defined by the [documentation policy](documentation.md). ## Architectural Invariants - `Runner.Run` reuses `Runner.Prepare` flow. -- CLI and HTTP currently instantiate `Runner` without a repairer. -- Artifact reading supports `inline` and `file` references. - Unknown input fields in config/prompt/profile/http JSON should be rejected by strict decoding. -- Raw API key values must not be accepted through config/HTTP payloads. +- Raw API key values must not be accepted through external configuration or + request payloads, and resolved secret values must not be emitted. ## Non-Goals