Completed the documentation update and removed the completed roadmap
This commit is contained in:
@@ -1,355 +0,0 @@
|
||||
# Documentation Roadmap
|
||||
|
||||
## Purpose
|
||||
|
||||
This roadmap defines the work required to bring scriptorium's documentation into compliance with `docs/policy/documentation.md` and the current implementation. It is a planning document only; future agents should use it to update the canonical documentation without describing unimplemented behavior outside `docs/roadmap/`.
|
||||
|
||||
## Repository Documentation Inventory
|
||||
|
||||
- `README.md` - keep and rewrite. It is currently a full manual covering config, CLI, HTTP, prompt/profile authoring, examples, and build commands; policy says README should be a short orientation page with a quickstart and links.
|
||||
- `architecture.md` - move/merge/delete after rewrite. It overlaps with `docs/policy/architecture.md`, contains "should" guidance and future extension notes outside `docs/roadmap/`, and references architecture that is partly stale or aspirational.
|
||||
- `docs/policy/documentation.md` - keep and lightly update only if policy itself changes. It is the controlling documentation policy.
|
||||
- `docs/policy/architecture.md` - keep and lightly update. It is the canonical development architecture policy, but some package-layout defaults do not exactly match this repository (`internal/adapter/...` versus policy examples such as `internal/adapters/...`).
|
||||
- `docs/policy/development.md` - create new. Required by policy for projects maintained by humans and LLM coding agents.
|
||||
- `docs/config/config-yml.md` - merge into `docs/config.md`. The content mostly matches code but lives in the wrong canonical home.
|
||||
- `docs/config/prompt-definitions.md` - merge into `docs/config.md`. The field reference is mostly accurate, but it needs caveats about `repair_attempts`, repeated message roles, schema path resolution, and examples.
|
||||
- `docs/config/profile-definitions.md` - merge into `docs/config.md`. It must stop implying that every profile field is sent to the provider; the current OpenAI-compatible client does not send `reasoning_effort` or `extra_params`.
|
||||
- `docs/config/schema-definitions.md` - merge into `docs/config.md` or link from it. Schema behavior is implemented, but the canonical config reference should own this material.
|
||||
- `docs/cli.md` - create new. Required by policy for the implemented CLI.
|
||||
- `docs/operations.md` - create new. Required by policy for this CLI/service application; scope should cover normal operation, config paths, secret handling, stdout/stderr, exit codes, HTTP serving, and the fact that there is no durable run state/resume behavior.
|
||||
- `docs/troubleshooting.md` - create new. Recommended by policy and justified by implemented failure modes in parser, config, prompt/profile loading, artifact reading, validation, LLM calls, and HTTP error mapping.
|
||||
- `docs/internal/` - create new. Required by policy for this modular application.
|
||||
- `docs/integrations/narratio.md` - keep and rewrite. It documents an actual CLI integration contract, but it includes future extension notes outside roadmap and illustrative prompt IDs that are not all present in the repository.
|
||||
- `docs/integrations/http-api.md` - create new. The implemented `POST /v1/runs` API is an external integration contract and should not live in README.
|
||||
- `docs/integrations/openai-compatible-chat.md` - create new. The outbound LLM contract is important and implemented in `internal/llm/openai_compatible_client.go`.
|
||||
- `examples/config.yml` - keep and lightly update if paths move. It is a valid app config example for the current root `prompts/`, `profiles/`, and `schemas/` directories.
|
||||
- `examples/fixtures/transcript.md` - keep. It is used by integration tests.
|
||||
- `examples/fixtures/glossary.yml` - keep. It is used by integration tests.
|
||||
- `prompts/` - keep as maintained sample prompt library for now; recommended to move or mirror under `examples/` only if tests and docs are updated together.
|
||||
- `profiles/` - keep as maintained sample profile library for now; recommended to move or mirror under `examples/` only if tests and docs are updated together.
|
||||
- `schemas/` - keep as maintained sample schema library for now; recommended to move or mirror under `examples/` only if tests and docs are updated together.
|
||||
- `local-test/` - delete, move out of the repository, or explicitly exclude from maintained docs. It contains ad hoc local artifacts and provider profiles; it should not be linked from canonical docs unless promoted to maintained examples with tests and secret review.
|
||||
|
||||
## Policy Compliance Assessment
|
||||
|
||||
Required documents that are missing:
|
||||
|
||||
- `docs/cli.md`
|
||||
- `docs/config.md`
|
||||
- `docs/operations.md`
|
||||
- `docs/internal/`
|
||||
- `docs/policy/development.md`
|
||||
|
||||
Recommended documents that should be added:
|
||||
|
||||
- `docs/troubleshooting.md`
|
||||
- Validated examples under `examples/` beyond the current config and fixtures, especially command examples that can be checked with `render`.
|
||||
- Integration docs for the implemented HTTP API and outbound OpenAI-compatible chat API.
|
||||
|
||||
Documents that exist but are stale or in the wrong canonical home:
|
||||
|
||||
- `README.md` duplicates material that belongs in `docs/cli.md`, `docs/config.md`, `docs/integrations/`, and `docs/internal/`.
|
||||
- `docs/config/*.md` should be merged into `docs/config.md`.
|
||||
- `architecture.md` should be merged into `docs/policy/architecture.md`, `docs/internal/`, or `docs/roadmap/`, then removed.
|
||||
- `docs/integrations/narratio.md` should remain under integrations but must be narrowed to implemented CLI behavior and actual integration guidance.
|
||||
|
||||
Content that appears to describe deprecated, historical, planned, or unimplemented behavior outside `docs/roadmap/`:
|
||||
|
||||
- `architecture.md` has future extension notes for S3 artifact references, additional LLM providers, streaming, batch execution, database-backed repositories, profile versioning, and HTTP render endpoints.
|
||||
- `architecture.md` and `README.md` describe bounded repair as if it is generally active. The code has an injected repairer hook, but the CLI and HTTP server construct `Runner` without a repairer, so production commands do not currently perform repair attempts.
|
||||
- `README.md` says "additional output formats can be added later"; this belongs in roadmap only.
|
||||
- `README.md` references `go build -o scriptorium ./cmd/scriptorium`, which is valid, but the README should not be the build/test manual after `docs/policy/development.md` exists.
|
||||
- `docs/integrations/narratio.md` has "Future Extension Notes" and examples for prompt IDs not present in the repository, such as `dnd.structured_events`, `dnd.glossary_suggestions`, and `dnd.player_summary`.
|
||||
- Any docs implying S3 artifact support should be removed from current-behavior docs. `domain.ArtifactRefS3` exists, but `artifact.CompositeReader` supports only `inline` and `file`.
|
||||
|
||||
Examples that are missing, stale, invalid, or untested:
|
||||
|
||||
- `examples/config.yml` points at root `prompts/`, `profiles/`, and `schemas/`; it is valid for repository-root execution but should be tested or explicitly checked.
|
||||
- There are no copyable CLI example scripts or expected-output files under `examples/`.
|
||||
- The maintained prompt/profile/schema examples live outside `examples/`; this is usable, but policy prefers copyable examples under `examples/`.
|
||||
- `local-test/` appears unmaintained and should not be treated as documentation.
|
||||
|
||||
Links that are likely stale or need verification:
|
||||
|
||||
- Existing links from README to docs should be rewritten after canonical files are created.
|
||||
- Any references to `docs/config/config-yml.md`, `docs/config/prompt-definitions.md`, `docs/config/profile-definitions.md`, or `docs/config/schema-definitions.md` should be updated after those files are merged.
|
||||
- References to default config paths must use the implemented search order: `/usr/local/etc/scriptorium/config.yml`, then `/etc/scriptorium/config.yml`.
|
||||
|
||||
## Target Documentation Set
|
||||
|
||||
### `README.md`
|
||||
|
||||
- Audience: users, administrators, operators.
|
||||
- Purpose: concise orientation and shortest useful command.
|
||||
- Canonical scope: project purpose, elevator pitch, quickstart, and links.
|
||||
- Recommended outline: description; why scriptorium exists; shortest useful `scriptorium render` or `scriptorium run` example; documentation links.
|
||||
- Source of truth: `cmd/scriptorium/main.go`, `internal/adapter/cli/run.go`, `examples/config.yml`, `prompts/generic.markdown_summary.yaml`.
|
||||
- Acceptance criteria: no full flag reference; no HTTP schema; no prompt/profile field tables; no future work; all links point to existing target docs.
|
||||
|
||||
### `docs/cli.md`
|
||||
|
||||
- Audience: users, administrators, operators.
|
||||
- Purpose: complete CLI reference and common workflows.
|
||||
- Canonical scope: commands, flags, outputs, exit codes, and command examples.
|
||||
- Recommended outline: shortest useful command; command overview; `run`; `render`; `serve`; flag reference; input and variable mapping syntax; output behavior; exit codes; common workflows.
|
||||
- Source of truth: `internal/adapter/cli/run.go`, `internal/adapter/cli/run_test.go`, `internal/format/prepared_run.go`, `cmd/scriptorium/main.go`.
|
||||
- Acceptance criteria: documents real flags only; notes deprecated aliases `--prompt-id` and `--profile-id`; documents that `render` does not currently accept `--schema-dir`; documents stdout/stderr split and exit code `2` for validation failure.
|
||||
|
||||
### `docs/config.md`
|
||||
|
||||
- Audience: administrators, operators, advanced users.
|
||||
- Purpose: canonical reference for app config, prompt definitions, profiles, and schemas.
|
||||
- Canonical scope: all implemented YAML/JSON file formats and precedence rules.
|
||||
- Recommended outline: config file discovery and precedence; minimal app config; production-oriented app config; app config reference; prompt definition reference; profile definition reference; schema behavior; secrets handling; maintained examples.
|
||||
- Source of truth: `internal/config/config.go`, `internal/defaults/defaults.go`, `internal/promptdef/filesystem_repository.go`, `internal/profile/filesystem_repository.go`, `internal/validate/standard_validator.go`, repository `prompts/`, `profiles/`, `schemas/`, and config/profile/prompt tests.
|
||||
- Acceptance criteria: replaces split `docs/config/*.md`; documents strict YAML decoding; documents raw API key rejection; documents `schema_dir` default `.`; documents prompt `content_file` relative to prompt YAML; states `content_type` is metadata only; does not claim operational repair unless a repairer is configured.
|
||||
|
||||
### `docs/operations.md`
|
||||
|
||||
- Audience: administrators, operators.
|
||||
- Purpose: operational use of CLI and HTTP service.
|
||||
- Canonical scope: normal workflow, filesystem expectations, config deployment, secrets, logs/output, validation behavior, and recovery from failed runs.
|
||||
- Recommended outline: normal run/render workflow; config and library directories; environment variables for API keys; serving HTTP; output and stderr summaries; validation failure handling; no durable state/resume/archive behavior; safe recovery steps.
|
||||
- Source of truth: `internal/adapter/cli/run.go`, `internal/adapter/http/handler.go`, `internal/config/config.go`, `internal/llm/openai_compatible_client.go`, `internal/usecase/runner.go`.
|
||||
- Acceptance criteria: makes clear scriptorium does not persist run state; does not invent cleanup/archive/resume; documents that HTTP has no built-in authentication and should be deployed behind trusted controls.
|
||||
|
||||
### `docs/troubleshooting.md`
|
||||
|
||||
- Audience: administrators, operators.
|
||||
- Purpose: safe diagnosis and fixes for recurring implemented failure modes.
|
||||
- Canonical scope: symptoms, likely causes, diagnostics, safe fixes, and links.
|
||||
- Recommended outline: missing config; missing prompt/profile dirs; unknown flags; prompt/profile load failures; missing input files; template render failures; missing API-key environment values; LLM non-2xx/malformed responses; JSON/schema validation failures; HTTP error codes.
|
||||
- Source of truth: `internal/adapter/cli/run_test.go`, `internal/adapter/http/handler_test.go`, `internal/config/config_test.go`, `internal/promptdef/repository_test.go`, `internal/profile/repository_test.go`, `internal/validate/standard_validator_test.go`, `internal/llm/openai_compatible_client_test.go`.
|
||||
- Acceptance criteria: every entry includes symptom, likely cause, diagnostic step, safe fix, and links to canonical CLI/config/operations docs.
|
||||
|
||||
### `docs/policy/architecture.md`
|
||||
|
||||
- Audience: developers, LLM coding agents.
|
||||
- Purpose: controlling development architecture and invariants.
|
||||
- Canonical scope: development principles, boundaries, invariants, non-goals.
|
||||
- Recommended outline: keep current policy shape; add scriptorium-specific package map or link to `docs/internal/`; clarify no orchestration creep; clarify current adapters.
|
||||
- Source of truth: existing policy, `internal/` package layout, `architecture.md`.
|
||||
- Acceptance criteria: remains policy-oriented; does not become user docs; future work stays in roadmap; no stale package names.
|
||||
|
||||
### `docs/policy/development.md`
|
||||
|
||||
- Audience: developers, LLM coding agents.
|
||||
- Purpose: contributor workflow and change checklist.
|
||||
- Canonical scope: repository layout, build/test commands, coding conventions, dependency policy, adding config/CLI/adapters, updating examples/docs.
|
||||
- Recommended outline: repository layout; common commands; coding conventions; dependency policy; how to add config fields; how to add CLI flags; how to add adapters; how to update examples; documentation expectations.
|
||||
- Source of truth: `go.mod`, `cmd/scriptorium/main.go`, `internal/adapter/cli/run.go`, `internal/config/config.go`, `docs/policy/architecture.md`, existing tests.
|
||||
- Acceptance criteria: includes `go test ./...`; references `go build ./cmd/scriptorium`; tells contributors to update docs and tests with behavior changes.
|
||||
|
||||
### `docs/internal/runner.md`
|
||||
|
||||
- Audience: developers, LLM coding agents.
|
||||
- Purpose: implemented core prepare/run behavior.
|
||||
- Canonical scope: `Runner.Prepare`, `Runner.Run`, profile selection, runtime merge, artifact loading, rendering, structured output setup, validation, repair hook boundary.
|
||||
- Recommended outline: purpose; inputs/outputs; prepare flow; run flow; boundary contracts; failure behavior; tests; invariants.
|
||||
- Source of truth: `internal/usecase/runner.go`, `internal/usecase/repairer.go`, `internal/usecase/runner_test.go`, `internal/usecase/integration_test.go`.
|
||||
- Acceptance criteria: states CLI/HTTP currently construct `Runner` without a repairer; documents validation content failures versus runtime validation errors; no provider-specific details except through ports.
|
||||
|
||||
### `docs/internal/adapters.md`
|
||||
|
||||
- Audience: developers, LLM coding agents.
|
||||
- Purpose: implemented adapter boundaries.
|
||||
- Canonical scope: CLI adapter, HTTP adapter, filesystem repositories, artifact reader, prompt renderer, OpenAI-compatible LLM client, validator, prepared-run formatter.
|
||||
- Recommended outline: adapter map; inputs/outputs; config fields used; external dependencies; failure behavior; tests to inspect.
|
||||
- Source of truth: `internal/adapter/cli`, `internal/adapter/http`, `internal/promptdef`, `internal/profile`, `internal/artifact`, `internal/prompt`, `internal/llm`, `internal/validate`, `internal/format`.
|
||||
- Acceptance criteria: documents only implemented adapters; states `inline` and `file` artifact refs are supported and S3 is not; states OpenAI request fields actually sent.
|
||||
|
||||
### `docs/integrations/http-api.md`
|
||||
|
||||
- Audience: developers, LLM coding agents, API clients.
|
||||
- Purpose: implemented inbound HTTP contract.
|
||||
- Canonical scope: `POST /v1/runs`, request/response shape, raw output opt-in, error mapping, validation-failed status behavior.
|
||||
- Recommended outline: scope; endpoint; request fields; response fields; error responses; validation behavior; security/auth note.
|
||||
- Source of truth: `internal/adapter/http/dto.go`, `internal/adapter/http/handler.go`, `internal/adapter/http/handler_test.go`.
|
||||
- Acceptance criteria: no unimplemented render endpoint; no built-in auth claim; unknown JSON fields rejected; raw API key fields rejected by strict JSON.
|
||||
|
||||
### `docs/integrations/openai-compatible-chat.md`
|
||||
|
||||
- Audience: developers, LLM operators, LLM adapter maintainers.
|
||||
- Purpose: implemented outbound LLM API contract.
|
||||
- Canonical scope: OpenAI-compatible chat completions request/response subset and provider-level structured output behavior.
|
||||
- Recommended outline: endpoint construction; request fields sent; auth header from `api_key_env`; timeout behavior; response expectations; error handling; unsupported profile fields.
|
||||
- Source of truth: `internal/llm/openai_compatible_client.go`, `internal/llm/openai_compatible_client_test.go`, `internal/usecase/runner.go`.
|
||||
- Acceptance criteria: says endpoint appends `/chat/completions`; says empty first choice content is malformed; says `reasoning_effort` and `extra_params` are not currently serialized into the outbound request.
|
||||
|
||||
### `docs/integrations/narratio.md`
|
||||
|
||||
- Audience: developers, LLM coding agents maintaining Narratio integration.
|
||||
- Purpose: CLI subprocess contract for Narratio.
|
||||
- Canonical scope: how Narratio should call implemented `scriptorium run` and `scriptorium render`.
|
||||
- Recommended outline: purpose; assumptions; command shapes; inputs/vars; profile selection; runtime overrides; config behavior; environment handling; output handling; exit statuses; security notes; non-goals.
|
||||
- Source of truth: `internal/adapter/cli/run.go`, `internal/adapter/cli/run_test.go`, `docs/cli.md`, `docs/config.md`.
|
||||
- Acceptance criteria: removes future extensions; labels any Narratio-specific prompt IDs as external examples only or removes them; links to canonical CLI/config docs.
|
||||
|
||||
## File-by-File Rewrite Guidance
|
||||
|
||||
- `README.md`: cover what scriptorium does and show one minimal command. Avoid field tables, complete flag lists, HTTP schema, internal package details, future extensions, and long examples. Link to `docs/cli.md`, `docs/config.md`, `docs/operations.md`, `docs/troubleshooting.md`, and `docs/integrations/`.
|
||||
- `docs/cli.md`: cover `run`, `render`, `serve`, flags, output behavior, and exit codes. Avoid duplicating prompt/profile YAML field references; link to `docs/config.md`. Inspect CLI parser tests before writing examples. Do not carry forward README's claim that render supports `--schema-dir`.
|
||||
- `docs/config.md`: cover app config, prompt YAML, profile YAML, and schema behavior. Avoid provider API details except where needed for profile fields; link to OpenAI integration doc. Do not carry forward claims that `repair_attempts` enables repair for normal CLI/HTTP runs unless code later wires a repairer.
|
||||
- `docs/operations.md`: cover deployed operation and recovery boundaries. Avoid inventing durable state, resume behavior, cleanup, backups, or archives. State that rerunning a command is the recovery model.
|
||||
- `docs/troubleshooting.md`: use tested errors and behavior. Avoid exposing internal wrapped error details that HTTP intentionally suppresses. Link to CLI/config/operations instead of repeating full references.
|
||||
- `docs/policy/architecture.md`: preserve policy authority and update only scriptorium-specific facts. Avoid copying the long historical `architecture.md` wholesale. Move future extension ideas to roadmap docs only.
|
||||
- `docs/policy/development.md`: cover contributor mechanics and how to update behavior safely. Avoid user-facing manuals. Include tests and docs update expectations.
|
||||
- `docs/internal/runner.md`: explain implemented core flow and invariants. Avoid CLI flag tables and HTTP DTO detail; link to adapter docs.
|
||||
- `docs/internal/adapters.md`: explain implemented adapter boundaries and tests. Avoid proposing new adapters. Do not imply `ArtifactRefS3` works.
|
||||
- `docs/integrations/http-api.md`: document only `POST /v1/runs`. Avoid documenting a render/prepare HTTP endpoint.
|
||||
- `docs/integrations/openai-compatible-chat.md`: document the outbound request subset. Avoid documenting unsupported OpenAI fields or provider-specific options unless code sends them.
|
||||
- `docs/integrations/narratio.md`: keep it as a subprocess contract. Avoid future work, S3, HTTP-as-primary-path, and undeployed prompt IDs as current examples.
|
||||
- `architecture.md`: after target docs exist, delete it or replace it with a short pointer to `docs/policy/architecture.md` and `docs/internal/`. Do not leave future notes in this root file.
|
||||
- `docs/config/*.md`: after `docs/config.md` exists and links are updated, delete these split files or replace them with pointers only if backwards-compatible links are necessary.
|
||||
|
||||
## Examples Plan
|
||||
|
||||
Existing maintained examples:
|
||||
|
||||
- `examples/config.yml`: minimal app config pointing at root prompt/profile/schema libraries. Validity check: `go test ./internal/config ./internal/adapter/cli` and `go run ./cmd/scriptorium render --config ./examples/config.yml --prompt generic.markdown_summary --input transcript=./examples/fixtures/transcript.md --format json`. Link from README, `docs/config.md`, and `docs/cli.md`.
|
||||
- `examples/fixtures/transcript.md`: sample transcript input. Validity check: used by `internal/usecase/integration_test.go` and render smoke command. Link from README and `docs/cli.md`.
|
||||
- `examples/fixtures/glossary.yml`: sample optional glossary input. Validity check: used by `internal/usecase/integration_test.go`. Link from `docs/config.md` and examples section in `docs/cli.md`.
|
||||
- `prompts/generic.markdown_summary.yaml`: sample markdown prompt. Validity check: render smoke command. Link from config docs until or unless it is moved under `examples/`.
|
||||
- `prompts/generic.structured_events.yaml` plus `schemas/structured_events.schema.json`: sample JSON-schema prompt. Validity check: `go test ./internal/usecase`. Link from `docs/config.md`.
|
||||
- `profiles/local-fast.yaml` and `profiles/local-quality.yaml`: sample profiles. Validity check: profile repository tests plus integration test. Link from `docs/config.md`, with a note that `local-quality` requires `SCRIPTORIUM_API_KEY` because it sets `api_key_env`.
|
||||
|
||||
Recommended example additions, all based on implemented behavior:
|
||||
|
||||
- `examples/render-markdown-summary.sh`: copyable render smoke command using `generic.markdown_summary`. Expected check: run script or equivalent `go run` command exits `0`. Link from README and `docs/cli.md`.
|
||||
- `examples/http-run.json`: copyable `POST /v1/runs` request body using `inline` or `file` artifact refs. Expected check: parse as JSON and keep aligned with `internal/adapter/http/dto.go`. Link from `docs/integrations/http-api.md`.
|
||||
- `examples/prompts/`, `examples/profiles/`, `examples/schemas/`: optional future move or mirror of maintained sample libraries. Expected check: update integration tests and `examples/config.yml` together. This is recommended for policy alignment but should be done as its own implementation stage to avoid breaking tests.
|
||||
|
||||
Do not document `local-test/` as maintained examples.
|
||||
|
||||
## Internal Documentation Plan
|
||||
|
||||
### Core runner
|
||||
|
||||
- Path: `docs/internal/runner.md`
|
||||
- Purpose: explain implemented prepare/run lifecycle.
|
||||
- Inputs and outputs: `domain.RunRequest`, `domain.PreparedRun`, `domain.RunResult`, `domain.GenerateRequest`.
|
||||
- Boundaries: usecase owns profile selection, runtime merge, artifact resolution orchestration, rendering orchestration, structured output setup, validation, and run metadata; adapters own transport/config parsing.
|
||||
- Config fields used: none directly; adapters pass resolved repositories, validators, and request values.
|
||||
- Adapters used: promptdef repository, profile repository, artifact reader, prompt renderer, LLM client, validator, optional injected repairer.
|
||||
- Failure behavior: invalid request, prompt/profile load, artifact load, render failure, LLM failure, validation runtime failure; validation content failures return a result.
|
||||
- Tests to inspect before changing: `internal/usecase/runner_test.go`, `internal/usecase/integration_test.go`.
|
||||
- Architectural invariants: `Run` reuses `Prepare`; no resolved API key values in prepared data; repair attempts bounded and only possible when a repairer is injected; no orchestration creep.
|
||||
|
||||
### Adapters and repositories
|
||||
|
||||
- Path: `docs/internal/adapters.md`
|
||||
- Purpose: explain implemented external boundaries.
|
||||
- Inputs and outputs: CLI args/stdout/stderr/exit codes; HTTP JSON DTOs; YAML prompt/profile/config files; file/inline artifacts; OpenAI-compatible HTTP requests; prepared-run text/JSON output.
|
||||
- Boundaries: adapters translate external forms into domain requests/results and must not own domain decisions.
|
||||
- Config fields used: `prompt_dir`, `profile_dir`, `schema_dir`, `server.addr`, `defaults.render_format`; profile `endpoint`, `model`, generation fields, timeout, `api_key_env`.
|
||||
- Adapters used: CLI, HTTP, filesystem repositories, artifact reader, Go template renderer, OpenAI-compatible client, standard validator, prepared-run formatter.
|
||||
- Failure behavior: strict YAML/JSON decoding, unknown fields rejected, unsupported artifact refs rejected, LLM non-2xx/malformed responses become errors.
|
||||
- Tests to inspect before changing: adapter, repository, artifact, renderer, LLM, validator, and formatter tests under `internal/**`.
|
||||
- Architectural invariants: no raw API keys; no S3 docs until reader exists; OpenAI client sends only implemented request fields.
|
||||
|
||||
### Validation and structured output
|
||||
|
||||
- Path: include in `docs/internal/runner.md` or create `docs/internal/validation.md` if the section grows.
|
||||
- Purpose: explain `none`, `basic`, `json`, `json_schema`, schema loading, and provider-level JSON schema request setup.
|
||||
- Inputs and outputs: `domain.Artifact`, `domain.OutputContract`, `domain.ValidationResult`, `domain.StructuredOutputSpec`.
|
||||
- Boundaries: validator checks output; runner creates provider-level structured output spec for `json_schema`; OpenAI adapter serializes `response_format`.
|
||||
- Config fields used: `schema_dir`; prompt `output.schema_path`, `output.validation_mode`, `output.format`.
|
||||
- Adapters used: standard validator and OpenAI-compatible client.
|
||||
- Failure behavior: invalid generated JSON is validation failure; missing/invalid schema file is runtime validation error before or during run preparation.
|
||||
- Tests to inspect before changing: `internal/validate/standard_validator_test.go`, `internal/usecase/runner_test.go`, `internal/llm/openai_compatible_client_test.go`.
|
||||
- Architectural invariants: schema docs must load before `json_schema` LLM request; schema paths resolve relative to `schema_dir`.
|
||||
|
||||
## Integration Documentation Plan
|
||||
|
||||
### HTTP API
|
||||
|
||||
- Path: `docs/integrations/http-api.md`
|
||||
- External system or contract: inbound HTTP clients of scriptorium.
|
||||
- Current usage in scriptorium: `scriptorium serve` exposes `POST /v1/runs`.
|
||||
- Version or compatibility notes: route is `/v1/runs`; request decoding rejects unknown JSON fields.
|
||||
- What should be documented: request fields, `file` and `inline` artifact refs, model overrides, raw output opt-in, response shape, error codes, validation-failed `200 OK`, no built-in auth.
|
||||
- What should not be documented: unimplemented render endpoint, streaming, batch, authentication middleware, remote artifact storage.
|
||||
|
||||
### OpenAI-compatible chat completions
|
||||
|
||||
- Path: `docs/integrations/openai-compatible-chat.md`
|
||||
- External system or contract: outbound OpenAI-compatible `/chat/completions` API.
|
||||
- Current usage in scriptorium: `OpenAICompatibleClient.Generate` posts chat messages and optional JSON schema response format.
|
||||
- Version or compatibility notes: compatibility is defined by the subset used in code, not by a pinned OpenAI API version.
|
||||
- What should be documented: endpoint construction, request fields, auth header behavior, timeout behavior, expected response shape, error handling, structured output payload.
|
||||
- What should not be documented: provider features not serialized by code, retries, streaming, tool calls, reasoning controls, or extra provider params.
|
||||
|
||||
### Narratio CLI subprocess
|
||||
|
||||
- Path: `docs/integrations/narratio.md`
|
||||
- External system or contract: Narratio calling scriptorium as a subprocess.
|
||||
- Current usage in scriptorium: public CLI commands `run` and `render`.
|
||||
- Version or compatibility notes: contract should be tied to implemented CLI flags and exit codes.
|
||||
- What should be documented: command construction, config use, input files, vars, profile overrides, timeout override, output paths, stdout/stderr handling, exit status semantics.
|
||||
- What should not be documented: future S3 support, future HTTP primary integration, unimplemented prompt IDs as current examples, or Narratio stage state internals.
|
||||
|
||||
JSON Schema is important but does not need a separate integration doc in the first migration; keep schema behavior in `docs/config.md` and internal validation docs unless compatibility issues require a dedicated page later.
|
||||
|
||||
## Recommended Implementation Sequence
|
||||
|
||||
### Stage 1: Canonical README, CLI, and Config
|
||||
|
||||
- Goal: make user-facing docs accurate and move reference material to canonical homes.
|
||||
- Files to create/update/delete/move: rewrite `README.md`; create `docs/cli.md`; create `docs/config.md`; leave old `docs/config/*.md` temporarily with pointers or delete them only after links are updated.
|
||||
- Repository areas to inspect: `cmd/scriptorium/main.go`, `internal/adapter/cli/run.go`, `internal/adapter/cli/run_test.go`, `internal/config`, `internal/promptdef`, `internal/profile`, `internal/validate`, `examples/config.yml`, `prompts/`, `profiles/`, `schemas/`.
|
||||
- Acceptance criteria: README is short; CLI flags match parser; config paths and precedence match code; no future work outside roadmap; no repair claims beyond implemented behavior.
|
||||
- Suggested validation commands: `go test ./...`; `go run ./cmd/scriptorium render --config ./examples/config.yml --prompt generic.markdown_summary --input transcript=./examples/fixtures/transcript.md --format json`; `rg -n "future|planned|may be added|can be added later|S3|streaming|batch" README.md docs/cli.md docs/config.md`.
|
||||
- One-prompt size: yes, if old split config files are deleted or replaced with pointers in the same change.
|
||||
|
||||
### Stage 2: Operations and Troubleshooting
|
||||
|
||||
- Goal: document operational behavior and known failure modes.
|
||||
- Files to create/update/delete/move: create `docs/operations.md`; create `docs/troubleshooting.md`; update README links.
|
||||
- Repository areas to inspect: CLI and HTTP adapters, config tests, LLM client tests, validator tests, prompt/profile repository tests.
|
||||
- Acceptance criteria: no invented state/resume/backup behavior; troubleshooting entries are actionable and link to canonical docs; HTTP no-auth caveat is clear.
|
||||
- Suggested validation commands: `go test ./internal/adapter/cli ./internal/adapter/http ./internal/config ./internal/llm ./internal/validate`; `rg -n "resume|archive|backup|cleanup|state" docs/operations.md docs/troubleshooting.md`.
|
||||
- One-prompt size: yes.
|
||||
|
||||
### Stage 3: Development Policy and Internal Docs
|
||||
|
||||
- Goal: give developers and LLM agents accurate package boundaries and invariants.
|
||||
- Files to create/update/delete/move: create `docs/policy/development.md`; update `docs/policy/architecture.md`; create `docs/internal/runner.md`; create `docs/internal/adapters.md`; optionally create `docs/internal/validation.md`.
|
||||
- Repository areas to inspect: all `internal/` packages, `go.mod`, root `architecture.md`, tests.
|
||||
- Acceptance criteria: package names match repository; current adapters only; future extension ideas absent except links to roadmap; repairer hook boundary is accurate.
|
||||
- Suggested validation commands: `go test ./...`; `rg -n "should expose|may be added|future|S3|batch|streaming|database-backed|additional providers" docs/policy docs/internal`.
|
||||
- One-prompt size: maybe split into two prompts if `docs/internal/` becomes too large.
|
||||
|
||||
### Stage 4: Integration Docs
|
||||
|
||||
- Goal: move external contracts out of README and make integration docs precise.
|
||||
- Files to create/update/delete/move: create `docs/integrations/http-api.md`; create `docs/integrations/openai-compatible-chat.md`; rewrite `docs/integrations/narratio.md`; update README and CLI/config links.
|
||||
- Repository areas to inspect: `internal/adapter/http`, `internal/llm`, `internal/usecase`, CLI tests, HTTP tests, LLM tests.
|
||||
- Acceptance criteria: HTTP docs cover only `POST /v1/runs`; OpenAI docs cover only serialized fields; Narratio docs include only implemented CLI integration and no future notes.
|
||||
- Suggested validation commands: `go test ./internal/adapter/http ./internal/llm ./internal/adapter/cli`; `rg -n "POST /v1/renders|S3|future|later|batch|streaming" docs/integrations`.
|
||||
- One-prompt size: yes.
|
||||
|
||||
### Stage 5: Examples and Link Cleanup
|
||||
|
||||
- Goal: make examples policy-compliant and verify links after moves.
|
||||
- Files to create/update/delete/move: optionally add `examples/render-markdown-summary.sh`; optionally add `examples/http-run.json`; decide whether to move or mirror `prompts/`, `profiles/`, `schemas/` under `examples/`; delete or exclude `local-test/`; remove or replace root `architecture.md`; delete obsolete `docs/config/*.md` if not already removed.
|
||||
- Repository areas to inspect: `examples/`, `prompts/`, `profiles/`, `schemas/`, `internal/usecase/integration_test.go`, docs links.
|
||||
- Acceptance criteria: examples are copyable, secret-free, and tested where practical; no links to deleted docs; no maintained docs link to `local-test/`.
|
||||
- Suggested validation commands: `go test ./...`; `go run ./cmd/scriptorium render --config ./examples/config.yml --prompt generic.markdown_summary --input transcript=./examples/fixtures/transcript.md --format text`; `rg -n "docs/config/|architecture.md|local-test|dnd.structured_events|dnd.glossary_suggestions|dnd.player_summary" README.md docs examples`.
|
||||
- One-prompt size: split if moving prompt/profile/schema assets because tests and paths must be updated carefully.
|
||||
|
||||
## Validation Plan
|
||||
|
||||
- Run `go test ./...` after documentation changes that touch examples, paths, command examples, or config references.
|
||||
- Smoke-test the documented render quickstart with `go run ./cmd/scriptorium render --config ./examples/config.yml --prompt generic.markdown_summary --input transcript=./examples/fixtures/transcript.md --format json`.
|
||||
- If documenting JSON-schema render/run examples, set `SCRIPTORIUM_API_KEY` or use a profile without `api_key_env`; `Runner.Prepare` validates the named environment variable.
|
||||
- Validate CLI flags against `internal/adapter/cli/run.go` and parser tests, especially `render` lacking `--schema-dir` and `serve` rejecting runtime override flags.
|
||||
- Validate app config examples against `internal/config/config.go` strict YAML decoding.
|
||||
- Validate prompt/profile examples against `internal/promptdef/filesystem_repository.go` and `internal/profile/filesystem_repository.go`.
|
||||
- Validate HTTP request examples against `internal/adapter/http/dto.go` strict JSON decoding.
|
||||
- Run grep checks for stale or roadmap-only terms outside `docs/roadmap/`: `future`, `planned`, `may be added`, `can be added later`, `S3`, `streaming`, `batch`, `database-backed`, `render endpoint`, and prompt IDs not present in `prompts/`.
|
||||
- Run grep checks for stale paths after file moves: `docs/config/`, `architecture.md`, and `local-test`.
|
||||
- No automated documentation link checker is currently configured; perform manual link review or add a link checker in a separate roadmap item if desired.
|
||||
|
||||
## Open Questions
|
||||
|
||||
No open questions block the documentation migration. The recommended path is to document the current implementation conservatively, move future ideas into `docs/roadmap/`, and avoid claiming production behavior for hooks that are present in code but not wired into CLI or HTTP adapters.
|
||||
Reference in New Issue
Block a user