Update documentation and testing policies and add a migration plan to cleanly separate the scriptorium CLI from the promptkit internals
This commit is contained in:
510
docs/roadmap/documentation.md
Normal file
510
docs/roadmap/documentation.md
Normal file
@@ -0,0 +1,510 @@
|
||||
# Documentation Compliance Roadmap
|
||||
|
||||
## Status
|
||||
|
||||
Proposed implementation plan. This document records the findings of the
|
||||
documentation audit performed after adoption of the canonical-ownership policy.
|
||||
The revisions described here are not yet implemented.
|
||||
|
||||
## Objective
|
||||
|
||||
Bring the current Scriptorium documentation into compliance with
|
||||
`docs/policy/documentation.md` before beginning the Promptkit migration.
|
||||
|
||||
The refresh should:
|
||||
|
||||
- give every topic one canonical owner;
|
||||
- remove parallel definitions of volatile contracts;
|
||||
- correct current factual discrepancies;
|
||||
- preserve useful contributor and operational guidance in the appropriate
|
||||
documents;
|
||||
- establish the missing internal overview and ADR history;
|
||||
- leave Promptkit and other future behavior in roadmap and ADR documents until
|
||||
implemented.
|
||||
|
||||
This roadmap covers documentation organization and current-behavior accuracy.
|
||||
A substantive review of `docs/policy/testing.md` remains separate.
|
||||
|
||||
## Audit Baseline
|
||||
|
||||
The audit covered:
|
||||
|
||||
- `README.md`;
|
||||
- all Markdown files under `docs/`;
|
||||
- all maintained files under `examples/`;
|
||||
- the CLI parser and output behavior;
|
||||
- app-config shapes and defaults;
|
||||
- HTTP DTOs, limits, and error mappings;
|
||||
- the public Go facade;
|
||||
- prompt, profile, schema, artifact, renderer, runner, and LLM behavior;
|
||||
- the embedded built-in profile assets.
|
||||
|
||||
At the time of the audit:
|
||||
|
||||
- all local Markdown link targets existed;
|
||||
- `go test ./...` passed;
|
||||
- the README/CLI render command completed successfully;
|
||||
- `examples/render-markdown-summary.sh` completed successfully;
|
||||
- `go run ./examples/go-library/prepare` completed successfully.
|
||||
|
||||
The dominant problem is duplicated ownership rather than broad factual
|
||||
staleness. CLI, configuration, HTTP, validation, state, and security behavior
|
||||
are repeated across contracts, operations, troubleshooting, integrations,
|
||||
consumer guides, architecture, and internal documents.
|
||||
|
||||
## Required Accuracy Corrections
|
||||
|
||||
Make these corrections while revising the owning documents:
|
||||
|
||||
1. `docs/policy/architecture.md` says Scriptorium has three entry paths but
|
||||
lists four. Distinguish the three executable entry paths from the public Go
|
||||
package, or describe four total entry paths.
|
||||
2. The built-in profile catalog in `docs/config.md` omits
|
||||
`deepseek-4-flash`. Reconcile the complete catalog with
|
||||
`internal/profile/builtin/assets/`.
|
||||
3. `docs/config.md` marks prompt `output.repair_attempts` as required. The
|
||||
loader permits omission and resolves it to zero; document it as optional
|
||||
with an effective default of zero.
|
||||
4. `docs/consumers/pkg-scriptorium.md` says every input map key must match a
|
||||
declared prompt input. The renderer enforces required declared inputs and
|
||||
resolves names actually referenced by templates, but it does not reject
|
||||
every undeclared extra input. Describe the implemented boundary.
|
||||
5. `docs/troubleshooting.md` says validation errors can be inspected in CLI
|
||||
stderr. The CLI success summary reports the number of validation errors, not
|
||||
their detailed messages. Correct the diagnostic guidance or separately
|
||||
change the product before documenting richer output.
|
||||
6. `docs/integrations/openai-compatible-chat.md` describes the `session_id`
|
||||
limit as characters. The implementation counts Unicode code points; use the
|
||||
canonical terminology consistently.
|
||||
7. The same integration document makes a time-sensitive statement about
|
||||
provider-supported service-tier values even though Scriptorium accepts and
|
||||
forwards any non-empty value. Remove the provider catalog claim or cite and
|
||||
version an intentionally maintained external contract.
|
||||
8. The `render` and `serve` flag sections in `docs/cli.md` list
|
||||
`--prompt-dir` both as an effective requirement and again as an optional
|
||||
flag. Keep one complete flag entry and separately explain how the
|
||||
requirement may be satisfied.
|
||||
9. Troubleshooting command examples containing placeholders such as
|
||||
`<prompt-id>` are not directly copyable shell commands. Use clearly defined
|
||||
shell variables, concrete maintained examples, or prose diagnostic steps.
|
||||
|
||||
## Canonical Structure Changes
|
||||
|
||||
### Create `docs/internal/overview.md`
|
||||
|
||||
Create the canonical implemented-component inventory and move the concrete
|
||||
package map out of architecture.
|
||||
|
||||
The overview should:
|
||||
|
||||
- list current public, command, adapter, domain, use-case, source, format,
|
||||
validation, and LLM components;
|
||||
- give each component a short implemented responsibility;
|
||||
- link to focused internal documents and relevant external contracts;
|
||||
- avoid restating global architecture rules or user-facing behavior.
|
||||
|
||||
Update `docs/development.md` to route general repository-orientation work
|
||||
through the overview once it exists.
|
||||
|
||||
### Establish `docs/adr/`
|
||||
|
||||
Create the ADR directory under the policy already defined in
|
||||
`docs/policy/documentation.md`.
|
||||
|
||||
Record the significant accepted documentation-ownership decision in an initial
|
||||
ADR if historical rationale is useful. The Promptkit split ADR remains part of
|
||||
Step 2 in `docs/roadmap/migration.md` and should not be pulled ahead of its
|
||||
gate merely to populate the directory.
|
||||
|
||||
ADRs must own decision context, alternatives, rationale, and consequences.
|
||||
Roadmaps must continue to own implementation status and sequencing.
|
||||
|
||||
### Consolidate Troubleshooting Into Operations
|
||||
|
||||
The ownership table assigns recovery to `docs/operations.md` and does not give
|
||||
troubleshooting a separate canonical owner. Fold the useful
|
||||
symptom/diagnosis/safe-recovery material from `docs/troubleshooting.md` into a
|
||||
concise operations section, then remove `docs/troubleshooting.md`.
|
||||
|
||||
The consolidated material should:
|
||||
|
||||
- organize failures by operational task rather than duplicate every contract
|
||||
field and status;
|
||||
- link exact CLI syntax and exit codes to `docs/cli.md`;
|
||||
- link config fields and defaults to `docs/config.md`;
|
||||
- link HTTP codes and schemas to `docs/api.md`;
|
||||
- describe only diagnostic and recovery actions locally.
|
||||
|
||||
Update README, development, integration, and other links after the removal.
|
||||
|
||||
## File-By-File Revision Catalog
|
||||
|
||||
### `README.md`
|
||||
|
||||
- Retain the product description and one tested end-to-end quickstart.
|
||||
- Keep the quickstart linked to the complete CLI contract.
|
||||
- Replace the repeated example inventory with a short link to the maintained
|
||||
examples section in the appropriate canonical contract, or make each retained
|
||||
path a direct link.
|
||||
- Keep the documentation list navigational; do not summarize contracts there.
|
||||
- Remove the troubleshooting link if troubleshooting is consolidated into
|
||||
operations.
|
||||
|
||||
### `docs/development.md`
|
||||
|
||||
- Retain only contributor orientation, task-specific routing, and baseline
|
||||
validation.
|
||||
- Add `docs/internal/overview.md` to the appropriate reading paths once it
|
||||
exists.
|
||||
- Update the operations/troubleshooting row after consolidation.
|
||||
- Link to actual ADRs when created; do not duplicate their decisions.
|
||||
- Confirm that all detailed recipes removed from the former development guide
|
||||
have an internal owner before declaring the refresh complete.
|
||||
|
||||
### `docs/policy/architecture.md`
|
||||
|
||||
- Fix the entry-path count.
|
||||
- Replace the concrete package inventory with a link to
|
||||
`docs/internal/overview.md`.
|
||||
- Remove config precedence, field-level behavior, exact routes, and exact
|
||||
import-path contracts except for the smallest orientation summary and links
|
||||
to their owners.
|
||||
- Remove the local testing checklist and link to
|
||||
`docs/policy/testing.md`. Component-specific test inventories remain in
|
||||
internal docs.
|
||||
- Remove the local documentation checklist and link to
|
||||
`docs/policy/documentation.md`.
|
||||
- Keep normative system shape, ownership, dependency direction, state
|
||||
philosophy, security properties, error-handling principles, invariants, and
|
||||
non-goals.
|
||||
- Review the coding and dependency rules removed from the former development
|
||||
guide. Preserve still-valid normative rules here without listing the current
|
||||
dependency inventory, which is concrete implementation information.
|
||||
- Move implementation facts such as current repairer wiring to the relevant
|
||||
internal component document unless they are intentionally elevated to
|
||||
architecture invariants.
|
||||
|
||||
### `docs/policy/documentation.md`
|
||||
|
||||
- No structural rewrite is currently required.
|
||||
- Recheck its ownership table after the troubleshooting consolidation and ADR
|
||||
creation.
|
||||
- Keep future policy refinements in this document rather than distributing
|
||||
documentation rules across architecture or development.
|
||||
|
||||
### `docs/policy/testing.md`
|
||||
|
||||
- It is the sole owner of global testing philosophy and sufficiency rules.
|
||||
- Remove competing global testing guidance from architecture and other docs.
|
||||
- Leave the planned substantive testing-policy review for its separate work
|
||||
cycle.
|
||||
|
||||
### `docs/cli.md`
|
||||
|
||||
- Keep the complete command, argument, flag, output, and exit-code contract.
|
||||
- Remove the repeated definition of config precedence and link to
|
||||
`docs/config.md`.
|
||||
- Move profile source and override semantics that are not CLI invocation
|
||||
semantics to `docs/config.md`.
|
||||
- Resolve duplicate `--prompt-dir` entries in the `render` and `serve`
|
||||
sections.
|
||||
- Verify every registered flag, deprecated alias, requirement, zero-value
|
||||
behavior, and output destination against `internal/adapter/cli/run.go`.
|
||||
- Clarify timeout conversion for subsecond Go durations if that behavior is
|
||||
intended to remain public.
|
||||
- Keep only compact workflow examples and link to maintained scripts under
|
||||
`examples/`.
|
||||
|
||||
### `docs/config.md`
|
||||
|
||||
- Keep app-config discovery, precedence, fields, defaults, validation,
|
||||
credential-supply mechanisms, prompt/profile formats, and selectable profile
|
||||
catalog as the canonical contract.
|
||||
- Replace complete inline config, prompt, and profile files with the smallest
|
||||
useful snippets and links to copyable files under `examples/`.
|
||||
- If a production-oriented complete config remains useful, move it into
|
||||
`examples/` and link it rather than maintaining a second complete copy.
|
||||
- Correct `repair_attempts` optionality.
|
||||
- Add `deepseek-4-flash` and reconcile every built-in profile ID, model, and
|
||||
credential-variable name with embedded assets.
|
||||
- Establish a low-friction way to prevent catalog drift, such as a generated
|
||||
catalog section or a focused consistency check, if its maintenance value
|
||||
justifies the cost.
|
||||
- Keep schema configuration and prompt/profile file-format rules here; move
|
||||
implementation mechanics to internal source docs.
|
||||
- Reduce the HTTP artifact-reference section to configuration meaning and link
|
||||
request shapes/statuses to `docs/api.md` and runtime security handling to
|
||||
`docs/operations.md`.
|
||||
- Keep the secret-supply mechanism here; move deployment permissions and
|
||||
sensitive-runtime handling to operations.
|
||||
|
||||
### `docs/api.md`
|
||||
|
||||
- Keep routes, media types, request/response schemas, strict JSON behavior,
|
||||
status codes, HTTP retry semantics, and HTTP artifact-access outcomes.
|
||||
- Replace repeated app-config defaults and CLI flag syntax with links to
|
||||
`docs/config.md` and `docs/cli.md`.
|
||||
- Retain HTTP limit effects but let configuration own field defaults and
|
||||
precedence.
|
||||
- Keep the lexical containment and symlink behavior as externally observable
|
||||
API/security behavior; operations may summarize its deployment consequence
|
||||
and link back.
|
||||
- Reduce request and response examples to compact contract-bearing shapes and
|
||||
link to `examples/http-run.json`.
|
||||
- Reverify all DTO fields, omission behavior, limit responses, and error mappings
|
||||
against `internal/adapter/http/`.
|
||||
|
||||
### `docs/operations.md`
|
||||
|
||||
- Rewrite as a task-oriented runbook rather than a secondary CLI, config, and
|
||||
API reference.
|
||||
- Keep deployment layout, process permissions, sensitive runtime artifact
|
||||
handling, normal workflow, service exposure, capacity planning, recovery,
|
||||
and the stateless rerun model.
|
||||
- Link exact commands and exit codes to `docs/cli.md`.
|
||||
- Link config fields, defaults, validation modes, and credential-supply
|
||||
mechanics to `docs/config.md`.
|
||||
- Link HTTP route, status, schema, and limit semantics to `docs/api.md`.
|
||||
- Remove the repeated maintained-example inventory and link to the owning
|
||||
examples section.
|
||||
- Incorporate the useful troubleshooting material and then delete
|
||||
`docs/troubleshooting.md`.
|
||||
|
||||
### `docs/troubleshooting.md`
|
||||
|
||||
- Correct the CLI validation-error diagnostic during migration.
|
||||
- Move useful symptom, diagnostic, and safe-fix material into
|
||||
`docs/operations.md`.
|
||||
- Remove repeated definitions of flags, fields, defaults, HTTP codes, routes,
|
||||
and validation semantics; link to their canonical contracts.
|
||||
- Delete this file after all useful recovery guidance and inbound links have
|
||||
been handled.
|
||||
|
||||
### `docs/consumers/api.md`
|
||||
|
||||
- Keep integration-surface selection, minimal consumer workflow, and consumer
|
||||
responsibilities.
|
||||
- Retain one minimal Go example as permitted instructional content and link to
|
||||
the complete package contract and maintained example.
|
||||
- Replace repeated CLI exit and HTTP status definitions with links to their
|
||||
canonical contracts.
|
||||
- Replace repeated deployment input and credential definitions with links to
|
||||
configuration and operations.
|
||||
- Keep retry and artifact-retention decisions as consumer responsibilities
|
||||
without restating wire semantics.
|
||||
|
||||
### `docs/consumers/pkg-scriptorium.md`
|
||||
|
||||
- Keep the import path, public constructors, options, types, errors, workflows,
|
||||
and public security boundary as the canonical Go package contract.
|
||||
- Correct the input-name validation statement.
|
||||
- Reverify public fields, JSON behavior, option precedence, nil behavior,
|
||||
source containment, direct-key handling, and sentinel errors against the root
|
||||
package.
|
||||
- Keep a minimal package example and link to
|
||||
`examples/go-library/prepare`; avoid duplicating complete maintained source.
|
||||
- Link prompt/profile/schema file formats to `docs/config.md` rather than
|
||||
redefining them.
|
||||
|
||||
### `docs/integrations/subprocess.md`
|
||||
|
||||
- Narrow this document to subprocess-specific compatibility and process
|
||||
integration concerns.
|
||||
- Link command syntax, flags, stdout/stderr behavior, and exit codes to
|
||||
`docs/cli.md` rather than maintaining parallel definitions.
|
||||
- Link config search and field semantics to `docs/config.md`.
|
||||
- Retain process isolation, environment propagation, stream capture, output
|
||||
ownership, cancellation/termination expectations, and security guidance when
|
||||
these are specific to subprocess consumers.
|
||||
- Move task-oriented surface-selection advice to `docs/consumers/api.md` if it
|
||||
is currently duplicated.
|
||||
|
||||
### `docs/integrations/openai-compatible-chat.md`
|
||||
|
||||
- Keep only the outbound HTTP wire contract: endpoint construction, request
|
||||
payload, authentication header, timeouts, response subset, and unsupported
|
||||
protocol behavior.
|
||||
- Remove the concrete implementation-file introduction and internal error
|
||||
sentinel catalog.
|
||||
- Correct the `session_id` unit to Unicode code points.
|
||||
- Remove or deliberately source/version the provider service-tier claim.
|
||||
- Reverify reserved `extra_params` keys, structured-output envelope,
|
||||
cache-control encoding, usage mapping, and response requirements against
|
||||
`internal/llm/openai_compatible_client.go`.
|
||||
- Link runner schema preparation to internal runner documentation rather than
|
||||
explaining orchestration locally.
|
||||
|
||||
### `docs/internal/overview.md` (new)
|
||||
|
||||
- Own the complete current component and package inventory.
|
||||
- Link each component to focused internal docs and external contracts.
|
||||
- Include the command entry point and public facade without redefining their
|
||||
external behavior.
|
||||
- Absorb the still-useful concrete repository-layout material removed from the
|
||||
old development guide and architecture policy.
|
||||
|
||||
### `docs/internal/adapters.md`
|
||||
|
||||
- Keep implementation flow, collaborators, translation boundaries, wiring,
|
||||
error-mapping mechanisms, and relevant tests.
|
||||
- Remove exact flag lists, config field definitions, route schemas, response
|
||||
codes, and exit-code definitions; link their canonical owners.
|
||||
- Keep only package-local invariants rather than repeating global architecture.
|
||||
- Incorporate the former contributor recipes for adding app-config fields, CLI
|
||||
flags, and adapter capabilities.
|
||||
- Describe how to verify affected adapter contracts without duplicating the
|
||||
global testing policy.
|
||||
|
||||
### `docs/internal/runner.md`
|
||||
|
||||
- Keep prepare/run flow, dependency boundaries, state transitions, failure
|
||||
categories, repair mechanics, hashing/validation coordination, and relevant
|
||||
tests.
|
||||
- Remove the app-config field list and link to adapter/config documentation.
|
||||
- Keep internal sentinel and collaborator information, but link public error
|
||||
behavior to the package/API contracts.
|
||||
- Distinguish package-local guarantees from global architectural invariants.
|
||||
- Add a focused change recipe for runner orchestration when useful.
|
||||
|
||||
### `docs/internal/sources.md`
|
||||
|
||||
- Keep loader implementations, source precedence mechanics, containment
|
||||
implementation, failure categories, collaborators, and relevant tests.
|
||||
- Link YAML/JSON field definitions and defaults to `docs/config.md`.
|
||||
- Link HTTP-visible artifact outcomes to `docs/api.md` and deployment handling
|
||||
to operations.
|
||||
- Incorporate the former recipe for updating prompt, profile, schema, and
|
||||
built-in-profile assets.
|
||||
- Distinguish implementation mechanics from external source contracts.
|
||||
|
||||
### `docs/internal/llm.md` (new)
|
||||
|
||||
Create a focused internal document if the implementation material removed from
|
||||
the outbound integration contract remains useful.
|
||||
|
||||
It should own:
|
||||
|
||||
- client construction and internal collaborator boundaries;
|
||||
- internal request mapping and timeout selection;
|
||||
- internal error categories;
|
||||
- relevant tests and package-local change guidance.
|
||||
|
||||
It must link to `docs/integrations/openai-compatible-chat.md` for the wire
|
||||
contract rather than repeat payload definitions.
|
||||
|
||||
### `docs/roadmap/migration.md`
|
||||
|
||||
- Retain migration status, gates, sequencing, and task breakdowns.
|
||||
- Mark the documentation-refresh gate complete only after this roadmap's
|
||||
completion criteria are satisfied.
|
||||
- After the Promptkit split ADR is accepted, replace duplicated decision
|
||||
rationale and architectural ownership detail with a concise summary and ADR
|
||||
link where practical.
|
||||
- Update paths and document ownership affected by this refresh.
|
||||
|
||||
### `examples/`
|
||||
|
||||
- Keep complete copyable artifacts here rather than in reference documents.
|
||||
- Verify `config.yml` and `config.full.yml` through the real config loader.
|
||||
- Keep prompt/profile/schema examples covered by representative repository and
|
||||
engine tests.
|
||||
- Keep the render script and Go package example runnable from the repository
|
||||
root.
|
||||
- Validate `http-run.json` structurally against the HTTP DTO contract without
|
||||
requiring a live model endpoint.
|
||||
- Add a separate production-oriented config example only if that maintained
|
||||
artifact has clear value; otherwise remove the duplicate production block
|
||||
from config documentation.
|
||||
- Preserve the fixtures as non-sensitive synthetic data.
|
||||
|
||||
## Implementation Sequence
|
||||
|
||||
### Stage 1: Establish Canonical Developer Structure
|
||||
|
||||
1. Create the initial documentation ADR if retained.
|
||||
2. Create `docs/internal/overview.md`.
|
||||
3. Rewrite architecture around normative ownership and invariants.
|
||||
4. Update `docs/development.md` routing.
|
||||
5. Rescue still-valid development rules and assign every removed recipe.
|
||||
|
||||
**Gate:** Contributor orientation, architecture, component inventory, policies,
|
||||
and decision history have distinct owners and no stale links.
|
||||
|
||||
### Stage 2: Correct And Consolidate External Contracts
|
||||
|
||||
1. Revise `docs/config.md` and correct its known factual issues.
|
||||
2. Revise `docs/cli.md`.
|
||||
3. Revise `docs/api.md`.
|
||||
4. Revise both consumer guides.
|
||||
5. Revise both integration contracts.
|
||||
6. Verify each volatile contract against code and maintained assets.
|
||||
|
||||
**Gate:** Each externally observable field, default, flag, route, status, import
|
||||
path, and wire behavior has one authoritative definition.
|
||||
|
||||
### Stage 3: Refocus Internal Documentation
|
||||
|
||||
1. Revise adapter, runner, and source docs.
|
||||
2. Add the internal LLM document if warranted.
|
||||
3. Incorporate the former subsystem change recipes.
|
||||
4. Remove global policy and external-contract duplication.
|
||||
|
||||
**Gate:** Internal docs explain implementation and package-local guarantees,
|
||||
link external contracts, and contain the detailed contributor recipes needed
|
||||
for safe changes.
|
||||
|
||||
### Stage 4: Rewrite Operations And Consolidate Recovery Guidance
|
||||
|
||||
1. Rewrite operations as a runbook.
|
||||
2. Migrate useful troubleshooting content.
|
||||
3. Delete `docs/troubleshooting.md`.
|
||||
4. Update all inbound links and the development reading guide.
|
||||
|
||||
**Gate:** Operations owns deployment and recovery without serving as a second
|
||||
CLI, config, or API reference.
|
||||
|
||||
### Stage 5: Normalize Examples And Orientation
|
||||
|
||||
1. Move any remaining complete copyable artifacts into `examples/`.
|
||||
2. Update README navigation and example links.
|
||||
3. Run maintained smoke examples.
|
||||
4. Add only high-value automated checks justified by the testing policy.
|
||||
|
||||
**Gate:** Examples are canonical, runnable, secret-free, linked from their
|
||||
owning references, and not duplicated as complete files in prose docs.
|
||||
|
||||
### Stage 6: Final Compliance Pass
|
||||
|
||||
1. Validate all local links and paths.
|
||||
2. Search for duplicated volatile values across non-owning documents.
|
||||
3. Reconcile flags, config fields/defaults, API DTOs/codes, public types/errors,
|
||||
and built-in profiles with implementation.
|
||||
4. Run `go test ./...`, `go vet ./...`, and the documented build command.
|
||||
5. Run the maintained render script and Go package example.
|
||||
6. Confirm that non-roadmap current docs contain implemented behavior only.
|
||||
7. Record completion in this roadmap and update the documentation gate in
|
||||
`docs/roadmap/migration.md`.
|
||||
|
||||
**Gate:** Code, tests, examples, contracts, internal documentation, operations,
|
||||
policies, and roadmap status agree.
|
||||
|
||||
## Completion Criteria
|
||||
|
||||
The documentation refresh is complete when:
|
||||
|
||||
- every topic in the ownership table has one canonical owner;
|
||||
- `docs/internal/overview.md` exists and architecture no longer owns the package
|
||||
inventory;
|
||||
- significant accepted decisions have an ADR owner;
|
||||
- all known factual discrepancies in this audit are corrected;
|
||||
- subsystem recipes from the former development guide have been preserved in
|
||||
relevant internal docs;
|
||||
- operations contains the retained recovery guidance and the standalone
|
||||
troubleshooting document is removed;
|
||||
- contracts and internal docs link to one another instead of maintaining
|
||||
parallel volatile definitions;
|
||||
- maintained examples are runnable and not duplicated as complete prose
|
||||
examples;
|
||||
- local links, validation commands, tests, and smoke checks pass;
|
||||
- the documentation-refresh gate in the Promptkit migration roadmap is marked
|
||||
complete.
|
||||
292
docs/roadmap/migration.md
Normal file
292
docs/roadmap/migration.md
Normal file
@@ -0,0 +1,292 @@
|
||||
# Promptkit Migration Roadmap
|
||||
|
||||
## Status
|
||||
|
||||
Accepted plan. This document describes proposed work that is not yet
|
||||
implemented.
|
||||
|
||||
## Objective
|
||||
|
||||
Split the current repository into two projects:
|
||||
|
||||
- **Promptkit**: the reusable Go framework, public Go facade, execution engine,
|
||||
source and validation support, OpenAI-compatible client, extension
|
||||
interfaces, and built-in execution-profile registry.
|
||||
- **Scriptorium**: a slim runnable application that imports Promptkit and
|
||||
provides the CLI and HTTP interfaces.
|
||||
|
||||
Scriptorium will become another downstream Promptkit consumer rather than the
|
||||
owner of the framework.
|
||||
|
||||
## Compatibility And Migration Policy
|
||||
|
||||
This is an intentionally breaking change.
|
||||
|
||||
- New and migrated Go consumers must import Promptkit instead of Scriptorium.
|
||||
- Scriptorium will not retain type aliases, forwarding packages, deprecated
|
||||
facade APIs, or other source-compatibility shims.
|
||||
- Existing consumers may continue using a previously tagged Scriptorium module
|
||||
version until they are migrated.
|
||||
- The migration does not need to preserve compatibility between intermediate
|
||||
development states. Each completed phase must instead leave the affected
|
||||
repository internally consistent and tested.
|
||||
- Promptkit should initially preserve the useful shape and behavior of the
|
||||
current public Go facade where doing so reduces extraction risk. Broader API
|
||||
redesign should follow the split unless required to establish the new
|
||||
boundary.
|
||||
|
||||
## Target Ownership
|
||||
|
||||
Promptkit should own application-neutral framework behavior:
|
||||
|
||||
- public engine, request, result, option, extension, and error APIs;
|
||||
- prompt-definition loading and rendering;
|
||||
- execution profiles, overlays, and the built-in profile registry;
|
||||
- artifact-loading interfaces and general-purpose `file` and `inline` support;
|
||||
- schema loading and output validation;
|
||||
- LLM client boundaries and the OpenAI-compatible implementation;
|
||||
- preparation and execution orchestration;
|
||||
- framework and execution defaults.
|
||||
|
||||
Scriptorium should own executable and transport concerns:
|
||||
|
||||
- the `scriptorium` command and its `run`, `render`, and `serve` interfaces;
|
||||
- CLI parsing, output formatting, exit codes, and process behavior;
|
||||
- application-config discovery and CLI precedence;
|
||||
- HTTP routing, request and response DTOs, limits, and error/status mapping;
|
||||
- HTTP artifact-root and deployment security policy;
|
||||
- server and adapter defaults;
|
||||
- executable examples, operations guidance, and transport documentation.
|
||||
|
||||
The intended dependency direction is:
|
||||
|
||||
```text
|
||||
Scriptorium CLI and HTTP adapters
|
||||
|
|
||||
v
|
||||
Promptkit
|
||||
|
|
||||
v
|
||||
consumer-supplied sources and clients
|
||||
```
|
||||
|
||||
Scriptorium must use Promptkit's public API. It must not depend on Promptkit
|
||||
implementation packages or reproduce framework orchestration.
|
||||
|
||||
## Migration Steps
|
||||
|
||||
### Step 1: Refresh And Synchronize Documentation
|
||||
|
||||
Perform a repository-wide documentation refresh before migration development.
|
||||
|
||||
At minimum:
|
||||
|
||||
- reconcile all current-behavior documentation with the code, tests, examples,
|
||||
defaults, and current public contracts;
|
||||
- introduce the planned documentation-policy updates;
|
||||
- establish an architecture decision record policy and canonical ADR location;
|
||||
- resolve stale, duplicated, or misplaced material;
|
||||
- validate documentation links and maintained examples;
|
||||
- leave future migration behavior in `docs/roadmap/` until implemented.
|
||||
|
||||
**Gate:** Do not begin architectural migration work until the documentation
|
||||
refresh and policy updates are merged and the repository has an agreed,
|
||||
accurate baseline.
|
||||
|
||||
### Step 2: Record The Architectural Decision And Detailed Boundary
|
||||
|
||||
Create an ADR, under the policy established in Step 1, that records:
|
||||
|
||||
- the decision to split Promptkit from Scriptorium;
|
||||
- the target ownership and dependency direction;
|
||||
- the selected Promptkit repository and Go module paths;
|
||||
- the breaking-change and versioning policy;
|
||||
- ownership of configuration fields and defaults;
|
||||
- artifact-reader and HTTP containment responsibilities;
|
||||
- local multi-repository development and release coordination;
|
||||
- documentation ownership after the split.
|
||||
|
||||
Use the ADR to resolve any remaining public-boundary decisions before code is
|
||||
moved.
|
||||
|
||||
**Gate:** The ADR is accepted, and every existing package, public contract,
|
||||
configuration category, and maintained asset has a target owner.
|
||||
|
||||
### Step 3: Characterize Existing Framework Behavior
|
||||
|
||||
Strengthen or add contract-focused tests where needed so extraction can be
|
||||
verified without relying on package placement.
|
||||
|
||||
Preserve coverage of:
|
||||
|
||||
- `Prepare` and `Run` behavior;
|
||||
- prompt, profile, execution-default, and request-override precedence;
|
||||
- presence-aware numeric overrides;
|
||||
- built-in profile fallback and custom-profile overlays;
|
||||
- strict YAML and JSON decoding;
|
||||
- prompt, profile, schema, and artifact source behavior;
|
||||
- structured-output requests and output validation;
|
||||
- validation failures versus validation runtime errors;
|
||||
- secret handling and redaction;
|
||||
- public error classification;
|
||||
- HTTP artifact restrictions and transport mappings.
|
||||
|
||||
**Gate:** Current framework and adapter contracts are represented by passing
|
||||
tests sufficient to detect behavioral regressions during the split.
|
||||
|
||||
### Step 4: Make Scriptorium Adapters Consume The Public Facade
|
||||
|
||||
Within the current repository, refactor the CLI and HTTP adapters to use the
|
||||
public framework facade rather than constructing or importing internal runner
|
||||
components directly.
|
||||
|
||||
Add only the minimum public capabilities needed to support this boundary. These
|
||||
may include:
|
||||
|
||||
- a small `Run`/`Prepare` consumer interface;
|
||||
- injectable artifact-reading behavior for Scriptorium's restricted HTTP
|
||||
policy;
|
||||
- source options currently available only through internal constructors;
|
||||
- prepared-run formatting based on public types;
|
||||
- stable public error classification required by CLI and HTTP mappings.
|
||||
|
||||
Do not broadly export internal repositories, domain types, or use-case
|
||||
implementations.
|
||||
|
||||
**Gate:** The CLI and HTTP adapters use only the public framework API for
|
||||
framework behavior, and all tests and documented smoke commands pass.
|
||||
|
||||
### Step 5: Create The Promptkit Repository
|
||||
|
||||
Create the Promptkit repository and Go module as an explicit out-of-band
|
||||
operation.
|
||||
|
||||
Establish:
|
||||
|
||||
- repository access, ownership, and branch protections;
|
||||
- the module path selected by the ADR;
|
||||
- baseline development, architecture, documentation, and release policies;
|
||||
- CI for build, test, vet, and other agreed checks;
|
||||
- an initial package layout centered on a small public facade with internal
|
||||
implementation packages;
|
||||
- a local development workflow for coordinated Promptkit and Scriptorium
|
||||
changes, using a workspace or temporary uncommitted module replacement where
|
||||
appropriate.
|
||||
|
||||
Do not commit local filesystem `replace` directives to release branches.
|
||||
|
||||
**Gate:** The Promptkit repository exists, is accessible to maintainers, has
|
||||
working CI and policy scaffolding, and can receive the extracted framework.
|
||||
Do not begin cross-repository extraction until this out-of-band work is
|
||||
confirmed complete.
|
||||
|
||||
### Step 6: Extract And Stabilize Promptkit
|
||||
|
||||
Move the application-neutral framework and built-in profile assets into
|
||||
Promptkit. Preserve implementation packages as internal where practical.
|
||||
|
||||
The initial public API should remain focused on the established engine workflow
|
||||
and the source and client extension points required by real consumers. Avoid
|
||||
combining the extraction with unrelated API redesign.
|
||||
|
||||
Move or recreate the relevant:
|
||||
|
||||
- framework implementation;
|
||||
- public package tests and framework contract tests;
|
||||
- built-in profile assets and registry tests;
|
||||
- Go consumer examples;
|
||||
- framework, consumer, configuration-format, and integration documentation.
|
||||
|
||||
Verify that Promptkit can be built, tested, and consumed independently of the
|
||||
Scriptorium repository.
|
||||
|
||||
**Gate:** Promptkit independently provides the agreed framework contract,
|
||||
passes its CI checks, and has a tagged version that Scriptorium and other
|
||||
consumers can import.
|
||||
|
||||
### Step 7: Slim Scriptorium And Adopt Promptkit
|
||||
|
||||
Update Scriptorium to import the tagged Promptkit module and remove the
|
||||
framework implementation and public Go facade that Promptkit replaces.
|
||||
|
||||
Retain only Scriptorium-owned executable and transport behavior. In particular:
|
||||
|
||||
- wire CLI and HTTP requests through Promptkit's public API;
|
||||
- keep application config and transport defaults in Scriptorium;
|
||||
- keep restricted HTTP artifact policy in Scriptorium while injecting it
|
||||
through Promptkit's supported boundary;
|
||||
- remove obsolete framework packages, tests, and documentation;
|
||||
- update Scriptorium examples and docs to describe the CLI and HTTP application;
|
||||
- direct Go framework consumers to Promptkit without providing compatibility
|
||||
aliases or forwarding APIs.
|
||||
|
||||
**Gate:** Scriptorium builds and passes all tests using a tagged Promptkit
|
||||
dependency, contains no duplicate framework implementation, and its current
|
||||
documentation describes only the slimmed application.
|
||||
|
||||
### Step 8: Migrate Downstream Consumers To Promptkit
|
||||
|
||||
Inventory downstream Go consumers and migrate each from the Scriptorium package
|
||||
to Promptkit. This work may occur in external repositories and must be tracked
|
||||
explicitly.
|
||||
|
||||
For each consumer:
|
||||
|
||||
- update module imports and dependencies;
|
||||
- adapt to any intentionally changed public API;
|
||||
- run its tests and relevant integration or smoke checks;
|
||||
- confirm configuration, source, validation, and error behavior;
|
||||
- release or deploy the migrated consumer through its normal process.
|
||||
|
||||
Consumers that cannot migrate immediately may remain pinned to the last
|
||||
framework-bearing Scriptorium tag. No compatibility work is required in the new
|
||||
Scriptorium project for those consumers.
|
||||
|
||||
**Gate:** All in-scope downstream consumers are either migrated and verified or
|
||||
explicitly recorded as remaining on the previous Scriptorium version with an
|
||||
owner and follow-up plan. Do not declare the ecosystem migration complete until
|
||||
the required out-of-band consumer changes are confirmed.
|
||||
|
||||
### Step 9: Complete Release And Documentation Cutover
|
||||
|
||||
Complete the coordinated project transition:
|
||||
|
||||
- publish Promptkit before dependent Scriptorium releases;
|
||||
- release the breaking Scriptorium version against the tagged Promptkit
|
||||
dependency;
|
||||
- publish migration guidance that maps the former Scriptorium Go API to
|
||||
Promptkit;
|
||||
- update cross-project links, examples, package documentation, and release
|
||||
notes;
|
||||
- verify that no release artifact depends on local workspaces or replacements;
|
||||
- archive completed roadmap material according to the documentation policy in
|
||||
effect at that time.
|
||||
|
||||
**Gate:** Promptkit and Scriptorium are independently releasable, their
|
||||
documentation has distinct and accurate ownership, and the migration status of
|
||||
all identified downstream consumers is recorded.
|
||||
|
||||
## Cross-Cutting Constraints
|
||||
|
||||
- Preserve the invariant that execution orchestration remains narrow and
|
||||
application-neutral.
|
||||
- Keep adapter-specific decisions out of Promptkit.
|
||||
- Keep Scriptorium dependent only on Promptkit's supported public API.
|
||||
- Preserve strict external decoding, error classification, validation
|
||||
semantics, and secret redaction throughout the migration.
|
||||
- Keep each repository buildable and testable at merged phase boundaries.
|
||||
- Coordinate cross-repository changes through tagged dependencies and explicit
|
||||
gates rather than assuming atomic commits across repositories.
|
||||
- Document only implemented behavior outside roadmap files.
|
||||
|
||||
## Completion Criteria
|
||||
|
||||
The migration is complete when:
|
||||
|
||||
- Promptkit is the independent owner of the reusable framework and built-in
|
||||
profiles;
|
||||
- Scriptorium is a slim CLI and HTTP consumer of Promptkit;
|
||||
- Scriptorium no longer exposes or maintains the former public Go framework;
|
||||
- all required downstream migrations and external repository work have been
|
||||
completed or explicitly dispositioned;
|
||||
- both repositories build, test, document, version, and release independently.
|
||||
Reference in New Issue
Block a user