scriptorium/docs/roadmap/migration.md

# Promptkit Migration Roadmap

## Status

Accepted plan. Steps 1 through 4 are complete. Steps 5 through 9 remain
proposed and are not yet implemented.

## Objective

Split the current repository into two projects:

- **Promptkit**: the reusable Go framework, public Go facade, execution engine,
  source and validation support, OpenAI-compatible client, extension
  interfaces, and built-in execution-profile registry.
- **Scriptorium**: a slim runnable application that imports Promptkit and
  provides the CLI and HTTP interfaces.

Scriptorium will become another downstream Promptkit consumer rather than the
owner of the framework.

## Compatibility And Migration Policy

This is an intentionally breaking change.

- New and migrated Go consumers must import Promptkit instead of Scriptorium.
- Scriptorium will not retain type aliases, forwarding packages, deprecated
  facade APIs, or other source-compatibility shims.
- Existing consumers may continue using a previously tagged Scriptorium module
  version until they are migrated.
- The migration does not need to preserve compatibility between intermediate
  development states. Each completed phase must instead leave the affected
  repository internally consistent and tested.
- Promptkit should initially preserve the useful shape and behavior of the
  current public Go facade where doing so reduces extraction risk. Broader API
  redesign should follow the split unless required to establish the new
  boundary.

## Target Ownership

Promptkit should own application-neutral framework behavior:

- public engine, request, result, option, extension, and error APIs;
- prompt-definition loading and rendering;
- execution profiles, overlays, and the built-in profile registry;
- artifact-loading interfaces and general-purpose `file` and `inline` support;
- schema loading and output validation;
- LLM client boundaries and the OpenAI-compatible implementation;
- preparation and execution orchestration;
- framework and execution defaults.

Scriptorium should own executable and transport concerns:

- the `scriptorium` command and its `run`, `render`, and `serve` interfaces;
- CLI parsing, output formatting, exit codes, and process behavior;
- application-config discovery and CLI precedence;
- HTTP routing, request and response DTOs, limits, and error/status mapping;
- HTTP artifact-root and deployment security policy;
- server and adapter defaults;
- executable examples, operations guidance, and transport documentation.

The intended dependency direction is:

```text
Scriptorium CLI and HTTP adapters
                |
                v
             Promptkit
                |
                v
     consumer-supplied sources and clients
```

Scriptorium must use Promptkit's public API. It must not depend on Promptkit
implementation packages or reproduce framework orchestration.

## Migration Steps

### Step 1: Refresh And Synchronize Documentation

Perform a repository-wide documentation refresh before migration development.

At minimum:

- reconcile all current-behavior documentation with the code, tests, examples,
  defaults, and current public contracts;
- introduce the planned documentation-policy updates;
- establish an architecture decision record policy and canonical ADR location;
- resolve stale, duplicated, or misplaced material;
- validate documentation links and maintained examples;
- leave future migration behavior in `docs/roadmap/` until implemented.

**Gate:** Do not begin architectural migration work until the documentation
refresh and policy updates are merged and the repository has an agreed,
accurate baseline.

**Gate status:** Complete as of 2026-07-26. The completed documentation
refresh, follow-up verification, and layered-timeout correction remain recorded
in repository history. Step 1 remains complete after that validation.

### Step 2: Record The Architectural Decision And Detailed Boundary

Create an ADR, under the policy established in Step 1, that records:

- the decision to split Promptkit from Scriptorium;
- the target ownership and dependency direction;
- the selected Promptkit repository and Go module paths;
- the breaking-change and versioning policy;
- ownership of configuration fields and defaults;
- artifact-reader and HTTP containment responsibilities;
- local multi-repository development and release coordination;
- documentation ownership after the split.

Use the ADR to resolve any remaining public-boundary decisions before code is
moved.

**Gate:** The ADR is accepted, and every existing package, public contract,
configuration category, and maintained asset has a target owner.

**Gate status:** Complete as of 2026-07-26.
[ADR 0002: Split Promptkit From Scriptorium](../adr/0002-split-promptkit-from-scriptorium.md)
is accepted and records the required ownership and coordination decisions.

### Step 3: Characterize Existing Framework Behavior

Strengthen or add contract-focused tests where needed so extraction can be
verified without relying on package placement.

The completed Step 3 gate records the accepted implementation scope and
intended completion state.

Preserve coverage of:

- `Prepare` and `Run` behavior;
- prompt, profile, execution-default, and request-override precedence;
- presence-aware numeric overrides;
- built-in profile fallback and custom-profile overlays;
- strict YAML and JSON decoding;
- prompt, profile, schema, and artifact source behavior;
- structured-output requests and output validation;
- validation failures versus validation runtime errors;
- secret handling and redaction;
- public error classification;
- HTTP artifact restrictions and transport mappings.

**Gate:** Current framework and adapter contracts are represented by passing
tests sufficient to detect behavioral regressions during the split.

**Gate status:** Complete as of 2026-07-27. The framework contract corpus,
public `Engine` characterization, ownership audit, full test and vet suites,
temporary executable build, and maintained offline examples passed. Step 4 is
next.

### Step 4: Make Scriptorium Adapters Consume The Public Facade

Within the current repository, refactor the CLI and HTTP adapters to use the
public framework facade rather than constructing or importing internal runner
components directly.

Add only the minimum public capabilities needed to support this boundary. These
may include:

- a small `Run`/`Prepare` consumer interface;
- injectable artifact-reading behavior for Scriptorium's restricted HTTP
  policy;
- source options currently available only through internal constructors;
- prepared-run formatting based on public types;
- stable public error classification required by CLI and HTTP mappings.

Do not broadly export internal repositories, domain types, or use-case
implementations.

**Gate:** The CLI and HTTP adapters use only the public framework API for
framework behavior, and all tests and documented smoke commands pass.

**Gate status:** Complete as of 2026-07-28. CLI `run`, `render`, and `serve`,
the HTTP handler, and prepared-run formatting use the public facade; the
restricted HTTP reader is injected through the public extension point. The
post-implementation public-error, deterministic MIME, and recursive
dependency-guard corrections passed full tests, vet, build, race checks,
maintained examples, and configuration smoke checks. Step 5 repository
creation is next.

### Step 5: Create The Promptkit Repository

Create the Promptkit repository and Go module as an explicit out-of-band
operation.

Establish:

- repository access, ownership, and branch protections;
- the module path selected by the ADR;
- baseline development, architecture, documentation, and release policies;
- documented maintainer-run test, vet, build, formatting, documentation-link,
  and repository-hygiene validation;
- a tag-based release procedure for source commits and semantic Go module tags,
  without runnable binaries or binary packaging;
- an initial package layout centered on a small public facade with internal
  implementation packages;
- a local development workflow for coordinated Promptkit and Scriptorium
  changes, using a workspace or temporary uncommitted module replacement where
  appropriate.

Do not commit local filesystem `replace` directives to release branches.
[ADR 0003](../adr/0003-use-maintainer-run-validation-and-tag-only-releases-for-promptkit.md)
records the controlling Promptkit validation and release decision.

**Gate:** The Promptkit repository exists, is accessible to maintainers, has
documented maintainer-run validation and tag-based release procedures, has the
required policy scaffolding, and can receive the extracted framework. Do not
begin cross-repository extraction until this out-of-band work is confirmed
complete.

### Step 6: Extract And Stabilize Promptkit

Move the application-neutral framework and built-in profile assets into
Promptkit. Preserve implementation packages as internal where practical.

The initial public API should remain focused on the established engine workflow
and the source and client extension points required by real consumers. Avoid
combining the extraction with unrelated API redesign.

Move or recreate the relevant:

- framework implementation;
- public package tests and framework contract tests;
- built-in profile assets and registry tests;
- Go consumer examples;
- framework, consumer, configuration-format, and integration documentation.

Verify that Promptkit can be built, tested, and consumed independently of the
Scriptorium repository.

**Gate:** Promptkit independently provides the agreed framework contract,
passes its documented validation, and has published its first versioned tag
before Scriptorium or another consumer adopts it, as required by
[ADR 0003](../adr/0003-use-maintainer-run-validation-and-tag-only-releases-for-promptkit.md).

### Step 7: Slim Scriptorium And Adopt Promptkit

Update Scriptorium to import the tagged Promptkit module and remove the
framework implementation and public Go facade that Promptkit replaces.

Retain only Scriptorium-owned executable and transport behavior. In particular:

- wire CLI and HTTP requests through Promptkit's public API;
- keep application config and transport defaults in Scriptorium;
- keep restricted HTTP artifact policy in Scriptorium while injecting it
  through Promptkit's supported boundary;
- remove obsolete framework packages, tests, and documentation;
- update Scriptorium examples and docs to describe the CLI and HTTP application;
- direct Go framework consumers to Promptkit without providing compatibility
  aliases or forwarding APIs.

**Gate:** Scriptorium builds and passes all tests using a tagged Promptkit
dependency, contains no duplicate framework implementation, and its current
documentation describes only the slimmed application.

### Step 8: Migrate Downstream Consumers To Promptkit

Inventory downstream Go consumers and migrate each from the Scriptorium package
to Promptkit. This work may occur in external repositories and must be tracked
explicitly.

For each consumer:

- update module imports and dependencies;
- adapt to any intentionally changed public API;
- run its tests and relevant integration or smoke checks;
- confirm configuration, source, validation, and error behavior;
- release or deploy the migrated consumer through its normal process.

Consumers that cannot migrate immediately may remain pinned to the last
framework-bearing Scriptorium tag. No compatibility work is required in the new
Scriptorium project for those consumers.

**Gate:** All in-scope downstream consumers are either migrated and verified or
explicitly recorded as remaining on the previous Scriptorium version with an
owner and follow-up plan. Do not declare the ecosystem migration complete until
the required out-of-band consumer changes are confirmed.

### Step 9: Complete Release And Documentation Cutover

Complete the coordinated project transition:

- publish Promptkit before dependent Scriptorium releases;
- release the breaking Scriptorium version against the tagged Promptkit
  dependency;
- publish migration guidance that maps the former Scriptorium Go API to
  Promptkit;
- update cross-project links, examples, package documentation, and release
  notes;
- verify that no release artifact depends on local workspaces or replacements;
- archive completed roadmap material according to the documentation policy in
  effect at that time.

**Gate:** Promptkit and Scriptorium are independently releasable, their
documentation has distinct and accurate ownership, and the migration status of
all identified downstream consumers is recorded.

## Cross-Cutting Constraints

- Preserve the invariant that execution orchestration remains narrow and
  application-neutral.
- Keep adapter-specific decisions out of Promptkit.
- Keep Scriptorium dependent only on Promptkit's supported public API.
- Preserve strict external decoding, error classification, validation
  semantics, and secret redaction throughout the migration.
- Keep each repository buildable and testable at merged phase boundaries.
- Coordinate cross-repository changes through tagged dependencies and explicit
  gates rather than assuming atomic commits across repositories.
- Document only implemented behavior outside roadmap files.

## Completion Criteria

The migration is complete when:

- Promptkit is the independent owner of the reusable framework and built-in
  profiles;
- Scriptorium is a slim CLI and HTTP consumer of Promptkit;
- Scriptorium no longer exposes or maintains the former public Go framework;
- all required downstream migrations and external repository work have been
  completed or explicitly dispositioned;
- both repositories build, test, document, version, and release independently.