scriptorium/docs/roadmap/step6.md

# Migration Step 6: Extract And Stabilize Promptkit

## Status

Proposed scope. Migration Step 5 is complete, the Promptkit repository
foundation is available, and framework extraction has not yet begun.

## Purpose

Establish Promptkit as the independent owner of Scriptorium's reusable
prompt-execution framework, public Go facade, built-in profile registry,
framework tests, consumer example, and framework documentation.

This step delivers and publishes the first usable Promptkit library without
changing Scriptorium to consume it. The
[main migration roadmap](migration.md) owns the overall sequence.
[Step 6 implementation plan](implementation.md) owns execution staging and
validation gates; this document owns the desired end state and completion
policy.
[ADR 0002](../adr/0002-split-promptkit-from-scriptorium.md) owns the project
boundary, breaking-change policy, module path, versioning direction, and
cross-repository release order.
[ADR 0003](../adr/0003-use-maintainer-run-validation-and-tag-only-releases-for-promptkit.md)
owns Promptkit's maintainer-run validation and tag-only release model.

## Confirmed Migration Policy

- Promptkit's module and public import path remain
  `gitea.maximumdirect.net/eric/promptkit`.
- The public package is `promptkit` at the module root.
- This migration is intentionally breaking. Promptkit does not provide aliases,
  forwarding packages, or another compatibility layer for the former
  Scriptorium import path.
- The initial Promptkit facade preserves the useful exported shape and
  observable behavior of the characterized Scriptorium facade, except for
  project and package identity changes required by the new module.
- Extraction must not become an unrelated public API redesign. Broader changes
  follow the split unless correctness or the accepted ownership boundary
  requires them.
- Promptkit owns application-neutral framework behavior. Scriptorium continues
  to own executable, CLI, HTTP-server, application-configuration, deployment,
  and process concerns.
- Promptkit must remain independently buildable and testable without a
  Scriptorium checkout, Go workspace, local replacement, or unpublished
  dependency.
- Promptkit's first release is `v0.1.0`. The tag must be published before
  Scriptorium or another consumer adopts Promptkit.
- Scriptorium adoption and removal of its former framework implementation
  belong to Step 7, not this step.

## Extraction Model And Transitional Ownership

Step 6 establishes the framework in Promptkit while leaving the current
Scriptorium module operational until Step 7. Because the repositories cannot
land one atomic cross-repository change, the Scriptorium framework code and
current Go-consumer documentation remain temporarily in place during this
step.

At Step 6 completion:

- Promptkit is the canonical project for new framework development and the
  tagged library contract.
- Scriptorium still contains its pre-cutover framework copy solely so the
  executable and existing module remain buildable before Step 7.
- New framework features and unrelated framework refactors do not land in the
  Scriptorium copy during this transition.
- A necessary defect or security correction discovered before Step 7 is
  applied consistently to Promptkit and to any still-executed Scriptorium copy,
  with contract tests protecting the shared observable behavior.
- No source file, test, asset, or documentation in either repository reads from
  the other repository's working tree.

This temporary duplication is a migration state, not the target architecture.
Step 7 removes the Scriptorium framework copy after adopting the tagged
Promptkit module.

## Target Promptkit Repository State

Promptkit becomes a complete reusable Go library with:

- a supported root public facade;
- internal framework implementation packages;
- embedded built-in execution profiles;
- strict prompt, profile, and schema loading;
- ordinary inline and caller-selected file artifact reading;
- prompt rendering, preparation, generation, validation, and error
  classification;
- a provider-neutral model-client boundary and built-in OpenAI-compatible
  client;
- contract-focused, deterministic, offline tests;
- current consumer, format, integration, and internal documentation;
- a maintained, offline Go consumer example; and
- a published `v0.1.0` Go module tag.

Promptkit has no runnable command, HTTP service, binary release, hosted CI
configuration, application-config discovery, deployment policy, or
Scriptorium dependency.

## Public Go Facade

### Package Identity

The existing Scriptorium root facade is extracted to package `promptkit`.
Exported GoDoc, package comments, examples, formatted request summaries, and
other project-identifying text use Promptkit terminology. In particular,
`String` and `GoString` representations identify `promptkit.RunRequest` and
`promptkit.GenerateRequest`, not the former package.

No import-path compatibility package is created in either repository.

### Engine Construction And Sources

Promptkit provides:

- `NewEngine(Config, ...Option)`;
- `Engine.Prepare`;
- `Engine.Run`;
- directory-backed prompt, profile, and schema sources through `Config`;
- `WithPromptFS` and `WithPromptFile`;
- `WithProfileFS`, `WithProfileFile`, and `WithProfiles`;
- `WithSchemaFS` and `WithSchemaFile`;
- `WithArtifactReader`; and
- `WithLLMClient`.

Nil options retain the characterized behavior. Invalid construction and invalid
extension values preserve the public `ErrInvalidConfig` category.

Prompt definitions, profiles, schemas, source selection, overlay precedence,
and strict-decoding behavior remain application-neutral Promptkit concerns.
Scriptorium configuration-file discovery and CLI precedence do not move with
them.

### Public Values And Extension Points

The initial root facade includes the currently characterized:

- request, prepared-run, result, artifact, execution-target, output-contract,
  validation, usage, rendered-prompt, cache-control, and structured-output
  values;
- serialized enum values for artifact types, output formats, validation modes,
  validation statuses, cache control, and structured output;
- presence-aware numeric request overrides;
- `ArtifactReader` extension point and `File`, `Inline`, and `InlineWithURI`
  helpers;
- `LLMClient`, `GenerateRequest`, and `GenerateResponse` extension boundary;
- in-memory `Profile`, `OpenAICompatibleProfileConfig`, and
  `OpenAICompatibleProfile`; and
- public sentinel errors used by Go consumers to classify failures through
  `errors.Is`.

Exported names, signatures, serialized values, copying and mutation isolation,
nil handling, and error relationships match the characterized Scriptorium
facade unless the module/package rename necessarily changes the observable
text. `ErrProfileRequired` and `ErrAPIKeyEnvMissing` continue also to match
`ErrInvalidRequest`.

The root facade does not expose internal domain types, repositories, renderers,
validators, concrete model clients, adapter DTOs, or assembly constructors.
No speculative extension point or public subpackage is introduced.

### Preparation And Execution Contract

Promptkit preserves the characterized engine workflow:

- `Prepare` validates and loads the prompt and profile, resolves execution
  settings and credentials, loads any structured-output schema, reads input
  artifacts, renders messages, computes hashes, and returns prepared state
  without calling a model.
- `Run` reuses the same preparation flow, makes one generation call, creates
  the output artifact, validates generated content, and returns result,
  validation, usage, hash, and timing metadata.
- Generated-content validation failures remain successful calls with a failed
  validation result. Schema access, compilation, and validator runtime failures
  remain errors.
- Optional internal repair behavior and its tests remain framework-owned, but
  extraction does not add a new public repair API.
- The framework creates no durable run store, checkpoint, archive, or resume
  workflow.

## Framework Package Ownership

The implementation may reorganize files when needed for clean Promptkit
boundaries, but the completed library owns these current Scriptorium
responsibilities:

| Current Scriptorium package or file group | Promptkit responsibility |
| --- | --- |
| Root facade Go files and facade tests | Root `promptkit` package, public construction, values, options, conversions, error mapping, redaction, and facade contracts. |
| `internal/domain` | Framework request, preparation, execution, result, validation, artifact, profile, and model-client domain values. |
| `internal/usecase` | Preparation and execution orchestration, setting resolution, validation coordination, and internal repair boundary. |
| `internal/filecatalog` | Deterministic YAML discovery and source-root helpers. |
| `internal/promptdef` | Strict prompt-definition loading from filesystem and `fs.FS` sources. |
| `internal/profile` | Strict execution-profile loading, validation, filesystem and `fs.FS` sources, and overlay behavior. |
| `internal/profile/builtin` | Embedded built-in profile registry and assets. |
| `internal/prompt` | Go-template prompt rendering and cache-control handling. |
| `internal/artifact` | Ordinary inline and unrestricted caller-selected file artifact reading. |
| `internal/validate` | Basic, JSON, and JSON Schema validation plus filesystem and `fs.FS` schema access. |
| `internal/llm` | Provider-neutral internal client boundary and OpenAI-compatible implementation. |
| Framework-owned members of `internal/defaults` | Schema fallback, execution defaults, output artifact and content types, OpenAI request path, and built-in client transport cap. |
| Framework contract and package test data | Repository-local Promptkit fixtures that support the moved public and internal test suites. |

Internal packages use the Promptkit module path and may import other Promptkit
internal packages only in the dependency direction defined by its architecture
policy. They do not import Scriptorium.

## Defaults And Settings

Promptkit owns the behavior and defaults used by its engine:

- empty `SchemaDir` uses the framework's current `.` fallback;
- execution temperature, token, top-p, and generation-timeout defaults;
- output artifact name and content types;
- the OpenAI-compatible chat-completions path;
- the built-in client's ten-minute transport-cap fallback; and
- source and validation defaults that do not depend on an application
  transport.

The execution-setting hierarchy remains request override, then non-zero profile
value, then framework default. Numeric request fields remain presence-aware so
explicit zero is distinct from omission.

Scriptorium retains ownership of its server address, request and response byte
limits, HTTP read-header timeout, CLI defaults, render-format defaults, and
application configuration precedence. Those values do not enter Promptkit.

## Source, Profile, And Built-In Registry Behavior

Promptkit preserves:

- strict YAML decoding and rejection of unknown fields;
- prompt selection by ID and optional version rather than filename;
- prompt `content_file` resolution relative to its owning source and
  containment within `fs.FS` roots;
- profile selection by ID, validation ranges, raw-key rejection, and
  `api_key_env` behavior;
- in-memory profile precedence over an explicit file or filesystem source,
  followed by built-ins;
- fallback to built-ins only for a genuine profile-not-found result, without
  hiding primary-source decoding or validation failures;
- schema path resolution and the distinction between validation mismatch and
  operational failure; and
- ordinary inline/file artifact hashing, content-type handling, copying, and
  failure behavior.

The complete current embedded built-in profile registry moves into Promptkit.
All assets remain embedded, strict-loader-valid, uniquely identifiable, and
covered by registry tests. No built-in asset is silently added, removed, or
semantically changed merely as part of extraction.

Promptkit's ordinary file reader is intentionally not a deployment sandbox. It
may read a path deliberately selected by an in-process consumer. Rooted
containment, byte limits, denial policy, and symlink/deployment policy for the
Scriptorium HTTP interface remain in Scriptorium.

## OpenAI-Compatible Client

Promptkit owns the characterized outbound client contract:

- endpoint construction and `POST /chat/completions`;
- rendered messages, session IDs, cache-control blocks, structured-output
  payloads, and flattened provider-specific parameters;
- reserved-field protection and JSON-compatible extra-parameter validation;
- direct request API-key precedence over environment lookup;
- model and execution-setting mapping, including explicit numeric zero values;
- response decoding and token-usage mapping;
- non-success and malformed-response classification without exposing provider
  response bodies; and
- no implicit retry, tool-call, or durable-session behavior.

Timeout enforcement remains layered:

- a positive supplied `HTTPClient.Timeout` is the transport cap;
- otherwise a positive `Config.Timeout` supplies the transport cap;
- otherwise the internal ten-minute cap applies;
- a positive effective `timeout_seconds` creates a per-generation context
  deadline;
- an explicit request value of zero disables only the generation deadline; and
- the earliest caller-context deadline, transport cap, or positive generation
  deadline terminates the call.

Supplied HTTP clients are cloned and not mutated. Cancellation and transport
failures retain the public generation-error category.

## Application Concerns That Do Not Move

Step 6 does not copy or recreate these Scriptorium-owned components in
Promptkit:

- `cmd/scriptorium`;
- `internal/adapter/cli`;
- `internal/adapter/http`, including the restricted HTTP artifact reader;
- `internal/config`;
- `internal/format`;
- CLI and HTTP DTOs, parsing, output, status mapping, and transport limits;
- configuration-file discovery and CLI-over-configuration precedence;
- process cancellation, logging, server construction, or operations behavior;
- executable packaging, subprocess integration, or deployment guidance; or
- Scriptorium-specific executable examples and configuration files.

Use of `net/http` for Promptkit's outbound model client and the public
`*http.Client` construction option remains valid framework behavior. It does
not authorize an inbound HTTP service or transport adapter in Promptkit.

## Dependencies And Repository Independence

Promptkit adds only dependencies required by the extracted implementation. The
initial extraction retains the currently validated YAML and JSON Schema library
versions unless a correctness issue requires an explicit change. Module
metadata is tidied and includes only direct and transitive dependencies used by
Promptkit.

The completed Promptkit module:

- has no dependency on `gitea.maximumdirect.net/eric/scriptorium`;
- contains no import of a Scriptorium package;
- contains no committed `go.work`, `go.work.sum`, or local filesystem
  `replace`;
- does not read Scriptorium-relative paths from code, tests, examples, or
  documentation;
- does not require Scriptorium environment variables, commands, or build
  artifacts; and
- builds and tests successfully from an independent clean checkout.

Temporary local workspace or replacement configuration may be used while
developing the cross-repository extraction, subject to Promptkit's development
guide. It is removed before acceptance and does not appear in a commit or tag.

## Test And Fixture Ownership

Tests move with the framework behavior they protect. Promptkit owns:

- public facade construction, source, profile, execution, validation,
  extension, timeout, copying, redaction, and error-classification tests;
- runner and domain contract tests;
- prompt, profile, built-in registry, schema, artifact, renderer, file-catalog,
  and OpenAI-compatible client tests;
- deterministic test doubles for model and external boundaries;
- package-local prompt, profile, schema, and artifact test data; and
- representative end-to-end library preparation and execution coverage.

Tests are rewritten only for Promptkit identity, repository-local paths, clean
package boundaries, or an explicitly documented correction. They do not lose
meaningful behavioral coverage merely because internal files move.

Promptkit tests remain deterministic, offline, parallel-safe, and independent
of credentials, paid APIs, live providers, mutable services, Scriptorium, and
test execution order. HTTP client tests use controlled local transports or
test servers as appropriate.

Scriptorium's CLI, HTTP, restricted-reader, application-config, prepared-output
formatting, executable dependency-guard, and process tests remain in
Scriptorium. Step 6 does not move those transport tests into Promptkit.

## Documentation And Examples

Promptkit's durable documentation describes the implemented library at Step 6
completion and owns the framework contracts. At minimum:

- `README.md` presents Promptkit as usable, includes a minimal current
  orientation or quickstart, and no longer says framework behavior is
  unimplemented.
- Go declarations and GoDoc own exact exported names, signatures, fields,
  values, and error contracts.
- `docs/consumers/pkg-promptkit.md` provides task-oriented construction,
  preparation, execution, source, profile, credential, extension, and error
  guidance without duplicating exact Go declarations.
- `docs/formats.md` owns prompt-definition, profile, schema-reference,
  validation, execution-setting, built-in-profile, and credential file-format
  contracts.
- `docs/integrations/openai-compatible-chat.md` owns the outbound HTTP wire and
  timeout contract.
- `docs/internal/runner.md`, `docs/internal/sources.md`, and
  `docs/internal/llm.md` describe the implemented internal behavior and tests.
- `docs/internal/overview.md` inventories every implemented package and links
  to its canonical contract or focused internal document.
- `docs/development.md`, the architecture, documentation, and testing policies,
  and `docs/release.md` are reconciled with the implemented package tree and
  validation workflow.

Promptkit's documentation policy is updated to assign one canonical owner to
framework formats. Documentation uses Promptkit package names and import paths,
does not claim CLI or inbound HTTP ownership, and does not rely on temporary
Scriptorium roadmap files for its current public contract.

Promptkit contains a maintained Go consumer example under
`examples/go-library/prepare`. The example:

- imports Promptkit at its canonical module path;
- is self-contained within the Promptkit repository;
- uses only implemented public APIs;
- performs an offline representative preparation flow without credentials or
  a live model call;
- does not read Scriptorium's examples or working tree; and
- is executed as part of acceptance validation.

Any example-specific prompt, profile, schema, or fixture asset belongs to
Promptkit and is minimal, synthetic, secret-free, and maintained with the
example. Scriptorium's executable examples remain in Scriptorium until their
Step 7 documentation cutover.

Because Scriptorium still implements and exposes its old facade during this
transition, its current Go and framework documentation remains in place through
Step 6. Step 7 replaces that material with Scriptorium application guidance and
links Go consumers to Promptkit.

## Release Outcome

After the extracted library and documentation satisfy every acceptance check,
Promptkit publishes annotated semantic Go module tag `v0.1.0` using its
documented release procedure.

The tag:

- identifies the validated Promptkit commit;
- contains no workspace, replacement, generated binary, or release artifact;
- is available from the configured Promptkit origin;
- resolves to the independently validated source commit; and
- is the version that Step 7 will use when Scriptorium adopts Promptkit.

No Scriptorium dependency update, consumer release, Promptkit binary, or
hosting-provider-specific release artifact is part of this release.

## Required Validation Outcome

From an independent clean Promptkit checkout with no active workspace or local
replacement, maintainers must successfully run:

```sh
go test ./...
go test -race ./...
go vet ./...
go build ./...
gofmt -l $(git ls-files '*.go')
git diff --check
```

The formatting command produces no paths. Acceptance also requires:

- `go list -m` reports module
  `gitea.maximumdirect.net/eric/promptkit` and the intended Go version;
- the root package reports name `promptkit` and the canonical import path;
- `go mod tidy` leaves module metadata clean;
- the module graph and all source imports contain no Scriptorium dependency;
- every package test and representative library workflow is offline and
  deterministic;
- the maintained Go consumer example runs successfully and produces its
  documented output shape;
- all embedded built-in profiles load and the registry tests pass;
- all maintained Markdown links, Go snippets, import paths, commands, and
  repository-relative paths are valid;
- current Promptkit documents agree on package ownership, behavior, validation,
  and release state;
- no template residue, stale Scriptorium package identity, real credential,
  private fixture, or unsupported future claim remains;
- no command package, inbound HTTP adapter, application config, workspace,
  replacement, hosted CI configuration, generated binary, or unrelated
  artifact is present;
- Git status is clean before tagging; and
- the published `v0.1.0` tag passes the verification required by
  `docs/release.md`.

The current Scriptorium repository must continue to pass:

```sh
go test ./...
go vet ./...
go build ./cmd/scriptorium
```

Scriptorium documentation links and maintained examples also remain valid.
These checks confirm that Step 6 did not break the pre-cutover executable while
Promptkit was established independently.

## Out Of Scope

Step 6 does not:

- update Scriptorium imports or `go.mod` to consume Promptkit;
- delete the Scriptorium root facade or framework packages;
- remove or redirect Scriptorium's current Go-consumer documentation;
- migrate CLI, HTTP, application configuration, prepared-output formatting, or
  restricted HTTP artifact behavior;
- migrate downstream consumers;
- provide a Scriptorium compatibility facade;
- redesign the public engine around a new workflow;
- add retries, tool calls, durable sessions, checkpoints, or multi-step
  orchestration;
- add a public repair API without a separately accepted need;
- add hosted Promptkit CI, binary packaging, or executable releases;
- introduce placeholder public packages or speculative extension points;
- upgrade unrelated dependencies; or
- begin the Step 7 Scriptorium cutover before `v0.1.0` is published and
  verified.

## Completion Criteria

Step 6 is complete when:

- Promptkit independently implements the application-neutral framework
  contract owned by ADR 0002;
- the root `promptkit` facade exposes the characterized engine, values,
  sources, profiles, extension points, and public error identities without
  compatibility shims or speculative exports;
- preparation, execution, profile/default precedence, presence-aware
  overrides, artifact loading, rendering, structured output, validation,
  timeout layering, redaction, and error classification retain their
  characterized behavior;
- all framework packages, tests, fixtures, and the complete built-in profile
  registry are owned and maintained within Promptkit;
- Scriptorium-only executable, adapter, application-config, restricted-reader,
  formatting, deployment, and process concerns are absent from Promptkit;
- Promptkit's module dependencies are minimal, tidy, and independent of
  Scriptorium;
- current consumer, format, integration, internal, contributor, architecture,
  testing, and release documentation has one coherent Promptkit-specific
  ownership model;
- the maintained Promptkit Go example is self-contained, offline, and passing;
- the complete Promptkit validation suite, race suite, documentation checks,
  repository-hygiene checks, and release checks pass from a clean independent
  checkout;
- Scriptorium remains independently buildable, tested, and documented in its
  pre-cutover state;
- Promptkit `v0.1.0` is published and verified against the validated commit;
- no Scriptorium or downstream consumer has adopted an unpublished Promptkit
  revision or a local replacement; and
- `migration.md` records Step 6 as complete and identifies the tagged
  Scriptorium adoption and framework-removal work in Step 7 as the next gate.

Migration Step 7 must not begin until these criteria are satisfied.

## Lifecycle

This feature roadmap is a temporary cross-repository migration artifact. It
remains in Scriptorium while Step 6 is active and may be reduced to a completion
record or removed after the migration status and durable contracts have moved
to their canonical owners.