scriptorium/docs/roadmap/step3.md

# Migration Step 3: Framework Characterization

## Status

Completed on 2026-07-27. The target state below is characterized by the
passing baseline; its ordered implementation record is in
[implementation.md](implementation.md).

## Purpose

Characterize the framework and adapter contracts that must survive the
Promptkit extraction. The work should make those contracts portable across the
future repository boundary without duplicating behavior at every test layer or
changing production APIs ahead of Migration Step 4.

The [testing policy](../policy/testing.md) governs test value, ownership, and
sufficiency. The [accepted split decision](../adr/0002-split-promptkit-from-scriptorium.md)
governs which project will own each behavior.

## Baseline Findings

The current suite already provides broad coverage at the public facade,
framework package, CLI, HTTP, source, validation, and model-client boundaries.
Step 3 is therefore a portability and risk-closing exercise rather than a
general coverage expansion.

The principal extraction risk is fixture ownership:

- public `Engine` tests rely extensively on executable assets under
  `examples/`;
- the assembled runner integration test also reads those assets; and
- the accepted split leaves executable examples in Scriptorium while moving
  the public facade and framework tests to Promptkit.

Those dependencies would either prevent the tests from moving or create
unwanted cross-repository fixture coupling. A smaller set of public-boundary
gaps also remains around complete setting precedence, cancellation, malformed
injected-client behavior, and reserved provider parameters.

## Target State

The completed baseline provides:

- tests destined for Promptkit use only Promptkit-destined testdata or
  fixtures generated within the test;
- Scriptorium's maintained executable examples remain independently validated
  by Scriptorium-owned checks;
- each behavior named in the main migration roadmap has one clear test owner;
- representative public `Engine` tests protect assembled framework behavior;
- focused package tests continue to own strict parsing, source mechanics,
  validation rules, and provider mapping without higher-level duplication;
- Scriptorium adapter tests continue to protect CLI, HTTP, containment, limits,
  and transport mappings; and
- no production API or package boundary has changed as part of Step 3.

## Policy Choices

### Framework-Owned Testdata

Promptkit-destined tests will use a compact framework contract corpus under root
`testdata/` or a purpose-built fixture created inside the test. The shared
corpus will contain only the assets needed to express durable framework
behavior:

- one ordinary prompt definition;
- one JSON Schema structured-output prompt definition;
- representative execution profiles;
- one JSON Schema document; and
- small file-artifact inputs.

The corpus should exercise directory-backed loading and relative prompt content
where those behaviors matter. Use small inline `fstest.MapFS` or temporary
fixtures for cases that do not benefit from shared files.

The complete executable example tree will not be copied. Testdata will remain
minimal, synthetic, secret-free, and distinct from user-facing examples.

### Test Boundaries And Consolidation

Representative assembled behavior belongs at the public `Engine` boundary.
Focused parsing, source, validation, provider, and adapter mechanics remain
with their package-level owners. Existing tests should be consolidated when a
new public contract test would otherwise duplicate the same risk.

The assembled runner integration behavior will be protected through the public
facade rather than through a second test tied to internal domain and runner
types. Promptkit-destined tests will not read Scriptorium-owned `examples/`
assets. Scriptorium adapter and maintained-example checks may continue to do so
where the example itself is the contract under test.

### Required Public Characterization

The public suite will characterize:

- an assembled directory-backed `Engine.Run` workflow with file inputs,
  structured output, schema validation, hashes, usage, and timing;
- framework-default, profile, and request execution-setting precedence;
- explicit numeric-zero propagation and target-presence metadata;
- caller-context cancellation at the outbound generation boundary and its
  public error classification;
- nil responses from injected model clients; and
- reserved provider parameters failing before a provider call.

These tests will use deterministic synchronization, real local collaborators
where inexpensive, and fakes only at the model-provider boundary.

## Target Test Ownership

| Test category | Future disposition |
| --- | --- |
| Public `Engine`, facade, model-client extension, and public error contracts | Move to Promptkit. |
| Framework domain, runner, prompt, profile, built-in registry, general artifact, validation, and LLM package tests | Move with their Promptkit-owned implementation. |
| CLI parsing, application configuration, prepared-run formatting, and process behavior | Remain in Scriptorium. |
| HTTP DTOs, strict JSON, limits, response mapping, and rooted artifact containment | Remain in Scriptorium. |
| Tests that construct internal runners or classify internal framework sentinels from Scriptorium adapters | Preserve their observable assertions, then rewrite against the public Promptkit boundary during Step 4. |
| Maintained Go consumer example | Move to Promptkit. |
| Maintained executable configuration, render, HTTP, and fixture examples | Remain in Scriptorium. |

Existing focused tests remain the owners of:

- strict prompt, profile, and application YAML decoding;
- strict HTTP JSON decoding;
- prompt, profile, schema, and artifact source mechanics;
- built-in profile validation and overlay fallback;
- structured-output encoding and schema validation;
- validation content failures versus operational failures;
- credential handling and redaction;
- OpenAI-compatible wire behavior; and
- HTTP artifact restrictions and transport mappings.

Coverage will be added only if the ownership audit identifies a consequential
behavior with no credible existing owner.

## Required Validation Outcome

The characterized baseline must pass the complete Go test and vet suites, a
temporary-output executable build, repeated public contract tests, both
maintained application configurations, maintained render and Go consumer
examples, the maintained HTTP request-example check, documentation-link
validation, and whitespace validation. All checks must remain offline and
independent of real credentials.

## Out Of Scope

Step 3 does not:

- add artifact-reader, repository, validator, or other production extension
  APIs;
- refactor CLI or HTTP adapters to consume the public facade;
- move restricted HTTP artifact behavior out of its current package;
- create the Promptkit repository or change the Go module path;
- move implementation packages between repositories;
- create compatibility aliases or forwarding APIs;
- redesign the public facade; or
- add tests solely to increase a coverage percentage.

Those changes belong to later migration steps.

## Completion Criteria

Step 3 is complete when:

- Promptkit-destined tests have no dependency on Scriptorium-owned executable
  examples;
- the targeted public contract gaps are covered with deterministic tests;
- the full roadmap behavior list has a clear, non-duplicative test owner;
- tests that require Step 4 rewrites are explicitly identified;
- no production behavior or public API changed;
- all required validation passes; and
- the main migration roadmap records the Step 3 gate as complete.

Migration Step 4 must not begin until these criteria are satisfied.

## Lifecycle

This completed implementation roadmap remains the concise characterization
record for the migration. Repository history retains the detailed implementation
record.