scriptorium/docs/roadmap/step3.md

Migration Step 3: Framework Characterization

Status

Completed on 2026-07-27. The target state below is characterized by the passing baseline; its ordered implementation record is in implementation.md.

Purpose

Characterize the framework and adapter contracts that must survive the Promptkit extraction. The work should make those contracts portable across the future repository boundary without duplicating behavior at every test layer or changing production APIs ahead of Migration Step 4.

The testing policy governs test value, ownership, and sufficiency. The accepted split decision governs which project will own each behavior.

Baseline Findings

The current suite already provides broad coverage at the public facade, framework package, CLI, HTTP, source, validation, and model-client boundaries. Step 3 is therefore a portability and risk-closing exercise rather than a general coverage expansion.

The principal extraction risk is fixture ownership:

• public Engine tests rely extensively on executable assets under examples/;
• the assembled runner integration test also reads those assets; and
• the accepted split leaves executable examples in Scriptorium while moving the public facade and framework tests to Promptkit.

Those dependencies would either prevent the tests from moving or create unwanted cross-repository fixture coupling. A smaller set of public-boundary gaps also remains around complete setting precedence, cancellation, malformed injected-client behavior, and reserved provider parameters.

Target State

The completed baseline provides:

• tests destined for Promptkit use only Promptkit-destined testdata or fixtures generated within the test;
• Scriptorium's maintained executable examples remain independently validated by Scriptorium-owned checks;
• each behavior named in the main migration roadmap has one clear test owner;
• representative public Engine tests protect assembled framework behavior;
• focused package tests continue to own strict parsing, source mechanics, validation rules, and provider mapping without higher-level duplication;
• Scriptorium adapter tests continue to protect CLI, HTTP, containment, limits, and transport mappings; and
• no production API or package boundary has changed as part of Step 3.

Policy Choices

Framework-Owned Testdata

Promptkit-destined tests will use a compact framework contract corpus under root testdata/ or a purpose-built fixture created inside the test. The shared corpus will contain only the assets needed to express durable framework behavior:

• one ordinary prompt definition;
• one JSON Schema structured-output prompt definition;
• representative execution profiles;
• one JSON Schema document; and
• small file-artifact inputs.

The corpus should exercise directory-backed loading and relative prompt content where those behaviors matter. Use small inline fstest.MapFS or temporary fixtures for cases that do not benefit from shared files.

The complete executable example tree will not be copied. Testdata will remain minimal, synthetic, secret-free, and distinct from user-facing examples.

Test Boundaries And Consolidation

Representative assembled behavior belongs at the public Engine boundary. Focused parsing, source, validation, provider, and adapter mechanics remain with their package-level owners. Existing tests should be consolidated when a new public contract test would otherwise duplicate the same risk.

The assembled runner integration behavior will be protected through the public facade rather than through a second test tied to internal domain and runner types. Promptkit-destined tests will not read Scriptorium-owned examples/ assets. Scriptorium adapter and maintained-example checks may continue to do so where the example itself is the contract under test.

Required Public Characterization

The public suite will characterize:

• an assembled directory-backed Engine.Run workflow with file inputs, structured output, schema validation, hashes, usage, and timing;
• framework-default, profile, and request execution-setting precedence;
• explicit numeric-zero propagation and target-presence metadata;
• caller-context cancellation at the outbound generation boundary and its public error classification;
• nil responses from injected model clients; and
• reserved provider parameters failing before a provider call.

These tests will use deterministic synchronization, real local collaborators where inexpensive, and fakes only at the model-provider boundary.

Target Test Ownership

Test category	Future disposition
Public Engine, facade, model-client extension, and public error contracts	Move to Promptkit.
Framework domain, runner, prompt, profile, built-in registry, general artifact, validation, and LLM package tests	Move with their Promptkit-owned implementation.
CLI parsing, application configuration, prepared-run formatting, and process behavior	Remain in Scriptorium.
HTTP DTOs, strict JSON, limits, response mapping, and rooted artifact containment	Remain in Scriptorium.
Tests that construct internal runners or classify internal framework sentinels from Scriptorium adapters	Preserve their observable assertions, then rewrite against the public Promptkit boundary during Step 4.
Maintained Go consumer example	Move to Promptkit.
Maintained executable configuration, render, HTTP, and fixture examples	Remain in Scriptorium.

Existing focused tests remain the owners of:

• strict prompt, profile, and application YAML decoding;
• strict HTTP JSON decoding;
• prompt, profile, schema, and artifact source mechanics;
• built-in profile validation and overlay fallback;
• structured-output encoding and schema validation;
• validation content failures versus operational failures;
• credential handling and redaction;
• OpenAI-compatible wire behavior; and
• HTTP artifact restrictions and transport mappings.

Coverage will be added only if the ownership audit identifies a consequential behavior with no credible existing owner.

Required Validation Outcome

The characterized baseline must pass the complete Go test and vet suites, a temporary-output executable build, repeated public contract tests, both maintained application configurations, maintained render and Go consumer examples, the maintained HTTP request-example check, documentation-link validation, and whitespace validation. All checks must remain offline and independent of real credentials.

Out Of Scope

Step 3 does not:

• add artifact-reader, repository, validator, or other production extension APIs;
• refactor CLI or HTTP adapters to consume the public facade;
• move restricted HTTP artifact behavior out of its current package;
• create the Promptkit repository or change the Go module path;
• move implementation packages between repositories;
• create compatibility aliases or forwarding APIs;
• redesign the public facade; or
• add tests solely to increase a coverage percentage.

Those changes belong to later migration steps.

Completion Criteria

Step 3 is complete when:

• Promptkit-destined tests have no dependency on Scriptorium-owned executable examples;
• the targeted public contract gaps are covered with deterministic tests;
• the full roadmap behavior list has a clear, non-duplicative test owner;
• tests that require Step 4 rewrites are explicitly identified;
• no production behavior or public API changed;
• all required validation passes; and
• the main migration roadmap records the Step 3 gate as complete.

Migration Step 4 must not begin until these criteria are satisfied.

Lifecycle

This completed implementation roadmap remains the concise characterization record for the migration. Repository history retains the detailed implementation record.