scriptorium/docs/roadmap/implementation.md

Migration Step 4 Implementation Plan

Status

Follow-up remediation is required. The public-facade adapter boundary is implemented, but the post-implementation review found an error-classification gap, a machine-dependent HTTP artifact test, and an incomplete architecture guard. Migration Step 4 is not fully revalidated until the stages below pass.

Objective

Complete the target state in the Step 4 public-facade adapter roadmap by correcting the review findings without changing the intended CLI, HTTP, or public Go contracts.

The accepted split decision owns the long-term Promptkit/Scriptorium boundary. The testing policy governs test value and ownership. The documentation policy governs completion bookkeeping and canonical ownership.

Constraints

• Keep this work in the Scriptorium repository. Do not modify the sibling Promptkit repository or infer that Migration Step 5 is complete merely from its local presence.
• Preserve the public ArtifactReader, error sentinel, CLI, HTTP, containment, artifact-size, formatting, timeout, credential, and redaction contracts.
• Do not add compatibility shims, begin framework extraction, change the module path, or add dependencies.
• Keep tests deterministic, offline, independent of machine MIME databases, and focused at the narrowest stable owner.
• Preserve unrelated working-tree changes.

Completed Work

The original five implementation stages established the Step 4 boundary:

• the root facade now exposes ArtifactReader and WithArtifactReader, copies mutable artifact bodies, handles nil responses, and exposes the two specific public request-error identities required by HTTP;
• the restricted artifact reader and its containment and byte-limit policy now belong to the HTTP adapter and use public artifact types;
• prepared-run formatting and CLI run and render consume public prepared, request, and result values;
• the HTTP handler consumes a local interface expressed in public types, and CLI serve injects the restricted reader into the public engine;
• the legacy restricted framework reader was removed, current-behavior documentation was reconciled, and an initial dependency guard was added; and
• the initial validation pass covered the full suite, vet, build, race and repeated tests, maintained examples, configuration loading, links, and whitespace.

The post-implementation review confirmed the production dependency direction but found that the completion gate needs the corrections below.

Stage 1: Preserve The Active Public Error Category

Correct mapPublicError so a public error returned by an injected collaborator does not suppress the public category added by the operation currently being performed.

The current early return from hasPublicError is too broad: for example, an artifact reader that returns ErrInvalidRequest is wrapped by the runner's internal artifact-load category, but the facade returns before adding public ErrArtifactLoad. The corrected mapping must:

• first classify the internal operation or source category present in the complete chain;
• add the corresponding public category even when another public sentinel is already nested in the chain;
• retain the original collaborator error and every existing internal cause for errors.Is;
• preserve the dual ErrInvalidRequest plus specific-sentinel behavior for ErrProfileRequired and ErrAPIKeyEnvMissing; and
• return an already-public or otherwise unclassified error unchanged when no internal category requires translation.

Remove the blanket public-error short circuit rather than adding an artifact-specific exception. Keep publicErrorFor responsible for selecting the outer public category from internal identities, with specific causes checked before broad internal categories. Do not parse error text or change public method signatures.

Regression Tests

Extend the public engine contract tests at the real Engine.Prepare and Engine.Run boundaries:

• make an injected artifact reader return ErrInvalidRequest and assert that the result matches both ErrArtifactLoad and the original ErrInvalidRequest;
• make an injected LLM client return a different public sentinel, such as ErrArtifactLoad, and assert that the result matches both ErrLLMGenerate and the original sentinel; and
• retain the existing generic reader-error, nil-response, cancellation, profile-required, and missing-credential cases without duplicating them.

The second case protects the shared translation rule rather than only the reported artifact example. Do not add a cross-product of every public sentinel and failure category.

Run:

go test .
go test -count=20 -run 'Test.*(ArtifactReader|PublicError|LLM)' .

Stage 1 Gate

• Every categorized runner failure gains the public category for its active operation even when a nested collaborator error is already public.
• Nested custom error identities remain observable through errors.Is.
• Existing broad and specific public error behavior remains unchanged.

Stage 2: Make HTTP Artifact MIME Coverage Deterministic

Repair TestRestrictedArtifactReaderReadsContainedFiles without changing the implemented MIME policy. The contract remains:

• call mime.TypeByExtension for a file extension; and
• use text/plain only when that lookup returns an empty string.

Replace the .md fixture with an extension in Go's built-in MIME table, such as .html, and assert the corresponding non-empty built-in result. Retain the separate unknown-extension case that asserts the text/plain fallback. Keep the contained relative and absolute path cases and all metadata assertions.

Do not hard-code a platform-specific Markdown media type, consult system MIME files directly, mutate the process-global MIME registry, or weaken the test to accept either value. Change production code only if the deterministic test demonstrates that it does not implement the stated lookup-and-fallback rule.

Run:

go test ./internal/adapter/http
go test -count=20 ./internal/adapter/http

Stage 2 Gate

• The HTTP package passes repeatedly on a machine whose MIME database has no Markdown registration.
• The test independently protects successful MIME inference and the unknown extension fallback.
• Artifact metadata, containment, and byte-limit behavior are unchanged.

Stage 3: Strengthen The Public-Boundary Dependency Guard

Update internal/adapter/dependency_test.go so the durable guard proves the entire intended production boundary rather than only today's flat directory layout.

Recursive Source Inspection

Walk these roots recursively with filepath.WalkDir:

• internal/adapter/cli;
• internal/adapter/http; and
• internal/format.

Inspect every non-test .go file under those roots. Skip directories only when they are not Go source owned by the repository; do not silently skip nested adapter or formatter packages. Continue parsing imports with the standard library and parser.ImportsOnly.

Forbidden Package Families

Represent each Promptkit-destined internal dependency as a package-family root:

• internal/domain;
• internal/usecase;
• internal/promptdef;
• internal/prompt;
• internal/profile;
• internal/validate;
• internal/llm; and
• internal/artifact.

Reject an import when it equals a forbidden root or begins with that root plus /. The internal/profile rule therefore covers internal/profile/builtin and any future descendant. Do not reject the public root package or Scriptorium-owned adapter, configuration, defaults, and formatter packages.

Diagnostics must identify the importing file and complete forbidden import path. Continue excluding _test.go files from the durable production architecture rule; test-boundary review remains a separate source inspection.

Guard Regression Test

Extract only the small scanning or classification helper needed to exercise the guard. Using t.TempDir, create a nested Go source fixture that imports a descendant of a forbidden package family and assert that the guard reports it. Also include an allowed public-facade import so the test proves it is not overbroad. Keep this fixture test in the existing dependency-test file rather than creating another architecture-test owner.

Run:

go test ./internal/adapter -run TestScriptoriumAdaptersUseOnlyPublicFrameworkBoundary
go test -count=20 ./internal/adapter

Stage 3 Gate

• Nested production packages cannot bypass the scan.
• Current or future descendants of forbidden framework packages cannot bypass package-family matching.
• Allowed public and Scriptorium-owned dependencies remain accepted.
• Failures remain precise and actionable.

Stage 4: Revalidate And Reconcile Completion Records

After Stages 1 through 3 pass, run the complete Step 4 validation from the Scriptorium repository:

go test ./...
go vet ./...
build_output="$(mktemp -d)"
go build -o "$build_output/scriptorium" ./cmd/scriptorium
go test -race . ./internal/adapter/http
go test -count=20 . ./internal/adapter ./internal/adapter/cli ./internal/adapter/http ./internal/format
go test ./internal/adapter/http -run TestMaintainedHTTPRunExampleMatchesRequestContract
bash ./examples/render-markdown-summary.sh
go run ./cmd/scriptorium render \
  --config ./examples/config.full.yml \
  --prompt generic.markdown_summary \
  --input transcript=./examples/fixtures/transcript.md \
  --input glossary=./examples/fixtures/glossary.yml \
  --format text
go run ./examples/go-library/prepare
git diff --check

Also:

• pass both maintained configuration files through the real configuration loader;
• inspect production and test imports under the adapter and formatter roots;
• validate every local Markdown link changed by the remediation;
• confirm no credential, private data, dependency, module, workspace, or local replace change was introduced; and
• inspect the final diff for unrelated work.

Once every check passes:

1. update the Step 4 completion summary in step4.md to mention the post-implementation error and guard corrections without turning the feature roadmap into an implementation log;
2. update the Step 4 gate in migration.md so its validation claim accurately reflects the successful follow-up pass;
3. mark this implementation plan complete and move the remediation stages into the completed-work summary; and
4. leave the Step 5 gate unchanged unless its out-of-band repository, governance, policy, and CI requirements have been independently confirmed.

Do not begin extraction or modify Promptkit as part of this stage.

Stage 4 Gate

• Every Step 4 completion criterion is satisfied.
• The public error contract is protected against nested public-sentinel collisions.
• The default suite is deterministic across machine MIME configurations.
• The dependency guard enforces recursive package-family boundaries.
• Full tests, vet, build, race checks, repeated tests, maintained examples, configuration loading, links, and whitespace validation pass.
• Scriptorium's completion records accurately describe the validated state.

Open Questions

None.