scriptorium/docs/roadmap/builtins.md

Built-In Profiles Roadmap

This roadmap defines the target behavior for adding built-in execution profiles to Scriptorium.

Built-in profiles are useful for CLI, HTTP, subprocess, and library consumers, and they provide a clean foundation for public-package ergonomics.

Motivation

Scriptorium currently requires a profile source for every run path. That is appropriate for fully custom deployments, but it creates unnecessary setup for common model targets where stable profile definitions can be shipped with the application.

Built-in profiles should let callers select standard profile IDs without creating local profile files. Users and downstream applications should still be able to override any built-in profile by providing a custom profile with the same ID.

Target Behavior

Scriptorium should include a built-in set of execution profiles compiled into the binary/package.

Profile lookup should use this precedence:

1. user-provided or downstream-provided profiles;
2. built-in profiles;
3. profile-not-found error.

If a user profile and a built-in profile share the same ID, the user profile wins. This is intentional override behavior and should not be treated as a duplicate-profile error.

Duplicate profile IDs within the user profile source should remain invalid. Duplicate profile IDs within the built-in profile set should be prevented by tests. Duplicate IDs across the user source and built-in source are valid because they express override intent.

Once built-ins exist, profile_dir should no longer be required for CLI, HTTP, or public library engine construction. When no custom profile source is configured, Scriptorium should use the built-in profile repository alone. When a custom profile source is configured, Scriptorium should overlay it on top of the built-in repository.

Architecture

Built-in profiles should be modeled as another implementation of the existing profile.Repository boundary.

Recommended repository structure:

• filesystem or custom profile repository for user-provided profiles;
• built-in profile repository backed by embedded profile YAML;
• overlay repository that checks the primary repository first and falls back to built-ins only when the primary returns profile.ErrProfileNotFound.

The runner should continue to depend only on profile.Repository. It should not know whether a selected profile came from a file, a built-in definition, or a future public-package source.

Built-In Repository

Built-in definitions should be stored as normal profile YAML and embedded into the binary with Go embed.

Recommended package shape:

• internal/profile/builtin owns embedded built-in profile assets and exposes a repository constructor.
• built-in profile files live under that package in a stable asset directory.
• the built-in repository reuses the same strict decoding and validation rules as normal profiles.

Using YAML for built-ins keeps the built-in profile format aligned with the documented profile format and lets maintainers add stable definitions without duplicating profile construction logic in Go.

FS Repository

The implementation should introduce or reuse an fs.FS-based profile repository rather than making the built-in loader special-purpose.

That repository supports:

• embedded built-in profile assets;
• embedded or virtual profile sources in public library work;
• fixture-based tests without temporary directory setup where useful.

The existing filesystem repository can remain as a thin path-based adapter, or it can delegate internally to the fs.FS repository where that is clean and maintainable.

Overlay Repository

An overlay repository should compose two repositories:

• primary: user-provided, custom, or downstream profile source;
• fallback: built-in profile source.

Lookup behavior:

• return the primary result if primary lookup succeeds;
• if primary returns profile.ErrProfileNotFound, try fallback;
• if primary returns any other error, return that error and do not try fallback;
• return fallback result or fallback error.

This preserves strict validation of user profile sources. A malformed selected user profile should not silently fall through to a built-in with the same ID.

CLI And HTTP Behavior

The CLI and HTTP server should no longer require profile_dir once built-in profiles are available.

Expected behavior:

• profile_dir omitted: built-ins are available.
• profile_dir provided: profiles from that directory override built-ins with the same ID.
• selected profile ID present only in built-ins: run succeeds.
• selected profile ID present in both custom profiles and built-ins: custom profile is used.
• selected profile ID missing from both sources: existing profile-not-found behavior is preserved.
• selected profile ID matches a malformed custom profile: profile-load failure is returned, not fallback to built-in.

Configuration and CLI documentation should describe profile_dir as optional once built-in profiles are available.

Public Library Interaction

This feature should support the current public package behavior and the production library roadmap.

For the current public engine, ProfileDir should become optional once built-ins exist. A caller that does not configure a custom profile directory should still be able to use built-in profile IDs.

The production library roadmap may add fs.FS, single-file, and in-memory profile sources. Those sources should become overlay primaries above the same built-in repository.

Credential behavior for built-ins should follow the active execution path:

• current CLI/HTTP behavior may continue to use api_key_env in profile definitions;
• the public library API may supply direct API-key values without changing built-in profile IDs;
• built-in profile files must never contain raw API keys.

Scope

In scope:

• built-in execution profile assets;
• strict validation of all built-in profiles;
• fs.FS profile repository support where needed for embedded assets;
• overlay profile repository with user-over-built-in precedence;
• optional profile_dir for CLI, HTTP, and public engine construction;
• tests for lookup precedence, override behavior, duplicate handling, and error behavior;
• documentation of CLI/config/profile behavior.

Out of scope:

• changing the profile YAML format;
• accepting raw API keys in profile YAML;
• adding a mutable runtime profile registry;
• adding a provider/model catalog that must track rapidly changing model availability;
• changing prompt default_profile semantics beyond allowing built-in IDs;
• implementing the broader fs.FS prompt/schema/library source work from docs/roadmap/library.md.

Acceptance Criteria

• Scriptorium can run or render using a built-in profile ID with no configured profile_dir.
• CLI run, CLI render, and HTTP serve no longer fail solely because profile_dir is omitted.
• A profile in profile_dir overrides a built-in profile with the same ID.
• Duplicate profile IDs inside profile_dir remain invalid.
• Duplicate profile IDs inside the built-in profile set are caught by tests.
• A malformed selected custom profile does not fall back to a built-in profile with the same ID.
• Profile-not-found behavior remains clear when an ID exists in neither custom profiles nor built-ins.
• Built-in profiles are loaded through the same validation rules as file-based profiles.
• Existing runtime override behavior continues to apply to built-in profiles.
• Existing CLI, HTTP, and public error mapping remains consistent with current profile-load and profile-not-found semantics.