scriptorium/docs/roadmap/implementation.md

Built-In Profiles And Library API Implementation Plan

This plan implements the target states in:

• docs/roadmap/builtins.md
• docs/roadmap/library.md

Audience: LLM coding agents implementing the work in order. Review and follow docs/policy/architecture.md, docs/policy/development.md, and docs/policy/documentation.md before changing code.

Global Constraints

• Implement built-in profiles before the public library production upgrades.
• Keep orchestration in internal/usecase; adapters and the public package should translate inputs and wire components.
• Keep internal/* packages internal. Public package types must remain facade types.
• Do not add a mutable global profile registry.
• Do not add a credential resolver or secret-manager abstraction.
• Do not accept raw API keys in YAML/JSON config, profile files, prompt files, CLI flags, or HTTP request bodies.
• Do not emit raw API keys in prepared output, run results, logs, or examples.
• Prefer standard library APIs. Do not add dependencies unless a later implementation prompt explicitly approves one.
• Keep each stage passing go test ./... before moving to the next stage.

Stage 1: Profile Repository Foundations

Goal: add reusable profile repository primitives that support built-ins without changing runner behavior.

Implementation Steps

1. Add an fs.FS-backed profile repository in internal/profile.

   • Constructor shape should be similar to NewFSRepository(fsys fs.FS, root string) Repository.
   • It must scan YAML files recursively below root.
   • It must use the same strict YAML decoding, profile validation, raw api_key rejection, and duplicate-ID behavior as the existing filesystem repository.
   • It must return the existing profile package sentinel errors where applicable.

2. Refactor shared profile-loading behavior.

   • Avoid duplicating validation and metadata logic between filesystem and fs.FS repositories.
   • The existing NewFilesystemRepository(dir) API should remain available.
   • It may either delegate to the fs.FS repository through os.DirFS or share unexported loader helpers.

3. Add an overlay profile repository in internal/profile.

   • Constructor shape should be similar to NewOverlayRepository(primary, fallback Repository) Repository.
   • Lookup must return the primary result when primary succeeds.
   • Lookup must fall back only when errors.Is(err, profile.ErrProfileNotFound) for the primary.
   • Lookup must return primary load/validation errors directly and must not fall back after those errors.
   • Nil repository inputs should be handled deliberately. Prefer treating nil primary as "no primary" and requiring a non-nil fallback for built-in-only operation.

Tests

Add focused tests under internal/profile.

Required coverage:

• NewFSRepository loads valid profiles from nested directories.
• NewFSRepository rejects unknown YAML fields.
• NewFSRepository rejects raw api_key in the selected profile.
• NewFSRepository ignores raw api_key in non-selected profiles, matching current filesystem behavior.
• NewFSRepository rejects duplicate IDs within one source.
• NewFilesystemRepository still satisfies all existing repository tests.
• NewOverlayRepository returns primary matches before fallback matches.
• NewOverlayRepository falls back on primary not found.
• NewOverlayRepository does not fall back after a primary invalid YAML/profile/raw-key error.
• NewOverlayRepository returns not found when both sources miss.

Verification

Run:

go test ./internal/profile
go test ./...

Stage 2: Built-In Profile Assets And Wiring

Goal: compile built-in profiles into Scriptorium and make profile lookup use user-over-built-in precedence.

Implementation Steps

1. Add an internal/profile/builtin package.

   • Store built-in profile YAML files in a stable asset directory under that package.
   • Use Go embed to compile those files into the binary/package.
   • Expose a constructor such as builtin.NewRepository() profile.Repository.
   • The repository should use the fs.FS profile repository from Stage 1.

2. Add built-in profile validation tests.

   • Tests should load every built-in profile through the real profile loader.
   • Tests should fail if the built-in profile set contains duplicate IDs.
   • Tests should fail if any built-in profile contains raw api_key.

3. Add built-in profiles.

   • Use docs/roadmap/profiles/ as the source catalog for the initial built-in profile set.
   • Copy those YAML files into the built-in profile asset directory, preserving provider subdirectories unless the implementation has a clear reason to flatten them.
   • Do not invent a broad provider/model catalog.
   • Do not add raw API keys.
   • Built-in profiles may use api_key_env for CLI/HTTP compatibility when the provider requires authentication.

4. Wire repositories through a small helper.

   • Add an internal helper near adapter wiring, or in internal/profile, that returns:
     • built-in repository only when no custom profile source is configured;
     • overlay repository when a custom profile source is configured.
   • The runner should still receive only a profile.Repository.

5. Make profile_dir optional.

   • CLI run, CLI render, and HTTP serve argument/config validation should require prompt_dir but no longer require profile_dir.
   • Public NewEngine should no longer reject an empty Config.ProfileDir.
   • When profile_dir is empty, wire only built-ins.
   • When profile_dir is non-empty, wire filesystem profiles over built-ins.

Tests

Add or update tests under internal/adapter/cli, internal/adapter/http, root public package tests, and profile/builtin tests.

Required coverage:

• CLI parse/config tests accept missing profile_dir when prompt_dir is present.
• HTTP serve parse/config tests accept missing profile_dir when prompt_dir is present.
• Public NewEngine accepts missing ProfileDir.
• A built-in profile ID can be selected with no custom profile_dir.
• A prompt default_profile can refer to a built-in profile ID.
• A custom profile in profile_dir overrides a built-in with the same ID.
• A malformed selected custom profile does not fall back to a built-in with the same ID.
• A missing profile ID still maps to the existing profile-not-found behavior.
• Existing duplicate-ID tests for filesystem profiles continue to fail within the custom source.

Documentation

After code behavior exists, update non-roadmap docs:

• docs/config.md: document profile_dir as optional and describe built-in fallback/override behavior.
• docs/cli.md: remove claims that --profile-dir is required.
• docs/internal/adapters.md: document built-in profile repository composition.
• Any affected examples or README snippets that imply profile_dir is mandatory.

Do not document built-in profile IDs outside implemented assets.

Verification

Run:

go test ./internal/profile ./internal/adapter/cli ./internal/adapter/http .
go test ./...
go run ./cmd/scriptorium render \
  --config ./examples/config.yml \
  --prompt generic.markdown_summary \
  --input transcript=./examples/fixtures/transcript.md \
  --input glossary=./examples/fixtures/glossary.yml \
  --format json

Also run a new smoke command that uses a built-in profile without --profile-dir once a concrete built-in profile ID is available.

Stage 3: Public API Credential Value

Goal: support the production library credential model: a direct API-key Go value, without adding a resolver or accepting raw keys in serialized config.

Public API Decision

Use a single public credential-supply method:

type RunRequest struct {
	// existing fields...
	APIKey string `json:"-"`
}

Do not add Config.APIKey, WithAPIKey, or a credential resolver in this stage. A request-level value avoids storing secrets on long-lived engines and supports per-tenant callers.

Implementation Steps

1. Add direct API-key plumbing through internal request/target types.

   • Add an internal direct API-key field where needed, with json:"-" and yaml:"-" tags.
   • Convert public RunRequest.APIKey into the internal request.
   • Carry the value to the effective execution target used for LLM generation.
   • Do not include the value in prepared/run public results, formatted output, logs, hashes, or docs examples.

2. Update credential validation.

   • If a selected/effective profile requires authentication and a direct API key is provided, do not require the environment variable to be set for the public path.
   • Preserve existing CLI/HTTP behavior that uses api_key_env.
   • Preserve existing errors for missing environment variables in CLI/HTTP paths.

3. Update the OpenAI-compatible LLM client.

   • Prefer the direct API-key value when present.
   • Fall back to existing api_key_env behavior for CLI/HTTP compatibility.
   • Never serialize or log the direct API-key value.

4. Update public LLM injection conversion.

   • Do not expose the raw API key to injected public LLMClient implementations unless that is strictly necessary for custom LLM execution.
   • If custom LLM clients need the key, expose it only on the public GenerateRequest with json:"-" and document that fake/test clients should avoid logging it.

Tests

Required coverage:

• Public Run can call the default OpenAI-compatible client path with a direct API key without requiring the configured api_key_env environment variable.
• CLI/HTTP behavior using api_key_env still works.
• Missing credentials still fail clearly when a selected profile requires authentication and neither direct key nor usable env value is available.
• Public PreparedRun and RunResult JSON do not include the direct API key.
• Formatted prepared output does not include the direct API key.
• Direct API-key values are not included in hashes.
• Injected fake LLM tests either receive no key or receive it only through a json:"-" field, depending on the implementation choice above.

Verification

Run:

go test ./internal/llm ./internal/usecase ./internal/adapter/cli ./internal/adapter/http .
go test ./...

Stage 4: Public Asset Source Options

Goal: let library consumers load standard Scriptorium prompt/profile/schema assets from directories, single files, and fs.FS sources.

Public API

Keep existing Config directory fields working. Add options:

func WithPromptFS(fsys fs.FS, root string) Option
func WithPromptFile(path string) Option
func WithProfileFS(fsys fs.FS, root string) Option
func WithProfileFile(path string) Option
func WithSchemaFS(fsys fs.FS, root string) Option
func WithSchemaFile(path string) Option

Rules:

• Directory Config fields remain the compatibility path.
• Explicit options override the corresponding Config directory field.
• Prompt source is required.
• Profile source is optional because built-in profiles exist.
• Schema source is optional and should default to the current schema default behavior when not configured.
• Nil fs.FS values, empty required roots, and invalid option combinations return ErrInvalidConfig.

Implementation Steps

1. Add fs.FS prompt-definition support.

   • Implement an fs.FS prompt repository that preserves existing strict prompt YAML behavior.
   • Preserve prompt lookup by YAML id, not path.
   • Preserve duplicate prompt ID errors within one source.
   • Resolve content_file relative to the prompt file's directory inside the same source.
   • Keep the existing filesystem repository API; it may delegate to the fs.FS implementation.

2. Add public prompt source wiring.

   • Config.PromptDir wires the filesystem repository.
   • WithPromptFS wires the fs.FS repository.
   • WithPromptFile(path) wires a single-file source and must still select by prompt YAML id.

3. Add public profile source wiring.

   • Reuse the Stage 1 profile fs.FS repository.
   • WithProfileFS and WithProfileFile become overlay primaries above built-ins.
   • Empty profile source still means built-ins only.

4. Add schema fs.FS support.

   • Extend or wrap the standard validator so schema files can be loaded from an fs.FS source.
   • WithSchemaFS should preserve existing schema_path semantics.
   • WithSchemaFile(path) should expose the file by its base name; prompts using it should set schema_path to that base name.

5. Keep adapter scope narrow.

   • This stage is for the public package and shared repositories/validators.
   • Do not change CLI/HTTP request shapes for prompt or schema fs.FS sources.

Tests

Required coverage:

• Public Prepare works with prompt definitions from embed.FS.
• content_file references resolve relative to the prompt file in embed.FS.
• Public Prepare works with WithPromptFile.
• Public Run or Prepare works with WithProfileFS over built-ins.
• Public Run or Prepare works with WithProfileFile over built-ins.
• Public structured-output schema validation works with WithSchemaFS.
• WithSchemaFile works when the prompt's schema_path is the schema file base name.
• Explicit source options override Config directory fields.
• Invalid/nil source options return ErrInvalidConfig.
• Existing filesystem prompt/profile/schema behavior remains unchanged.

Verification

Run:

go test ./internal/promptdef ./internal/profile ./internal/validate .
go test ./...

Stage 5: Public In-Memory Profiles And Profile Templates

Goal: allow library callers to provide typed profile values and construct stable OpenAI-compatible profile templates without generating YAML.

Public API

Add a public profile facade aligned with the profile YAML contract:

type Profile struct {
	ID              string
	Endpoint        string
	Model           string
	Temperature     float64
	MaxTokens       int
	TopP            float64
	TimeoutSeconds  int
	ServiceTier     string
	ReasoningEffort string
	APIKeyRequired  bool
	ExtraParams     map[string]any
}

func WithProfiles(profiles ...Profile) Option

Add an OpenAI-compatible template constructor:

type OpenAICompatibleProfileConfig struct {
	ID              string
	Endpoint        string
	Model           string
	APIKeyRequired  bool
	Temperature     float64
	MaxTokens       int
	TopP            float64
	TimeoutSeconds  int
	ServiceTier     string
	ReasoningEffort string
	ExtraParams     map[string]any
}

func OpenAICompatibleProfile(cfg OpenAICompatibleProfileConfig) Profile

Rules:

• WithProfiles profiles override built-ins with the same ID.
• If WithProfiles and a file/FS profile source are both configured, in-memory profiles have highest precedence, then file/FS profiles, then built-ins.
• Public profile values must not include raw API-key fields.
• Use APIKeyRequired to indicate whether RunRequest.APIKey is required for the public path. Internal conversion may map this to the existing auth-required/profile credential model without exposing raw keys.

Implementation Steps

1. Add a profile repository for public in-memory Profile values.

   • Validate with the same effective rules as YAML profiles.
   • Reject duplicate IDs within the provided values.
   • Deep-copy ExtraParams across public/internal boundaries.

2. Compose profile sources in public NewEngine.

   • Highest: in-memory public profiles.
   • Next: configured profile file/FS/directory source.
   • Fallback: built-in profiles.

3. Add template constructor conversion.

   • OpenAICompatibleProfile should be a convenience constructor only.
   • It should not register global state.
   • It should not maintain a broad model catalog.

4. Ensure direct API-key behavior works with in-memory/template profiles.

   • Profiles marked APIKeyRequired should require RunRequest.APIKey for public default LLM execution.
   • Profiles not marked APIKeyRequired should not require an API key.

Tests

Required coverage:

• Public Prepare/Run uses WithProfiles without any profile files.
• In-memory profiles override built-ins.
• In-memory profiles override file/FS profile sources when IDs collide.
• Duplicate in-memory profile IDs return ErrInvalidConfig.
• Template-created profiles execute through the same path as normal profiles.
• Template-created profiles requiring an API key work with RunRequest.APIKey.
• Template-created profiles that do not require an API key work without one.
• ExtraParams in public profiles are deep-copied and isolated from caller mutation.

Verification

Run:

go test .
go test ./...

Stage 6: Public Documentation And Examples

Goal: document implemented behavior in canonical locations after code exists.

Documentation

Update:

• README.md: add a short pointer to library usage and built-in profiles without turning the README into a manual.
• docs/config.md: document optional profile_dir, built-in override behavior, and implemented built-in profile IDs.
• docs/cli.md: document optional --profile-dir and built-in profile selection.
• docs/internal/adapters.md: document repository composition and public library adapter surface.
• docs/consumers/api.md: describe public consumer surfaces at a high level.
• docs/consumers/pkg-scriptorium.md: document root package usage, source options, direct API-key value, errors, and examples.
• docs/integrations/openai-compatible-chat.md: update only if direct API-key plumbing changes provider-call semantics.

Do not document unimplemented roadmap behavior outside docs/roadmap/.

Examples

Add or update copyable examples that require no real provider credentials:

• prepare with built-in profile and no profile_dir;
• public library prepare from embed.FS;
• public library run with injected fake LLM;
• public library run with typed/template profile and direct API key, using a fake/local provider path so no real secret is required.

Examples must be secret-free.

Tests And Smoke Commands

Required verification:

go test ./...
go run ./cmd/scriptorium render \
  --config ./examples/config.yml \
  --prompt generic.markdown_summary \
  --input transcript=./examples/fixtures/transcript.md \
  --input glossary=./examples/fixtures/glossary.yml \
  --format json

Also run smoke commands for any new examples, such as:

go run ./examples/go-library/prepare

If an example relies on a specific built-in profile ID, use an ID from docs/roadmap/profiles/ and include that smoke command in the implementing change.

Final Completion Checklist

Before marking the combined feature complete:

1. go test ./... passes.
2. Existing CLI render smoke command passes.
3. At least one smoke command proves built-in profile lookup works without profile_dir.
4. Public package examples compile/run without real provider credentials.
5. profile_dir is optional in CLI, HTTP, and public engine construction.
6. Custom profiles override built-ins with the same ID.
7. Malformed selected custom profiles do not fall back to built-ins.
8. Built-in profiles use the same validation rules as file profiles.
9. Public RunRequest.APIKey is the documented library credential method.
10. Raw API keys do not appear in YAML/JSON config, prepared output, run results, logs, hashes, or examples.
11. Non-roadmap docs describe only implemented behavior.