Add configurable transient retries for CLI execution

docs/cli.md

@@ -64,6 +64,7 @@ Optional flags:
 | `--top-p <float>` | Runtime top-p override. |
 | `--reasoning-effort <value>` | Runtime reasoning-effort override. |
 | `--timeout <duration>` | Runtime timeout override using Go duration syntax. |
+| `--transient-retries <integer>` | Nonnegative additional transient attempt count for this request; `0` disables additional attempts. |
 
 Deprecated aliases: `--prompt-id` for `--prompt`, and `--profile-id` for
 `--profile`.
@@ -77,6 +78,10 @@ zero-second override. The timeout layers are defined in the
 [Promptkit outbound integration contract](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.11.0/docs/integrations/openai-compatible-chat.md#timeout-and-cancellation).
 With the built-in client, each transient retry gets a fresh generation timeout;
 this flag does not limit the complete run.
+Omitting `--transient-retries` inherits the application setting or Promptkit's
+current default of two additional attempts. The flag overrides only this
+request; it does not alter the configured engine. Structural repair generations
+have separate transient retry budgets.
 
 There is no raw API-key flag. Use `--api-key-env`.
 
@@ -107,13 +112,15 @@ scriptorium render [flags]
 `--config`, `--prompt-dir`, `--profile-dir`, `--prompt-version`, `--profile`,
 `--input`, `--var`, `--out`, `--llm-base-url`,
 `--model`, `--api-key-env`, `--temperature`, `--max-tokens`, `--top-p`,
-`--reasoning-effort`, `--session-id`, `--timeout`, and `--format text|json`. Their meanings match the corresponding
+`--reasoning-effort`, `--session-id`, `--timeout`, `--transient-retries`, and `--format text|json`. Their meanings match the corresponding
 `run` flags; `--format` selects prepared-run output and otherwise uses
 `defaults.render_format`.
 
 The same deprecated aliases and numeric/timeout behavior as `run` apply.
 The same session and reasoning inheritance, replacement, and clearing behavior
 also applies.
+`render` validates the retry override but does not generate model output or
+include a retry field in prepared output.
 `render` does not accept `--schema-dir`; configure `schema_dir` through the
 configuration file. It resolves profiles and schemas as part of preparation but
 does not call an LLM.

docs/config.md

@@ -40,6 +40,7 @@ do not override a prior value. Raw API-key fields are not accepted.
 | `server.max_response_bytes` | `16777216` | Maximum encoded HTTP response bytes; `0` disables the limit. |
 | `defaults.render_format` | `text` | Default prepared-run output format: `text` or `json`. |
 | `backends` | unset | Optional mapping of custom Promptkit backend IDs to engine-scoped connection and capacity settings. |
+| `transient_retries` | Promptkit default | Optional nonnegative count of additional built-in transient generation attempts. Omitted or `null` delegates to Promptkit; `0` disables additional attempts. |
 
 The size fields must be zero or greater. The [HTTP API](api.md) defines how
 each limit is enforced and reported. `server.artifact_root` configures an HTTP
@@ -120,12 +121,13 @@ A positive Promptkit `repair_attempts` budget can add provider calls, latency,
 token use, and cost; see the tagged format reference for its permitted values
 and validation-mode requirements.
 
-Promptkit's built-in client now makes up to two additional attempts for eligible
+Promptkit's built-in client makes up to two additional attempts for eligible
-transient HTTP statuses by default. This can increase latency, request count,
+transient HTTP statuses by default. The application `transient_retries` setting
-and cost. Scriptorium currently uses that upstream default without an
+changes that budget for its constructed engine; an explicit `run` request
-application-level retry setting. Each initial or repair generation has its own
+override takes precedence. This can increase latency, request count, and cost.
-transient budget; an injected client retains its own retry behavior. The tagged
+Each initial or repair generation has its own transient budget; an injected
-integration contract owns eligibility and timing.
+client retains its own retry behavior. The tagged integration contract owns
+eligibility and timing.
 
 Promptkit's
 [OpenAI-compatible integration contract](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.11.0/docs/integrations/openai-compatible-chat.md)

docs/internal/adapters.md

@@ -33,6 +33,10 @@ described by its tagged
 and profile selection, optional file inputs, variables, and presence-aware
 execution overrides. Omitted framework settings remain zero values so Promptkit
 resolves its own defaults and definition-required inputs.
+The application retry setting enters shared engine construction, while a
+run/render flag enters only the mapped request. The server uses the same engine
+setting and retains one backend admission state. Render prepares without
+generation.
 
 `inspect prompt` maps the selected ID and optional version to
 `promptkit.Engine.InspectPrompt`, then formats Scriptorium-owned inspection

docs/operations.md

@@ -112,8 +112,11 @@ call.
 Promptkit's built-in client now allows two additional attempts for eligible
 transient provider statuses by default. Account for added calls, latency, and
 cost when planning capacity. Each repair generation has its own transient
-budget, separate from the structural repair count. The current Scriptorium
+budget, separate from the structural repair count. Configure the engine budget
-interfaces use this upstream retry default. Existing generation timeouts apply
+with [`transient_retries`](config.md); `run` and `render` accept a request
+override described in the [CLI contract](cli.md). Omission inherits the engine
+setting and then Promptkit's default, while explicit zero disables additional
+attempts. Existing generation timeouts apply
 per attempt, while the caller context covers the complete operation. Provider
 `Retry-After` timing can exceed Promptkit's ordinary backoff cap; consult the
 [outbound integration contract](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.11.0/docs/integrations/openai-compatible-chat.md)

docs/roadmap/implementation.md

@@ -107,7 +107,7 @@ are handled; no new runtime flags or report schema are claimed in current docs.
 
 ## Stage 2: Wire Retry Configuration And CLI Overrides
 
-**Status: Pending.**
+**Status: Complete.**
 
 Selected interface: top-level application YAML `transient_retries`, and
 `--transient-retries <integer>` on `run` and `render`. This is not a model

internal/adapter/cli/inspect.go

@@ -151,7 +151,7 @@ func resolveAppSettingsForProfileInspection(cfg *profileInspectionConfig) (engin
 	if err != nil {
 		return engineSettings{}, err
 	}
-	return engineSettings{profileDir: settings.ProfileDir, backends: settings.Backends}, nil
+	return engineSettings{profileDir: settings.ProfileDir, backends: settings.Backends, transientRetries: settings.TransientRetries}, nil
 }
 
 func parsePromptInspectionArgs(args []string) (*promptInspectionConfig, error) {
@@ -196,5 +196,5 @@ func resolveAppSettingsForPromptInspection(cfg *promptInspectionConfig) (engineS
 	if strings.TrimSpace(settings.PromptDir) == "" {
 		return engineSettings{}, errors.New(errPromptDirRequired)
 	}
-	return engineSettings{promptDir: settings.PromptDir, backends: settings.Backends}, nil
+	return engineSettings{promptDir: settings.PromptDir, backends: settings.Backends, transientRetries: settings.TransientRetries}, nil
 }

internal/adapter/cli/run.go

@@ -51,6 +51,8 @@ type runConfig struct {
 	schemaDir                  string
 	backends                   []appconfig.BackendSettings
 	timeout                    time.Duration
+	transientRetries           int
+	configuredTransientRetries *int
 
 	defaultRenderFormat renderformat.PreparedRunOutputFormat
 
@@ -62,6 +64,7 @@ type runConfig struct {
 	topPSet             bool
 	reasoningEffortSet  bool
 	timeoutSet          bool
+	transientRetriesSet bool
 }
 
 type renderConfig struct {
@@ -81,6 +84,7 @@ type serveConfig struct {
 	maxArtifactBytes int64
 	maxResponseBytes int64
 	backends         []appconfig.BackendSettings
+	transientRetries *int
 }
 
 type commonCommandSettings struct {
@@ -94,6 +98,7 @@ type commonCommandSettings struct {
 	maxResponseBytes    int64
 	defaultRenderFormat renderformat.PreparedRunOutputFormat
 	backends            []appconfig.BackendSettings
+	transientRetries    *int
 }
 
 type listFlag []string
@@ -335,6 +340,7 @@ func parseServeArgs(args []string) (*serveConfig, error) {
 	cfg.maxArtifactBytes = settings.maxArtifactBytes
 	cfg.maxResponseBytes = settings.maxResponseBytes
 	cfg.backends = settings.backends
+	cfg.transientRetries = settings.transientRetries
 
 	if err := validateRequiredLibraryDirs(cfg.promptDir); err != nil {
 		return nil, err
@@ -369,6 +375,7 @@ func registerExecutionRequestFlags(fs *flag.FlagSet, cfg *runConfig) {
 	fs.Float64Var(&cfg.topP, "top-p", 0, "optional top_p override")
 	fs.StringVar(&cfg.reasoningEffort, "reasoning-effort", "", "optional reasoning effort override")
 	fs.DurationVar(&cfg.timeout, "timeout", 0, "LLM request timeout")
+	fs.IntVar(&cfg.transientRetries, "transient-retries", 0, "additional transient generation attempts")
 	fs.StringVar(&cfg.promptID, "prompt-id", "", "deprecated alias for --prompt")
 	fs.StringVar(&cfg.profileID, "profile-id", "", "deprecated alias for --profile")
 }
@@ -392,6 +399,7 @@ func finalizeExecutionRequestConfig(fs *flag.FlagSet, cfg *runConfig) error {
 	cfg.schemaDir = settings.schemaDir
 	cfg.defaultRenderFormat = settings.defaultRenderFormat
 	cfg.backends = settings.backends
+	cfg.configuredTransientRetries = settings.transientRetries
 
 	if err := validateRequiredLibraryDirs(cfg.promptDir); err != nil {
 		return err
@@ -414,6 +422,10 @@ func finalizeExecutionRequestConfig(fs *flag.FlagSet, cfg *runConfig) error {
 	cfg.topPSet = flagWasSet(fs, "top-p")
 	cfg.reasoningEffortSet = flagWasSet(fs, "reasoning-effort")
 	cfg.timeoutSet = flagWasSet(fs, "timeout")
+	cfg.transientRetriesSet = flagWasSet(fs, "transient-retries")
+	if cfg.transientRetriesSet && cfg.transientRetries < 0 {
+		return errors.New("--transient-retries must be greater than or equal to 0")
+	}
 	return nil
 }
 
@@ -541,6 +553,7 @@ func resolveCommonSettings(fs *flag.FlagSet, configPath string, overrides appcon
 		maxResponseBytes:    settings.MaxResponseBytes,
 		defaultRenderFormat: settings.DefaultRenderFormat,
 		backends:            settings.Backends,
+		transientRetries:    settings.TransientRetries,
 	}, nil
 }
 
@@ -556,14 +569,15 @@ type engineSettings struct {
 	profileDir       string
 	schemaDir        string
 	backends         []appconfig.BackendSettings
+	transientRetries *int
 }
 
 func (c runConfig) engineSettings() engineSettings {
-	return engineSettings{promptDir: c.promptDir, profileDir: c.profileDir, schemaDir: c.schemaDir, backends: c.backends}
+	return engineSettings{promptDir: c.promptDir, profileDir: c.profileDir, schemaDir: c.schemaDir, backends: c.backends, transientRetries: c.configuredTransientRetries}
 }
 
 func (c serveConfig) engineSettings() engineSettings {
-	return engineSettings{promptDir: c.promptDir, profileDir: c.profileDir, schemaDir: c.schemaDir, backends: c.backends}
+	return engineSettings{promptDir: c.promptDir, profileDir: c.profileDir, schemaDir: c.schemaDir, backends: c.backends, transientRetries: c.transientRetries}
 }
 
 func newEngine(settings engineSettings, options ...promptkit.Option) (*promptkit.Engine, error) {
@@ -583,6 +597,7 @@ func newEngine(settings engineSettings, options ...promptkit.Option) (*promptkit
 		PromptDir:        settings.promptDir,
 		ProfileDir:       settings.profileDir,
 		SchemaDir:        settings.schemaDir,
+		TransientRetries: settings.transientRetries,
 	}, engineOptions...)
 	if err != nil {
 		return nil, fmt.Errorf("engine initialization from application configuration: %w", err)
@@ -641,6 +656,10 @@ func buildRunRequestFromConfig(cfg *runConfig) (promptkit.RunRequest, error) {
 		}
 	}
 
+	var transientRetries *int
+	if cfg.transientRetriesSet {
+		transientRetries = &cfg.transientRetries
+	}
 	return promptkit.RunRequest{
 		PromptID:         cfg.promptID,
 		PromptVersion:    cfg.promptVersion,
@@ -649,6 +668,7 @@ func buildRunRequestFromConfig(cfg *runConfig) (promptkit.RunRequest, error) {
 		Inputs:           inputs,
 		Vars:             varMappings,
 		Execution:        modelOverride,
+		TransientRetries: transientRetries,
 	}, nil
 }
 
@@ -753,8 +773,8 @@ func runErrorMessage(err error) string {
 
 func printUsage(w io.Writer) {
 	fmt.Fprintln(w, "usage: scriptorium <run|render|serve|inspect> ...")
-	fmt.Fprintln(w, "  run:   scriptorium run [--config PATH] [--prompt-dir DIR] [--profile-dir DIR] --prompt ID [--prompt-version VERSION] [--input name=path] [--profile ID] [--session-id ID] [--llm-base-url URL] [--model NAME] [--api-key-env ENV] [--temperature N] [--max-tokens N] [--top-p N] [--reasoning-effort VALUE] [--var k=v] [--out path] [--timeout 10m]")
+	fmt.Fprintln(w, "  run:   scriptorium run [--config PATH] [--prompt-dir DIR] [--profile-dir DIR] --prompt ID [--prompt-version VERSION] [--input name=path] [--profile ID] [--session-id ID] [--llm-base-url URL] [--model NAME] [--api-key-env ENV] [--temperature N] [--max-tokens N] [--top-p N] [--reasoning-effort VALUE] [--var k=v] [--out path] [--timeout 10m] [--transient-retries N]")
-	fmt.Fprintln(w, "  render: scriptorium render [--config PATH] [--prompt-dir DIR] [--profile-dir DIR] --prompt ID [--prompt-version VERSION] [--input name=path] [--profile ID] [--session-id ID] [--llm-base-url URL] [--model NAME] [--api-key-env ENV] [--temperature N] [--max-tokens N] [--top-p N] [--reasoning-effort VALUE] [--var k=v] [--format text|json] [--out path] [--timeout 10m]")
+	fmt.Fprintln(w, "  render: scriptorium render [--config PATH] [--prompt-dir DIR] [--profile-dir DIR] --prompt ID [--prompt-version VERSION] [--input name=path] [--profile ID] [--session-id ID] [--llm-base-url URL] [--model NAME] [--api-key-env ENV] [--temperature N] [--max-tokens N] [--top-p N] [--reasoning-effort VALUE] [--var k=v] [--format text|json] [--out path] [--timeout 10m] [--transient-retries N]")
 	fmt.Fprintf(w, "  serve: scriptorium serve [--config PATH] [--addr %s] [--prompt-dir DIR] [--profile-dir DIR] [--schema-dir DIR] [--artifact-root DIR] [--max-request-bytes N] [--max-artifact-bytes N] [--max-response-bytes N]\n", defaults.HTTPAddrDefault)
 	fmt.Fprintln(w, "  inspect prompt: scriptorium inspect prompt --prompt ID [--prompt-version VERSION] [--config PATH] [--prompt-dir DIR] [--format text|json] [--out path]")
 	fmt.Fprintln(w, "  inspect profile: scriptorium inspect profile --profile ID [--config PATH] [--profile-dir DIR] [--format text|json] [--out path]")

internal/adapter/cli/run_test.go

@@ -1258,6 +1258,81 @@ func TestPromptkitDefinitionsRenderThroughCLI(t *testing.T) {
 	}
 }
 
+func TestTransientRetrySettingsAndCLIOverride(t *testing.T) {
+	lib := newCLITestLibrary(t)
+	inputPath := lib.writeInputFile(t, "transcript.md", "hello")
+	writePromptFile(t, lib.promptDir, "retry.prompt", "local-default")
+	var hits int32
+	provider := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+		if atomic.AddInt32(&hits, 1) == 1 {
+			http.Error(w, "temporary", http.StatusServiceUnavailable)
+			return
+		}
+		w.Header().Set("Content-Type", "application/json")
+		_, _ = w.Write([]byte(`{"choices":[{"message":{"role":"assistant","content":"recovered"}}]}`))
+	}))
+	defer provider.Close()
+	writeProfileFile(t, lib.profileDir, "local-default", provider.URL+"/v1", "local-model")
+	configPath := writeAppConfigFile(t, fmt.Sprintf("prompt_dir: %q\nprofile_dir: %q\ntransient_retries: 1\n", lib.promptDir, lib.profileDir))
+	args := []string{"--config", configPath, "--prompt", "retry.prompt", "--input", "transcript=" + inputPath}
+
+	serve, err := parseServeArgs([]string{"--config", configPath})
+	if err != nil || serve.transientRetries == nil || *serve.transientRetries != 1 {
+		t.Fatalf("server did not inherit retry setting: cfg=%+v err=%v", serve, err)
+	}
+	if _, err := newEngine(serve.engineSettings()); err != nil {
+		t.Fatalf("server engine configuration: %v", err)
+	}
+
+	code, stdout, stderr := runCLICommand(t, runCommand, args)
+	if code != ExitOK || stdout != "recovered" || atomic.LoadInt32(&hits) != 2 {
+		t.Fatalf("configured retry was not applied: code=%d stdout=%q stderr=%q hits=%d", code, stdout, stderr, atomic.LoadInt32(&hits))
+	}
+	atomic.StoreInt32(&hits, 0)
+	code, stdout, stderr = runCLICommand(t, runCommand, append(append([]string{}, args...), "--transient-retries", "0"))
+	if code != ExitRuntimeError || stdout != "" || atomic.LoadInt32(&hits) != 1 {
+		t.Fatalf("explicit zero did not override configuration: code=%d stdout=%q stderr=%q hits=%d", code, stdout, stderr, atomic.LoadInt32(&hits))
+	}
+
+	for _, tc := range []struct {
+		name string
+		args []string
+		want *int
+	}{
+		{name: "omitted", args: args},
+		{name: "zero", args: append(append([]string{}, args...), "--transient-retries", "0"), want: intPointer(0)},
+		{name: "positive", args: append(append([]string{}, args...), "--transient-retries", "3"), want: intPointer(3)},
+	} {
+		t.Run(tc.name, func(t *testing.T) {
+			cfg, err := parseRunArgs(tc.args)
+			if err != nil {
+				t.Fatal(err)
+			}
+			req, err := buildRunRequestFromConfig(cfg)
+			if err != nil {
+				t.Fatal(err)
+			}
+			if !reflect.DeepEqual(req.TransientRetries, tc.want) {
+				t.Fatalf("request retry override: got %v, want %v", req.TransientRetries, tc.want)
+			}
+		})
+	}
+	for _, invalid := range []string{"-1", "wrong"} {
+		if _, err := parseRunArgs(append(append([]string{}, args...), "--transient-retries", invalid)); err == nil {
+			t.Fatalf("run accepted invalid retry count %q", invalid)
+		}
+		if _, err := parseRenderArgs(append(append([]string{}, args...), "--transient-retries", invalid)); err == nil {
+			t.Fatalf("render accepted invalid retry count %q", invalid)
+		}
+	}
+	code, stdout, stderr = runCLICommand(t, renderCommand, append(append([]string{}, args...), "--transient-retries", "0", "--format", "json"))
+	if code != ExitOK || strings.Contains(stdout, "transient_retries") {
+		t.Fatalf("render retry override changed prepared output: code=%d stdout=%q stderr=%q", code, stdout, stderr)
+	}
+}
+
+func intPointer(value int) *int { return &value }
+
 func TestPromptkitPromptInspectionSelectsVersionsAndContracts(t *testing.T) {
 	fixtureRoot := definitionFixtureRoot(t)
 	configPath := writeDefinitionFixtureConfig(t, fixtureRoot, true)

internal/config/config.go

@@ -36,6 +36,7 @@ type Config struct {
 	PromptDir        string                   `yaml:"prompt_dir"`
 	ProfileDir       string                   `yaml:"profile_dir"`
 	SchemaDir        string                   `yaml:"schema_dir"`
+	TransientRetries *strictInteger           `yaml:"transient_retries"`
 	Server           ServerConfig             `yaml:"server"`
 	Defaults         DefaultsConfig           `yaml:"defaults"`
 	Backends         map[string]BackendConfig `yaml:"backends"`
@@ -49,6 +50,20 @@ type BackendConfig struct {
 	QueueCapacity    *int           `yaml:"queue_capacity"`
 }
 
+type strictInteger int
+
+func (value *strictInteger) UnmarshalYAML(node *yaml.Node) error {
+	if node.Tag != "!!int" {
+		return fmt.Errorf("expected integer, got %s", node.Tag)
+	}
+	var decoded int
+	if err := node.Decode(&decoded); err != nil {
+		return err
+	}
+	*value = strictInteger(decoded)
+	return nil
+}
+
 type BackendSettings struct {
 	ID               string
 	Endpoint         string
@@ -82,6 +97,7 @@ type AppSettings struct {
 	MaxResponseBytes    int64
 	DefaultRenderFormat renderformat.PreparedRunOutputFormat
 	Backends            []BackendSettings
+	TransientRetries    *int
 }
 
 // CLIOverrides can be applied after config load to enforce precedence.
@@ -224,6 +240,13 @@ func decodeConfig(raw []byte) (Config, error) {
 
 func applyConfig(base AppSettings, cfg Config) (AppSettings, error) {
 	out := base
+	if cfg.TransientRetries != nil {
+		if *cfg.TransientRetries < 0 {
+			return AppSettings{}, fmt.Errorf("%w: transient_retries must be greater than or equal to 0", ErrInvalidConfig)
+		}
+		value := int(*cfg.TransientRetries)
+		out.TransientRetries = &value
+	}
 
 	if v := strings.TrimSpace(cfg.PromptDir); v != "" {
 		out.PromptDir = filepath.Clean(v)

internal/config/config_test.go

@@ -239,6 +239,44 @@ server:
 	}
 }
 
+func TestLoadConfigPreservesTransientRetryPresence(t *testing.T) {
+	for _, tc := range []struct {
+		name string
+		body string
+		want *int
+	}{
+		{name: "omitted", body: "prompt_dir: ./prompts\n"},
+		{name: "null", body: "transient_retries: null\n"},
+		{name: "zero", body: "transient_retries: 0\n", want: intPointer(0)},
+		{name: "positive", body: "transient_retries: 3\n", want: intPointer(3)},
+	} {
+		t.Run(tc.name, func(t *testing.T) {
+			settings, err := LoadConfig(writeConfigFile(t, "config.yml", tc.body), true)
+			if err != nil {
+				t.Fatal(err)
+			}
+			if !reflect.DeepEqual(settings.TransientRetries, tc.want) {
+				t.Fatalf("retry setting: got %v, want %v", settings.TransientRetries, tc.want)
+			}
+		})
+	}
+	for _, tc := range []struct {
+		body string
+		want error
+	}{
+		{body: "transient_retries: -1\n", want: ErrInvalidConfig},
+		{body: "transient_retries: wrong\n", want: ErrInvalidConfigYAML},
+		{body: "transient_retries: 1.5\n", want: ErrInvalidConfigYAML},
+	} {
+		_, err := LoadConfig(writeConfigFile(t, "config.yml", tc.body), true)
+		if !errors.Is(err, tc.want) {
+			t.Fatalf("body %q: expected %v, got %v", tc.body, tc.want, err)
+		}
+	}
+}
+
+func intPointer(value int) *int { return &value }
+
 func TestLoadConfigRejectsNegativeHTTPSizeLimits(t *testing.T) {
 	tests := []struct {
 		name string