promptkit/docs/internal/llm.md

Internal Model Client

Purpose

This document describes Promptkit's internal model-client implementation. The architecture policy owns the library boundary, and the OpenAI-compatible chat integration owns the observable outbound HTTP contract. The framework format reference owns the profile and prompt settings consumed by the client.

The concrete client remains under internal/llm. The root engine assembles it as the default implementation behind Promptkit's public client boundary.

Components And Flow

Client is the provider-neutral generation boundary consumed by later orchestration. OpenAICompatibleClient is the built-in implementation. It uses internal domain values for rendered prompts, execution targets, structured output, responses, and token usage.

Construction validates the configured base URL and clones any supplied http.Client so Promptkit can apply its timeout default without mutating the caller's client. Generation then:

1. validates request-level timeout and endpoint requirements;
2. maps the internal request into the OpenAI-compatible chat payload;
3. validates and merges extra parameters;
4. resolves authentication;
5. performs the outbound request under the applicable deadlines; and
6. decodes the first response choice and token usage.

The implementation has no retry loop, tool-call support, provider catalog, inbound HTTP behavior, or durable session store.

Failure Categories

The package preserves distinct error identities for invalid client configuration, invalid generation requests, request execution failures, non-success provider statuses, and malformed successful responses. Provider response bodies are not included in non-success errors.

Caller cancellation and deadline failures during the outbound request are reported as request execution failures. The runner classifies these identities without depending on HTTP status mapping.

Test Ownership

The OpenAI-compatible client tests own configuration, client cloning, deterministic deadline precedence, authentication, request and response mapping, malformed data, error identity, cancellation, and response-body suppression. They use local test servers and test transports; the default suite makes no live or paid provider requests.