diff --git a/README.md b/README.md index 8559433..84ee51c 100644 --- a/README.md +++ b/README.md @@ -33,10 +33,13 @@ boundary and constraints that framework work must preserve. ## Release Guidance -Consumers upgrading from `v0.5.0` to `v0.6.0` should read the -[v0.6.0 changelog and migration guide](docs/releases/v0.6.0.md). +Consumers upgrading from `v0.6.0` to `v0.7.0` should read the +[v0.7.0 changelog and migration guide](docs/releases/v0.7.0.md). Earlier adopters can consult the +[v0.6.0 changelog and migration guide](docs/releases/v0.6.0.md). + +Consumers upgrading from `v0.4.0` to `v0.5.0` can consult the [v0.5.0 changelog and migration guide](docs/releases/v0.5.0.md). Consumers upgrading from `v0.3.0` to `v0.4.0` should read the diff --git a/docs/releases/v0.7.0.md b/docs/releases/v0.7.0.md new file mode 100644 index 0000000..ab86ca6 --- /dev/null +++ b/docs/releases/v0.7.0.md @@ -0,0 +1,155 @@ +# Promptkit v0.7.0 + +This supplemental changelog and migration guide summarizes the consumer-facing +changes from `v0.6.0` to `v0.7.0`. The annotated `v0.7.0` tag is the +authoritative release record. Exact current contracts belong to the linked +GoDoc and durable documentation. + +## Summary + +`v0.7.0` expands provider integration and profile composition while making +credential and generation-failure handling more flexible: + +- Promptkit now includes the `rakestrawhome` backend and its Gemma profile; +- built-in generation failures expose bounded structured provider details; +- an unavailable optional API-key environment source no longer prevents a + request from reaching an upstream that permits unauthenticated access; and +- profiles can inherit from and selectively refine another profile. + +## Compatibility + +This release adds public declarations and fields but removes none. Existing +keyed configuration literals and ordinary `errors.Is` handling continue to +work. + +Adding `BaseProfileID` to `Profile` and `OpenAICompatibleProfileConfig` changes +their struct shape. Consumers using positional composite literals for either +type must convert them to keyed literals. Existing keyed literals require no +change. + +The `rakestrawhome` backend ID is now built in and reserved. A consumer that +previously registered that exact ID with `WithBackend` must remove its manual +registration before upgrading. Other custom backend registrations are +unchanged. + +When an optional backend, profile, or request `APIKeyEnv` is unset, empty, or +whitespace-only, the built-in client now omits `Authorization` and sends the +request. Previously this condition could fail before transport. Set +`Profile.APIKeyRequired` when missing credentials must remain a local +preflight error. + +Provider non-success responses continue to match `ErrLLMGenerate`. Their +rendered wording is not a compatibility contract; consumers can now use +`errors.As` with `*GenerationError` when structured status information is +needed. + +## Upgrade + +Update the module dependency with: + +```sh +go get gitea.maximumdirect.net/eric/promptkit@v0.7.0 +go mod tidy +``` + +Remove any manual `rakestrawhome` backend registration, convert positional +profile literals to keyed literals, and run the consuming project's ordinary +and race-enabled tests. + +## Rakestrawhome Built-In Backend And Profile + +Every engine now includes the reserved `rakestrawhome` backend, identified by +`BackendRakestrawHome`. The built-in `rakestrawhome-gemma-4-31b` profile +selects that backend. Consumers can use the maintained endpoint, credential, +capacity, and model defaults without registering either definition themselves. + +See the [built-in backend and profile catalogs](../formats.md#built-in-backends) +and the [consumer adoption example](../consumers/pkg-promptkit.md#use-the-rakestrawhome-built-in-profile) +for the current contracts. + +## Structured Generation Errors + +Non-2xx responses from the built-in OpenAI-compatible client now return an +immutable `*GenerationError`. Consumers can inspect the HTTP status and any +safely extracted provider code, type, or message while retaining the ordinary +generation-error category: + +```go +var generationErr *promptkit.GenerationError +if errors.As(err, &generationErr) { + status := generationErr.StatusCode() + _ = status +} +``` + +Provider fields are bounded and normalized but remain untrusted and may +contain sensitive request or schema details. Default and Go-syntax formatting +omit those fields. Applications must apply their own disclosure policy before +logging or presenting accessor values. + +See the [`GenerationError` GoDoc](../../generation_error.go), the +[consumer error-handling guide](../consumers/pkg-promptkit.md#handle-errors), +and the [OpenAI-compatible response contract](../integrations/openai-compatible-chat.md#response-handling). + +## Optional Credential Sources + +`APIKeyEnv` names an optional environment lookup source unless the selected +profile explicitly sets `APIKeyRequired`. When neither a direct request key nor +a usable environment value exists, the built-in client omits the bearer header +and handles the upstream response normally. This supports local and other +OpenAI-compatible providers that permit unauthenticated requests without +hiding an authentication error returned by a provider that requires one. + +The [credential format reference](../formats.md#credentials), the +[`Backend` GoDoc](../../backends.go), the +[`ExecutionTargetOverride` GoDoc](../../types.go), and the +[authentication integration contract](../integrations/openai-compatible-chat.md#authentication) +define the current precedence and availability rules. + +## Profile Inheritance + +YAML profiles can name one parent with `base_profile`; in-memory profiles use +`Profile.BaseProfileID`, and `OpenAICompatibleProfileConfig` forwards the same +field. A profile can act as an application-owned alias of a built-in or refine +selected inherited settings: + +```go +promptkit.WithProfiles(promptkit.Profile{ + ID: "weather-light", + BaseProfileID: "deepseek-4-flash", + ReasoningEffort: "high", +}) +``` + +Base lookup observes the existing source precedence. Chains are linear, +cycle-safe, and resolved afresh for ordinary operations. Prepared execution +freezes the fully resolved target. The selected leaf ID remains public while +effective execution settings reflect the resolved chain. + +See the [profile inheritance format reference](../formats.md#profile-inheritance), +the [consumer alias example](../consumers/pkg-promptkit.md#alias-a-built-in-profile), +and the [`Profile` GoDoc](../../types.go) for exact merge and validation +behavior. + +## Public API Changes + +The release adds: + +- `BackendRakestrawHome`; +- `GenerationError`, including `StatusCode`, `ProviderCode`, `ProviderType`, + `ProviderMessage`, `Error`, `GoString`, and `Unwrap`; +- `Profile.BaseProfileID`; and +- `OpenAICompatibleProfileConfig.BaseProfileID`. + +No public declaration was removed. + +## Consumer Action + +- Remove a manual backend registration whose ID is exactly `rakestrawhome`. +- Convert positional `Profile` or `OpenAICompatibleProfileConfig` literals to + keyed literals. +- Set `Profile.APIKeyRequired` where a missing credential must fail locally + instead of reaching the provider unauthenticated. +- Treat `GenerationError` provider fields as untrusted and potentially + sensitive when adopting the new accessors. +- Run consumer ordinary and race-enabled tests after updating the module. diff --git a/docs/roadmap/future.md b/docs/roadmap/future.md index 1bfaf79..6e10ebe 100644 --- a/docs/roadmap/future.md +++ b/docs/roadmap/future.md @@ -40,12 +40,11 @@ consumers. ### Public bounded output repair -After the codebase-audit remediations are complete, Promptkit should make its -bounded output-repair capability available through the public engine. A -consumer should be able to request a limited number of corrective generation -attempts when JSON or JSON Schema output fails content validation, without -having to reproduce Promptkit's generation, validation, capacity, and result- -accounting orchestration. +Promptkit should make its bounded output-repair capability available through +the public engine. A consumer should be able to request a limited number of +corrective generation attempts when JSON or JSON Schema output fails content +validation, without having to reproduce Promptkit's generation, validation, +capacity, and result-accounting orchestration. - Repair is validation recovery, not a general provider retry, failover, or backoff policy. Transport failures, cancellation, and operational schema or @@ -62,10 +61,6 @@ accounting orchestration. - Ordinary and prepared execution should expose coherent behavior, including cancellation, frozen prepared state, error identity, and capacity lifetime. -Select this work only after the accepted audit findings affecting shared -execution invariants, validation, orchestration, transport, and repair -internals have been remediated. - ## Entry Format Use a short heading followed by a concise summary. Add focused bullets when diff --git a/docs/roadmap/implementation.md b/docs/roadmap/implementation.md deleted file mode 100644 index 08b05d2..0000000 --- a/docs/roadmap/implementation.md +++ /dev/null @@ -1,477 +0,0 @@ -# Profile Inheritance Implementation Plan - -## Purpose - -Implement the profile-inheritance feature defined by the -[profile inheritance roadmap](profile-inheritance.md). This plan is the -decision-complete execution sequence for a coding agent. Follow the repository -[architecture](../policy/architecture.md), [testing](../policy/testing.md), and -[documentation](../policy/documentation.md) policies and the task-specific -reading guide in [`docs/development.md`](../development.md) throughout the -work. - -## Target Outcome - -A downstream consumer can define a YAML or in-memory profile whose -`base_profile`/`BaseProfileID` names another profile, including a Promptkit -built-in. Promptkit resolves the complete one-parent chain through the assembled -profile catalog, merges root-to-leaf profile settings, resolves the final -backend, and then applies request overrides exactly as it does for a standalone -profile. - -The selected leaf ID remains the public profile identity. Ordinary operations -resolve sources afresh, while prepared execution freezes the resolved target. -Missing dependencies, cycles, excessive depth, and incomplete resolved profiles -fail as profile-load errors without changing the distinct not-found contract for -an absent directly selected profile. - -## Fixed Decisions - -- YAML uses `base_profile`; the public Go fields are - `Profile.BaseProfileID` and - `OpenAICompatibleProfileConfig.BaseProfileID`. -- One profile may name one base. Bases may form a linear chain of at most 32 - profiles, including the selected leaf. -- The resolver performs every lookup through the complete raw composite - repository. Each base therefore observes the ordinary in-memory, configured, - fallback, and built-in precedence. -- The resolving repository is stateless and does not cache results across - calls. It is applied once, outside the fully assembled raw overlay chain. -- Profile definitions with a base may omit `model` and both connection fields. - Standalone definitions retain their current completeness requirements. -- Root-to-leaf merge behavior follows existing profile presence semantics: - nonblank strings and nonzero numbers override; blank strings and zero numbers - inherit; a nonempty `extra_params` map replaces the complete inherited map; - and `APIKeyRequired` is a sticky logical OR. -- `id` always comes from the selected leaf. `base_profile` is cleared from the - resolved profile before it reaches the use-case layer. -- Backend and endpoint remain independent fields. A child backend replaces the - inherited backend ID without implicitly clearing an inherited endpoint; a - child endpoint replaces only the endpoint. -- This work introduces no profile-level clearing syntax, presence pointers, - multiple inheritance, source-qualified references, public resolution graph, - new public error sentinel, or provider-wire change. -- A missing directly selected profile continues to expose - `ErrProfileNotFound`. Once the leaf exists, a missing base is an invalid - selected profile and must expose `ErrProfileLoad` without also matching - `ErrProfileNotFound`. -- Existing raw-source failures and context cancellation remain discoverable - through `errors.Is` where they are currently preserved. The missing-base - branch is the one deliberate exception: its diagnostic names the missing ID - and chain but does not wrap the internal not-found sentinel. -- No release notes, version tags, commits, or pushes are part of these stages. - -## Execution Rules - -- Complete the stages in numerical order. Each stage is sized for one - gpt-5.6-terra implementation prompt and must finish its focused verification - before the next begins. -- At the start of every stage, inspect the working tree and preserve unrelated - changes. The feature roadmap is intentional planning state and must remain. -- Treat intermediate stages as an unreleased partial implementation. Do not - describe the feature in durable current-state prose until Stage 4. -- Keep inheritance logic in `internal/profile`. Do not add recursive profile - lookup, chain merging, or source-precedence logic to `internal/usecase` or the - root public facade. -- Use real in-memory repositories and synthetic `fs.FS` sources where practical. - Tests must remain deterministic, offline, parallel-safe, and independent of - provider credentials. -- Update existing test owners when their validation contract changes; do not - retain contradictory expectations or duplicate the full inheritance matrix - at higher layers. -- Use `apply_patch` for edits, format every changed Go file, and review the - complete stage diff before proceeding. - -## Stage 1: Add Profile Definition Surfaces And Separate Local Validation - -### Objective - -Represent a base-profile reference in every supported profile input while -preserving strict decoding, standalone-profile compatibility, and eager local -validation. At the end of this stage repositories can publish locally valid -derived definitions, but the engine does not yet resolve them. - -### Implementation - -1. Add `BaseProfileID string` to `internal/domain.ExecutionProfile` with the - YAML tag `base_profile`. Keep it out of execution targets and provider - request types. -2. Add `BaseProfileID string` to the public `Profile` immediately after `ID`. - Its GoDoc must state that: - - `WithProfiles` trims the value; - - a nonblank value permits required target fields to be inherited; - - reference existence and resolved completeness are checked only when the - profile is selected or inspected; and - - blank means the profile remains standalone. -3. Add the matching field to `OpenAICompatibleProfileConfig`, document that it - maps to `Profile.BaseProfileID`, and copy it in - `OpenAICompatibleProfile`. Preserve the constructor's shallow-copy and - deferred deep-validation behavior for `ExtraParams`. -4. Map and trim `Profile.BaseProfileID` in `toDomainProfile`. Ensure the memory - repository continues to return caller-independent `ExtraParams` and does not - expose shared mutable profile state. -5. Centralize the duplicated file-profile and in-memory-profile normalization - rules in `internal/profile` rather than adding a third validation path: - - provide an internal-package exported helper named - `NormalizeAndValidateDefinition` so the root facade can validate a - converted in-memory `domain.ExecutionProfile` without owning profile - parsing rules; - - trim `ID`, `BaseProfileID`, `BackendID`, and `Endpoint` in that helper; - - require a nonblank `ID` and validate a supplied endpoint and all execution - setting bounds for every definition; - - when `BaseProfileID` is blank, retain the existing requirements for a - nonblank model and at least one nonblank backend or endpoint; - - when `BaseProfileID` is nonblank, allow model, backend, and endpoint to be - omitted so they can be inherited; and - - do not add unrelated normalization, environment-name policy, clearing - behavior, or backend registry lookup. -6. Replace `normalizeAndValidateProfile` in - `internal/profile/filesystem_repository.go` and - `normalizeAndValidatePublicProfile` in `profiles.go` with the shared helper. - Preserve each caller's current public/internal error wrapping, selected-file - path diagnostic, raw-key prohibition, strict single-document rule, and - duplicate-ID behavior. Remove imports and helpers made obsolete by the - centralization. -7. Keep the raw repositories raw: file, `fs.FS`, memory, overlay, and built-in - repositories return `BaseProfileID` but do not recursively load it in this - stage. - -### Tests - -Update the narrow existing owners: - -1. In `internal/profile/repository_test.go`, extend the filesystem/`fs.FS` - source-parity coverage to prove that: - - strict YAML accepts `base_profile`, trims it, and publishes it; - - an alias containing only `id` and `base_profile` is locally valid; - - a derived profile still rejects an invalid supplied endpoint, negative or - out-of-range setting, malformed `extra_params`, raw `api_key`, duplicate - ID, and extra YAML document through their existing identities; and - - a standalone profile missing connection fields or model remains invalid. - Modify existing missing-field expectations rather than duplicating their - entire matrices. -2. In root package tests, extend the existing `WithProfiles` validation owner - to show that an incomplete profile with a nonblank `BaseProfileID` is - accepted by `NewEngine`, while the equivalent standalone profile retains its - current `ErrInvalidConfig` result. Do not attempt to select the derived - profile until the resolver is implemented. -3. Extend `TestOpenAICompatibleProfileMapsEveryField` so - `BaseProfileID` is included in the constructor's exact field mapping. -4. Preserve existing tests for defensive copying, duplicates, standalone - profiles, overlay precedence, built-ins, and strict decoding. - -### Verification - -```sh -gofmt -w internal/domain/domain.go internal/profile/*.go profiles.go types.go -go test ./internal/profile -go test ./internal/profile/builtin -go test ./... -run 'Profile|OpenAICompatibleProfile' -go test ./... -git diff --check -``` - -Stage 1 is complete when all supported definition surfaces carry the normalized -base ID, locally valid aliases can be stored, standalone validation is -unchanged, and no repository performs inheritance resolution yet. - -**Status:** Complete. - -## Stage 2: Implement The Stateless Resolving Repository - -### Objective - -Add the internal component that resolves one complete profile chain, merges it -deterministically, enforces safety bounds, and returns one ordinary resolved -profile to its caller. - -### Implementation - -1. Add `internal/profile/resolving_repository.go` with: - - an unexported `resolvingRepository` containing one underlying raw - `Repository`; - - `NewResolvingRepository(source Repository) Repository`; and - - an unexported constant setting the maximum chain length to 32, counting - the selected leaf. -2. Make the wrapper safe for concurrent calls by keeping the source immutable - and all chain, visited-ID, and merge state local to `GetProfile`. Do not add a - cache, mutex, process-global registry, or source snapshot. -3. Resolve one request using this exact flow: - - reject a nil underlying repository or blank requested ID as - `ErrInvalidProfile`; - - trim and look up the directly requested ID through the underlying raw - repository; - - if that first lookup fails, return its error unchanged so direct - `ErrProfileNotFound` behavior remains intact; - - reject a nil profile returned without an error as `ErrInvalidProfile`; - - follow each normalized nonblank `BaseProfileID` through the same underlying - repository, recording IDs and profiles from leaf toward root; - - check `ctx.Err()` between lookups; - - detect a repeated ID before loading it again and return an - `ErrInvalidProfile` cycle diagnostic containing the `a -> b -> a` chain; - - reject a reference that would make the chain exceed 32 profiles with an - `ErrInvalidProfile` diagnostic containing the traversed chain and next ID; - - when a base lookup returns `ErrProfileNotFound`, return a new - `ErrInvalidProfile` error naming the missing base and chain without - wrapping `ErrProfileNotFound`; and - - for any other base error, add `ErrInvalidProfile`, base ID, and chain - context while preserving the source error and cancellation identity with - wrapping. -4. Merge the collected definitions from root to leaf into a new value. Use one - focused helper with these rules: - - retain the selected leaf `ID` and clear final `BaseProfileID`; - - replace `BackendID`, `Endpoint`, `Model`, `ServiceTier`, - `ReasoningEffort`, and `APIKeyEnv` only for a nonblank child value; - - replace `Temperature`, `MaxTokens`, `TopP`, and `TimeoutSeconds` only for a - nonzero child value; - - set `APIKeyRequired` to the logical OR of every definition in the chain; - - replace `ExtraParams` only when the child map is nonempty; otherwise - inherit the current map; and - - defensively deep-copy every retained or replacement `ExtraParams` map by - using the existing bounded `internal/jsonvalue` owner. Never mutate a - repository-returned profile or map. Treat an unexpected copy failure as - `ErrInvalidProfile`, preserve the copy error, and include the selected - chain in the diagnostic. -5. Add an unexported final validator in `internal/profile` that requires a - nonblank model and at least one backend or endpoint on the merged profile, - and reuses the common endpoint and execution-setting validation. Wrap final - failures with `ErrInvalidProfile` and the selected chain. The resolver must - not perform backend registry membership lookup; that remains in - `internal/usecase` after profile resolution. -6. Return a caller-owned resolved profile on every successful call. A - standalone profile passes through the same final validation and copy path, - ensuring the wrapper does not create separate standalone and inherited - execution contracts. - -### Tests - -Create `internal/profile/resolving_repository_test.go` and use small synthetic -repositories plus the real overlay repository where it materially protects -composition: - -1. Prove a three-level root/middle/leaf chain merges representative values for - every rule: inherited values, string and numeric replacements, independent - backend/endpoint behavior, sticky `APIKeyRequired`, complete-map - `ExtraParams` replacement, leaf identity, and cleared resolution metadata. - Use one well-structured table or fixture rather than one test per field. -2. Prove an empty child `ExtraParams` inherits the base map, a nonempty child - map replaces rather than key-merges it, and mutating the returned nested - values cannot mutate the raw repository or a later result. -3. Prove the same composite repository resolves a base from a lower-precedence - source and honors a higher-precedence shadow of that base ID. Do not add - source-specific resolution to the wrapper. -4. Cover the safety/error matrix through the resolving repository boundary: - - directly selected missing ID preserves `ErrProfileNotFound`; - - missing base matches `ErrInvalidProfile` and does not match - `ErrProfileNotFound`; - - direct and indirect cycles are rejected with useful chains; - - exactly 32 profiles are accepted and a 33rd is rejected; - - nil repository results and incomplete final profiles are invalid; - - non-not-found base errors retain their original identity; and - - cancellation before or during traversal remains discoverable and stops - later lookups. -5. Prove freshness and concurrency without timing sleeps: change a controlled - raw repository between sequential calls and observe the new base value, then - issue concurrent independent resolutions and verify results do not share - mutable maps. Use synchronization owned by the fake; do not race on a plain - map. -6. Preserve the existing raw repository and overlay tests unchanged except - where Stage 1 intentionally changed local completeness rules. - -### Verification - -```sh -gofmt -w internal/profile/*.go -go test ./internal/profile -run 'Resolving|Inheritance|Profile' -go test -race ./internal/profile -go test ./... -git diff --check -``` - -Stage 2 is complete when a standalone internal wrapper can safely and freshly -resolve every accepted chain into one complete, copied profile with the fixed -merge and error contracts. - -**Status:** Complete. - -## Stage 3: Assemble Inheritance And Prove Public Workflows - -### Objective - -Place the resolver at the correct engine boundary and prove that YAML, -in-memory, built-in, inspection, ordinary preparation, runtime overrides, and -prepared execution compose as intended through the public facade. - -### Implementation - -1. Update `newProfileRepository` in `engine.go` to: - - assemble the raw repository in its current precedence order—built-ins, - fallback, configured source or `Config.ProfileDir`, then in-memory; - - wrap that complete raw repository exactly once with - `profile.NewResolvingRepository`; and - - return the wrapper to `usecase.NewRunner`. - Do not wrap individual sources, because that would prevent a higher-level - child from referencing a lower-level base and would change shadowing rules. -2. Keep `internal/usecase.resolveProfileSelection` structurally unchanged. It - should continue to request one profile, preserve the requested normalized ID - as selection identity, resolve the final backend once, and then apply the - ordinary backend/profile/request precedence. Do not pass `BaseProfileID` - into `ExecutionTarget`, `PreparedRun`, `RunResult`, injected `LLMClient` - requests, stable JSON, hashes, or provider payloads. -3. Confirm `APIKeyRequired` behavior after inheritance without introducing a - special credential path: a true value in any ancestor reaches the existing - profile-to-target merge, clears inherited profile/backend `APIKeyEnv`, and - still permits a direct request key or explicit request environment override. -4. Preserve the current error mapper. The resolver's use of - `ErrInvalidProfile` must make a missing/invalid base cross the use-case and - root boundaries as `ErrProfileLoad`; no new root sentinel or special-case - string matching is permitted. -5. Do not modify built-in profile assets merely to demonstrate inheritance. - Built-ins serve as valid bases without changing their own definitions. - -### Tests - -Use the smallest public tests that protect assembled behavior without repeating -the resolver's matrix: - -1. Add one external-package contract workflow that defines `weather-light` as - an in-memory alias/refinement of the built-in `deepseek-4-flash`. Compare - its inspected target with an inspection of the base rather than duplicating - every built-in literal. Assert: - - the child inspection reports `weather-light`; - - backend and model are inherited; - - explicit child reasoning and timeout values win; - - `Prepare` reports the child as `SelectedProfileID`; and - - a per-request reasoning or timeout override still wins over the child. -2. Exercise YAML through `WithProfileFS` with a minimal alias of a built-in and - prove it resolves through inspection or preparation. Parser tests own YAML - edge cases, so do not repeat them here. -3. Add a focused public error-identity table: - - a directly selected absent profile still matches `ErrProfileNotFound` and - not `ErrProfileLoad`; - - an existing child with an absent base matches `ErrProfileLoad` and not - `ErrProfileNotFound`; and - - one representative cycle matches `ErrProfileLoad` and includes its IDs. - Reuse or extend the existing profile-inspection public error owner where - that keeps the suite lean. -4. Prove fresh ordinary resolution and frozen prepared execution with one - controlled mutable `fs.FS` workflow: - - prepare an execution while its base supplies model A; - - change the base source to model B after preparation; - - run the prepared handle through an injected deterministic client and - observe model A; and - - perform a new inspection or ordinary preparation and observe model B. - Keep all mutation sequential or synchronized and make no provider call. -5. Add only a narrow credential assertion if existing use-case tests do not - already prove the final inherited `APIKeyRequired` target behavior. The - resolver test owns sticky merging; current credential tests own availability, - precedence, redaction, and prepared rechecking. -6. Run all existing source-precedence, fallback, malformed-profile, built-in, - public JSON, runtime-override, credential, and prepared-execution tests. Fix - production behavior rather than weakening those contracts. - -### Verification - -```sh -gofmt -w engine.go engine_test.go public_contract_test.go prepared_execution_contract_test.go profiles.go types.go -go test ./... -run 'Profile|Inheritance|PreparedExecution|RuntimeOverride' -go test ./internal/profile -go test ./internal/usecase -go test -race ./... -go vet ./... -go build ./... -git diff --check -``` - -Stage 3 is complete when the engine resolves aliases and refinements across all -supported sources, public identities and errors match the roadmap, ordinary -operations remain fresh, and prepared execution remains frozen. - -**Status:** Complete. - -## Stage 4: Publish Canonical Documentation And Complete Validation - -### Objective - -Align every canonical contract owner with the implemented feature, remove -temporary ambiguity, and validate the release candidate comprehensively. - -### Documentation - -1. Finalize exact GoDoc for `Profile.BaseProfileID` and - `OpenAICompatibleProfileConfig.BaseProfileID`, plus nearby `Profile`, - `OpenAICompatibleProfile`, and `WithProfiles` text whose standalone - completeness or validation-timing statements changed. GoDoc owns the public - fields and construction contract; do not reproduce internal traversal. -2. Update `docs/formats.md` as the exact YAML contract owner: - - add `base_profile` to the field table and example; - - distinguish locally complete standalone profiles from derived aliases; - - document root-to-leaf override rules, complete-map `extra_params` - replacement, sticky `APIKeyRequired`, independent backend/endpoint - behavior, the 32-profile limit, and lack of clearing syntax; - - document cross-source lookup precedence and shadowing; and - - distinguish a missing selected profile from an invalid selected chain. -3. Update `docs/internal/sources.md` to describe the raw composite catalog, - the one outer resolving repository, fresh per-operation traversal, - cycle/depth protection, final validation, and prepared-state freezing. Link - to the format reference for exact consumer syntax and merge rules. -4. Update the `internal/profile` entry in `docs/internal/overview.md` because - the concrete package now resolves inherited definitions in addition to - loading, validating, and overlaying raw sources. Do not change architecture - policy because package ownership and dependency direction are unchanged. -5. Add one concise task-oriented example to - `docs/consumers/pkg-promptkit.md` showing an application-owned alias of a - built-in and a selected child override. Link to the format reference and - public GoDoc for details rather than duplicating the complete contract. -6. Do not update the README, examples, release documents, release procedure, - future/deferred catalogs, or built-in catalog unless the implementation has - made a concrete current-state statement there inaccurate. Do not copy the - roadmap into durable documentation. - -### Final Validation - -Run the complete maintainer workflow from -[`docs/development.md#maintainer-validation`](../development.md#maintainer-validation): - -```sh -go test ./... -go test -race ./... -go vet ./... -go build ./... -go run ./examples/go-library/prepare -go run ./examples/go-library/run -``` - -Then run the documented tracked-Go formatting check, repository-relative -Markdown link checker, workspace/vendor/module-replacement checks, unstaged and -staged whitespace checks, ignored-file check, credential scan, and -`git diff --check`. Inspect the complete diff and confirm: - -- only `base_profile`/`BaseProfileID` were added to consumer configuration; -- raw repositories remain source-local and the resolver wraps the full catalog - once; -- chain resolution is bounded, cycle-safe, fresh, and concurrency-safe; -- merge behavior covers every profile field without mutating source values; -- direct missing and dependent missing profiles preserve distinct public error - identities; -- selected IDs, backend resolution, runtime overrides, credentials, and - prepared freezing retain their stated ownership; -- resolution metadata never reaches stable execution JSON, hashes, injected - generation targets, or provider payloads; -- tests remain lean, deterministic, offline, and owned at the narrowest stable - boundary; and -- durable documentation describes only implemented behavior with one canonical - owner per exact contract. - -Stage 4 is complete when all focused and repository-wide checks pass and the -implementation, GoDoc, format reference, internal documentation, and consumer -guidance agree. - -**Status:** Complete. - -## Open Questions - -None. Naming, supported sources, lookup precedence, merge behavior, connection -semantics, credential inheritance, validation timing, chain bounds, error -identity, freshness, prepared freezing, package ownership, test boundaries, and -documentation ownership are fixed by the feature roadmap and this plan. diff --git a/docs/roadmap/profile-inheritance.md b/docs/roadmap/profile-inheritance.md deleted file mode 100644 index 63a4a59..0000000 --- a/docs/roadmap/profile-inheritance.md +++ /dev/null @@ -1,224 +0,0 @@ -# Profile Inheritance Roadmap - -## Purpose - -Allow a Promptkit execution profile to derive from another profile. This gives -downstream consumers stable, application-owned profile IDs without requiring -them to copy a built-in or shared profile's model and execution settings. - -For example, Weatherreporter should be able to select `weather-light` from its -prompt definitions while defining that profile as an alias or refinement of a -Promptkit built-in: - -```yaml -id: weather-light -base_profile: deepseek-4-flash -reasoning_effort: high -timeout_seconds: 120 -``` - -Changing only `base_profile` to another profile later should redirect every -prompt that selects `weather-light`, without requiring changes to deployed -prompt definitions. The inherited profile remains ordinary Promptkit -configuration rather than application-specific routing logic. - -## Target End State - -- File and `fs.FS` profile definitions may use the optional `base_profile` - field to name one parent profile. -- In-memory profiles expose the equivalent `Profile.BaseProfileID` field. - `OpenAICompatibleProfileConfig` exposes and forwards the same field so its - convenience constructor remains feature-complete. -- A profile containing only `id` and `base_profile` is a valid semantic alias. - Fields needed for an executable target may be inherited instead of repeated. -- Base profiles may come from any configured profile source, including the - embedded built-in catalog. Resolution uses the engine's complete assembled - profile catalog and its existing source precedence. -- A child may refine selected inherited settings using the ordinary profile - fields. After inheritance resolves, backend defaults and request overrides - retain their existing precedence. -- `Prepare`, `PrepareExecution`, `Run`, and `InspectProfile` all use the same - inheritance behavior. Prepared execution freezes the fully resolved target - and does not reopen the profile chain when it later runs. -- Public results continue to report the selected child ID, such as - `weather-light`, while effective backend, endpoint, model, credentials, and - execution settings reflect the resolved chain. - -## Profile Reference Semantics - -Each profile may name at most one direct base. Bases may themselves inherit, -so aliases and refinements can form a linear chain. Multiple inheritance and -merging an array of profiles are outside this feature. - -Every ID in a chain is resolved through the same composite catalog used for an -ordinary profile selection. Existing precedence therefore applies separately -to each lookup: - -1. in-memory profiles; -2. the ordinary configured profile source; -3. the application fallback profile source; and -4. the embedded built-in catalog. - -A higher-precedence definition of a base ID intentionally shadows a lower- -precedence definition, just as it would if selected directly. References are -not source-qualified, and there is no special syntax for bypassing an override -to select a lower-precedence or specifically built-in definition. A profile -cannot extend a shadowed definition with its own ID; that is a self-cycle. - -Profile chains are resolved afresh for each ordinary preparation, execution, -or inspection operation, preserving the current fresh-source behavior. The -resolver does not cache a chain across operations. Mutable sources are not -promised a transactional snapshot across separate file reads; callers that -need a frozen result use prepared execution. - -## Override Semantics - -Inheritance combines profile definitions from the root base to the selected -leaf. The leaf's `id` is always retained, and `base_profile` is resolution -metadata rather than an execution setting. - -Child fields use the profile conventions already exposed by Promptkit: - -- nonblank `backend`, `endpoint`, `model`, `service_tier`, - `reasoning_effort`, and `api_key_env` values replace inherited values; -- nonzero `temperature`, `max_tokens`, `top_p`, and `timeout_seconds` values - replace inherited values; -- omitted, blank, or zero fields inherit, according to the existing profile - contract; -- a nonempty `extra_params` map replaces the complete inherited map rather - than merging individual keys; and -- `APIKeyRequired: true` is inherited and remains sticky through the chain. - It retains the existing security behavior of clearing inherited profile or - backend environment sources and requiring a direct request key or an - explicit request `APIKeyEnv` override. - -This feature does not introduce profile-level presence pointers or clearing -syntax. A derived profile cannot use a blank string, numeric zero, false, or an -empty map to clear an inherited value because those states already mean -"unspecified" for profile configuration. Consumers can use the existing -presence-aware runtime overrides when they need an explicit zero or an empty -reasoning value. A future clearing syntax may be considered independently if -real consumer demand emerges. - -Connection fields retain their existing independent meanings. A child endpoint -may override an inherited endpoint without changing backend identity, and a -child backend replaces an inherited backend ID. Other fields continue to -inherit unless the child supplies their ordinary nonblank or nonzero override. - -## Validation And Failure Behavior - -Profile parsing and registration must distinguish local validity from resolved -completeness: - -- IDs, base IDs, supplied endpoints, numeric bounds, extra parameters, and - other values that already have source-local rules remain validated at their - owning input boundary. -- A standalone profile with no base continues to require a model and at least - one backend or endpoint. -- A profile with a base may omit those required target fields, because the - resolved chain may provide them. -- The completely merged profile must satisfy the same target invariants as a - current standalone profile before backend resolution and use. -- In-memory profiles without a base retain their current `NewEngine` - validation behavior. Reference existence, cycles, and resolved completeness - are evaluated when an inherited profile is selected or inspected, so file - and in-memory profiles share one resolution contract and fresh lower sources - are not frozen at engine construction. - -Resolution must detect direct and indirect cycles and impose a maximum chain -length of 32 profiles, including the selected leaf. The diagnostic should name -the relevant profile chain without exposing internal package values. - -Failure identity distinguishes the requested profile from its dependencies: - -- an absent directly selected profile retains `ErrProfileNotFound`; -- an existing selected profile whose base is absent, malformed, cyclic, - excessively deep, or incomplete fails with `ErrProfileLoad`, not - `ErrProfileNotFound`; and -- backend lookup and resolved-setting failures retain their existing - `ErrProfileLoad` behavior. - -A malformed higher-precedence definition remains authoritative and stops -fallback. Inheritance must not silently skip a broken base to use another -definition or partially resolve a chain. - -## Architectural Ownership - -Inheritance resolution belongs in `internal/profile`, around the fully -assembled composite repository. Individual filesystem, `fs.FS`, in-memory, -and built-in repositories continue to own discovery, strict decoding, local -normalization, defensive copying, and source precedence; they must not resolve -bases independently within their own source. - -The root facade assembles the raw composite catalog and the resolving profile -boundary. `internal/usecase` continues to request one selected profile and -resolve its backend; it should receive an already combined, caller-owned -profile rather than implementing recursion or source traversal itself. - -`internal/domain` may carry the normalized base ID and any internal information -needed to distinguish locally supplied fields, but those internal values must -not leak through public execution targets or provider requests. Final target -resolution still follows: - -1. framework defaults; -2. the backend selected by the resolved profile; -3. the fully resolved profile; and -4. request runtime overrides. - -## Public And Format Surface - -The feature adds only the smallest consumer-facing configuration needed for -one-parent inheritance: - -- YAML `base_profile` in the profile format; -- `Profile.BaseProfileID`; and -- `OpenAICompatibleProfileConfig.BaseProfileID`. - -Exact public behavior belongs in the new fields' GoDoc once implemented. The -profile format reference owns YAML syntax, merge rules, chain limits, and -validation behavior. Consumer guidance should show one concise alias/refinement -workflow and link to those canonical contracts rather than duplicating them. - -No new engine method, profile registry mutation API, public resolver, public -inheritance graph, or result provenance field is required. Existing profile -selection through prompt defaults and `RunRequest.ProfileID` remains unchanged. - -## Quality And Documentation End State - -Testing should follow the repository's lean ownership model: - -- profile parser and registration tests own the new field, local validation, - and defensive copying; -- profile resolver tests own cross-source lookup, precedence, chain merging, - cycle and depth protection, missing bases, and resolved completeness; -- use-case tests retain ownership of backend and runtime-override precedence - without duplicating the full inheritance matrix; and -- one representative external-package workflow should prove that a downstream - alias of a built-in profile reports the child ID and inherited effective - target through preparation or inspection. - -All tests remain deterministic, offline, parallel-safe, and independent of -real provider credentials. Existing standalone-profile, overlay, malformed- -selection, built-in, credential, and prepared-execution contracts must remain -green. - -The implemented feature's exact GoDoc, `docs/formats.md`, -`docs/internal/sources.md`, and focused consumer guidance must agree. The -internal component inventory reflects inheritance resolution as a concrete -`internal/profile` responsibility. Release documentation remains a separate -release-preparation concern. - -## Non-Goals - -- Multiple inheritance or ordered profile mixins. -- Deep or key-by-key merging of `extra_params`. -- Source-qualified references or a special built-in namespace. -- Extending a shadowed lower-precedence profile with the same ID. -- Per-request changes to the base-profile relationship. -- Profile-level clearing syntax or presence-aware scalar fields. -- Caching resolved chains across operations or watching sources for changes. -- Changing backend registry, concurrency, credential, runtime-override, - provider-wire, or prompt-selection semantics beyond applying them to the - resolved profile. -- Consumer-specific configuration discovery, deployment migration, or routing - policy. diff --git a/docs/roadmap/structured-generation-errors.md b/docs/roadmap/structured-generation-errors.md deleted file mode 100644 index d159915..0000000 --- a/docs/roadmap/structured-generation-errors.md +++ /dev/null @@ -1,260 +0,0 @@ -# Structured Generation Errors - -## Purpose - -Promptkit should give downstream applications actionable, machine-readable -details when the built-in OpenAI-compatible client receives a non-success HTTP -response. Today the client reports only the status code and discards the -provider response body. This makes ordinary configuration failures, such as an -unsupported strict JSON Schema keyword, unnecessarily difficult to diagnose. - -This feature supplies bounded facts about the provider response. It does not -make retry, presentation, or logging decisions for consumers. - -## Target End State - -Every non-2xx response received by Promptkit's built-in OpenAI-compatible -client becomes a public typed generation error. A consumer can use -`errors.As` to obtain the HTTP status and any safely extracted provider fields, -and `errors.Is` continues to match `ErrLLMGenerate`. - -The typed contract is available from both `Run` and `RunPrepared`. It is not -produced during preparation, which performs no model request. Successful -responses, transport failures before a response is received, cancellation, -capacity failures, validation failures, and nil responses from injected model -clients retain their existing categories and behavior. - -An unusable response body never hides the known HTTP status. Empty, malformed, -unrecognized, unreadable, or oversized bodies therefore produce the same typed -error with status-only detail rather than falling back to an unstructured -error or becoming a malformed-success response. - -## Public Contract - -The root package exposes an immutable `GenerationError` type with unexported -state and these read-only accessors: - -- `StatusCode() int` returns the received HTTP status code; -- `ProviderCode() string` returns a normalized provider code, when present; -- `ProviderType() string` returns a normalized provider error type, when - present; and -- `ProviderMessage() string` returns the bounded normalized diagnostic message, - when present. - -The engine returns a `*GenerationError`, so the idiomatic inspection form is: - -```go -var generationErr *promptkit.GenerationError -if errors.As(err, &generationErr) { - status := generationErr.StatusCode() - message := generationErr.ProviderMessage() - _, _ = status, message -} -``` - -There is no public constructor or mutation API. The type implements `error`, -unwraps to `ErrLLMGenerate`, and provides safe ordinary and Go-syntax -formatting. `Error()` and `GoString()` include the HTTP status but no provider- -controlled code, type, or message. Consumers must use the accessors -deliberately when they want provider details and must not classify failures by -matching error text. - -The zero value and a nil `*GenerationError` receiver are safe: accessors return -zero or empty values, formatting returns a generic redacted generation-failure -description, and unwrapping still identifies `ErrLLMGenerate`. Engine-produced -values always have the non-2xx status received from the provider. The type has -no stable JSON representation. - -All provider-derived strings remain untrusted even after normalization. GoDoc -must warn consumers that provider fields can contain sensitive request or -schema fragments and must not be logged, displayed, or returned to another -caller without an application-appropriate disclosure policy. - -## Recognized Provider Envelope - -Promptkit recognizes only the conventional OpenAI-compatible top-level error -object: - -```json -{ - "error": { - "message": "diagnostic text", - "type": "invalid_request_error", - "code": "unsupported_parameter" - } -} -``` - -The envelope must be one JSON document followed only by JSON whitespace. The -top-level `error` value must be an object. Unknown top-level and error-object -fields are ignored. The optional supported fields are interpreted -independently: - -- `message` and `type` must be JSON strings; -- `code` may be a JSON string or number and is exposed as normalized text; - numeric codes retain their validated JSON number text without floating-point - coercion; and -- `null`, booleans, arrays, objects, or otherwise invalid values are treated as - absent for that field. - -An invalid optional field does not discard other valid supported fields. An -absent `error` object, malformed or multiply framed JSON, or an object with no -usable supported fields simply leaves all provider accessors empty while -preserving the typed status error. - -Promptkit does not expose `param`, metadata objects, nested causes, headers, or -provider-specific extensions in this feature. - -## Bounded Reading And Normalization - -Non-success bodies have a separate fixed limit of 64 KiB (65,536 bytes). This -is intentionally much smaller than the successful completion-body limit while -remaining large enough for useful schema diagnostics. - -- A declared `Content-Length` above the limit is rejected without reading the - body for detail extraction. -- Otherwise Promptkit reads at most one byte beyond the limit so streamed, - chunked, and underreported bodies are bounded. -- A body over the limit contributes no provider fields; Promptkit does not - parse or retain a prefix as though it were a complete envelope. -- Read failures likewise discard provider fields while preserving the status. -- The response body is closed on every outcome and is not drained beyond the - bounded read. - -Extracted strings are converted to valid UTF-8, trimmed, and made single-line: -invalid UTF-8 is replaced, and runs of Unicode whitespace, control characters, -and formatting controls are replaced with one ASCII space. Empty normalized -values are treated as absent. - -Normalized provider codes and types are retained only when they contain at -most 256 Unicode code points. Longer values are omitted rather than truncated -so consumers never classify on a fabricated partial identifier. A provider -message is limited to 4,096 Unicode code points; a longer normalized message is -truncated at a code-point boundary with a visible ellipsis inside that limit. -The raw response body and pre-normalized strings are never exposed or retained -in the public error. - -## Error Propagation And Compatibility - -- Every built-in-client non-2xx response matches `ErrLLMGenerate` and supports - `errors.As` to `*GenerationError`, including status-only cases. -- The internal model client retains its non-success-status identity for its - own package tests. The use-case layer remains provider-neutral and continues - to add only its generation category. -- The root error boundary converts only the built-in transport's structured - status error. It does not parse arbitrary error text, inspect consumer error - fields, or fabricate HTTP details for an injected `LLMClient`. -- Errors returned by injected clients remain in the chain exactly as today. - If an injected client deliberately returns an existing `*GenerationError`, - its identity may pass through ordinary wrapping, but Promptkit does not - construct or enrich one on that client's behalf. -- Existing cancellation and deadline identities, capacity errors, validation - behavior, repair behavior, and successful response decoding remain - unchanged. -- This is an additive public API. Existing consumers that use - `errors.Is(err, ErrLLMGenerate)` continue to work; consumers should not rely - on the previous rendered wording of non-success errors. - -## Architecture And Ownership - -The provider-envelope parser and bounded body reader belong in `internal/llm`, -which owns the OpenAI-compatible transport. The internal transport error owns -only normalized status facts and continues to match the package's existing -non-success-status sentinel. - -The use-case package does not gain HTTP DTOs, status policy, or a provider- -specific branch. Its existing wrapping carries the internal error to the root -facade. The root error mapper recognizes the internal structured status error -and constructs the public `GenerationError` without exposing an internal type -or raw cause through public fields. No transport error is added to -`internal/domain`. - -The public type and its exact Go semantics are owned by its declaration and -GoDoc. The -[OpenAI-compatible integration contract](../integrations/openai-compatible-chat.md) -owns recognized wire shapes, limits, and observable response behavior. The -[internal model-client document](../internal/llm.md) owns implementation flow, -internal failure categories, and test ownership. Architecture policy does not -need a new package or dependency rule for this feature. - -## Documentation End State - -Canonical documentation at the target state has these responsibilities: - -- the `GenerationError` declaration and GoDoc define the exact public methods, - formatting, unwrapping, zero-value behavior, and trust boundary; -- `Engine.Run` and `Engine.RunPrepared` GoDoc identify the typed error without - duplicating its accessor contract; -- the consumer guide includes one short `errors.As` example and links to the - public declaration; -- the integration document replaces its status-only description with the - bounded envelope contract; and -- the internal model-client document describes parsing, conversion ownership, - and narrow test owners. - -The architecture policy, framework format reference, and built-in backend -catalog do not duplicate this API or wire contract. - -## Verification Expectations - -Verification protects each behavior at its narrowest stable owner: - -- internal model-client tests cover recognized string and numeric codes, - independent optional-field handling, unknown fields, empty and malformed - envelopes, single-document framing, read failures, declared and streamed - size boundaries, body closure, normalization, field limits, and absence of - raw provider content from rendered errors; -- root error-boundary tests cover conversion to the immutable public type, - every accessor, `errors.Is`, `errors.As`, and safe `%v`, `%+v`, and `%#v` - formatting; -- one representative ordinary run and one prepared run prove that the built-in - transport contract crosses the assembled engine boundary, without repeating - the complete parser matrix; -- existing injected-client tests continue to prove preservation of consumer - error identity without fabricated provider details; and -- all tests use controlled transports or local servers and never contact a - live or paid provider. - -Security limits and their exact boundaries are contractual enough to warrant -literal boundary tests. Higher-level tests should remain representative and -must not duplicate the internal transport matrix. - -## Acceptance Criteria - -- A consumer can distinguish an HTTP 400 from other generation failures and - deliberately obtain a bounded provider explanation when one is available. -- The same typed error remains available through ordinary and prepared - execution and still satisfies `errors.Is(err, ErrLLMGenerate)`. -- Default and Go-syntax error formatting cannot disclose any provider-derived - string or raw response content. -- Empty, malformed, unreadable, unrecognized, and oversized bodies preserve a - typed status-only error. -- No read, retained field, or formatted representation can exceed its stated - bound, and the body is closed on every outcome. -- Existing success, cancellation, capacity, validation, repair, and injected- - client contracts remain unchanged. -- Current-state documentation changes only when the implementation exists and - follows the repository's canonical ownership policy. - -## Non-Goals - -This feature does not add: - -- retryability classification, retry loops, backoff, failover, or routing; -- parsing of success bodies as errors or changes to successful-response limits; -- provider-specific envelope variants beyond the conventional top-level - `error` object; -- response headers such as `Retry-After`, raw bodies, request data, endpoints, - credentials, schema documents, generated content, or provider metadata; -- logging, telemetry, redaction policy for downstream applications, HTTP status - mapping for consumer servers, or user-facing presentation; -- translation or enrichment of arbitrary injected-client errors; or -- a new public package, public constructor, mutable error value, or transport - type in the domain model. - -## Open Questions - -None. The public type direction, accessor surface, formatting and error-chain -behavior, envelope scope, normalization, safety limits, fallback behavior, -layer ownership, compatibility boundaries, documentation ownership, and test -boundaries are fixed by this roadmap.