# PromptKit v0.8.0 Upgrade ## Status Proposed. ## Purpose Upgrade Notarius from PromptKit v0.5.0 to v0.8.0 and deliberately adopt the useful correctness, profile-composition, provider-diagnostic, backend, and structured-output-repair capabilities introduced in PromptKit v0.6.0, v0.7.0, and v0.8.0. The upgrade should improve structured-output reliability without confusing PromptKit's bounded deterministic repair with Notarius's existing stage retry budget or the future feedback-aware semantic-validation loop. PromptKit types and provider behavior must remain behind Notarius's transport-neutral LLM boundary. ## Current State Notarius currently pins PromptKit v0.5.0. Its production adapter prepares one frozen execution, records credential-redacted details, and runs that same prepared value. It maps PromptKit capacity failures to an application-owned error, maps failed structured validation to `ErrInvalidStructuredOutput`, and returns PromptKit's raw validated bytes and usage metadata. Every maintained production prompt uses JSON Schema validation and currently declares `repair_attempts: 0`. Notarius stage bindings separately expose `retries`, which reruns a complete stage operation after an error or rejected candidate. The two mechanisms have different ownership and must remain independent. Notarius also maintains: - embedded prompt, schema, and fallback-profile filesystems; - operator profile-file and profile-directory sources; - one optional conventional `local` backend registration; - explicit profile preflight through PromptKit inspection; - one application-wide scheduled LLM client around the PromptKit adapter; - PromptKit profile-source fingerprints for checkpoint safety; and - redacted debug and manifest provenance at application-owned boundaries. The upgrade must preserve those established responsibilities while revising the pinned integration contract and any behavior affected by the three intervening releases. This roadmap is based on PromptKit's pinned release guides for [v0.6.0](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.8.0/docs/releases/v0.6.0.md), [v0.7.0](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.8.0/docs/releases/v0.7.0.md), and [v0.8.0](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.8.0/docs/releases/v0.8.0.md), plus the public API and format documentation at the v0.8.0 tag. ## Target End State - `go.mod` and `go.sum` pin PromptKit v0.8.0 without a local replacement or vendored copy. - Every maintained PromptKit prompt and profile prepares successfully under v0.8.0's stricter validation and source-loading rules. - Eligible Notarius structured completions use one PromptKit corrective call by default after a structurally invalid response. Operators can explicitly set a value from zero through three for a configured pipeline, with a more local LLM-backed binding override where needed. - PromptKit repair remains an inner operation within one Notarius stage attempt. It never consumes or replenishes the binding's `retries` budget. - A successful repaired result exposes cumulative usage and the actual repair count to Notarius's application-owned response and debug models. A repaired success is not itself a warning. - Exhausted PromptKit validation remains an invalid structured-output result, preserving the final candidate and diagnostics for debug and for any applicable outer Notarius stage policy. Invalid structured output is never accepted merely because the repair budget was exhausted. - Profile inheritance, the built-in Rakestrawhome backend/profile, optional credential behavior, and structured generation errors work through the existing Notarius PromptKit boundary and are accurately documented. - Provider-specific PromptKit types do not escape `internal/framework/llm`. - Checkpoint identity, effective configuration, redacted summaries, and debug provenance reflect every execution-affecting repair or profile change. - Current documentation pins and describes v0.8.0; future Notarius semantic validation retries remain roadmap behavior rather than being conflated with this dependency upgrade. ## Release-by-Release Adoption ### PromptKit v0.6.0: Correctness, Safety, And Efficiency PromptKit v0.6.0 adds no public declarations, but intentionally rejects several formerly permissive or ambiguous inputs. The upgrade must audit Notarius's embedded and operator-facing integration against these rules: - YAML `id` and `version` metadata, rather than filenames, define prompt and profile identity. - Prompt `content_file` paths are exact, relative, contained paths; built-in file artifacts must resolve to regular files. - execution controls, output contracts, and repair budgets must be finite and within their documented ranges; - provider endpoints must be absolute HTTP or HTTPS URLs with a host and no user information, query, or fragment; - JSON documents and successful provider responses contain exactly one value; - successful provider responses are bounded to 16 MiB; and - JSON-compatible values are bounded for depth and expansion. Notarius should rely on PromptKit for these rules rather than duplicate its parsers or internal limits. Existing Notarius validation may retain a narrower application rule where it has independent value, but overlapping validation must agree with PromptKit and must not accept a value PromptKit will reject later. The upgrade automatically receives operation-local schema-plan reuse, artifact-text memoization, improved cancellation checks, and transport error identity preservation. Notarius should verify these changes through its real adapter boundary and avoid adding a second cache or response-body layer that would duplicate PromptKit's ownership. ### PromptKit v0.7.0: Profiles, Backend Access, And Generation Errors #### Profile Inheritance Operator profiles may use `base_profile` to alias or selectively refine a built-in, fallback, or higher-precedence operator profile. Notarius must pass profile sources through unchanged and let PromptKit own parent lookup, merge rules, source precedence, cycle detection, and fully resolved prepared targets. Preflight inspection must resolve inherited profiles through the same source and backend composition used at execution. The selected leaf profile ID remains the public profile identity, while effective backend, endpoint, model, and reasoning provenance reflect the resolved chain. Notarius must not implement a second inheritance parser. The existing complete `dnd-extraction` fallback remains a standalone profile: PromptKit v0.8.0 does not provide a built-in `openai/gpt-5.6-luna` profile that would be an appropriate parent. Documentation should nevertheless explain how operators can use inheritance for environment-specific workload profiles and should link to PromptKit's pinned format contract rather than duplicate its field-by-field merge algorithm. Checkpoint safety must cover inherited behavior. Operator file/directory digests already cover changes to definitions in those sources, fallback asset digests cover application parents, and the PromptKit built-in catalog marker must change from its v0.5.0 identity to v0.8.0 so a changed built-in parent cannot reuse an incompatible checkpoint. #### Rakestrawhome Backend And Profile PromptKit's reserved `rakestrawhome` backend and `rakestrawhome-gemma-4-31b` profile become available without Notarius-specific registration. Notarius must not register or shadow the reserved backend ID. Profile preflight, backend-capacity reporting, scheduling, generation, and provenance should work for it through the same generic paths used by OpenRouter and `local`. The D&D default remains `dnd-extraction`; this upgrade does not silently move a production workload to Rakestrawhome. Operator documentation should identify the built-in profile as an available selection and link to PromptKit for its endpoint, credential environment, model, and capacity defaults. #### Optional Credentials An absent or blank optional `APIKeyEnv` now causes PromptKit to omit the `Authorization` header and send the request. Notarius must not restore the old failure behavior by pre-reading provider credential environment variables or by adding provider-specific authentication logic. Profile inspection may report an explicit `APIKeyRequired` policy without reading the credential, and execution remains the boundary at which that requirement is enforced. For optional profiles, an authentication-requiring provider may instead return a structured 401 or 403 generation failure. The configuration and operations documentation must explain this distinction. Notarius does not currently expose PromptKit's in-memory profile-registration API to operators, and PromptKit's filesystem profile format does not expose `APIKeyRequired`; therefore Notarius must not promise that an operator profile can force local credential preflight. Operators should provision the named environment variable, while Notarius should preserve the provider's structured authentication failure when it is absent. Notarius must continue to document mechanisms and environment-variable names, never secret values. #### Structured Generation Errors The adapter should recognize `*promptkit.GenerationError` with `errors.As` and translate useful information into an immutable, provider-neutral Notarius error classification. At minimum, retain the HTTP status code so callers and future retry policy can distinguish transport success with provider rejection from other generation failures. PromptKit's provider code, type, and message accessors are bounded but remain untrusted and potentially sensitive. They must never appear automatically in ordinary CLI output, warnings, manifests, checkpoint identity, or cache data. If retained for an explicitly requested debug trace, they must pass through Notarius's known-secret and bearer redaction and remain clearly identified as untrusted provider diagnostics. Default error formatting should continue to use a bounded, redacted application-owned message. Capacity and cancellation retain their current more specific classifications and precedence. This upgrade does not add automatic provider-error retry classification; it only preserves safe structured data needed for diagnosis and later policy. ### PromptKit v0.8.0: Bounded Structured-Output Repair #### Default Policy Every maintained production prompt whose output is consumed as structured data should declare one repair attempt. All current production prompts use eligible JSON Schema validation, so no current prompt needs a zero default merely because of its output mode. One repair means at most one corrective generation after the initial candidate. PromptKit reconstructs the immutable original conversation and appends only the latest invalid assistant candidate and latest deterministic validation diagnostics. It preserves the selected target, direct session ID, provider-native structured-output contract, and backend capacity policy. This shape preserves the original cacheable prompt prefix and avoids accumulating unbounded failed history. The default is deliberately small. A single repair captures the common case in which a capable model can correct malformed JSON or a schema violation after receiving an exact diagnostic, while bounding the extra latency and cost of a single structured completion. #### Configuration Contract The public configuration is an optional, presence-aware `structured_output_repair_attempts` integer at pipeline scope and at each LLM-backed module or validator binding. Its effective precedence is: 1. the binding value, when present; 2. the pipeline value, when present; and 3. the selected prompt's declared `repair_attempts` value. The value must be from zero through three. Explicit zero disables PromptKit repair at that scope. A deterministic binding must reject the field because it cannot perform structured LLM repair. Validator bindings may use it only when the selected validator is LLM-backed. Shorthand module bindings continue to inherit the pipeline or prompt default. The long, provider-neutral name is intentional: it distinguishes PromptKit's inner structural repair from the existing binding `retries` field, which owns complete stage attempts, without exposing a dependency name in generic pipeline contracts. The effective value must survive file parsing, cloning, redacted summaries, pipeline resolution, and pipeline digest construction without pointer aliasing or loss of presence. It must affect checkpoint identity because it can change the selected result, latency, token usage, and provider cost. #### Adapter Contract The transport-neutral structured-completion request should carry an optional application-owned structural-repair budget. No `promptkit.OutputContract` or other PromptKit type may cross the adapter boundary. PromptKit v0.8.0 request validation replaces the complete prompt output contract rather than merging one field. When Notarius has a configured override, the adapter must therefore inspect the selected prompt, copy its normalized declared format, validation mode, and schema path, change only the repair count, and supply that complete contract on the prepared request. A nil override continues to use the prompt declaration directly. Inspection and preparation must use the same immutable engine sources; a small adapter-local cache keyed by normalized prompt ID and version is acceptable but not required without measured need. This approach prevents configuration from accidentally dropping JSON Schema validation, avoids duplicating schema paths in pipeline YAML, and keeps prompt assets authoritative for every output-contract field other than the explicit operator override. The transport-neutral structured-completion response should report the actual number of PromptKit repair calls. PromptKit's returned token usage is already cumulative and must be passed through without re-summing it. Debug records should distinguish the configured budget from the actual count. Ordinary run manifests need not gain raw prompt or response data merely to report repairs; any durable aggregate should be added only if it has a clear consumer contract. #### Result And Failure Semantics - A valid initial candidate returns normally with zero actual repairs. - A valid corrected candidate returns normally with cumulative usage and its positive actual repair count. It does not emit a warning solely because a repair occurred. - Exhausting the repair budget returns PromptKit's final candidate and failed validation result. The adapter maps this to `ErrInvalidStructuredOutput`, preserves the response and debug material, and does not decode or accept the candidate. - An explicitly empty or whitespace-only candidate participates in the declared structural validation and repair flow. Missing, `null`, or non-string provider content remains a malformed provider response. - A generation failure during a corrective call is an operational generation failure and uses the same safe structured-error adaptation as an initial generation failure. - Context cancellation remains authoritative throughout the initial and corrective calls. PromptKit repair happens inside one scheduled `CompleteStructured` operation. The Notarius scheduler holds one permit for that logical operation while PromptKit performs its initial and serial corrective calls; PromptKit reacquires its own selected-backend capacity for each corrective generation. Because corrective calls are serial, this cannot expand actual concurrent provider work beyond the number of admitted Notarius operations, but documentation must stop describing the Notarius permit as a separate admission event for every internal repair call. One `CompleteStructured` invocation with effective PromptKit repair budget `R` may make at most `R + 1` provider calls. If one stage attempt makes `C` structured-completion invocations, a binding with `retries: N` has an upper bound of `(N + 1) * C * (R + 1)` provider calls; `C` may itself be a bounded, data-dependent module property, as it is for batched semantic reconciliation. LLM-backed validators have their own corresponding invocation counts, budgets, and costs. These formulas are upper bounds, not promises that every failure is retryable or that every attempt reaches the provider. ## Profile And Prompt Source Compatibility The upgrade must preserve Notarius's source precedence: an operator source, then registered application fallback profiles, then PromptKit built-ins. A selected malformed definition remains authoritative and fails rather than falling through. Parent resolution introduced by profile inheritance observes that same precedence. All embedded prompt manifests, shared content fragments, response schemas, and fallback profiles must be prepared or inspected offline under v0.8.0. The review should specifically catch: - IDs inferred accidentally from filenames; - stale or escaping `content_file` paths; - missing or non-regular embedded artifacts; - repair values outside zero through three or paired with ineligible validation; - schemas or examples that are not exact single JSON documents; - unsupported endpoint forms; and - JSON-compatible variables or profile extras that exceed upstream bounds. No prompt prose, schema shape, durable D&D artifact contract, or default D&D model should change merely to exercise the dependency. Prompt manifests should change only as needed to enable the adopted repair default and satisfy v0.8.0 contracts. ## Provenance, Debugging, And Security - Update the opaque PromptKit built-in profile-catalog identity from v0.5.0 to v0.8.0. Do not hash or publish PromptKit's internal catalog bytes. - Ensure a prompt's repair default remains covered by its existing prompt asset fingerprint and a configured effective override remains covered by the resolved pipeline digest. - Preserve selected leaf profile identity while recording the inherited effective target already exposed by PromptKit inspection and prepared details. - Add actual structural-repair count and, when useful, the configured budget to application-owned debug material. Token totals remain PromptKit's cumulative values. - Do not generate a warning for a successful repair. Repair exhaustion is an invalid-output failure, while provider rejection is a generation failure. - Never expose raw provider diagnostic fields without explicit debug capture and application redaction. Do not place them in normal errors or durable summaries. - Preserve context and transport error identity sufficiently for `errors.Is`-based cancellation and deadline handling after adapting the external error. ## Documentation And Examples Implementation should update current-state documentation only when the new behavior lands: - `docs/integrations/pkg-promptkit.md` must pin v0.8.0 and define the revised prepared-execution, repair, profile-inheritance, backend, credential, and error-adaptation boundary. - `docs/config.md` must own the repair configuration fields, precedence, allowed range, explicit-zero behavior, profile inheritance availability, and optional credential semantics. - `docs/operations.md` must explain structural repair cost, timeout and concurrency effects, credential failures, and its distinction from stage retries. - `docs/internal/llm.md` must describe adapter contract replacement, actual repair metadata, error adaptation, source compatibility, and scheduling. - `docs/internal/pipeline.md` must describe how effective repair configuration is resolved and how inner repair differs from outer stage attempts. - `docs/policy/architecture.md` should receive only the durable ownership rule: PromptKit owns bounded deterministic structural repair within one completion, while Notarius owns stage attempts and semantic validation policy. Detailed fields and retry formulas belong in their canonical configuration and operations documents. Update maintained configuration examples only if the public Notarius configuration contract changes. A short inheritance illustration may remain in the configuration reference; do not create a complete example solely to copy PromptKit's upstream profile catalog. All upstream links must point to the v0.8.0 tag. Historical release or archived roadmap references should remain historical. No ADR is required solely to pin a newer dependency. The durable separation between PromptKit structural repair and Notarius semantic stage retries should be stated in architecture documentation now; the more extensive future validation state machine still warrants the separate ADR already identified in `future.md` when that work is promoted. ## Validation And Acceptance Criteria The implementation is complete when: - the repository builds and tests against PromptKit v0.8.0 with no replacement directive, workspace dependency, or vendored source; - every maintained prompt and profile prepares or inspects successfully under the v0.8.0 source, path, endpoint, output-contract, and JSON-value rules; - an invalid first JSON Schema candidate followed by a valid correction returns the valid raw output, cumulative usage, and actual repair count through the Notarius adapter; - repair exhaustion returns the final raw candidate and debug material with an error matching `ErrInvalidStructuredOutput`; - a corrective generation failure retains safe generation classification and provider status without leaking untrusted provider detail; - explicit empty content follows structural validation rather than being misclassified by Notarius; - repair configuration is presence-aware, range checked, rejected on deterministic bindings, resolved with documented precedence, and included in effective pipeline identity; - inherited profiles resolve consistently during preflight and execution, and changes to any relevant operator, fallback, or built-in parent invalidate checkpoint reuse; - the Rakestrawhome built-in profile reaches generic preflight, scheduling, and provenance paths without application-specific registration; - optional missing credentials and explicitly required credentials behave as documented without contacting real providers in tests; - cancellation, timeout, backend capacity, prepared-execution snapshot, session ID, raw-output, debug-redaction, and existing profile provenance behavior remain intact; - maintained examples validate successfully; and - canonical documentation contains no active v0.5.0 pin or claim that PromptKit is always single-pass. Tests should follow `docs/policy/testing.md`: exercise observable Notarius contracts with offline fake clients or `httptest` boundaries, and do not copy PromptKit's entire internal repair test suite or assert its exact correction message prose. The dependency's internal wording is not a Notarius contract. ## Non-Goals - Implementing Notarius's future feedback-aware semantic stage-retry loop. - Adding the D&D combat-scene semantic validator. - Redesigning warning policy or treating successful structural repair as a warning. - Adding provider transport retries or deciding which HTTP statuses should consume a stage retry. - Exposing PromptKit request, response, profile, validation, capacity, or error types outside the LLM adapter. - Changing durable artifact schemas, D&D prompt semantics, the D&D default model, or the fixed pipeline shape. - Reimplementing PromptKit profile inheritance, schema validation, response bounds, repair conversations, backend admission, or provider parsing inside Notarius. ## Decisions ### 1. Default Structured-Output Repair Budget **Decision: default to one repair attempt.** Set every maintained eligible production prompt to `repair_attempts: 1`. One corrective call is a strong fit for Notarius because every current production LLM response has a strict JSON Schema contract, smaller cost-effective models are a deliberate deployment target, and a precise structural diagnostic often makes one retry materially more successful. The budget is paid only after a structurally invalid candidate and remains tightly bounded. **Alternative considered: retain zero by default.** This preserves single-pass cost and latency and requires operators to opt in. It is preferable for an environment where every additional request is expensive or where upstream provider-native schema enforcement already produces negligible invalid output. It is less suitable as the Notarius default because one malformed response can otherwise discard substantial completed pipeline work. **Alternative considered: default to two.** This may improve recovery for weak models, but it doubles the worst-case corrective cost relative to the selected default and compounds with outer stage retries. It should be an operator choice supported by configuration, not the initial default, unless observational evidence shows that the second correction has a worthwhile marginal success rate. ### 2. Repair Override Scope **Decision: support both pipeline and LLM-backed binding overrides.** Use the presence-aware `structured_output_repair_attempts` field and precedence defined above. A pipeline value provides the convenient one-line control the operator requested, while a binding value permits an expensive normalizer or future LLM-backed validator to use a deliberately different budget. This mirrors Notarius's established pipeline/binding profile inheritance and scales without editing embedded prompts. **Alternative considered: support only a pipeline override.** This is smaller to implement and document and still permits global enablement or disablement for one pipeline. Its drawback is that one exceptional prompt cannot opt out or request a larger budget without changing an embedded asset for every pipeline. **Alternative considered: expose one global value under the top-level `promptkit` configuration.** This makes client construction simple, but applies the same budget to unrelated pipelines and leaks an execution policy into the dependency configuration block. It is less compositional than pipeline-owned policy and therefore not recommended. ### 3. Retention Of Provider-Supplied Generation Details **Decision: retain status in the application-owned error contract and retain redacted provider code, type, and message only in explicitly requested debug traces.** Status is useful for diagnosis and future retry policy without usually containing sensitive data. The other fields can materially explain a 400 response but may echo request or schema content, so they belong only in the already-sensitive debug surface after Notarius redaction. **Alternative considered: retain only HTTP status and discard all provider fields.** This is the safest and smallest policy and still improves typed failure handling. It sacrifices potentially decisive provider diagnostics, leaving an operator with less information when a provider returns a terse status and the problem cannot be reproduced easily. **Alternative considered: include bounded provider code and type in normal errors while keeping message debug-only.** Codes and types are often stable and less sensitive than messages, but PromptKit explicitly classifies every provider field as untrusted. Promoting them to ordinary output creates a disclosure and compatibility burden that is not currently justified.