From 55247c47ab2e6f959e2e661313489751a28fcff0 Mon Sep 17 00:00:00 2001 From: Eric Rakestraw Date: Tue, 25 Aug 2026 19:11:30 +0000 Subject: [PATCH] Plan the PromptKit 0.8 upgrade --- docs/roadmap/future.md | 36 +- docs/roadmap/implementation.md | 1132 +++++++++++++++++++++----------- docs/roadmap/promptkit-v0.8.md | 520 +++++++++++++++ 3 files changed, 1269 insertions(+), 419 deletions(-) create mode 100644 docs/roadmap/promptkit-v0.8.md diff --git a/docs/roadmap/future.md b/docs/roadmap/future.md index ad413ec6..21844f02 100644 --- a/docs/roadmap/future.md +++ b/docs/roadmap/future.md @@ -15,34 +15,11 @@ is warranted. ### 1. Upgrade To PromptKit v0.8.0 -- Upgrade the PromptKit dependency and follow the upstream v0.8.0 release - guide, including its now-active `repair_attempts` behavior and compatibility - checks for prompt definitions, prepared execution, validation results, - capacity accounting, error adaptation, debug data, and cumulative token - usage. -- Use PromptKit's bounded repair loop for eligible structural output failures. - A repair must resend the immutable original prompt followed by the latest - defective assistant response and one bounded user correction message. It - must not accumulate the history of every defective candidate. -- Review every maintained LLM prompt's structural repair budget. Prefer a - small positive budget where a corrected structured response can be useful, - while retaining zero where a second call would be inappropriate or where - observational evidence does not justify it. -- Keep PromptKit repair attempts distinct from Notarius stage retries. - PromptKit repairs a response that fails its deterministic `basic`, `json`, or - `json_schema` contract within one stage attempt; Notarius may later retry a - complete stage candidate after application validation. Document and test the - multiplicative maximum provider-call cost created by both configured - budgets, including PromptKit repair calls made by any LLM-backed validators. - Attribute generation count, token usage, latency, and failure provenance to - the producer, validator, PromptKit repair, and Notarius correction layers so - operators can explain the actual cost of a run. -- Treat exhaustion of PromptKit structural repair as the absence of a usable - structured candidate. The default terminal policy is to fail the run after - the applicable Notarius stage-attempt budget is also exhausted. A future - configurable alternative may reject the affected stage or lane where the - pipeline can represent that outcome, but it must never accept undecodable or - structurally invalid output merely with a warning. +This item has been promoted to the standalone +[PromptKit v0.8.0 Upgrade](promptkit-v0.8.md) roadmap. That document owns the +release-by-release compatibility review, adopted features, structured-repair +policy, target integration boundary, acceptance criteria, and settled design +decisions. ### 2. Feedback-Aware Stage Validation Retries @@ -245,8 +222,7 @@ The implemented source-backed core and initial D&D registry adoption are described by [Module Internals](../internal/modules.md#semantic-reconciliation) and [D&D Module Internals](../internal/dnd.md#semantic-registry-reconciliation). -The [Semantic Reconciliation Roadmap](semantic-reconciliation.md) retains the -original feature scope; the sections below keep broader extensions deferred. +The sections below keep broader extensions deferred. ### Large-Collection Semantic Reconciliation diff --git a/docs/roadmap/implementation.md b/docs/roadmap/implementation.md index 68e834a0..a8c4969d 100644 --- a/docs/roadmap/implementation.md +++ b/docs/roadmap/implementation.md @@ -1,428 +1,782 @@ -# Source-Only Release Implementation Plan +# PromptKit v0.8.0 Upgrade Implementation Plan ## Purpose -Implement the target state defined by -[Source-Only Releases](source-releases.md): immutable source tags with checked-in -release notes, a diagnostic version interface, strong shared candidate checks, -validation-only tag CI, Linux support, best-effort macOS compilation, and no -packaged binaries or Windows support. +Implement the target state defined by the +[PromptKit v0.8.0 Upgrade](promptkit-v0.8.md): adopt the useful PromptKit +v0.6.0, v0.7.0, and v0.8.0 changes; enable one bounded structural correction +by default; expose pipeline and binding overrides; preserve safe provider +diagnostics; and keep PromptKit behind Notarius's transport-neutral LLM +boundary. This plan is ordered. Each numbered stage is one implementation prompt for a gpt-5.6-terra coding agent. Complete and validate one stage before beginning -the next. Preserve all unrelated worktree changes, follow every policy under -`docs/policy/`, and update current-behavior documentation in the same stage as -the behavior it describes. +the next. Read `docs/development.md` and every policy under `docs/policy/` at +the start of each stage, inspect the current code and tests named by that +stage, preserve unrelated worktree changes, and update current-behavior +documentation in the same stage as the behavior it describes. -Do not create or push a release tag while implementing this plan. Do not invent -a release note for `v0.1.0`, `v0.2.0`, or `v0.3.0`. The first real release -under the completed procedure will add its own note in a separate release -operation. +Do not retire this plan or `promptkit-v0.8.md` during implementation. Keep both +until the completed work has passed a separate review. Do not implement the +future Notarius semantic-validation retry loop, D&D combat-scene validator, or +warning redesign as part of this plan. ## Decisions Fixed For Implementation -- Releases are stable `vMAJOR.MINOR.PATCH` source tags on `main`; prereleases - are unsupported initially. -- Tags are lightweight and immutable after publication. -- No release binaries, archives, checksums, signatures, containers, - package-manager entries, or Gitea release objects are produced. -- Linux is supported. Release checks compile Linux `amd64` and `arm64` with - `CGO_ENABLED=0`. -- macOS is best-effort. Release checks compile Darwin `amd64` and `arm64` with - `CGO_ENABLED=0`, without promising runtime CI or packaged output. -- Windows is unsupported and must not be added to build checks. -- Release notes begin with the first release made under the new procedure; - historical tags are left untouched. -- `notarius --version` is informational. Receipt and artifact contracts remain - authoritative for downstream compatibility. -- One checked-in POSIX shell command owns substantive source-candidate checks. - The release procedure and tag CI call it rather than maintaining duplicate - test/build matrices. -- Tag CI validates only. Pre-publication local guards remain mandatory because - tag CI cannot prevent an already-pushed tag. +- Pin `gitea.maximumdirect.net/eric/promptkit` v0.8.0 directly, with no + `replace`, workspace dependency, or vendored source. +- Every maintained eligible production prompt defaults to exactly one + PromptKit structural repair attempt. +- Add the exact configuration key + `structured_output_repair_attempts` at pipeline scope and on LLM-backed + module and validator bindings. +- Effective precedence is binding value, then pipeline value, then the prompt's + declared `repair_attempts` value. Omission inherits; explicit zero disables + structural repair at that scope. +- Accepted values are integers from zero through three. Explicit null and + non-integer values are invalid. An explicit binding value on a deterministic + module or validator is invalid. A pipeline value is applied only to selected + LLM-backed bindings and does not make deterministic bindings invalid. +- Keep file configuration version 4. This is an additive pre-release field and + does not require parallel versioned behavior. +- Use `StructuredOutputRepairAttempts *int` for presence-aware internal Go + fields. Clone pointers at every ownership boundary. +- A configured override never replaces schema identity, output format, or + validation mode. The PromptKit adapter calls `InspectPrompt`, copies the + complete normalized prompt-owned output contract, changes only + `RepairAttempts`, and supplies the complete replacement on `RunRequest`. + Do not add an inspection cache initially. +- PromptKit repair is internal to one `CompleteStructured` call and does not + consume or replenish a binding's existing `retries` budget. +- Add `RepairAttempts int` to Notarius's structured-completion response. It is + the actual corrective-call count reported by PromptKit; token usage remains + PromptKit's cumulative usage and must not be summed again. +- A valid repaired response is successful and produces no warning solely + because repair occurred. Exhausted structural validation maps to + `ErrInvalidStructuredOutput` with the final candidate and debug material + retained. +- Add an application-owned generation-error sentinel and typed status-bearing + error. PromptKit error types must not cross `internal/framework/llm`. +- HTTP status may appear in the application-owned generation error. Provider + code, type, and message are excluded from ordinary errors, warnings, + manifests, cache, and checkpoint identity; they may appear only in an + explicitly requested debug trace after Notarius redaction. +- Profile inheritance is owned entirely by PromptKit. Notarius passes sources + through, inspects and records the resolved target, and does not parse or merge + `base_profile` itself. +- PromptKit's built-in `rakestrawhome` backend and + `rakestrawhome-gemma-4-31b` profile are available generically. Notarius does + not register, shadow, or select them by default. +- Missing optional credential environment values are allowed to reach the + provider without `Authorization`; Notarius does not recreate v0.5.0's local + failure or add provider-specific authentication logic. +- No dependency-upgrade ADR is required. Update architecture only with the + durable ownership distinction between PromptKit structural repair and + Notarius stage/semantic validation policy. -## Stage 1: Add Build Version Resolution And `--version` ✅ +## Stage 1: Upgrade The Dependency And Establish A Clean v0.8.0 Baseline ### Goal -Add a small, testable build-information boundary and expose the public root -version flag without affecting existing command behavior. +Move the repository to PromptKit v0.8.0, resolve source-compatibility issues, +and establish a passing baseline before adopting new behavior. ### Implementation -1. Create `internal/buildinfo` with an exported link-time string variable named - `Override` and an exported resolver such as `Version() (string, error)`. - Keep this package independent of CLI and application packages. -2. Resolve the displayed value using this precedence: - 1. a nonempty `Override`; - 2. the main-module version returned by `runtime/debug.ReadBuildInfo`; then - 3. the literal `development`. -3. Accept a release value only when it matches the complete stable SemVer tag - form `^v(0|[1-9][0-9]*)\.(0|[1-9][0-9]*)\.(0|[1-9][0-9]*)$`. - Whitespace, prerelease/build suffixes, pseudo-versions, and arbitrary text - are not release versions. A nonempty invalid linker override is an error; - an empty, `(devel)`, pseudo-version, or otherwise non-release main-module - version falls back to `development`. -4. Add `--version` to the root dispatch in `internal/cli`. It is valid only as - the sole argument, writes exactly `notarius \n` to stdout, - writes nothing to stderr, and exits zero. Additional arguments are a syntax - error using the existing exit-2 and stderr conventions. An invalid linker - override is a runtime/build error using exit 1 and stderr. -5. Add the flag to root usage without changing the existing behavior of empty - arguments, help spellings, or subcommands. -6. Update `docs/cli.md` as the canonical public contract and - `docs/internal/cli.md` as the implementation owner. State that tagged - `go install` builds can obtain the main-module tag from Go build information, - controlled builds may inject - `gitea.maximumdirect.net/eric/notarius/internal/buildinfo.Override`, and an - ordinary unversioned checkout reports `development`. - -### Tests - -- Add table-driven `internal/buildinfo` tests for valid stable versions, - leading-zero rejection, whitespace, prerelease/build suffixes, pseudo- - versions, override precedence, invalid nonempty override, and development - fallback. Test the pure resolution decision rather than trying to mutate - process build information. -- Extend the CLI command-contract tests to cover exact stdout/stderr/exit - behavior for `--version`, extra arguments, and unchanged help/unknown-command - behavior. -- Do not snapshot the whole usage document solely for the new line; assert the - stable semantic fragments already owned by the CLI contract tests. - -### Validation - -```sh -go test ./internal/buildinfo ./internal/cli -go test ./... -go vet ./... -go build ./cmd/notarius -go run ./cmd/notarius --version -``` - -Build a temporary Linux host binary with: - -```sh -go build -trimpath \ - -ldflags '-X gitea.maximumdirect.net/eric/notarius/internal/buildinfo.Override=v0.0.0' \ - -o /path/to/temp/notarius ./cmd/notarius -``` - -and verify that its output is exactly `notarius v0.0.0`. - -### Acceptance Criteria - -- The public and internal documentation matches the implemented version - behavior. -- Ordinary builds print `notarius development`. -- A valid linker override prints the exact stable tag. -- Invalid linker content cannot masquerade as a release version. -- Existing CLI commands, help, streams, and exit classes remain unchanged. - -## Stage 2: Add One Reusable Source-Candidate Checker ✅ - -### Goal - -Create one repository-owned, offline validation command used identically by a -maintainer and release CI. - -### Implementation - -1. Create executable POSIX shell script `scripts/check-release-source.sh`. - Require exactly one positional argument containing a stable SemVer tag. The - script must locate and enter the repository root from its own checked-in - path so callers cannot accidentally validate another working directory. -2. Use `set -eu`, quote all expansions, reject invalid versions before using - them in paths or linker arguments, and use a freshly created temporary - directory for cross-build output. Install a cleanup trap scoped only to that - resolved temporary directory. -3. Keep release-note, branch, remote, clean-worktree, and tag-existence guards - out of this script. Those publication-specific checks belong in - `docs/release.md` and tag CI. This script owns the substantive source checks - that both workflows share. -4. Implement these checks in a clear fail-fast order: - - require the expected Notarius module path in `go.mod`; - - reject tracked `go.work` or `go.work.sum`, an existing `vendor` directory, - and any `replace` directive in `go.mod`; - - `GOWORK=off go test -count=1 ./...`; - - `GOWORK=off go test -race -count=1 ./...`; - - `GOWORK=off go vet ./...`; - - `GOWORK=off go build ./...`; - - `GOWORK=off go mod tidy -diff`; - - require no output from `gofmt -l` for tracked Go files; - - `git diff --check` and `git diff --cached --check`; - - validate `examples/dnd-minimal.config.yml` and - `examples/dnd-complete.config.yml` for pipeline `dnd-session` using the - built or `go run` Notarius command; - - build `./cmd/notarius` with `CGO_ENABLED=0` for Linux `amd64` and `arm64` - and Darwin `amd64` and `arm64`; and - - inject the supplied tag through `internal/buildinfo.Override` in every - cross-build. -5. Execute the built command and verify exact `--version` output when the - current host GOOS/GOARCH matches one of the four targets. Do not attempt to - execute a foreign target. -6. Do not compile for Windows, write output beneath the repository, contact an - LLM provider, require credentials, or mutate tracked files. +1. Re-read the upstream v0.6.0, v0.7.0, and v0.8.0 release guides and the + v0.8.0 package consumer and format documentation. Treat the pinned v0.8.0 + tag, not the sibling checkout's moving branch, as authoritative. +2. Update `go.mod` and `go.sum` to PromptKit v0.8.0 and run `go mod tidy` with + `GOWORK=off`. +3. Compile before making compatibility edits. Correct only actual source or + behavior incompatibilities. In particular: + - convert any positional `promptkit.Profile` or + `promptkit.OpenAICompatibleProfileConfig` literals to keyed literals; + - confirm Notarius does not register the newly reserved `rakestrawhome` + backend ID; and + - preserve `PrepareExecution`/`Details`/`RunPrepared` snapshot ownership, + `Discard`, session forwarding, reasoning override, profile preflight, + and capacity adaptation. +4. Change `promptKitBuiltinProfileCatalogID` in + `internal/framework/llm/promptkit_profile_fingerprint.go` from the v0.5.0 + catalog marker to an opaque v0.8.0 marker. Do not hash PromptKit internal + files or include catalog content in manifests. +5. Update `docs/integrations/pkg-promptkit.md` to pin and link v0.8.0 and to + state that this stage still leaves the production prompt-declared repair + budget at its current value. Do not document later configuration or default + behavior before it exists. +6. Update only those existing tests whose public PromptKit types or stable + v0.8.0 behavior genuinely changed. Do not rewrite tests merely to match + upstream diagnostic wording. ### Tests And Validation -- Run the script with `v0.0.0` as a synthetic build version. It does not require - or create a corresponding release note or Git tag. -- Exercise its cheap argument guards separately with missing, extra, malformed, - prerelease, and leading-zero versions. These failures must occur before Go - tests or builds begin. -- Confirm temporary outputs are removed on success and ordinary command - failure. Do not add a large shell-test framework solely for this script; - retain focused automated tests only if they protect a realistic failure that - is not more clearly covered by executing the checker itself. - ```sh -./scripts/check-release-source.sh v0.0.0 -git status --short -``` - -### Acceptance Criteria - -- One command runs every substantive source-candidate check required by the - feature roadmap. -- The command is deterministic, offline, credential-free, POSIX-compatible, - fail-fast, and safe with temporary paths. -- The Linux and Darwin target matrix succeeds and Windows is absent. -- Version injection and host-binary reporting are checked as part of the same - matrix. -- Successful execution leaves the worktree and index unchanged. - -## Stage 3: Add Validation-Only Tag CI ✅ - -### Goal - -Independently validate every newly pushed release tag without publishing or -mutating release state. - -### Implementation - -1. Add `.woodpecker/release.yml` triggered only by tag events. -2. Use the Go 1.25.5 container image to match the version currently declared - by `go.mod`. When the declared Go version changes in a future release, the - release pipeline image and release documentation must be reviewed in the - same change. -3. In one validation step: - - read the candidate version only from `CI_COMMIT_TAG`; - - require the stable SemVer form fixed above; - - require a nonempty `docs/releases/$CI_COMMIT_TAG.md`; - - require the exact heading `# Notarius $CI_COMMIT_TAG`; - - require exact `## Summary`, `## Compatibility`, `## Upgrade`, and - `## Changes` headings; and - - invoke `./scripts/check-release-source.sh "$CI_COMMIT_TAG"`. -4. Do not include a release plugin, API token, artifact upload, Gitea release - creation, archive/checksum step, Windows target, tag mutation, or retry that - could overwrite published state. -5. Keep the CI file thin: note/tag guards belong in it, while the substantive - source checks remain in the shared script. - -### Validation - -- Review the YAML trigger and commands against the repository's Woodpecker - syntax and the established Weatherreporter tag pipeline structure. -- Confirm every invoked path exists and the script is executable. -- Run the shared checker locally with `v0.0.0`. -- Search the new pipeline for release-plugin configuration, upload commands, - secrets, Windows targets, and mutation commands; none may be present. -- Run `git diff --check`. - -Do not push a synthetic tag merely to test this stage. The first real release -will exercise the remote trigger; local source validation and review provide -the pre-release confidence boundary. - -### Acceptance Criteria - -- Every stable release tag triggers the validation pipeline. -- Missing or malformed version-matched release notes fail before the expensive - source checks. -- CI calls the same substantive checker used locally. -- The pipeline cannot publish binaries, releases, checksums, or other assets - and requires no release secret. - -## Stage 4: Establish The Canonical Release Procedure And Documentation Policy ✅ - -### Goal - -Make the complete source-release workflow executable by a maintainer without -undocumented knowledge, and give release documentation an explicit canonical -home. - -### Implementation - -1. Create `docs/release.md`, adapted to Notarius's source-only model. It must - define: - - stable SemVer selection and the pre-`v1` compatibility policy; - - the exact `docs/releases/.md` template and validation guards; - - invocation of `./scripts/check-release-source.sh "$RELEASE_VERSION"`; - - manual review of changed Markdown links and unintended repository files; - - committing and pushing the release note and current documentation before - tagging; - - recording `RELEASE_COMMIT` from `HEAD^{commit}`; - - a copyable guard that requires `main`, a clean worktree/index, disabled Go - workspace use, an exact match between `RELEASE_COMMIT` and - `origin/main`, a matching note, and an unused local and remote tag; - - explicit lightweight-tag creation against `RELEASE_COMMIT`; - - pushing only - `refs/tags/$RELEASE_VERSION:refs/tags/$RELEASE_VERSION`; - - remote tag and tagged-note verification; - - a fresh `go install ...@"$RELEASE_VERSION"` or fresh exact-tag checkout - verification, including exact `--version` output; and - - immutable-tag failure and correction policy. -2. The procedure must say that the tag and checked-in note are the release and - that tag CI is validation-only. It must explicitly exclude binaries, - archives, checksums, signatures, containers, package-manager publication, - Gitea release objects, Windows, and retrospective notes for existing tags. -3. Document private-module installation through standard `GOPRIVATE` and Git - authentication mechanisms without including credentials or private - environment dumps. Do not make one maintainer's credential setup part of - the release contract. -4. Update `docs/policy/documentation.md`: - - add canonical-owner rows for `docs/release.md` and `docs/releases/`; - - state that release notes are historical summaries, not current-state - contract owners; - - state that the checked-in note at the immutable tag is the release record; - and - - require current canonical docs to change with behavior rather than using - release notes as substitutes. -5. Update `docs/development.md` with a release-preparation/tagging/verification - routing row pointing to `docs/release.md` and the relevant policies. -6. Do not create an empty placeholder release note or a retrospective note. - `docs/releases/` first becomes tracked when the next actual release note is - prepared. - -### Validation - -- Follow every local Markdown link added or changed in this stage. -- Execute every non-destructive candidate-validation command that does not - require an actual new release note, remote tag, or publication. -- Compare the procedure line by line with the shared checker and CI so their - tag syntax, note headings, target matrix, and validation ownership agree. -- Confirm the procedure never uses `git push --tags`, moves a published tag, - uploads an asset, or embeds credentials. -- Run `git diff --check`. - -### Acceptance Criteria - -- `docs/release.md` is sufficient to prepare, guard, tag, publish, verify, and - recover from a source release. -- Release procedure and release-note ownership are explicit and nonduplicative. -- The procedure calls the shared checker rather than restating its full command - matrix. -- No historical or placeholder release note is introduced. - -## Stage 5: Align Platform, Installation, And Operational Documentation ✅ - -### Goal - -Make the supported-platform and source-installation story discoverable in the -canonical current-state documents without duplicating the release procedure. - -### Implementation - -1. Update `README.md` with a concise source-installation section. Show - `go install gitea.maximumdirect.net/eric/notarius/cmd/notarius@` as a - version-pinned pattern and retain the existing minimal product quickstart. - Link release maintainers to `docs/release.md` rather than reproducing its - guards. -2. Update `docs/operations.md` with operator-facing source deployment facts: - Linux support, the Go version declared by `go.mod`, version pinning, exact - tag builds, and `notarius --version` as a diagnostic. Link command semantics - to `docs/cli.md` and maintainer publication mechanics to `docs/release.md`. -3. Update `docs/policy/architecture.md` with the durable platform and - distribution invariants: supported Linux deployment, best-effort macOS - development, unsupported Windows, and source-only distribution. Keep tag - commands and release mechanics out of architecture. -4. Review `docs/internal/overview.md` navigation after adding - `internal/buildinfo`. Add only the smallest component entry needed if the - current inventory would otherwise omit a meaningful implemented boundary; - do not inflate build information into a subsystem. -5. Confirm `docs/roadmap/future.md` no longer lists the active documented - release-process work. Retain packaged alpha artifacts as deferred work; the - source-only release feature does not permanently reject reconsideration. -6. Review the completed feature roadmap against the implementation and correct - only genuine target-state inconsistencies. Do not convert it into a - changelog or duplicate `docs/release.md`. - -### Validation - -- Verify all added and changed local Markdown links. -- Confirm commands agree with the implemented CLI and declared module path. -- Confirm no current-state document claims that Notarius publishes binary - assets or supports Windows. -- Run: - -```sh -go run ./cmd/notarius --version -go run ./cmd/notarius help +GOWORK=off go mod tidy -diff +GOWORK=off go test ./internal/framework/llm ./internal/cli +GOWORK=off go test ./... +GOWORK=off go vet ./... +GOWORK=off go build ./cmd/notarius git diff --check ``` ### Acceptance Criteria -- Users can discover how to install a pinned source release. -- Operators can identify the deployed version and understand the support - boundary. -- Architecture records durable platform/distribution policy without owning - maintainer release commands. -- Packaged artifacts remain clearly deferred rather than accidentally promised - or permanently prohibited. +- `go list -m gitea.maximumdirect.net/eric/promptkit` reports v0.8.0. +- There is no PromptKit `replace`, active Go workspace dependency, or vendor + tree. +- The adapter still uses one frozen prepared execution and all existing LLM + tests pass. +- Checkpoint profile identity includes the v0.8.0 built-in catalog marker. +- Current integration documentation pins v0.8.0 without claiming that + later stages are already active. +- The full ordinary test suite, vet, and command build pass. -## Stage 6: Final Release-System Verification ✅ +## Stage 2: Verify v0.6.0 Compatibility And Hardening ### Goal -Review the implemented feature as one system and prove that code, -documentation, shared validation, and CI converge on the same release model. +Audit Notarius's assets and boundary values against PromptKit v0.6.0's stricter +source, path, endpoint, JSON, and cancellation contracts, fixing only concrete +incompatibilities. -### Verification Work +### Implementation -1. Inspect all commits associated with Stages 1–5 and compare the result with - `docs/roadmap/source-releases.md` and this plan. -2. Run the shared candidate checker with synthetic version `v0.0.0`. This is a - build identity only; do not create a note or tag for it. -3. Independently run the repository-wide baseline checks if any are not already - performed by the shared checker: +1. Inspect `internal/framework/llm/asset_registry.go`, prompt/profile source + composition, all registered asset roots, the conventional local backend, + and their focused tests. +2. Exercise every production asset registry through PromptKit engine + construction and the existing production composition tests. Confirm that: + - YAML IDs and versions, not filenames, select definitions; + - every `content_file` path is exact, relative, contained, and points to a + regular embedded file; + - every schema and JSON asset is one complete JSON value; + - every current output contract is valid under v0.8.0; and + - unrelated malformed definitions do not create a second Notarius identity + or fallback mechanism. +3. Review local endpoint parsing and validation. Retain a narrower Notarius + rule only if it has independent application value; otherwise rely on + PromptKit's absolute HTTP/HTTPS URL contract. Never accept a value that the + adapter will later reject. +4. Review conversion of Notarius variables, inputs, profile extras, and debug + values at the adapter boundary for PromptKit's bounded JSON-compatible-value + rules. Do not add a second generic JSON walker or duplicate upstream numeric + limits. +5. Verify cancellation and deadline identity through existing adapter tests. + Add or refine one focused regression only if Notarius currently destroys an + `errors.Is`-relevant context or transport error that the application owns. +6. Do not add a cross-operation schema cache, artifact cache, provider-body + reader, or duplicate JSON framing validation; v0.6.0 owns those mechanisms. + +### Tests And Validation + +Run the focused asset, profile-source, and adapter packages, then the ordinary +and race-enabled suites: ```sh -go test ./... -go vet ./... -go build ./cmd/notarius +GOWORK=off go test ./internal/framework/llm ./internal/cli +GOWORK=off go test ./... +GOWORK=off go test -race ./... +git diff --check ``` -4. Verify exact development and injected release output: - - ordinary checkout: `notarius development`; - - injected `v0.0.0`: `notarius v0.0.0`; - - invalid linker override: nonzero exit, no false release identity. -5. Confirm both maintained D&D configurations validate offline. -6. Verify all added or changed local Markdown links and run - `git diff --check`. -7. Audit `.woodpecker/release.yml` and `docs/release.md` for agreement on tag - form, note location/headings, Go image/version expectations, and shared - checker use. -8. Confirm the repository contains no generated release binary, distribution - directory, checksum file, release credential, Windows target, release - plugin, synthetic release note, or new tag. -9. Review tests under the repository testing policy. Retain behavior-level - coverage for the public version contract and important validation guards; - remove redundant tests that merely duplicate the shared checker or CI text. -10. Report any remaining divergence as a concrete finding. Fix only in-scope - release-feature defects discovered during this verification; do not expand - into packaged distribution or unrelated cleanup. +Tests must remain offline and should validate Notarius's assembled boundary, +not reproduce PromptKit's internal path, JSON-depth, or response-size matrices. ### Acceptance Criteria -- All feature-roadmap acceptance criteria are met except creation of the first - real post-procedure release, which is intentionally a separate operator - action. -- Local validation and tag CI share one substantive source checker. -- Version reporting, source installation, platform policy, documentation - ownership, tag guards, and immutable failure handling are internally - consistent. -- The full test suite, vet, build, configuration validation, four-target cross- - build matrix, Markdown link review, and whitespace checks pass. -- The worktree contains no release side effects beyond the intended source, - automation, and documentation changes. +- Every maintained embedded prompt, schema, and fallback profile can be loaded + through the assembled v0.8.0 engine. +- Current local endpoint and JSON-compatible values either satisfy the stricter + upstream contract or fail during preparation with safe diagnostics. +- No duplicate PromptKit-owned cache, JSON, or response-bound mechanism is + introduced. +- Cancellation and deadline behavior remains discoverable at the Notarius + boundary. +- Ordinary and race-enabled tests pass. + +## Stage 3: Adopt Profile Inheritance, Rakestrawhome, And Optional Credentials + +### Goal + +Make the useful PromptKit v0.7.0 profile and backend behavior work through +Notarius's existing generic profile boundary without adding provider-specific +composition logic. + +### Implementation + +1. Inspect `promptkit_profiles.go`, `asset_registry.go`, profile fingerprinting, + CLI profile preflight, profile provenance recording, and their tests before + editing. +2. Add an offline integration test using a temporary operator profile source + whose leaf uses `base_profile`. Prove that: + - preflight reports the leaf ID; + - the effective backend, model, reasoning, and other inherited values match + the resolved PromptKit target; + - execution uses the same resolved target as inspection; and + - a missing parent or cycle fails before provider generation with a safe + profile-load diagnostic. + Do not duplicate PromptKit's entire field-by-field merge test matrix. +3. Add a checkpoint-safety test showing that changing a parent definition in + an operator profile directory changes Notarius's profile-source fingerprint + while profile content and paths remain absent from the fingerprint value. + Retain the v0.8.0 catalog marker as coverage for built-in-parent changes. +4. Verify `rakestrawhome-gemma-4-31b` through the ordinary profile inspector. + Assert its selected backend reaches Notarius's application-owned inspection + and provenance fields. Use a fake PromptKit client or transport if execution + coverage is needed; never contact the live service or require credentials. +5. Verify that Notarius registers no `rakestrawhome` override and that the + existing `local` registration remains independent. +6. Add one `httptest`-backed adapter integration test for a filesystem profile + with a missing optional `api_key_env`. The request must reach the test server + without an `Authorization` header. Add a focused in-memory PromptKit profile + test for `APIKeyRequired` only if needed to prove Notarius preserves upstream + preflight behavior; do not expose a new operator profile API. +7. Keep `assets/dnd/profiles/dnd-extraction.yaml` standalone and unchanged. No + matching v0.8.0 built-in profile owns its `openai/gpt-5.6-luna` target. +8. Update the current profile-source, deployment, and pinned-integration + sections in `docs/config.md`, `docs/operations.md`, + `docs/internal/llm.md`, and `docs/integrations/pkg-promptkit.md`. Link to the + pinned PromptKit format rules for inheritance. Explain that filesystem + profiles cannot express PromptKit's in-memory `APIKeyRequired` field and + that an optional missing credential may result in a provider 401/403. + +### Tests And Validation + +```sh +GOWORK=off go test ./internal/framework/llm ./internal/core/config ./internal/cli +GOWORK=off go test ./... +GOWORK=off go test -race ./internal/framework/llm ./internal/cli +git diff --check +``` + +### Acceptance Criteria + +- Inherited operator profiles resolve identically during preflight and + execution, with the leaf ID and effective target kept distinct. +- Parent changes invalidate checkpoint reuse without leaking profile content or + paths. +- Rakestrawhome is available through generic PromptKit profile handling and is + not selected by default or registered by Notarius. +- Missing optional credentials omit authorization and reach the controlled + test provider; explicitly required credentials retain upstream behavior. +- Current documentation accurately describes the implemented profile and + credential behavior without duplicating PromptKit's merge algorithm. + +## Stage 4: Adapt Structured Generation Errors Safely + +### Goal + +Use PromptKit v0.7.0's structured generation errors for stable status +classification and debug-only provider diagnostics without leaking PromptKit +types or sensitive provider text. + +### Implementation + +1. In `internal/framework/contracts`, add: + - `ErrLLMGeneration` as the provider-neutral generation-failure sentinel; + - an application-owned `LLMGenerationError` with private status and safe + diagnostic fields, `Error`, `Unwrap`, and `StatusCode` methods; and + - a constructor that accepts a nonnegative status and an already-redacted + diagnostic. Status zero means no HTTP status was available. + Ordinary callers may inspect status with `errors.As` and category with + `errors.Is`, but cannot obtain provider code, type, or message from the + error. +2. Add an application-owned `LLMDebugProviderError` with `status_code`, `code`, + `type`, and `message` fields, referenced optionally from + `LLMDebugResponse`. This is debug material, not a manifest or durable public + artifact contract. +3. In `PromptKitClient.CompleteStructured`, preserve precedence in this order: + caller context cancellation/deadline, PromptKit capacity error, structured + PromptKit generation error, then other PromptKit generation failures. + Map every generation failure to `ErrLLMGeneration`; map + `*promptkit.GenerationError` to `LLMGenerationError` with its status. + Never wrap or return the PromptKit error value itself. +4. Keep the ordinary diagnostic limited to PromptKit's safe default error + formatting after bearer and known-credential redaction. Do not append + `ProviderCode`, `ProviderType`, or `ProviderMessage` to it. +5. For an explicitly requested debug path, preserve prepared prompt details and + attach the PromptKit provider code, type, and message after: + - reading only the selected prepared target's `APIKeyEnv`, if any, to obtain + the exact known credential solely for redaction; + - applying `RedactSecrets` and the existing bearer/key-pattern redaction; + - retaining PromptKit's already-normalized bounds; and + - discarding the credential value immediately rather than storing it. + Do not scan unrelated environment variables. +6. Return prompt/debug material alongside the error so the existing debug LLM + wrapper can persist it only when debug recording is enabled. Confirm that + provider fields do not appear in ordinary error text, warnings, manifests, + cache, checkpoint data, or a run without debug output. +7. Refactor error mapping into small helpers if needed to keep + `CompleteStructured` readable; do not create provider-specific policy in + modules or the pipeline runner. +8. Update the error and observability sections of `docs/internal/llm.md` and + `docs/integrations/pkg-promptkit.md`. Keep operator disclosure rules in + `docs/operations.md` concise and link to the internal boundary where useful. + +### Tests And Validation + +- Use `httptest.Server` to return representative structured 400 and 503 + responses. Assert `errors.Is(ErrLLMGeneration)`, `errors.As` to the + application-owned type, and the exact status without asserting complete + human wording. +- Include a provider message containing the selected test credential and a + bearer-shaped value. Verify both are absent from the ordinary error and + debug artifact, while a non-sensitive marker appears only in the requested + debug trace. +- Retain existing capacity and context tests to prove their more specific + classifications still win. + +```sh +GOWORK=off go test ./internal/framework/contracts ./internal/framework/llm ./internal/framework/pipeline ./internal/cli +GOWORK=off go test ./... +GOWORK=off go test -race ./internal/framework/llm ./internal/framework/pipeline +git diff --check +``` + +### Acceptance Criteria + +- PromptKit generation errors never escape the adapter error chain. +- All generation failures match `ErrLLMGeneration`; structured non-success + responses expose only application-owned HTTP status to ordinary callers. +- Provider code, type, and message are available only in an explicitly + requested, redacted debug trace. +- Capacity and context classifications remain unchanged and more specific. +- Security tests prove selected credentials and bearer tokens are not leaked. + +## Stage 5: Add Adapter-Level Structured Repair Support + +### Goal + +Teach the transport-neutral completion boundary and PromptKit adapter to apply +an optional repair override and report actual repair behavior, without yet +exposing the setting in pipeline configuration. + +### Implementation + +1. Add `StructuredOutputRepairAttempts *int` to + `contracts.StructuredCompletionRequest`. Copy the pointed-to value wherever + requests are cloned or retained. +2. Add `RepairAttempts int` to `contracts.StructuredCompletionResponse`. It is + the actual number of corrective generation calls, not the configured budget + and not the number of total candidates. +3. Validate a non-nil request value as zero through three at the adapter + boundary so programmatic callers cannot bypass later file/config validation. +4. When the request value is nil, leave `promptkit.RunRequest.Validation` nil + so the prompt's complete contract remains authoritative. +5. When the value is non-nil: + - call `Engine.InspectPrompt(ctx, promptID, promptVersion)`; + - copy `PromptInspection.OutputContract` by value; + - replace only `RepairAttempts`; + - pass the complete copied contract as `RunRequest.Validation`; and + - prepare and execute exactly as before. + Do not infer or hard-code schema paths, validation modes, or formats. Do not + cache inspection in this stage. +6. Map `result.Validation.RepairAttempts` to the response and leave + `result.Usage` cumulative values unchanged. The existing debug validation + object and prepared output contract should show actual and configured values + respectively. +7. Preserve result semantics: + - valid initial and repaired candidates decode normally; + - repair exhaustion returns the final raw candidate/debug material with an + error matching `ErrInvalidStructuredOutput`; + - explicit empty or whitespace-only content follows PromptKit validation; + - missing/null/non-string content remains a generation/provider failure; + - corrective-call generation errors use Stage 4's application-owned mapping; + and + - context cancellation wins at every error boundary. +8. Keep `CompleteStructured` and its helpers provider neutral outside this + adapter package. Do not expose PromptKit validation or inspection types. +9. Update only the adapter-owned repair behavior in `docs/internal/llm.md` and + `docs/integrations/pkg-promptkit.md`. State that public pipeline configuration + and the production default are added by later stages of this plan. + +### Tests And Validation + +Add adapter-level behavioral tests using a deterministic fake PromptKit LLM: + +- nil override uses the prompt declaration; +- explicit zero overrides a positive prompt declaration without dropping its + JSON Schema contract; +- explicit one turns an invalid first candidate followed by a valid candidate + into one successful response with the final raw bytes, actual repair count + one, and cumulative usage; +- repair exhaustion returns the final candidate and validation diagnostics as + `ErrInvalidStructuredOutput`; +- explicit empty content is eligible for repair; +- a corrective generation failure maps through Stage 4; and +- invalid direct values below zero or above three fail before provider work. + +Do not assert PromptKit's exact assistant/user correction prose or copy its +full internal repair matrix. + +```sh +GOWORK=off go test ./internal/framework/contracts ./internal/framework/llm ./internal/framework/pipeline +GOWORK=off go test -race ./internal/framework/llm ./internal/framework/pipeline +GOWORK=off go test ./... +git diff --check +``` + +### Acceptance Criteria + +- The adapter changes only repair count when applying a request override. +- Nil and explicit zero remain distinct. +- Repaired success returns final raw output, cumulative usage, and actual count + without a warning. +- Exhaustion, empty content, corrective generation failure, and cancellation + match the target semantics. +- No PromptKit type crosses the LLM package boundary. + +## Stage 6: Propagate Repair Policy Through Framework Requests + +### Goal + +Carry an optional effective repair budget from each resolved stage or validator +binding to its module request without changing public file configuration yet. + +### Implementation + +1. Add `StructuredOutputRepairAttempts *int` alongside `LLMProfile` to every + stage request that can belong to an LLM-backed binding: + - `ParseRequest`; + - `ChunkRequest`; + - `TypedExtractionRequest`; + - `TypedMergeRequest`; + - `TypedNormalizeRequest`; + - `OutputRequest`; + - `TypedValidationRequest`; + - `ChunkValidationRequest`; and + - `SerializedValidationRequest`. +2. Add the same optional field to the erased/internal request carriers used by + registry builders, preparation, runner stage attempts, validator targets, + retry closures, and debug wrappers. Copy pointer values; never share a + mutable pointer owned by configuration. +3. At every runner stage invocation, obtain the value from the exact resolved + producer binding. At every validator invocation, obtain it from that exact + resolved validator binding. Do not use the producer's value for a validator + or vice versa. +4. Ensure all retry attempts for the same binding receive the same effective + structural-repair value. Do not decrement it in Notarius; PromptKit owns the + inner budget independently on each `CompleteStructured` call. +5. Extend `semanticreconcile.Request` with the optional field and carry it into + each generic reconciliation completion. A batched reconciliation may make + several completion calls; each call receives the same effective budget. +6. Update registry erasure/adaptation code for typed merge, normalize, and + validation requests so no field is lost. Preserve input/output support even + though current production input and output modules are deterministic. +7. Add focused framework tests for one chunk producer, one extraction + producer, one normalizer, and one LLM-backed validator. Verify exact pointer + value propagation and separation between producer and validator settings. + Do not add repetitive tests for every generic adapter. + +### Tests And Validation + +```sh +GOWORK=off go test ./internal/framework/contracts ./internal/framework/pipeline ./internal/framework/semanticreconcile +GOWORK=off go test -race ./internal/framework/pipeline ./internal/framework/semanticreconcile +GOWORK=off go test ./... +git diff --check +``` + +### Acceptance Criteria + +- Every stage and validator request can carry a detached optional repair value. +- The runner sources the value from the exact resolved binding. +- Producer and validator values cannot overwrite one another. +- Stage retries reuse but do not mutate or consume the inner repair budget. +- Semantic reconciliation forwards the budget to every one of its completion + calls. +- Existing behavior remains unchanged while all values are nil. + +## Stage 7: Forward Repair Policy From Every LLM-Backed Module + +### Goal + +Complete the internal end-to-end path by having every production LLM-backed +module forward its stage request value to `CompleteStructured`. + +### Implementation + +1. Inventory every production `CompleteStructured` call with code search before + editing. The expected current owners include: + - `dnd/scenes` chunking; + - the combat-turn, enemy-event, item-occurrence, item-registry, + location-occurrence, location-registry, NPC-occurrence, NPC-registry, + scene-description, and spell extractors; and + - generic semantic reconciliation used by the item, location, and NPC + registry normalizers. + Reconcile this list with the actual repository; do not omit a newly added + production caller merely because it is not named here. +2. In each direct caller, set + `StructuredCompletionRequest.StructuredOutputRepairAttempts` from the + corresponding stage request. Clone the pointer or use a small shared helper + if that reduces repeated ownership mistakes without moving domain logic. +3. Ensure D&D registry normalizers pass their typed normalize request value into + `semanticreconcile.Request`, and that the generic engine forwards it as + established in Stage 6. +4. Update existing module prompt-mapping tests that already inspect a captured + structured-completion request to assert the new field. Do not create a new + one-test-per-module suite solely to memorialize field plumbing; rely on the + existing request-contract tests plus a final complete call-site audit. +5. Search again after editing for production `CompleteStructured` calls and + verify each either forwards the field or documents why it cannot receive a + pipeline binding. Test-only fakes need only preserve the field when their + contract test depends on it. +6. Do not set a module-specific fallback value. Nil must reach the adapter so + the prompt declaration remains authoritative. + +### Tests And Validation + +Run focused D&D and semantic-reconciliation packages, then the full suite: + +```sh +GOWORK=off go test ./internal/modules/dnd/... ./internal/framework/semanticreconcile +GOWORK=off go test -race ./internal/modules/dnd/... ./internal/framework/semanticreconcile +GOWORK=off go test ./... +git diff --check +``` + +### Acceptance Criteria + +- Every production LLM-backed completion receives the exact stage or validator + repair value. +- No module invents a default or imports PromptKit. +- Registry normalizers preserve the value through semantic reconciliation. +- Existing request-contract tests remain concise and pass. +- A final call-site audit finds no silent production omission. + +## Stage 8: Add The Public Repair Configuration Contract + +### Goal + +Add presence-aware pipeline and binding configuration for +`structured_output_repair_attempts` without yet changing runtime resolution. + +### Implementation + +1. Add `StructuredOutputRepairAttempts *int` to + `pipeline.PipelineProfile` and `pipeline.ModuleBinding`, using + `json:"structured_output_repair_attempts,omitempty"`. +2. Add presence-aware YAML support at pipeline and object-binding scope: + - exact key `structured_output_repair_attempts`; + - integer values zero through three; + - explicit null, non-integer, and out-of-range values rejected with scoped + diagnostics; and + - scalar shorthand bindings continue to omit the binding override. + Preserve file configuration version 4. +3. Update every configuration clone, conversion, redaction, summary, and JSON + round-trip carrier. Copy pointers by value into newly allocated storage so + parsed, configured, and redacted values do not alias. +4. Preserve omission versus explicit zero through YAML parsing, profile + inheritance, module-binding object form, and JSON round trips. Keep scalar + shorthand bindings equivalent to omission. +5. Do not add a top-level `promptkit.repair_attempts` setting or CLI override. +6. Add concise parser and ownership tests. Defer execution-class checks, + effective precedence, resolved digests, and runtime forwarding to Stage 9, + where module metadata is available. + +### Tests And Validation + +At the parser/config boundary, test omitted, explicit zero, positive bounds, +negative, above-three, null, non-integer, scalar shorthand, cloning, redaction, +and JSON round-trip behavior. Use relational boundary tests for the allowed +range and avoid duplicating the same cases at every layer. + +```sh +GOWORK=off go test ./internal/core/config ./internal/cli +GOWORK=off go test -race ./internal/core/config +GOWORK=off go test ./... +go run ./cmd/notarius config validate \ + --config examples/dnd-minimal.config.yml \ + --pipeline dnd-session +go run ./cmd/notarius config validate \ + --config examples/dnd-complete.config.yml \ + --pipeline dnd-session +git diff --check +``` + +### Acceptance Criteria + +- The exact public field parses at pipeline and object-binding scope with the + fixed range. +- Nil and explicit zero remain distinguishable through parsing, cloning, + inheritance, redaction, summaries, and round trips. +- Both maintained configurations remain valid without requiring the new field. +- No runtime or prompt default has changed prematurely. + +## Stage 9: Resolve And Apply Repair Configuration + +### Goal + +Resolve the public field against module execution classes, incorporate the +effective value into pipeline identity, and connect it to the request plumbing +completed in Stages 6 and 7. + +### Implementation + +1. During resolution, compute the effective value for every selected binding: + - explicit binding value wins; + - otherwise an explicit pipeline value applies to an LLM-backed binding; + - otherwise leave nil for prompt-owned policy. + Apply the pipeline value to LLM-backed validators as well as producers. +2. Reject an explicit binding value on a deterministic module or deterministic + validator using the same execution-class knowledge used for `llm_profile`. + Do not reject a pipeline-level value merely because a selected pipeline also + contains deterministic bindings; simply do not apply it to those bindings. +3. Clone every resolved pointer so the parsed pipeline, resolved profile, + redacted summaries, and runner requests have distinct ownership. +4. Include the effective field in resolved pipeline JSON and digest input. A + change between nil, zero, and a positive value must change the resolved + digest when it changes an LLM-backed selected binding. Unselected lanes must + retain the repository's existing digest and selection semantics. +5. Pass the resolved value into the Stage 6 request field for every selected + input, chunk, extract, merge, normalize, output, and validator binding. +6. Update `docs/config.md` as the canonical field, range, and precedence + contract; `docs/internal/pipeline.md` as the resolution owner; and + `docs/operations.md` for the distinction from binding `retries`. The + prompt-owned production default remains unchanged until Stage 10. +7. Add focused resolution and runner tests. Cover representative execution + classes rather than repeating the same assertion for every module type. + +### Tests And Validation + +Test: + +- binding over pipeline over nil precedence; +- inheritance into each selected LLM-backed stage and validator; +- no inheritance into deterministic bindings; +- explicit deterministic-binding rejection; +- detached pointers; +- runner forwarding for representative producer and validator bindings; and +- digest changes for execution-relevant nil, zero, and positive changes. + +```sh +GOWORK=off go test ./internal/framework/pipeline ./internal/cli +GOWORK=off go test -race ./internal/framework/pipeline +GOWORK=off go test ./... +go run ./cmd/notarius config validate \ + --config examples/dnd-minimal.config.yml \ + --pipeline dnd-session +go run ./cmd/notarius config validate \ + --config examples/dnd-complete.config.yml \ + --pipeline dnd-session +git diff --check +``` + +### Acceptance Criteria + +- The exact public field has the fixed binding-over-pipeline-over-prompt + precedence for every selected LLM-backed producer and validator. +- Deterministic binding misuse fails during resolution before execution, while + a pipeline value coexists with deterministic bindings. +- Nil and explicit zero remain distinguishable through resolution, runtime, + summaries, and digests. +- A policy change invalidates checkpoint identity when it changes an effective + selected binding. +- Current configuration, pipeline, and operations documentation matches the + implemented behavior. + +## Stage 10: Enable The Default, Finish Documentation, And Verify The Feature + +### Goal + +Set the accepted production default of one repair, reconcile all canonical +documentation, and run the full repository verification pass. + +### Implementation + +1. Change `repair_attempts: 0` to `repair_attempts: 1` in every maintained + production prompt manifest that produces structured output, including the + generic semantic-reconciliation prompt and every D&D chunk, extraction, and + registry-normalization prompt. Do not mechanically change unrelated test + fixtures whose purpose is to exercise zero. +2. Inspect every production prompt output contract after the edit. Confirm that + each positive budget uses `basic`, `json`, or `json_schema`, remains no + greater than three, and retains its existing format and schema path. +3. Add or refine the smallest durable assembled-assets test that proves the + production engine can prepare the maintained prompts with the activated + contracts. Do not add a brittle test that asserts an exact prompt count, + file count, message prose, correction text, or asset length. The public + default may be tested at one canonical assembled boundary because its + literal value is an operational contract. +4. Confirm a successful repair does not create a warning and that exhausted + repair remains `ErrInvalidStructuredOutput`. Verify the debug prompt records + the configured contract, the debug response records actual repair count, + and cumulative usage is not double-counted. +5. Confirm scheduling behavior with one focused test or existing coverage: the + Notarius scheduled client admits one logical `CompleteStructured` operation + while PromptKit may make serial corrective provider calls inside it. Do not + attempt to reacquire a Notarius permit from inside PromptKit or add a second + scheduler. +6. Reconcile current-state documentation: + - `docs/integrations/pkg-promptkit.md` owns the pinned upstream boundary; + - `docs/config.md` owns field names, range, default, and precedence; + - `docs/operations.md` owns latency/cost, optional credentials, concurrency, + timeout, and the upper-bound formula; + - `docs/internal/llm.md` owns inspection-based contract replacement, + cumulative usage, actual repair count, generation errors, and debug data; + - `docs/internal/pipeline.md` owns effective policy propagation and the + separation from stage retries; and + - `docs/policy/architecture.md` adds only the durable rule that PromptKit + owns deterministic structural repair inside one completion while Notarius + owns stage attempts and semantic validation. +7. Remove current-behavior claims that PromptKit is v0.5.0, that every + production repair budget is zero, or that PromptKit is always single-pass. + Do not alter historical release notes or archived roadmaps. +8. Keep the maintained minimal and complete examples secret-free and valid. + They may omit the new field to demonstrate the default; do not add a + redundant complete profile or Rakestrawhome example merely to exercise an + upstream catalog entry. +9. Review `docs/roadmap/future.md` only for consistency. Leave the future + feedback-aware stage retry, combat-scene validator, and warning-reform work + unimplemented and clearly separate. + +### Tests And Validation + +Run focused tests first, then all repository checks: + +```sh +GOWORK=off go test ./internal/framework/llm ./internal/framework/pipeline ./internal/framework/semanticreconcile ./internal/modules/dnd/... +GOWORK=off go test ./... +GOWORK=off go test -race ./... +GOWORK=off go vet ./... +GOWORK=off go build ./cmd/notarius +GOWORK=off go mod tidy -diff +go run ./cmd/notarius config validate \ + --config examples/dnd-minimal.config.yml \ + --pipeline dnd-session +go run ./cmd/notarius config validate \ + --config examples/dnd-complete.config.yml \ + --pipeline dnd-session +git diff --check +``` + +Also perform focused repository searches that exclude `docs/roadmap/archive/` +and historical release notes: + +- no active v0.5.0 PromptKit pins or links remain; +- no maintained production prompt still declares `repair_attempts: 0`; +- every production `CompleteStructured` caller forwards the repair field; and +- no provider code, type, or message is added to ordinary errors, warnings, + manifests, cache, or checkpoint schemas. + +If the repository's source-release checker is available and the ordinary +checks above pass, run `./scripts/check-release-source.sh v0.0.0` as the final +integrated validation. It must not create a tag, release note, or repository +artifact. + +### Acceptance Criteria + +- Every maintained structured prompt defaults to one corrective call and can + be overridden to zero through three at pipeline or binding scope. +- A real assembled Notarius completion follows the PromptKit v0.8.0 repair + contract without changing prompt schema identity or cacheable prefix. +- Actual repair count, cumulative usage, error classification, debug-only + provider diagnostics, scheduling, and checkpoint identity match the feature + roadmap. +- Profile inheritance, Rakestrawhome availability, optional credentials, and + v0.6.0 hardening remain covered and documented. +- All canonical documentation describes implemented v0.8.0 behavior in its + assigned home and leaves future semantic validation work in the roadmap. +- Maintained examples validate, all ordinary/race/vet/build/module checks pass, + and the worktree contains no generated or sensitive artifacts. diff --git a/docs/roadmap/promptkit-v0.8.md b/docs/roadmap/promptkit-v0.8.md new file mode 100644 index 00000000..c98321e7 --- /dev/null +++ b/docs/roadmap/promptkit-v0.8.md @@ -0,0 +1,520 @@ +# PromptKit v0.8.0 Upgrade + +## Status + +Proposed. + +## Purpose + +Upgrade Notarius from PromptKit v0.5.0 to v0.8.0 and deliberately adopt the +useful correctness, profile-composition, provider-diagnostic, backend, and +structured-output-repair capabilities introduced in PromptKit v0.6.0, v0.7.0, +and v0.8.0. + +The upgrade should improve structured-output reliability without confusing +PromptKit's bounded deterministic repair with Notarius's existing stage retry +budget or the future feedback-aware semantic-validation loop. PromptKit types +and provider behavior must remain behind Notarius's transport-neutral LLM +boundary. + +## Current State + +Notarius currently pins PromptKit v0.5.0. Its production adapter prepares one +frozen execution, records credential-redacted details, and runs that same +prepared value. It maps PromptKit capacity failures to an application-owned +error, maps failed structured validation to `ErrInvalidStructuredOutput`, and +returns PromptKit's raw validated bytes and usage metadata. + +Every maintained production prompt uses JSON Schema validation and currently +declares `repair_attempts: 0`. Notarius stage bindings separately expose +`retries`, which reruns a complete stage operation after an error or rejected +candidate. The two mechanisms have different ownership and must remain +independent. + +Notarius also maintains: + +- embedded prompt, schema, and fallback-profile filesystems; +- operator profile-file and profile-directory sources; +- one optional conventional `local` backend registration; +- explicit profile preflight through PromptKit inspection; +- one application-wide scheduled LLM client around the PromptKit adapter; +- PromptKit profile-source fingerprints for checkpoint safety; and +- redacted debug and manifest provenance at application-owned boundaries. + +The upgrade must preserve those established responsibilities while revising +the pinned integration contract and any behavior affected by the three +intervening releases. + +This roadmap is based on PromptKit's pinned release guides for +[v0.6.0](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.8.0/docs/releases/v0.6.0.md), +[v0.7.0](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.8.0/docs/releases/v0.7.0.md), +and +[v0.8.0](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.8.0/docs/releases/v0.8.0.md), +plus the public API and format documentation at the v0.8.0 tag. + +## Target End State + +- `go.mod` and `go.sum` pin PromptKit v0.8.0 without a local replacement or + vendored copy. +- Every maintained PromptKit prompt and profile prepares successfully under + v0.8.0's stricter validation and source-loading rules. +- Eligible Notarius structured completions use one PromptKit corrective call by + default after a structurally invalid response. Operators can explicitly set + a value from zero through three for a configured pipeline, with a more local + LLM-backed binding override where needed. +- PromptKit repair remains an inner operation within one Notarius stage + attempt. It never consumes or replenishes the binding's `retries` budget. +- A successful repaired result exposes cumulative usage and the actual repair + count to Notarius's application-owned response and debug models. A repaired + success is not itself a warning. +- Exhausted PromptKit validation remains an invalid structured-output result, + preserving the final candidate and diagnostics for debug and for any + applicable outer Notarius stage policy. Invalid structured output is never + accepted merely because the repair budget was exhausted. +- Profile inheritance, the built-in Rakestrawhome backend/profile, optional + credential behavior, and structured generation errors work through the + existing Notarius PromptKit boundary and are accurately documented. +- Provider-specific PromptKit types do not escape `internal/framework/llm`. +- Checkpoint identity, effective configuration, redacted summaries, and debug + provenance reflect every execution-affecting repair or profile change. +- Current documentation pins and describes v0.8.0; future Notarius semantic + validation retries remain roadmap behavior rather than being conflated with + this dependency upgrade. + +## Release-by-Release Adoption + +### PromptKit v0.6.0: Correctness, Safety, And Efficiency + +PromptKit v0.6.0 adds no public declarations, but intentionally rejects several +formerly permissive or ambiguous inputs. The upgrade must audit Notarius's +embedded and operator-facing integration against these rules: + +- YAML `id` and `version` metadata, rather than filenames, define prompt and + profile identity. +- Prompt `content_file` paths are exact, relative, contained paths; built-in + file artifacts must resolve to regular files. +- execution controls, output contracts, and repair budgets must be finite and + within their documented ranges; +- provider endpoints must be absolute HTTP or HTTPS URLs with a host and no + user information, query, or fragment; +- JSON documents and successful provider responses contain exactly one value; +- successful provider responses are bounded to 16 MiB; and +- JSON-compatible values are bounded for depth and expansion. + +Notarius should rely on PromptKit for these rules rather than duplicate its +parsers or internal limits. Existing Notarius validation may retain a narrower +application rule where it has independent value, but overlapping validation +must agree with PromptKit and must not accept a value PromptKit will reject +later. + +The upgrade automatically receives operation-local schema-plan reuse, +artifact-text memoization, improved cancellation checks, and transport error +identity preservation. Notarius should verify these changes through its real +adapter boundary and avoid adding a second cache or response-body layer that +would duplicate PromptKit's ownership. + +### PromptKit v0.7.0: Profiles, Backend Access, And Generation Errors + +#### Profile Inheritance + +Operator profiles may use `base_profile` to alias or selectively refine a +built-in, fallback, or higher-precedence operator profile. Notarius must pass +profile sources through unchanged and let PromptKit own parent lookup, merge +rules, source precedence, cycle detection, and fully resolved prepared targets. + +Preflight inspection must resolve inherited profiles through the same source +and backend composition used at execution. The selected leaf profile ID remains +the public profile identity, while effective backend, endpoint, model, and +reasoning provenance reflect the resolved chain. Notarius must not implement a +second inheritance parser. + +The existing complete `dnd-extraction` fallback remains a standalone profile: +PromptKit v0.8.0 does not provide a built-in `openai/gpt-5.6-luna` profile that +would be an appropriate parent. Documentation should nevertheless explain how +operators can use inheritance for environment-specific workload profiles and +should link to PromptKit's pinned format contract rather than duplicate its +field-by-field merge algorithm. + +Checkpoint safety must cover inherited behavior. Operator file/directory +digests already cover changes to definitions in those sources, fallback asset +digests cover application parents, and the PromptKit built-in catalog marker +must change from its v0.5.0 identity to v0.8.0 so a changed built-in parent +cannot reuse an incompatible checkpoint. + +#### Rakestrawhome Backend And Profile + +PromptKit's reserved `rakestrawhome` backend and +`rakestrawhome-gemma-4-31b` profile become available without Notarius-specific +registration. Notarius must not register or shadow the reserved backend ID. +Profile preflight, backend-capacity reporting, scheduling, generation, and +provenance should work for it through the same generic paths used by OpenRouter +and `local`. + +The D&D default remains `dnd-extraction`; this upgrade does not silently move a +production workload to Rakestrawhome. Operator documentation should identify +the built-in profile as an available selection and link to PromptKit for its +endpoint, credential environment, model, and capacity defaults. + +#### Optional Credentials + +An absent or blank optional `APIKeyEnv` now causes PromptKit to omit the +`Authorization` header and send the request. Notarius must not restore the old +failure behavior by pre-reading provider credential environment variables or +by adding provider-specific authentication logic. + +Profile inspection may report an explicit `APIKeyRequired` policy without +reading the credential, and execution remains the boundary at which that +requirement is enforced. For optional profiles, an authentication-requiring +provider may instead return a structured 401 or 403 generation failure. The +configuration and operations documentation must explain this distinction. +Notarius does not currently expose PromptKit's in-memory profile-registration +API to operators, and PromptKit's filesystem profile format does not expose +`APIKeyRequired`; therefore Notarius must not promise that an operator profile +can force local credential preflight. Operators should provision the named +environment variable, while Notarius should preserve the provider's structured +authentication failure when it is absent. + +Notarius must continue to document mechanisms and environment-variable names, +never secret values. + +#### Structured Generation Errors + +The adapter should recognize `*promptkit.GenerationError` with `errors.As` and +translate useful information into an immutable, provider-neutral Notarius +error classification. At minimum, retain the HTTP status code so callers and +future retry policy can distinguish transport success with provider rejection +from other generation failures. + +PromptKit's provider code, type, and message accessors are bounded but remain +untrusted and potentially sensitive. They must never appear automatically in +ordinary CLI output, warnings, manifests, checkpoint identity, or cache data. +If retained for an explicitly requested debug trace, they must pass through +Notarius's known-secret and bearer redaction and remain clearly identified as +untrusted provider diagnostics. Default error formatting should continue to +use a bounded, redacted application-owned message. + +Capacity and cancellation retain their current more specific classifications +and precedence. This upgrade does not add automatic provider-error retry +classification; it only preserves safe structured data needed for diagnosis +and later policy. + +### PromptKit v0.8.0: Bounded Structured-Output Repair + +#### Default Policy + +Every maintained production prompt whose output is consumed as structured data +should declare one repair attempt. All current production prompts use eligible +JSON Schema validation, so no current prompt needs a zero default merely +because of its output mode. + +One repair means at most one corrective generation after the initial +candidate. PromptKit reconstructs the immutable original conversation and +appends only the latest invalid assistant candidate and latest deterministic +validation diagnostics. It preserves the selected target, direct session ID, +provider-native structured-output contract, and backend capacity policy. This +shape preserves the original cacheable prompt prefix and avoids accumulating +unbounded failed history. + +The default is deliberately small. A single repair captures the common case in +which a capable model can correct malformed JSON or a schema violation after +receiving an exact diagnostic, while bounding the extra latency and cost of a +single structured completion. + +#### Configuration Contract + +The public configuration is an optional, presence-aware +`structured_output_repair_attempts` integer at pipeline scope and at each +LLM-backed module or validator binding. Its effective precedence is: + +1. the binding value, when present; +2. the pipeline value, when present; and +3. the selected prompt's declared `repair_attempts` value. + +The value must be from zero through three. Explicit zero disables PromptKit +repair at that scope. A deterministic binding must reject the field because it +cannot perform structured LLM repair. Validator bindings may use it only when +the selected validator is LLM-backed. Shorthand module bindings continue to +inherit the pipeline or prompt default. + +The long, provider-neutral name is intentional: it distinguishes PromptKit's +inner structural repair from the existing binding `retries` field, which owns +complete stage attempts, without exposing a dependency name in generic +pipeline contracts. + +The effective value must survive file parsing, cloning, redacted summaries, +pipeline resolution, and pipeline digest construction without pointer aliasing +or loss of presence. It must affect checkpoint identity because it can change +the selected result, latency, token usage, and provider cost. + +#### Adapter Contract + +The transport-neutral structured-completion request should carry an optional +application-owned structural-repair budget. No `promptkit.OutputContract` or +other PromptKit type may cross the adapter boundary. + +PromptKit v0.8.0 request validation replaces the complete prompt output +contract rather than merging one field. When Notarius has a configured +override, the adapter must therefore inspect the selected prompt, copy its +normalized declared format, validation mode, and schema path, change only the +repair count, and supply that complete contract on the prepared request. A nil +override continues to use the prompt declaration directly. Inspection and +preparation must use the same immutable engine sources; a small adapter-local +cache keyed by normalized prompt ID and version is acceptable but not required +without measured need. + +This approach prevents configuration from accidentally dropping JSON Schema +validation, avoids duplicating schema paths in pipeline YAML, and keeps prompt +assets authoritative for every output-contract field other than the explicit +operator override. + +The transport-neutral structured-completion response should report the actual +number of PromptKit repair calls. PromptKit's returned token usage is already +cumulative and must be passed through without re-summing it. Debug records +should distinguish the configured budget from the actual count. Ordinary run +manifests need not gain raw prompt or response data merely to report repairs; +any durable aggregate should be added only if it has a clear consumer contract. + +#### Result And Failure Semantics + +- A valid initial candidate returns normally with zero actual repairs. +- A valid corrected candidate returns normally with cumulative usage and its + positive actual repair count. It does not emit a warning solely because a + repair occurred. +- Exhausting the repair budget returns PromptKit's final candidate and failed + validation result. The adapter maps this to + `ErrInvalidStructuredOutput`, preserves the response and debug material, and + does not decode or accept the candidate. +- An explicitly empty or whitespace-only candidate participates in the + declared structural validation and repair flow. Missing, `null`, or + non-string provider content remains a malformed provider response. +- A generation failure during a corrective call is an operational generation + failure and uses the same safe structured-error adaptation as an initial + generation failure. +- Context cancellation remains authoritative throughout the initial and + corrective calls. + +PromptKit repair happens inside one scheduled `CompleteStructured` operation. +The Notarius scheduler holds one permit for that logical operation while +PromptKit performs its initial and serial corrective calls; PromptKit +reacquires its own selected-backend capacity for each corrective generation. +Because corrective calls are serial, this cannot expand actual concurrent +provider work beyond the number of admitted Notarius operations, but +documentation must stop describing the Notarius permit as a separate admission +event for every internal repair call. + +One `CompleteStructured` invocation with effective PromptKit repair budget `R` +may make at most `R + 1` provider calls. If one stage attempt makes `C` +structured-completion invocations, a binding with `retries: N` has an upper +bound of `(N + 1) * C * (R + 1)` provider calls; `C` may itself be a bounded, +data-dependent module property, as it is for batched semantic reconciliation. +LLM-backed validators have their own corresponding invocation counts, budgets, +and costs. These formulas are upper bounds, not promises that every failure is +retryable or that every attempt reaches the provider. + +## Profile And Prompt Source Compatibility + +The upgrade must preserve Notarius's source precedence: an operator source, +then registered application fallback profiles, then PromptKit built-ins. A +selected malformed definition remains authoritative and fails rather than +falling through. Parent resolution introduced by profile inheritance observes +that same precedence. + +All embedded prompt manifests, shared content fragments, response schemas, and +fallback profiles must be prepared or inspected offline under v0.8.0. The +review should specifically catch: + +- IDs inferred accidentally from filenames; +- stale or escaping `content_file` paths; +- missing or non-regular embedded artifacts; +- repair values outside zero through three or paired with ineligible + validation; +- schemas or examples that are not exact single JSON documents; +- unsupported endpoint forms; and +- JSON-compatible variables or profile extras that exceed upstream bounds. + +No prompt prose, schema shape, durable D&D artifact contract, or default D&D +model should change merely to exercise the dependency. Prompt manifests should +change only as needed to enable the adopted repair default and satisfy v0.8.0 +contracts. + +## Provenance, Debugging, And Security + +- Update the opaque PromptKit built-in profile-catalog identity from v0.5.0 to + v0.8.0. Do not hash or publish PromptKit's internal catalog bytes. +- Ensure a prompt's repair default remains covered by its existing prompt asset + fingerprint and a configured effective override remains covered by the + resolved pipeline digest. +- Preserve selected leaf profile identity while recording the inherited + effective target already exposed by PromptKit inspection and prepared + details. +- Add actual structural-repair count and, when useful, the configured budget to + application-owned debug material. Token totals remain PromptKit's cumulative + values. +- Do not generate a warning for a successful repair. Repair exhaustion is an + invalid-output failure, while provider rejection is a generation failure. +- Never expose raw provider diagnostic fields without explicit debug capture + and application redaction. Do not place them in normal errors or durable + summaries. +- Preserve context and transport error identity sufficiently for + `errors.Is`-based cancellation and deadline handling after adapting the + external error. + +## Documentation And Examples + +Implementation should update current-state documentation only when the new +behavior lands: + +- `docs/integrations/pkg-promptkit.md` must pin v0.8.0 and define the revised + prepared-execution, repair, profile-inheritance, backend, credential, and + error-adaptation boundary. +- `docs/config.md` must own the repair configuration fields, precedence, + allowed range, explicit-zero behavior, profile inheritance availability, and + optional credential semantics. +- `docs/operations.md` must explain structural repair cost, timeout and + concurrency effects, credential failures, and its distinction from stage + retries. +- `docs/internal/llm.md` must describe adapter contract replacement, actual + repair metadata, error adaptation, source compatibility, and scheduling. +- `docs/internal/pipeline.md` must describe how effective repair configuration + is resolved and how inner repair differs from outer stage attempts. +- `docs/policy/architecture.md` should receive only the durable ownership rule: + PromptKit owns bounded deterministic structural repair within one completion, + while Notarius owns stage attempts and semantic validation policy. Detailed + fields and retry formulas belong in their canonical configuration and + operations documents. + +Update maintained configuration examples only if the public Notarius +configuration contract changes. A short inheritance illustration may remain in +the configuration reference; do not create a complete example solely to copy +PromptKit's upstream profile catalog. All upstream links must point to the +v0.8.0 tag. Historical release or archived roadmap references should remain +historical. + +No ADR is required solely to pin a newer dependency. The durable separation +between PromptKit structural repair and Notarius semantic stage retries should +be stated in architecture documentation now; the more extensive future +validation state machine still warrants the separate ADR already identified in +`future.md` when that work is promoted. + +## Validation And Acceptance Criteria + +The implementation is complete when: + +- the repository builds and tests against PromptKit v0.8.0 with no replacement + directive, workspace dependency, or vendored source; +- every maintained prompt and profile prepares or inspects successfully under + the v0.8.0 source, path, endpoint, output-contract, and JSON-value rules; +- an invalid first JSON Schema candidate followed by a valid correction returns + the valid raw output, cumulative usage, and actual repair count through the + Notarius adapter; +- repair exhaustion returns the final raw candidate and debug material with an + error matching `ErrInvalidStructuredOutput`; +- a corrective generation failure retains safe generation classification and + provider status without leaking untrusted provider detail; +- explicit empty content follows structural validation rather than being + misclassified by Notarius; +- repair configuration is presence-aware, range checked, rejected on + deterministic bindings, resolved with documented precedence, and included in + effective pipeline identity; +- inherited profiles resolve consistently during preflight and execution, and + changes to any relevant operator, fallback, or built-in parent invalidate + checkpoint reuse; +- the Rakestrawhome built-in profile reaches generic preflight, scheduling, and + provenance paths without application-specific registration; +- optional missing credentials and explicitly required credentials behave as + documented without contacting real providers in tests; +- cancellation, timeout, backend capacity, prepared-execution snapshot, + session ID, raw-output, debug-redaction, and existing profile provenance + behavior remain intact; +- maintained examples validate successfully; and +- canonical documentation contains no active v0.5.0 pin or claim that PromptKit + is always single-pass. + +Tests should follow `docs/policy/testing.md`: exercise observable Notarius +contracts with offline fake clients or `httptest` boundaries, and do not copy +PromptKit's entire internal repair test suite or assert its exact correction +message prose. The dependency's internal wording is not a Notarius contract. + +## Non-Goals + +- Implementing Notarius's future feedback-aware semantic stage-retry loop. +- Adding the D&D combat-scene semantic validator. +- Redesigning warning policy or treating successful structural repair as a + warning. +- Adding provider transport retries or deciding which HTTP statuses should + consume a stage retry. +- Exposing PromptKit request, response, profile, validation, capacity, or error + types outside the LLM adapter. +- Changing durable artifact schemas, D&D prompt semantics, the D&D default + model, or the fixed pipeline shape. +- Reimplementing PromptKit profile inheritance, schema validation, response + bounds, repair conversations, backend admission, or provider parsing inside + Notarius. + +## Decisions + +### 1. Default Structured-Output Repair Budget + +**Decision: default to one repair attempt.** Set every maintained +eligible production prompt to `repair_attempts: 1`. One corrective call is a +strong fit for Notarius because every current production LLM response has a +strict JSON Schema contract, smaller cost-effective models are a deliberate +deployment target, and a precise structural diagnostic often makes one retry +materially more successful. The budget is paid only after a structurally +invalid candidate and remains tightly bounded. + +**Alternative considered: retain zero by default.** This preserves single-pass +cost and latency and requires operators to opt in. It is preferable for an +environment where every additional request is expensive or where upstream +provider-native schema enforcement already produces negligible invalid output. +It is less suitable as the Notarius default because one malformed response can +otherwise discard substantial completed pipeline work. + +**Alternative considered: default to two.** This may improve recovery for +weak models, but it doubles the worst-case corrective cost relative to the +selected default and compounds with outer stage retries. It should be an +operator choice supported by configuration, not the initial default, unless +observational evidence shows that the second correction has a worthwhile +marginal success rate. + +### 2. Repair Override Scope + +**Decision: support both pipeline and LLM-backed binding overrides.** Use +the presence-aware `structured_output_repair_attempts` field and precedence +defined above. A pipeline value provides the convenient one-line control the +operator requested, while a binding value permits an expensive normalizer or +future LLM-backed validator to use a deliberately different budget. This +mirrors Notarius's established pipeline/binding profile inheritance and scales +without editing embedded prompts. + +**Alternative considered: support only a pipeline override.** This is smaller to +implement and document and still permits global enablement or disablement for +one pipeline. Its drawback is that one exceptional prompt cannot opt out or +request a larger budget without changing an embedded asset for every pipeline. + +**Alternative considered: expose one global value under the top-level +`promptkit` configuration.** This makes client construction simple, but applies +the same budget to unrelated pipelines and leaks an execution policy into the +dependency configuration block. It is less compositional than pipeline-owned +policy and therefore not recommended. + +### 3. Retention Of Provider-Supplied Generation Details + +**Decision: retain status in the application-owned error contract and +retain redacted provider code, type, and message only in explicitly requested +debug traces.** Status is useful for diagnosis and future retry policy without +usually containing sensitive data. The other fields can materially explain a +400 response but may echo request or schema content, so they belong only in the +already-sensitive debug surface after Notarius redaction. + +**Alternative considered: retain only HTTP status and discard all provider fields.** +This is the safest and smallest policy and still improves typed failure +handling. It sacrifices potentially decisive provider diagnostics, leaving an +operator with less information when a provider returns a terse status and the +problem cannot be reproduced easily. + +**Alternative considered: include bounded provider code and type in normal +errors while keeping message debug-only.** Codes and types are often stable and +less sensitive than messages, but PromptKit explicitly classifies every +provider field as untrusted. Promoting them to ordinary output creates a +disclosure and compatibility burden that is not currently justified.