Files
notarius/docs/roadmap/implementation.md

38 KiB

PromptKit v0.8.0 Upgrade Implementation Plan

Purpose

Implement the target state defined by the PromptKit v0.8.0 Upgrade: adopt the useful PromptKit v0.6.0, v0.7.0, and v0.8.0 changes; enable one bounded structural correction by default; expose pipeline and binding overrides; preserve safe provider diagnostics; and keep PromptKit behind Notarius's transport-neutral LLM boundary.

This plan is ordered. Each numbered stage is one implementation prompt for a gpt-5.6-terra coding agent. Complete and validate one stage before beginning the next. Read docs/development.md and every policy under docs/policy/ at the start of each stage, inspect the current code and tests named by that stage, preserve unrelated worktree changes, and update current-behavior documentation in the same stage as the behavior it describes.

Do not retire this plan or promptkit-v0.8.md during implementation. Keep both until the completed work has passed a separate review. Do not implement the future Notarius semantic-validation retry loop, D&D combat-scene validator, or warning redesign as part of this plan.

Decisions Fixed For Implementation

  • Pin gitea.maximumdirect.net/eric/promptkit v0.8.0 directly, with no replace, workspace dependency, or vendored source.
  • Every maintained eligible production prompt defaults to exactly one PromptKit structural repair attempt.
  • Add the exact configuration key structured_output_repair_attempts at pipeline scope and on LLM-backed module and validator bindings.
  • Effective precedence is binding value, then pipeline value, then the prompt's declared repair_attempts value. Omission inherits; explicit zero disables structural repair at that scope.
  • Accepted values are integers from zero through three. Explicit null and non-integer values are invalid. An explicit binding value on a deterministic module or validator is invalid. A pipeline value is applied only to selected LLM-backed bindings and does not make deterministic bindings invalid.
  • Keep file configuration version 4. This is an additive pre-release field and does not require parallel versioned behavior.
  • Use StructuredOutputRepairAttempts *int for presence-aware internal Go fields. Clone pointers at every ownership boundary.
  • A configured override never replaces schema identity, output format, or validation mode. The PromptKit adapter calls InspectPrompt, copies the complete normalized prompt-owned output contract, changes only RepairAttempts, and supplies the complete replacement on RunRequest. Do not add an inspection cache initially.
  • PromptKit repair is internal to one CompleteStructured call and does not consume or replenish a binding's existing retries budget.
  • Add RepairAttempts int to Notarius's structured-completion response. It is the actual corrective-call count reported by PromptKit; token usage remains PromptKit's cumulative usage and must not be summed again.
  • A valid repaired response is successful and produces no warning solely because repair occurred. Exhausted structural validation maps to ErrInvalidStructuredOutput with the final candidate and debug material retained.
  • Add an application-owned generation-error sentinel and typed status-bearing error. PromptKit error types must not cross internal/framework/llm.
  • HTTP status may appear in the application-owned generation error. Provider code, type, and message are excluded from ordinary errors, warnings, manifests, cache, and checkpoint identity; they may appear only in an explicitly requested debug trace after Notarius redaction.
  • Profile inheritance is owned entirely by PromptKit. Notarius passes sources through, inspects and records the resolved target, and does not parse or merge base_profile itself.
  • PromptKit's built-in rakestrawhome backend and rakestrawhome-gemma-4-31b profile are available generically. Notarius does not register, shadow, or select them by default.
  • Missing optional credential environment values are allowed to reach the provider without Authorization; Notarius does not recreate v0.5.0's local failure or add provider-specific authentication logic.
  • No dependency-upgrade ADR is required. Update architecture only with the durable ownership distinction between PromptKit structural repair and Notarius stage/semantic validation policy.

Stage 1: Upgrade The Dependency And Establish A Clean v0.8.0 Baseline

Goal

Move the repository to PromptKit v0.8.0, resolve source-compatibility issues, and establish a passing baseline before adopting new behavior.

Implementation

  1. Re-read the upstream v0.6.0, v0.7.0, and v0.8.0 release guides and the v0.8.0 package consumer and format documentation. Treat the pinned v0.8.0 tag, not the sibling checkout's moving branch, as authoritative.
  2. Update go.mod and go.sum to PromptKit v0.8.0 and run go mod tidy with GOWORK=off.
  3. Compile before making compatibility edits. Correct only actual source or behavior incompatibilities. In particular:
    • convert any positional promptkit.Profile or promptkit.OpenAICompatibleProfileConfig literals to keyed literals;
    • confirm Notarius does not register the newly reserved rakestrawhome backend ID; and
    • preserve PrepareExecution/Details/RunPrepared snapshot ownership, Discard, session forwarding, reasoning override, profile preflight, and capacity adaptation.
  4. Change promptKitBuiltinProfileCatalogID in internal/framework/llm/promptkit_profile_fingerprint.go from the v0.5.0 catalog marker to an opaque v0.8.0 marker. Do not hash PromptKit internal files or include catalog content in manifests.
  5. Update docs/integrations/pkg-promptkit.md to pin and link v0.8.0 and to state that this stage still leaves the production prompt-declared repair budget at its current value. Do not document later configuration or default behavior before it exists.
  6. Update only those existing tests whose public PromptKit types or stable v0.8.0 behavior genuinely changed. Do not rewrite tests merely to match upstream diagnostic wording.

Tests And Validation

GOWORK=off go mod tidy -diff
GOWORK=off go test ./internal/framework/llm ./internal/cli
GOWORK=off go test ./...
GOWORK=off go vet ./...
GOWORK=off go build ./cmd/notarius
git diff --check

Acceptance Criteria

  • go list -m gitea.maximumdirect.net/eric/promptkit reports v0.8.0.
  • There is no PromptKit replace, active Go workspace dependency, or vendor tree.
  • The adapter still uses one frozen prepared execution and all existing LLM tests pass.
  • Checkpoint profile identity includes the v0.8.0 built-in catalog marker.
  • Current integration documentation pins v0.8.0 without claiming that later stages are already active.
  • The full ordinary test suite, vet, and command build pass.

Stage 2: Verify v0.6.0 Compatibility And Hardening

Goal

Audit Notarius's assets and boundary values against PromptKit v0.6.0's stricter source, path, endpoint, JSON, and cancellation contracts, fixing only concrete incompatibilities.

Implementation

  1. Inspect internal/framework/llm/asset_registry.go, prompt/profile source composition, all registered asset roots, the conventional local backend, and their focused tests.
  2. Exercise every production asset registry through PromptKit engine construction and the existing production composition tests. Confirm that:
    • YAML IDs and versions, not filenames, select definitions;
    • every content_file path is exact, relative, contained, and points to a regular embedded file;
    • every schema and JSON asset is one complete JSON value;
    • every current output contract is valid under v0.8.0; and
    • unrelated malformed definitions do not create a second Notarius identity or fallback mechanism.
  3. Review local endpoint parsing and validation. Retain a narrower Notarius rule only if it has independent application value; otherwise rely on PromptKit's absolute HTTP/HTTPS URL contract. Never accept a value that the adapter will later reject.
  4. Review conversion of Notarius variables, inputs, profile extras, and debug values at the adapter boundary for PromptKit's bounded JSON-compatible-value rules. Do not add a second generic JSON walker or duplicate upstream numeric limits.
  5. Verify cancellation and deadline identity through existing adapter tests. Add or refine one focused regression only if Notarius currently destroys an errors.Is-relevant context or transport error that the application owns.
  6. Do not add a cross-operation schema cache, artifact cache, provider-body reader, or duplicate JSON framing validation; v0.6.0 owns those mechanisms.

Tests And Validation

Run the focused asset, profile-source, and adapter packages, then the ordinary and race-enabled suites:

GOWORK=off go test ./internal/framework/llm ./internal/cli
GOWORK=off go test ./...
GOWORK=off go test -race ./...
git diff --check

Tests must remain offline and should validate Notarius's assembled boundary, not reproduce PromptKit's internal path, JSON-depth, or response-size matrices.

Acceptance Criteria

  • Every maintained embedded prompt, schema, and fallback profile can be loaded through the assembled v0.8.0 engine.
  • Current local endpoint and JSON-compatible values either satisfy the stricter upstream contract or fail during preparation with safe diagnostics.
  • No duplicate PromptKit-owned cache, JSON, or response-bound mechanism is introduced.
  • Cancellation and deadline behavior remains discoverable at the Notarius boundary.
  • Ordinary and race-enabled tests pass.

Stage 3: Adopt Profile Inheritance, Rakestrawhome, And Optional Credentials

Goal

Make the useful PromptKit v0.7.0 profile and backend behavior work through Notarius's existing generic profile boundary without adding provider-specific composition logic.

Implementation

  1. Inspect promptkit_profiles.go, asset_registry.go, profile fingerprinting, CLI profile preflight, profile provenance recording, and their tests before editing.
  2. Add an offline integration test using a temporary operator profile source whose leaf uses base_profile. Prove that:
    • preflight reports the leaf ID;
    • the effective backend, model, reasoning, and other inherited values match the resolved PromptKit target;
    • execution uses the same resolved target as inspection; and
    • a missing parent or cycle fails before provider generation with a safe profile-load diagnostic. Do not duplicate PromptKit's entire field-by-field merge test matrix.
  3. Add a checkpoint-safety test showing that changing a parent definition in an operator profile directory changes Notarius's profile-source fingerprint while profile content and paths remain absent from the fingerprint value. Retain the v0.8.0 catalog marker as coverage for built-in-parent changes.
  4. Verify rakestrawhome-gemma-4-31b through the ordinary profile inspector. Assert its selected backend reaches Notarius's application-owned inspection and provenance fields. Use a fake PromptKit client or transport if execution coverage is needed; never contact the live service or require credentials.
  5. Verify that Notarius registers no rakestrawhome override and that the existing local registration remains independent.
  6. Add one httptest-backed adapter integration test for a filesystem profile with a missing optional api_key_env. The request must reach the test server without an Authorization header. Add a focused in-memory PromptKit profile test for APIKeyRequired only if needed to prove Notarius preserves upstream preflight behavior; do not expose a new operator profile API.
  7. Keep assets/dnd/profiles/dnd-extraction.yaml standalone and unchanged. No matching v0.8.0 built-in profile owns its openai/gpt-5.6-luna target.
  8. Update the current profile-source, deployment, and pinned-integration sections in docs/config.md, docs/operations.md, docs/internal/llm.md, and docs/integrations/pkg-promptkit.md. Link to the pinned PromptKit format rules for inheritance. Explain that filesystem profiles cannot express PromptKit's in-memory APIKeyRequired field and that an optional missing credential may result in a provider 401/403.

Tests And Validation

GOWORK=off go test ./internal/framework/llm ./internal/core/config ./internal/cli
GOWORK=off go test ./...
GOWORK=off go test -race ./internal/framework/llm ./internal/cli
git diff --check

Acceptance Criteria

  • Inherited operator profiles resolve identically during preflight and execution, with the leaf ID and effective target kept distinct.
  • Parent changes invalidate checkpoint reuse without leaking profile content or paths.
  • Rakestrawhome is available through generic PromptKit profile handling and is not selected by default or registered by Notarius.
  • Missing optional credentials omit authorization and reach the controlled test provider; explicitly required credentials retain upstream behavior.
  • Current documentation accurately describes the implemented profile and credential behavior without duplicating PromptKit's merge algorithm.

Stage 4: Adapt Structured Generation Errors Safely

Goal

Use PromptKit v0.7.0's structured generation errors for stable status classification and debug-only provider diagnostics without leaking PromptKit types or sensitive provider text.

Implementation

  1. In internal/framework/contracts, add:
    • ErrLLMGeneration as the provider-neutral generation-failure sentinel;
    • an application-owned LLMGenerationError with private status and safe diagnostic fields, Error, Unwrap, and StatusCode methods; and
    • a constructor that accepts a nonnegative status and an already-redacted diagnostic. Status zero means no HTTP status was available. Ordinary callers may inspect status with errors.As and category with errors.Is, but cannot obtain provider code, type, or message from the error.
  2. Add an application-owned LLMDebugProviderError with status_code, code, type, and message fields, referenced optionally from LLMDebugResponse. This is debug material, not a manifest or durable public artifact contract.
  3. In PromptKitClient.CompleteStructured, preserve precedence in this order: caller context cancellation/deadline, PromptKit capacity error, structured PromptKit generation error, then other PromptKit generation failures. Map every generation failure to ErrLLMGeneration; map *promptkit.GenerationError to LLMGenerationError with its status. Never wrap or return the PromptKit error value itself.
  4. Keep the ordinary diagnostic limited to PromptKit's safe default error formatting after bearer and known-credential redaction. Do not append ProviderCode, ProviderType, or ProviderMessage to it.
  5. For an explicitly requested debug path, preserve prepared prompt details and attach the PromptKit provider code, type, and message after:
    • reading only the selected prepared target's APIKeyEnv, if any, to obtain the exact known credential solely for redaction;
    • applying RedactSecrets and the existing bearer/key-pattern redaction;
    • retaining PromptKit's already-normalized bounds; and
    • discarding the credential value immediately rather than storing it. Do not scan unrelated environment variables.
  6. Return prompt/debug material alongside the error so the existing debug LLM wrapper can persist it only when debug recording is enabled. Confirm that provider fields do not appear in ordinary error text, warnings, manifests, cache, checkpoint data, or a run without debug output.
  7. Refactor error mapping into small helpers if needed to keep CompleteStructured readable; do not create provider-specific policy in modules or the pipeline runner.
  8. Update the error and observability sections of docs/internal/llm.md and docs/integrations/pkg-promptkit.md. Keep operator disclosure rules in docs/operations.md concise and link to the internal boundary where useful.

Tests And Validation

  • Use httptest.Server to return representative structured 400 and 503 responses. Assert errors.Is(ErrLLMGeneration), errors.As to the application-owned type, and the exact status without asserting complete human wording.
  • Include a provider message containing the selected test credential and a bearer-shaped value. Verify both are absent from the ordinary error and debug artifact, while a non-sensitive marker appears only in the requested debug trace.
  • Retain existing capacity and context tests to prove their more specific classifications still win.
GOWORK=off go test ./internal/framework/contracts ./internal/framework/llm ./internal/framework/pipeline ./internal/cli
GOWORK=off go test ./...
GOWORK=off go test -race ./internal/framework/llm ./internal/framework/pipeline
git diff --check

Acceptance Criteria

  • PromptKit generation errors never escape the adapter error chain.
  • All generation failures match ErrLLMGeneration; structured non-success responses expose only application-owned HTTP status to ordinary callers.
  • Provider code, type, and message are available only in an explicitly requested, redacted debug trace.
  • Capacity and context classifications remain unchanged and more specific.
  • Security tests prove selected credentials and bearer tokens are not leaked.

Stage 5: Add Adapter-Level Structured Repair Support

Goal

Teach the transport-neutral completion boundary and PromptKit adapter to apply an optional repair override and report actual repair behavior, without yet exposing the setting in pipeline configuration.

Implementation

  1. Add StructuredOutputRepairAttempts *int to contracts.StructuredCompletionRequest. Copy the pointed-to value wherever requests are cloned or retained.
  2. Add RepairAttempts int to contracts.StructuredCompletionResponse. It is the actual number of corrective generation calls, not the configured budget and not the number of total candidates.
  3. Validate a non-nil request value as zero through three at the adapter boundary so programmatic callers cannot bypass later file/config validation.
  4. When the request value is nil, leave promptkit.RunRequest.Validation nil so the prompt's complete contract remains authoritative.
  5. When the value is non-nil:
    • call Engine.InspectPrompt(ctx, promptID, promptVersion);
    • copy PromptInspection.OutputContract by value;
    • replace only RepairAttempts;
    • pass the complete copied contract as RunRequest.Validation; and
    • prepare and execute exactly as before. Do not infer or hard-code schema paths, validation modes, or formats. Do not cache inspection in this stage.
  6. Map result.Validation.RepairAttempts to the response and leave result.Usage cumulative values unchanged. The existing debug validation object and prepared output contract should show actual and configured values respectively.
  7. Preserve result semantics:
    • valid initial and repaired candidates decode normally;
    • repair exhaustion returns the final raw candidate/debug material with an error matching ErrInvalidStructuredOutput;
    • explicit empty or whitespace-only content follows PromptKit validation;
    • missing/null/non-string content remains a generation/provider failure;
    • corrective-call generation errors use Stage 4's application-owned mapping; and
    • context cancellation wins at every error boundary.
  8. Keep CompleteStructured and its helpers provider neutral outside this adapter package. Do not expose PromptKit validation or inspection types.
  9. Update only the adapter-owned repair behavior in docs/internal/llm.md and docs/integrations/pkg-promptkit.md. State that public pipeline configuration and the production default are added by later stages of this plan.

Tests And Validation

Add adapter-level behavioral tests using a deterministic fake PromptKit LLM:

  • nil override uses the prompt declaration;
  • explicit zero overrides a positive prompt declaration without dropping its JSON Schema contract;
  • explicit one turns an invalid first candidate followed by a valid candidate into one successful response with the final raw bytes, actual repair count one, and cumulative usage;
  • repair exhaustion returns the final candidate and validation diagnostics as ErrInvalidStructuredOutput;
  • explicit empty content is eligible for repair;
  • a corrective generation failure maps through Stage 4; and
  • invalid direct values below zero or above three fail before provider work.

Do not assert PromptKit's exact assistant/user correction prose or copy its full internal repair matrix.

GOWORK=off go test ./internal/framework/contracts ./internal/framework/llm ./internal/framework/pipeline
GOWORK=off go test -race ./internal/framework/llm ./internal/framework/pipeline
GOWORK=off go test ./...
git diff --check

Acceptance Criteria

  • The adapter changes only repair count when applying a request override.
  • Nil and explicit zero remain distinct.
  • Repaired success returns final raw output, cumulative usage, and actual count without a warning.
  • Exhaustion, empty content, corrective generation failure, and cancellation match the target semantics.
  • No PromptKit type crosses the LLM package boundary.

Stage 6: Propagate Repair Policy Through Framework Requests

Goal

Carry an optional effective repair budget from each resolved stage or validator binding to its module request without changing public file configuration yet.

Implementation

  1. Add StructuredOutputRepairAttempts *int alongside LLMProfile to every stage request that can belong to an LLM-backed binding:
    • ParseRequest;
    • ChunkRequest;
    • TypedExtractionRequest;
    • TypedMergeRequest;
    • TypedNormalizeRequest;
    • OutputRequest;
    • TypedValidationRequest;
    • ChunkValidationRequest; and
    • SerializedValidationRequest.
  2. Add the same optional field to the erased/internal request carriers used by registry builders, preparation, runner stage attempts, validator targets, retry closures, and debug wrappers. Copy pointer values; never share a mutable pointer owned by configuration.
  3. At every runner stage invocation, obtain the value from the exact resolved producer binding. At every validator invocation, obtain it from that exact resolved validator binding. Do not use the producer's value for a validator or vice versa.
  4. Ensure all retry attempts for the same binding receive the same effective structural-repair value. Do not decrement it in Notarius; PromptKit owns the inner budget independently on each CompleteStructured call.
  5. Extend semanticreconcile.Request with the optional field and carry it into each generic reconciliation completion. A batched reconciliation may make several completion calls; each call receives the same effective budget.
  6. Update registry erasure/adaptation code for typed merge, normalize, and validation requests so no field is lost. Preserve input/output support even though current production input and output modules are deterministic.
  7. Add focused framework tests for one chunk producer, one extraction producer, one normalizer, and one LLM-backed validator. Verify exact pointer value propagation and separation between producer and validator settings. Do not add repetitive tests for every generic adapter.

Tests And Validation

GOWORK=off go test ./internal/framework/contracts ./internal/framework/pipeline ./internal/framework/semanticreconcile
GOWORK=off go test -race ./internal/framework/pipeline ./internal/framework/semanticreconcile
GOWORK=off go test ./...
git diff --check

Acceptance Criteria

  • Every stage and validator request can carry a detached optional repair value.
  • The runner sources the value from the exact resolved binding.
  • Producer and validator values cannot overwrite one another.
  • Stage retries reuse but do not mutate or consume the inner repair budget.
  • Semantic reconciliation forwards the budget to every one of its completion calls.
  • Existing behavior remains unchanged while all values are nil.

Stage 7: Forward Repair Policy From Every LLM-Backed Module

Goal

Complete the internal end-to-end path by having every production LLM-backed module forward its stage request value to CompleteStructured.

Implementation

  1. Inventory every production CompleteStructured call with code search before editing. The expected current owners include:
    • dnd/scenes chunking;
    • the combat-turn, enemy-event, item-occurrence, item-registry, location-occurrence, location-registry, NPC-occurrence, NPC-registry, scene-description, and spell extractors; and
    • generic semantic reconciliation used by the item, location, and NPC registry normalizers. Reconcile this list with the actual repository; do not omit a newly added production caller merely because it is not named here.
  2. In each direct caller, set StructuredCompletionRequest.StructuredOutputRepairAttempts from the corresponding stage request. Clone the pointer or use a small shared helper if that reduces repeated ownership mistakes without moving domain logic.
  3. Ensure D&D registry normalizers pass their typed normalize request value into semanticreconcile.Request, and that the generic engine forwards it as established in Stage 6.
  4. Update existing module prompt-mapping tests that already inspect a captured structured-completion request to assert the new field. Do not create a new one-test-per-module suite solely to memorialize field plumbing; rely on the existing request-contract tests plus a final complete call-site audit.
  5. Search again after editing for production CompleteStructured calls and verify each either forwards the field or documents why it cannot receive a pipeline binding. Test-only fakes need only preserve the field when their contract test depends on it.
  6. Do not set a module-specific fallback value. Nil must reach the adapter so the prompt declaration remains authoritative.

Tests And Validation

Run focused D&D and semantic-reconciliation packages, then the full suite:

GOWORK=off go test ./internal/modules/dnd/... ./internal/framework/semanticreconcile
GOWORK=off go test -race ./internal/modules/dnd/... ./internal/framework/semanticreconcile
GOWORK=off go test ./...
git diff --check

Acceptance Criteria

  • Every production LLM-backed completion receives the exact stage or validator repair value.
  • No module invents a default or imports PromptKit.
  • Registry normalizers preserve the value through semantic reconciliation.
  • Existing request-contract tests remain concise and pass.
  • A final call-site audit finds no silent production omission.

Stage 8: Add The Public Repair Configuration Contract

Goal

Add presence-aware pipeline and binding configuration for structured_output_repair_attempts without yet changing runtime resolution.

Implementation

  1. Add StructuredOutputRepairAttempts *int to pipeline.PipelineProfile and pipeline.ModuleBinding, using json:"structured_output_repair_attempts,omitempty".
  2. Add presence-aware YAML support at pipeline and object-binding scope:
    • exact key structured_output_repair_attempts;
    • integer values zero through three;
    • explicit null, non-integer, and out-of-range values rejected with scoped diagnostics; and
    • scalar shorthand bindings continue to omit the binding override. Preserve file configuration version 4.
  3. Update every configuration clone, conversion, redaction, summary, and JSON round-trip carrier. Copy pointers by value into newly allocated storage so parsed, configured, and redacted values do not alias.
  4. Preserve omission versus explicit zero through YAML parsing, profile inheritance, module-binding object form, and JSON round trips. Keep scalar shorthand bindings equivalent to omission.
  5. Do not add a top-level promptkit.repair_attempts setting or CLI override.
  6. Add concise parser and ownership tests. Defer execution-class checks, effective precedence, resolved digests, and runtime forwarding to Stage 9, where module metadata is available.

Tests And Validation

At the parser/config boundary, test omitted, explicit zero, positive bounds, negative, above-three, null, non-integer, scalar shorthand, cloning, redaction, and JSON round-trip behavior. Use relational boundary tests for the allowed range and avoid duplicating the same cases at every layer.

GOWORK=off go test ./internal/core/config ./internal/cli
GOWORK=off go test -race ./internal/core/config
GOWORK=off go test ./...
go run ./cmd/notarius config validate \
  --config examples/dnd-minimal.config.yml \
  --pipeline dnd-session
go run ./cmd/notarius config validate \
  --config examples/dnd-complete.config.yml \
  --pipeline dnd-session
git diff --check

Acceptance Criteria

  • The exact public field parses at pipeline and object-binding scope with the fixed range.
  • Nil and explicit zero remain distinguishable through parsing, cloning, inheritance, redaction, summaries, and round trips.
  • Both maintained configurations remain valid without requiring the new field.
  • No runtime or prompt default has changed prematurely.

Stage 9: Resolve And Apply Repair Configuration

Goal

Resolve the public field against module execution classes, incorporate the effective value into pipeline identity, and connect it to the request plumbing completed in Stages 6 and 7.

Implementation

  1. During resolution, compute the effective value for every selected binding:
    • explicit binding value wins;
    • otherwise an explicit pipeline value applies to an LLM-backed binding;
    • otherwise leave nil for prompt-owned policy. Apply the pipeline value to LLM-backed validators as well as producers.
  2. Reject an explicit binding value on a deterministic module or deterministic validator using the same execution-class knowledge used for llm_profile. Do not reject a pipeline-level value merely because a selected pipeline also contains deterministic bindings; simply do not apply it to those bindings.
  3. Clone every resolved pointer so the parsed pipeline, resolved profile, redacted summaries, and runner requests have distinct ownership.
  4. Include the effective field in resolved pipeline JSON and digest input. A change between nil, zero, and a positive value must change the resolved digest when it changes an LLM-backed selected binding. Unselected lanes must retain the repository's existing digest and selection semantics.
  5. Pass the resolved value into the Stage 6 request field for every selected input, chunk, extract, merge, normalize, output, and validator binding.
  6. Update docs/config.md as the canonical field, range, and precedence contract; docs/internal/pipeline.md as the resolution owner; and docs/operations.md for the distinction from binding retries. The prompt-owned production default remains unchanged until Stage 10.
  7. Add focused resolution and runner tests. Cover representative execution classes rather than repeating the same assertion for every module type.

Tests And Validation

Test:

  • binding over pipeline over nil precedence;
  • inheritance into each selected LLM-backed stage and validator;
  • no inheritance into deterministic bindings;
  • explicit deterministic-binding rejection;
  • detached pointers;
  • runner forwarding for representative producer and validator bindings; and
  • digest changes for execution-relevant nil, zero, and positive changes.
GOWORK=off go test ./internal/framework/pipeline ./internal/cli
GOWORK=off go test -race ./internal/framework/pipeline
GOWORK=off go test ./...
go run ./cmd/notarius config validate \
  --config examples/dnd-minimal.config.yml \
  --pipeline dnd-session
go run ./cmd/notarius config validate \
  --config examples/dnd-complete.config.yml \
  --pipeline dnd-session
git diff --check

Acceptance Criteria

  • The exact public field has the fixed binding-over-pipeline-over-prompt precedence for every selected LLM-backed producer and validator.
  • Deterministic binding misuse fails during resolution before execution, while a pipeline value coexists with deterministic bindings.
  • Nil and explicit zero remain distinguishable through resolution, runtime, summaries, and digests.
  • A policy change invalidates checkpoint identity when it changes an effective selected binding.
  • Current configuration, pipeline, and operations documentation matches the implemented behavior.

Stage 10: Enable The Default, Finish Documentation, And Verify The Feature

Goal

Set the accepted production default of one repair, reconcile all canonical documentation, and run the full repository verification pass.

Implementation

  1. Change repair_attempts: 0 to repair_attempts: 1 in every maintained production prompt manifest that produces structured output, including the generic semantic-reconciliation prompt and every D&D chunk, extraction, and registry-normalization prompt. Do not mechanically change unrelated test fixtures whose purpose is to exercise zero.
  2. Inspect every production prompt output contract after the edit. Confirm that each positive budget uses basic, json, or json_schema, remains no greater than three, and retains its existing format and schema path.
  3. Add or refine the smallest durable assembled-assets test that proves the production engine can prepare the maintained prompts with the activated contracts. Do not add a brittle test that asserts an exact prompt count, file count, message prose, correction text, or asset length. The public default may be tested at one canonical assembled boundary because its literal value is an operational contract.
  4. Confirm a successful repair does not create a warning and that exhausted repair remains ErrInvalidStructuredOutput. Verify the debug prompt records the configured contract, the debug response records actual repair count, and cumulative usage is not double-counted.
  5. Confirm scheduling behavior with one focused test or existing coverage: the Notarius scheduled client admits one logical CompleteStructured operation while PromptKit may make serial corrective provider calls inside it. Do not attempt to reacquire a Notarius permit from inside PromptKit or add a second scheduler.
  6. Reconcile current-state documentation:
    • docs/integrations/pkg-promptkit.md owns the pinned upstream boundary;
    • docs/config.md owns field names, range, default, and precedence;
    • docs/operations.md owns latency/cost, optional credentials, concurrency, timeout, and the upper-bound formula;
    • docs/internal/llm.md owns inspection-based contract replacement, cumulative usage, actual repair count, generation errors, and debug data;
    • docs/internal/pipeline.md owns effective policy propagation and the separation from stage retries; and
    • docs/policy/architecture.md adds only the durable rule that PromptKit owns deterministic structural repair inside one completion while Notarius owns stage attempts and semantic validation.
  7. Remove current-behavior claims that PromptKit is v0.5.0, that every production repair budget is zero, or that PromptKit is always single-pass. Do not alter historical release notes or archived roadmaps.
  8. Keep the maintained minimal and complete examples secret-free and valid. They may omit the new field to demonstrate the default; do not add a redundant complete profile or Rakestrawhome example merely to exercise an upstream catalog entry.
  9. Review docs/roadmap/future.md only for consistency. Leave the future feedback-aware stage retry, combat-scene validator, and warning-reform work unimplemented and clearly separate.

Tests And Validation

Run focused tests first, then all repository checks:

GOWORK=off go test ./internal/framework/llm ./internal/framework/pipeline ./internal/framework/semanticreconcile ./internal/modules/dnd/...
GOWORK=off go test ./...
GOWORK=off go test -race ./...
GOWORK=off go vet ./...
GOWORK=off go build ./cmd/notarius
GOWORK=off go mod tidy -diff
go run ./cmd/notarius config validate \
  --config examples/dnd-minimal.config.yml \
  --pipeline dnd-session
go run ./cmd/notarius config validate \
  --config examples/dnd-complete.config.yml \
  --pipeline dnd-session
git diff --check

Also perform focused repository searches that exclude docs/roadmap/archive/ and historical release notes:

  • no active v0.5.0 PromptKit pins or links remain;
  • no maintained production prompt still declares repair_attempts: 0;
  • every production CompleteStructured caller forwards the repair field; and
  • no provider code, type, or message is added to ordinary errors, warnings, manifests, cache, or checkpoint schemas.

If the repository's source-release checker is available and the ordinary checks above pass, run ./scripts/check-release-source.sh v0.0.0 as the final integrated validation. It must not create a tag, release note, or repository artifact.

Acceptance Criteria

  • Every maintained structured prompt defaults to one corrective call and can be overridden to zero through three at pipeline or binding scope.
  • A real assembled Notarius completion follows the PromptKit v0.8.0 repair contract without changing prompt schema identity or cacheable prefix.
  • Actual repair count, cumulative usage, error classification, debug-only provider diagnostics, scheduling, and checkpoint identity match the feature roadmap.
  • Profile inheritance, Rakestrawhome availability, optional credentials, and v0.6.0 hardening remain covered and documented.
  • All canonical documentation describes implemented v0.8.0 behavior in its assigned home and leaves future semantic validation work in the roadmap.
  • Maintained examples validate, all ordinary/race/vet/build/module checks pass, and the worktree contains no generated or sensitive artifacts.