Files
notarius/docs/roadmap/implementation.md

783 lines
38 KiB
Markdown

# PromptKit v0.8.0 Upgrade Implementation Plan
## Purpose
Implement the target state defined by the
[PromptKit v0.8.0 Upgrade](promptkit-v0.8.md): adopt the useful PromptKit
v0.6.0, v0.7.0, and v0.8.0 changes; enable one bounded structural correction
by default; expose pipeline and binding overrides; preserve safe provider
diagnostics; and keep PromptKit behind Notarius's transport-neutral LLM
boundary.
This plan is ordered. Each numbered stage is one implementation prompt for a
gpt-5.6-terra coding agent. Complete and validate one stage before beginning
the next. Read `docs/development.md` and every policy under `docs/policy/` at
the start of each stage, inspect the current code and tests named by that
stage, preserve unrelated worktree changes, and update current-behavior
documentation in the same stage as the behavior it describes.
Do not retire this plan or `promptkit-v0.8.md` during implementation. Keep both
until the completed work has passed a separate review. Do not implement the
future Notarius semantic-validation retry loop, D&D combat-scene validator, or
warning redesign as part of this plan.
## Decisions Fixed For Implementation
- Pin `gitea.maximumdirect.net/eric/promptkit` v0.8.0 directly, with no
`replace`, workspace dependency, or vendored source.
- Every maintained eligible production prompt defaults to exactly one
PromptKit structural repair attempt.
- Add the exact configuration key
`structured_output_repair_attempts` at pipeline scope and on LLM-backed
module and validator bindings.
- Effective precedence is binding value, then pipeline value, then the prompt's
declared `repair_attempts` value. Omission inherits; explicit zero disables
structural repair at that scope.
- Accepted values are integers from zero through three. Explicit null and
non-integer values are invalid. An explicit binding value on a deterministic
module or validator is invalid. A pipeline value is applied only to selected
LLM-backed bindings and does not make deterministic bindings invalid.
- Keep file configuration version 4. This is an additive pre-release field and
does not require parallel versioned behavior.
- Use `StructuredOutputRepairAttempts *int` for presence-aware internal Go
fields. Clone pointers at every ownership boundary.
- A configured override never replaces schema identity, output format, or
validation mode. The PromptKit adapter calls `InspectPrompt`, copies the
complete normalized prompt-owned output contract, changes only
`RepairAttempts`, and supplies the complete replacement on `RunRequest`.
Do not add an inspection cache initially.
- PromptKit repair is internal to one `CompleteStructured` call and does not
consume or replenish a binding's existing `retries` budget.
- Add `RepairAttempts int` to Notarius's structured-completion response. It is
the actual corrective-call count reported by PromptKit; token usage remains
PromptKit's cumulative usage and must not be summed again.
- A valid repaired response is successful and produces no warning solely
because repair occurred. Exhausted structural validation maps to
`ErrInvalidStructuredOutput` with the final candidate and debug material
retained.
- Add an application-owned generation-error sentinel and typed status-bearing
error. PromptKit error types must not cross `internal/framework/llm`.
- HTTP status may appear in the application-owned generation error. Provider
code, type, and message are excluded from ordinary errors, warnings,
manifests, cache, and checkpoint identity; they may appear only in an
explicitly requested debug trace after Notarius redaction.
- Profile inheritance is owned entirely by PromptKit. Notarius passes sources
through, inspects and records the resolved target, and does not parse or merge
`base_profile` itself.
- PromptKit's built-in `rakestrawhome` backend and
`rakestrawhome-gemma-4-31b` profile are available generically. Notarius does
not register, shadow, or select them by default.
- Missing optional credential environment values are allowed to reach the
provider without `Authorization`; Notarius does not recreate v0.5.0's local
failure or add provider-specific authentication logic.
- No dependency-upgrade ADR is required. Update architecture only with the
durable ownership distinction between PromptKit structural repair and
Notarius stage/semantic validation policy.
## Stage 1: Upgrade The Dependency And Establish A Clean v0.8.0 Baseline ✅
### Goal
Move the repository to PromptKit v0.8.0, resolve source-compatibility issues,
and establish a passing baseline before adopting new behavior.
### Implementation
1. Re-read the upstream v0.6.0, v0.7.0, and v0.8.0 release guides and the
v0.8.0 package consumer and format documentation. Treat the pinned v0.8.0
tag, not the sibling checkout's moving branch, as authoritative.
2. Update `go.mod` and `go.sum` to PromptKit v0.8.0 and run `go mod tidy` with
`GOWORK=off`.
3. Compile before making compatibility edits. Correct only actual source or
behavior incompatibilities. In particular:
- convert any positional `promptkit.Profile` or
`promptkit.OpenAICompatibleProfileConfig` literals to keyed literals;
- confirm Notarius does not register the newly reserved `rakestrawhome`
backend ID; and
- preserve `PrepareExecution`/`Details`/`RunPrepared` snapshot ownership,
`Discard`, session forwarding, reasoning override, profile preflight,
and capacity adaptation.
4. Change `promptKitBuiltinProfileCatalogID` in
`internal/framework/llm/promptkit_profile_fingerprint.go` from the v0.5.0
catalog marker to an opaque v0.8.0 marker. Do not hash PromptKit internal
files or include catalog content in manifests.
5. Update `docs/integrations/pkg-promptkit.md` to pin and link v0.8.0 and to
state that this stage still leaves the production prompt-declared repair
budget at its current value. Do not document later configuration or default
behavior before it exists.
6. Update only those existing tests whose public PromptKit types or stable
v0.8.0 behavior genuinely changed. Do not rewrite tests merely to match
upstream diagnostic wording.
### Tests And Validation
```sh
GOWORK=off go mod tidy -diff
GOWORK=off go test ./internal/framework/llm ./internal/cli
GOWORK=off go test ./...
GOWORK=off go vet ./...
GOWORK=off go build ./cmd/notarius
git diff --check
```
### Acceptance Criteria
- `go list -m gitea.maximumdirect.net/eric/promptkit` reports v0.8.0.
- There is no PromptKit `replace`, active Go workspace dependency, or vendor
tree.
- The adapter still uses one frozen prepared execution and all existing LLM
tests pass.
- Checkpoint profile identity includes the v0.8.0 built-in catalog marker.
- Current integration documentation pins v0.8.0 without claiming that
later stages are already active.
- The full ordinary test suite, vet, and command build pass.
## Stage 2: Verify v0.6.0 Compatibility And Hardening ✅
### Goal
Audit Notarius's assets and boundary values against PromptKit v0.6.0's stricter
source, path, endpoint, JSON, and cancellation contracts, fixing only concrete
incompatibilities.
### Implementation
1. Inspect `internal/framework/llm/asset_registry.go`, prompt/profile source
composition, all registered asset roots, the conventional local backend,
and their focused tests.
2. Exercise every production asset registry through PromptKit engine
construction and the existing production composition tests. Confirm that:
- YAML IDs and versions, not filenames, select definitions;
- every `content_file` path is exact, relative, contained, and points to a
regular embedded file;
- every schema and JSON asset is one complete JSON value;
- every current output contract is valid under v0.8.0; and
- unrelated malformed definitions do not create a second Notarius identity
or fallback mechanism.
3. Review local endpoint parsing and validation. Retain a narrower Notarius
rule only if it has independent application value; otherwise rely on
PromptKit's absolute HTTP/HTTPS URL contract. Never accept a value that the
adapter will later reject.
4. Review conversion of Notarius variables, inputs, profile extras, and debug
values at the adapter boundary for PromptKit's bounded JSON-compatible-value
rules. Do not add a second generic JSON walker or duplicate upstream numeric
limits.
5. Verify cancellation and deadline identity through existing adapter tests.
Add or refine one focused regression only if Notarius currently destroys an
`errors.Is`-relevant context or transport error that the application owns.
6. Do not add a cross-operation schema cache, artifact cache, provider-body
reader, or duplicate JSON framing validation; v0.6.0 owns those mechanisms.
### Tests And Validation
Run the focused asset, profile-source, and adapter packages, then the ordinary
and race-enabled suites:
```sh
GOWORK=off go test ./internal/framework/llm ./internal/cli
GOWORK=off go test ./...
GOWORK=off go test -race ./...
git diff --check
```
Tests must remain offline and should validate Notarius's assembled boundary,
not reproduce PromptKit's internal path, JSON-depth, or response-size matrices.
### Acceptance Criteria
- Every maintained embedded prompt, schema, and fallback profile can be loaded
through the assembled v0.8.0 engine.
- Current local endpoint and JSON-compatible values either satisfy the stricter
upstream contract or fail during preparation with safe diagnostics.
- No duplicate PromptKit-owned cache, JSON, or response-bound mechanism is
introduced.
- Cancellation and deadline behavior remains discoverable at the Notarius
boundary.
- Ordinary and race-enabled tests pass.
## Stage 3: Adopt Profile Inheritance, Rakestrawhome, And Optional Credentials ✅
### Goal
Make the useful PromptKit v0.7.0 profile and backend behavior work through
Notarius's existing generic profile boundary without adding provider-specific
composition logic.
### Implementation
1. Inspect `promptkit_profiles.go`, `asset_registry.go`, profile fingerprinting,
CLI profile preflight, profile provenance recording, and their tests before
editing.
2. Add an offline integration test using a temporary operator profile source
whose leaf uses `base_profile`. Prove that:
- preflight reports the leaf ID;
- the effective backend, model, reasoning, and other inherited values match
the resolved PromptKit target;
- execution uses the same resolved target as inspection; and
- a missing parent or cycle fails before provider generation with a safe
profile-load diagnostic.
Do not duplicate PromptKit's entire field-by-field merge test matrix.
3. Add a checkpoint-safety test showing that changing a parent definition in
an operator profile directory changes Notarius's profile-source fingerprint
while profile content and paths remain absent from the fingerprint value.
Retain the v0.8.0 catalog marker as coverage for built-in-parent changes.
4. Verify `rakestrawhome-gemma-4-31b` through the ordinary profile inspector.
Assert its selected backend reaches Notarius's application-owned inspection
and provenance fields. Use a fake PromptKit client or transport if execution
coverage is needed; never contact the live service or require credentials.
5. Verify that Notarius registers no `rakestrawhome` override and that the
existing `local` registration remains independent.
6. Add one `httptest`-backed adapter integration test for a filesystem profile
with a missing optional `api_key_env`. The request must reach the test server
without an `Authorization` header. Add a focused in-memory PromptKit profile
test for `APIKeyRequired` only if needed to prove Notarius preserves upstream
preflight behavior; do not expose a new operator profile API.
7. Keep `assets/dnd/profiles/dnd-extraction.yaml` standalone and unchanged. No
matching v0.8.0 built-in profile owns its `openai/gpt-5.6-luna` target.
8. Update the current profile-source, deployment, and pinned-integration
sections in `docs/config.md`, `docs/operations.md`,
`docs/internal/llm.md`, and `docs/integrations/pkg-promptkit.md`. Link to the
pinned PromptKit format rules for inheritance. Explain that filesystem
profiles cannot express PromptKit's in-memory `APIKeyRequired` field and
that an optional missing credential may result in a provider 401/403.
### Tests And Validation
```sh
GOWORK=off go test ./internal/framework/llm ./internal/core/config ./internal/cli
GOWORK=off go test ./...
GOWORK=off go test -race ./internal/framework/llm ./internal/cli
git diff --check
```
### Acceptance Criteria
- Inherited operator profiles resolve identically during preflight and
execution, with the leaf ID and effective target kept distinct.
- Parent changes invalidate checkpoint reuse without leaking profile content or
paths.
- Rakestrawhome is available through generic PromptKit profile handling and is
not selected by default or registered by Notarius.
- Missing optional credentials omit authorization and reach the controlled
test provider; explicitly required credentials retain upstream behavior.
- Current documentation accurately describes the implemented profile and
credential behavior without duplicating PromptKit's merge algorithm.
## Stage 4: Adapt Structured Generation Errors Safely ✅
### Goal
Use PromptKit v0.7.0's structured generation errors for stable status
classification and debug-only provider diagnostics without leaking PromptKit
types or sensitive provider text.
### Implementation
1. In `internal/framework/contracts`, add:
- `ErrLLMGeneration` as the provider-neutral generation-failure sentinel;
- an application-owned `LLMGenerationError` with private status and safe
diagnostic fields, `Error`, `Unwrap`, and `StatusCode` methods; and
- a constructor that accepts a nonnegative status and an already-redacted
diagnostic. Status zero means no HTTP status was available.
Ordinary callers may inspect status with `errors.As` and category with
`errors.Is`, but cannot obtain provider code, type, or message from the
error.
2. Add an application-owned `LLMDebugProviderError` with `status_code`, `code`,
`type`, and `message` fields, referenced optionally from
`LLMDebugResponse`. This is debug material, not a manifest or durable public
artifact contract.
3. In `PromptKitClient.CompleteStructured`, preserve precedence in this order:
caller context cancellation/deadline, PromptKit capacity error, structured
PromptKit generation error, then other PromptKit generation failures.
Map every generation failure to `ErrLLMGeneration`; map
`*promptkit.GenerationError` to `LLMGenerationError` with its status.
Never wrap or return the PromptKit error value itself.
4. Keep the ordinary diagnostic limited to PromptKit's safe default error
formatting after bearer and known-credential redaction. Do not append
`ProviderCode`, `ProviderType`, or `ProviderMessage` to it.
5. For an explicitly requested debug path, preserve prepared prompt details and
attach the PromptKit provider code, type, and message after:
- reading only the selected prepared target's `APIKeyEnv`, if any, to obtain
the exact known credential solely for redaction;
- applying `RedactSecrets` and the existing bearer/key-pattern redaction;
- retaining PromptKit's already-normalized bounds; and
- discarding the credential value immediately rather than storing it.
Do not scan unrelated environment variables.
6. Return prompt/debug material alongside the error so the existing debug LLM
wrapper can persist it only when debug recording is enabled. Confirm that
provider fields do not appear in ordinary error text, warnings, manifests,
cache, checkpoint data, or a run without debug output.
7. Refactor error mapping into small helpers if needed to keep
`CompleteStructured` readable; do not create provider-specific policy in
modules or the pipeline runner.
8. Update the error and observability sections of `docs/internal/llm.md` and
`docs/integrations/pkg-promptkit.md`. Keep operator disclosure rules in
`docs/operations.md` concise and link to the internal boundary where useful.
### Tests And Validation
- Use `httptest.Server` to return representative structured 400 and 503
responses. Assert `errors.Is(ErrLLMGeneration)`, `errors.As` to the
application-owned type, and the exact status without asserting complete
human wording.
- Include a provider message containing the selected test credential and a
bearer-shaped value. Verify both are absent from the ordinary error and
debug artifact, while a non-sensitive marker appears only in the requested
debug trace.
- Retain existing capacity and context tests to prove their more specific
classifications still win.
```sh
GOWORK=off go test ./internal/framework/contracts ./internal/framework/llm ./internal/framework/pipeline ./internal/cli
GOWORK=off go test ./...
GOWORK=off go test -race ./internal/framework/llm ./internal/framework/pipeline
git diff --check
```
### Acceptance Criteria
- PromptKit generation errors never escape the adapter error chain.
- All generation failures match `ErrLLMGeneration`; structured non-success
responses expose only application-owned HTTP status to ordinary callers.
- Provider code, type, and message are available only in an explicitly
requested, redacted debug trace.
- Capacity and context classifications remain unchanged and more specific.
- Security tests prove selected credentials and bearer tokens are not leaked.
## Stage 5: Add Adapter-Level Structured Repair Support ✅
### Goal
Teach the transport-neutral completion boundary and PromptKit adapter to apply
an optional repair override and report actual repair behavior, without yet
exposing the setting in pipeline configuration.
### Implementation
1. Add `StructuredOutputRepairAttempts *int` to
`contracts.StructuredCompletionRequest`. Copy the pointed-to value wherever
requests are cloned or retained.
2. Add `RepairAttempts int` to `contracts.StructuredCompletionResponse`. It is
the actual number of corrective generation calls, not the configured budget
and not the number of total candidates.
3. Validate a non-nil request value as zero through three at the adapter
boundary so programmatic callers cannot bypass later file/config validation.
4. When the request value is nil, leave `promptkit.RunRequest.Validation` nil
so the prompt's complete contract remains authoritative.
5. When the value is non-nil:
- call `Engine.InspectPrompt(ctx, promptID, promptVersion)`;
- copy `PromptInspection.OutputContract` by value;
- replace only `RepairAttempts`;
- pass the complete copied contract as `RunRequest.Validation`; and
- prepare and execute exactly as before.
Do not infer or hard-code schema paths, validation modes, or formats. Do not
cache inspection in this stage.
6. Map `result.Validation.RepairAttempts` to the response and leave
`result.Usage` cumulative values unchanged. The existing debug validation
object and prepared output contract should show actual and configured values
respectively.
7. Preserve result semantics:
- valid initial and repaired candidates decode normally;
- repair exhaustion returns the final raw candidate/debug material with an
error matching `ErrInvalidStructuredOutput`;
- explicit empty or whitespace-only content follows PromptKit validation;
- missing/null/non-string content remains a generation/provider failure;
- corrective-call generation errors use Stage 4's application-owned mapping;
and
- context cancellation wins at every error boundary.
8. Keep `CompleteStructured` and its helpers provider neutral outside this
adapter package. Do not expose PromptKit validation or inspection types.
9. Update only the adapter-owned repair behavior in `docs/internal/llm.md` and
`docs/integrations/pkg-promptkit.md`. State that public pipeline configuration
and the production default are added by later stages of this plan.
### Tests And Validation
Add adapter-level behavioral tests using a deterministic fake PromptKit LLM:
- nil override uses the prompt declaration;
- explicit zero overrides a positive prompt declaration without dropping its
JSON Schema contract;
- explicit one turns an invalid first candidate followed by a valid candidate
into one successful response with the final raw bytes, actual repair count
one, and cumulative usage;
- repair exhaustion returns the final candidate and validation diagnostics as
`ErrInvalidStructuredOutput`;
- explicit empty content is eligible for repair;
- a corrective generation failure maps through Stage 4; and
- invalid direct values below zero or above three fail before provider work.
Do not assert PromptKit's exact assistant/user correction prose or copy its
full internal repair matrix.
```sh
GOWORK=off go test ./internal/framework/contracts ./internal/framework/llm ./internal/framework/pipeline
GOWORK=off go test -race ./internal/framework/llm ./internal/framework/pipeline
GOWORK=off go test ./...
git diff --check
```
### Acceptance Criteria
- The adapter changes only repair count when applying a request override.
- Nil and explicit zero remain distinct.
- Repaired success returns final raw output, cumulative usage, and actual count
without a warning.
- Exhaustion, empty content, corrective generation failure, and cancellation
match the target semantics.
- No PromptKit type crosses the LLM package boundary.
## Stage 6: Propagate Repair Policy Through Framework Requests ✅
### Goal
Carry an optional effective repair budget from each resolved stage or validator
binding to its module request without changing public file configuration yet.
### Implementation
1. Add `StructuredOutputRepairAttempts *int` alongside `LLMProfile` to every
stage request that can belong to an LLM-backed binding:
- `ParseRequest`;
- `ChunkRequest`;
- `TypedExtractionRequest`;
- `TypedMergeRequest`;
- `TypedNormalizeRequest`;
- `OutputRequest`;
- `TypedValidationRequest`;
- `ChunkValidationRequest`; and
- `SerializedValidationRequest`.
2. Add the same optional field to the erased/internal request carriers used by
registry builders, preparation, runner stage attempts, validator targets,
retry closures, and debug wrappers. Copy pointer values; never share a
mutable pointer owned by configuration.
3. At every runner stage invocation, obtain the value from the exact resolved
producer binding. At every validator invocation, obtain it from that exact
resolved validator binding. Do not use the producer's value for a validator
or vice versa.
4. Ensure all retry attempts for the same binding receive the same effective
structural-repair value. Do not decrement it in Notarius; PromptKit owns the
inner budget independently on each `CompleteStructured` call.
5. Extend `semanticreconcile.Request` with the optional field and carry it into
each generic reconciliation completion. A batched reconciliation may make
several completion calls; each call receives the same effective budget.
6. Update registry erasure/adaptation code for typed merge, normalize, and
validation requests so no field is lost. Preserve input/output support even
though current production input and output modules are deterministic.
7. Add focused framework tests for one chunk producer, one extraction
producer, one normalizer, and one LLM-backed validator. Verify exact pointer
value propagation and separation between producer and validator settings.
Do not add repetitive tests for every generic adapter.
### Tests And Validation
```sh
GOWORK=off go test ./internal/framework/contracts ./internal/framework/pipeline ./internal/framework/semanticreconcile
GOWORK=off go test -race ./internal/framework/pipeline ./internal/framework/semanticreconcile
GOWORK=off go test ./...
git diff --check
```
### Acceptance Criteria
- Every stage and validator request can carry a detached optional repair value.
- The runner sources the value from the exact resolved binding.
- Producer and validator values cannot overwrite one another.
- Stage retries reuse but do not mutate or consume the inner repair budget.
- Semantic reconciliation forwards the budget to every one of its completion
calls.
- Existing behavior remains unchanged while all values are nil.
## Stage 7: Forward Repair Policy From Every LLM-Backed Module ✅
### Goal
Complete the internal end-to-end path by having every production LLM-backed
module forward its stage request value to `CompleteStructured`.
### Implementation
1. Inventory every production `CompleteStructured` call with code search before
editing. The expected current owners include:
- `dnd/scenes` chunking;
- the combat-turn, enemy-event, item-occurrence, item-registry,
location-occurrence, location-registry, NPC-occurrence, NPC-registry,
scene-description, and spell extractors; and
- generic semantic reconciliation used by the item, location, and NPC
registry normalizers.
Reconcile this list with the actual repository; do not omit a newly added
production caller merely because it is not named here.
2. In each direct caller, set
`StructuredCompletionRequest.StructuredOutputRepairAttempts` from the
corresponding stage request. Clone the pointer or use a small shared helper
if that reduces repeated ownership mistakes without moving domain logic.
3. Ensure D&D registry normalizers pass their typed normalize request value into
`semanticreconcile.Request`, and that the generic engine forwards it as
established in Stage 6.
4. Update existing module prompt-mapping tests that already inspect a captured
structured-completion request to assert the new field. Do not create a new
one-test-per-module suite solely to memorialize field plumbing; rely on the
existing request-contract tests plus a final complete call-site audit.
5. Search again after editing for production `CompleteStructured` calls and
verify each either forwards the field or documents why it cannot receive a
pipeline binding. Test-only fakes need only preserve the field when their
contract test depends on it.
6. Do not set a module-specific fallback value. Nil must reach the adapter so
the prompt declaration remains authoritative.
### Tests And Validation
Run focused D&D and semantic-reconciliation packages, then the full suite:
```sh
GOWORK=off go test ./internal/modules/dnd/... ./internal/framework/semanticreconcile
GOWORK=off go test -race ./internal/modules/dnd/... ./internal/framework/semanticreconcile
GOWORK=off go test ./...
git diff --check
```
### Acceptance Criteria
- Every production LLM-backed completion receives the exact stage or validator
repair value.
- No module invents a default or imports PromptKit.
- Registry normalizers preserve the value through semantic reconciliation.
- Existing request-contract tests remain concise and pass.
- A final call-site audit finds no silent production omission.
## Stage 8: Add The Public Repair Configuration Contract ✅
### Goal
Add presence-aware pipeline and binding configuration for
`structured_output_repair_attempts` without yet changing runtime resolution.
### Implementation
1. Add `StructuredOutputRepairAttempts *int` to
`pipeline.PipelineProfile` and `pipeline.ModuleBinding`, using
`json:"structured_output_repair_attempts,omitempty"`.
2. Add presence-aware YAML support at pipeline and object-binding scope:
- exact key `structured_output_repair_attempts`;
- integer values zero through three;
- explicit null, non-integer, and out-of-range values rejected with scoped
diagnostics; and
- scalar shorthand bindings continue to omit the binding override.
Preserve file configuration version 4.
3. Update every configuration clone, conversion, redaction, summary, and JSON
round-trip carrier. Copy pointers by value into newly allocated storage so
parsed, configured, and redacted values do not alias.
4. Preserve omission versus explicit zero through YAML parsing, profile
inheritance, module-binding object form, and JSON round trips. Keep scalar
shorthand bindings equivalent to omission.
5. Do not add a top-level `promptkit.repair_attempts` setting or CLI override.
6. Add concise parser and ownership tests. Defer execution-class checks,
effective precedence, resolved digests, and runtime forwarding to Stage 9,
where module metadata is available.
### Tests And Validation
At the parser/config boundary, test omitted, explicit zero, positive bounds,
negative, above-three, null, non-integer, scalar shorthand, cloning, redaction,
and JSON round-trip behavior. Use relational boundary tests for the allowed
range and avoid duplicating the same cases at every layer.
```sh
GOWORK=off go test ./internal/core/config ./internal/cli
GOWORK=off go test -race ./internal/core/config
GOWORK=off go test ./...
go run ./cmd/notarius config validate \
--config examples/dnd-minimal.config.yml \
--pipeline dnd-session
go run ./cmd/notarius config validate \
--config examples/dnd-complete.config.yml \
--pipeline dnd-session
git diff --check
```
### Acceptance Criteria
- The exact public field parses at pipeline and object-binding scope with the
fixed range.
- Nil and explicit zero remain distinguishable through parsing, cloning,
inheritance, redaction, summaries, and round trips.
- Both maintained configurations remain valid without requiring the new field.
- No runtime or prompt default has changed prematurely.
## Stage 9: Resolve And Apply Repair Configuration ✅
### Goal
Resolve the public field against module execution classes, incorporate the
effective value into pipeline identity, and connect it to the request plumbing
completed in Stages 6 and 7.
### Implementation
1. During resolution, compute the effective value for every selected binding:
- explicit binding value wins;
- otherwise an explicit pipeline value applies to an LLM-backed binding;
- otherwise leave nil for prompt-owned policy.
Apply the pipeline value to LLM-backed validators as well as producers.
2. Reject an explicit binding value on a deterministic module or deterministic
validator using the same execution-class knowledge used for `llm_profile`.
Do not reject a pipeline-level value merely because a selected pipeline also
contains deterministic bindings; simply do not apply it to those bindings.
3. Clone every resolved pointer so the parsed pipeline, resolved profile,
redacted summaries, and runner requests have distinct ownership.
4. Include the effective field in resolved pipeline JSON and digest input. A
change between nil, zero, and a positive value must change the resolved
digest when it changes an LLM-backed selected binding. Unselected lanes must
retain the repository's existing digest and selection semantics.
5. Pass the resolved value into the Stage 6 request field for every selected
input, chunk, extract, merge, normalize, output, and validator binding.
6. Update `docs/config.md` as the canonical field, range, and precedence
contract; `docs/internal/pipeline.md` as the resolution owner; and
`docs/operations.md` for the distinction from binding `retries`. The
prompt-owned production default remains unchanged until Stage 10.
7. Add focused resolution and runner tests. Cover representative execution
classes rather than repeating the same assertion for every module type.
### Tests And Validation
Test:
- binding over pipeline over nil precedence;
- inheritance into each selected LLM-backed stage and validator;
- no inheritance into deterministic bindings;
- explicit deterministic-binding rejection;
- detached pointers;
- runner forwarding for representative producer and validator bindings; and
- digest changes for execution-relevant nil, zero, and positive changes.
```sh
GOWORK=off go test ./internal/framework/pipeline ./internal/cli
GOWORK=off go test -race ./internal/framework/pipeline
GOWORK=off go test ./...
go run ./cmd/notarius config validate \
--config examples/dnd-minimal.config.yml \
--pipeline dnd-session
go run ./cmd/notarius config validate \
--config examples/dnd-complete.config.yml \
--pipeline dnd-session
git diff --check
```
### Acceptance Criteria
- The exact public field has the fixed binding-over-pipeline-over-prompt
precedence for every selected LLM-backed producer and validator.
- Deterministic binding misuse fails during resolution before execution, while
a pipeline value coexists with deterministic bindings.
- Nil and explicit zero remain distinguishable through resolution, runtime,
summaries, and digests.
- A policy change invalidates checkpoint identity when it changes an effective
selected binding.
- Current configuration, pipeline, and operations documentation matches the
implemented behavior.
## Stage 10: Enable The Default, Finish Documentation, And Verify The Feature ✅
### Goal
Set the accepted production default of one repair, reconcile all canonical
documentation, and run the full repository verification pass.
### Implementation
1. Change `repair_attempts: 0` to `repair_attempts: 1` in every maintained
production prompt manifest that produces structured output, including the
generic semantic-reconciliation prompt and every D&D chunk, extraction, and
registry-normalization prompt. Do not mechanically change unrelated test
fixtures whose purpose is to exercise zero.
2. Inspect every production prompt output contract after the edit. Confirm that
each positive budget uses `basic`, `json`, or `json_schema`, remains no
greater than three, and retains its existing format and schema path.
3. Add or refine the smallest durable assembled-assets test that proves the
production engine can prepare the maintained prompts with the activated
contracts. Do not add a brittle test that asserts an exact prompt count,
file count, message prose, correction text, or asset length. The public
default may be tested at one canonical assembled boundary because its
literal value is an operational contract.
4. Confirm a successful repair does not create a warning and that exhausted
repair remains `ErrInvalidStructuredOutput`. Verify the debug prompt records
the configured contract, the debug response records actual repair count,
and cumulative usage is not double-counted.
5. Confirm scheduling behavior with one focused test or existing coverage: the
Notarius scheduled client admits one logical `CompleteStructured` operation
while PromptKit may make serial corrective provider calls inside it. Do not
attempt to reacquire a Notarius permit from inside PromptKit or add a second
scheduler.
6. Reconcile current-state documentation:
- `docs/integrations/pkg-promptkit.md` owns the pinned upstream boundary;
- `docs/config.md` owns field names, range, default, and precedence;
- `docs/operations.md` owns latency/cost, optional credentials, concurrency,
timeout, and the upper-bound formula;
- `docs/internal/llm.md` owns inspection-based contract replacement,
cumulative usage, actual repair count, generation errors, and debug data;
- `docs/internal/pipeline.md` owns effective policy propagation and the
separation from stage retries; and
- `docs/policy/architecture.md` adds only the durable rule that PromptKit
owns deterministic structural repair inside one completion while Notarius
owns stage attempts and semantic validation.
7. Remove current-behavior claims that PromptKit is v0.5.0, that every
production repair budget is zero, or that PromptKit is always single-pass.
Do not alter historical release notes or archived roadmaps.
8. Keep the maintained minimal and complete examples secret-free and valid.
They may omit the new field to demonstrate the default; do not add a
redundant complete profile or Rakestrawhome example merely to exercise an
upstream catalog entry.
9. Review `docs/roadmap/future.md` only for consistency. Leave the future
feedback-aware stage retry, combat-scene validator, and warning-reform work
unimplemented and clearly separate.
### Tests And Validation
Run focused tests first, then all repository checks:
```sh
GOWORK=off go test ./internal/framework/llm ./internal/framework/pipeline ./internal/framework/semanticreconcile ./internal/modules/dnd/...
GOWORK=off go test ./...
GOWORK=off go test -race ./...
GOWORK=off go vet ./...
GOWORK=off go build ./cmd/notarius
GOWORK=off go mod tidy -diff
go run ./cmd/notarius config validate \
--config examples/dnd-minimal.config.yml \
--pipeline dnd-session
go run ./cmd/notarius config validate \
--config examples/dnd-complete.config.yml \
--pipeline dnd-session
git diff --check
```
Also perform focused repository searches that exclude `docs/roadmap/archive/`
and historical release notes:
- no active v0.5.0 PromptKit pins or links remain;
- no maintained production prompt still declares `repair_attempts: 0`;
- every production `CompleteStructured` caller forwards the repair field; and
- no provider code, type, or message is added to ordinary errors, warnings,
manifests, cache, or checkpoint schemas.
If the repository's source-release checker is available and the ordinary
checks above pass, run `./scripts/check-release-source.sh v0.0.0` as the final
integrated validation. It must not create a tag, release note, or repository
artifact.
### Acceptance Criteria
- Every maintained structured prompt defaults to one corrective call and can
be overridden to zero through three at pipeline or binding scope.
- A real assembled Notarius completion follows the PromptKit v0.8.0 repair
contract without changing prompt schema identity or cacheable prefix.
- Actual repair count, cumulative usage, error classification, debug-only
provider diagnostics, scheduling, and checkpoint identity match the feature
roadmap.
- Profile inheritance, Rakestrawhome availability, optional credentials, and
v0.6.0 hardening remain covered and documented.
- All canonical documentation describes implemented v0.8.0 behavior in its
assigned home and leaves future semantic validation work in the roadmap.
- Maintained examples validate, all ordinary/race/vet/build/module checks pass,
and the worktree contains no generated or sensitive artifacts.