34 KiB
Feedback-Aware Validation Retry Implementation Plan
Status
Ready for implementation. This plan implements the target state defined by Feedback-Aware Stage Validation Retries. Its stages are ordered dependencies, and each stage is intentionally scoped for one gpt-5.6-terra implementation prompt.
The feature roadmap owns product intent and durable policy. This document owns implementation order, concrete boundaries, and stage-level verification. If a conflict is discovered, preserve the feature roadmap and the repository policies, stop the affected stage, and revise this plan rather than silently choosing a different architecture.
Settled Decisions
The implementation agent must treat these decisions as fixed:
- PromptKit v0.9.0 is the minimum and exact supported PromptKit release for
this work. Notarius uses
RunRequest.AppendedMessages; it does not create paired correction manifests or bypass PromptKit preparation and execution. - A correction request reconstructs the ordinary initial request and appends
exactly two messages: the latest exact producer response as
assistant, followed by one deterministic aggregate correction request asuser. Earlier correction turns never accumulate. - A correction-capable producer supplies the exact single LLM response that directly controlled the candidate. The initial protocol does not synthesize a model-facing projection of a compound artifact.
- The only supported producer correction protocol is
single_response_v1. An empty protocol means unsupported. Configuration rejects semantic-retry workflows for LLM-backed compound producers or any other producer unable to satisfysingle_response_v1. - The existing producer binding
retriesvalue is the sole outer stage budget. PromptKit repair attempts and validator execution retries are independent budgets. - Terminal policy is configured as pipeline defaults with field-by-field producer-binding overrides. Validators never own candidate disposition.
- Application defaults are
fail_runfor exhausted producer structural failure,fail_runfor exhausted semantic rejection, andwarn_continuefor exhausted validator execution failure. - All applicable validators run sequentially in configured order. Rejections and validator failures are aggregated; failures are not represented to the producer as candidate defects.
- Deterministic producers do not consume retries after rejection. An LLM-backed attempt that took a deterministic fast path and produced no model response is likewise not correctable for that candidate.
- Known rejected or structurally invalid output never advances. A candidate
accepted under
validator_failure: warn_continueadvances with explicit incomplete-validation provenance but is not checkpointed. - The D&D combat-scene validator and broader warning-system redesign remain out of scope.
Cross-Stage Implementation Rules
Apply these rules in every stage:
- Read the feature roadmap,
docs/development.md, alldocs/policy/documents, and the focused current documentation named by the stage before editing. - Inspect current code and tests rather than assuming paths or private helper layouts. Preserve unrelated work and existing public ordering, cancellation, scheduler, checkpoint, and sensitive-data invariants.
- Use transport-neutral types outside
internal/framework/llm. PromptKit types must not escape the adapter boundary. - Keep correction material attempt-local. Do not put raw assistant responses or correction text in ordinary errors, warnings, manifests, receipts, checkpoints, caches, or default debug summaries.
- Add the smallest durable tests that protect the stage's contracts. Do not duplicate PromptKit's own prepared-execution, copying, hashing, capacity, role-validation, or structural-repair test matrix.
- Use
gofmton changed Go files. Run the focused checks listed for the stage and fix failures before stopping. Do not proceed into the next stage in the same prompt. - Do not delete this plan or the feature roadmap during implementation. They are retired only after a final review confirms the complete target state.
Contract Names And Bounds
Use one vocabulary consistently across packages. Exact private helper names may follow local conventions, but the following contracts and values are not open for redesign:
CorrectionProtocolhas only the empty unsupported value andsingle_response_v1.- An application-owned
SemanticCorrectioncarries ownedAssistantResponse []byteandUserGuidance stringvalues. It is exposed on chunk, typed extract, typed merge, typed normalize, and structured completion requests as an optional pointer. - An application-owned
ModelCandidatecarries the owned exact response bytes andCorrectionProtocol. It is exposed on the corresponding producer results as an optional pointer and is never serialized as part of a durable artifact. ValidationResultgains optionalCorrectionGuidance. Operator-facingMessageis not copied into this field implicitly.- Each reason code is valid UTF-8, nonblank after trimming, and at most 128 bytes. Each validator-supplied correction guidance value is valid UTF-8, nonblank after trimming, and at most 4 KiB.
- The exact defective assistant response may be at most 1 MiB. The aggregate user correction message may be at most 64 KiB. The combined appended content may therefore be at most 1,114,112 bytes. Reject invalid UTF-8, empty/whitespace-only content, or over-limit content; never truncate it.
- Invalid producer-owned correction material is a framework contract error.
It is not a validator failure and cannot be converted to
reject_outputorwarn_continue. - Aggregate correction entries are ordered by validator order, contain the
stable reason code plus guidance, and collapse only byte-identical duplicate
(reason_code, correction_guidance)pairs while retaining first occurrence. A rejection without guidance receives bounded generic guidance based only on its reason code. - Terminal actions are closed enums: structural failure and semantic
rejection accept
fail_runorreject_output; validator failure acceptswarn_continueorfail_run.
Stage 1 — Adopt PromptKit v0.9.0
Goal
Upgrade the dependency without changing Notarius stage behavior, prove the existing integration remains compatible, and make the new upstream request primitive available to later stages.
Work
- Update
go.modandgo.sumto PromptKit v0.9.0 withgo getandgo mod tidy. Accept the catalog modules selected transitively by PromptKit; do not import or register them directly. - Review every Notarius
promptkit.RunRequestliteral and retain keyed form. Verify production prompt roles are limited tosystemanduser. - Update the conservative built-in-profile checkpoint marker in
internal/framework/llm/promptkit_profile_fingerprint.gofrom v0.8.0 to v0.9.0. Do not duplicate the separately versioned catalog module versions. - Update
docs/integrations/pkg-promptkit.mdonly to describe the newly implemented dependency version and unchanged current integration boundary. Include the strict role vocabulary and external catalog ownership, but do not document Notarius correction behavior yet.
Verification
- Run
go test ./internal/framework/llm ./internal/cli. - Run
go test ./...,go test -race ./internal/framework/llm,go vet ./..., andgo build ./cmd/notarius.
Exit Criteria
Notarius builds and passes its suite on PromptKit v0.9.0, its compatibility document and checkpoint marker name v0.9.0, and no Notarius package directly depends on either upstream catalog module. This stage is one Terra prompt.
Stage 2 — Record The Architecture And Add Transport-Neutral Contracts
Goal
Record the durable decision and introduce owned correction/candidate types without changing runtime retry behavior.
Work
- Add ADR-0014 under
docs/adr/using the repository ADR format. Record the three distinct retry budgets, complete validator aggregation, the fresh two-message correction protocol, exact-response producer capability, non-recursive validator failure handling, terminal-policy ownership and defaults, checkpoint conservatism, and sensitive-data constraints. - Add
CorrectionProtocol,SemanticCorrection, andModelCandidateto the domain-neutral framework contracts, with constructors/clone helpers that validate the settled bounds and defensively copy bytes. - Add optional correction input to
ChunkRequest,TypedExtractionRequest,TypedMergeRequest, andTypedNormalizeRequest. - Add optional model-candidate output to
ChunkPlanResult,TypedExtractionResult,TypedMergeResult, andTypedNormalizeResult. - Add
CorrectionGuidancetoValidationResultand centralize validation of reason codes and corrective guidance at the framework boundary. - Update type-erasure adapters, stored fakes, and clone paths so the new values retain ownership and cannot alias caller buffers. Do not declare production module capabilities or use the fields in the runner yet.
Verification
- Add contract-level tests for accepted values, UTF-8 and size rejection, defensive copying, nil behavior, and typed-erasure preservation.
- Run
go test ./internal/framework/contracts ./internal/framework/pipeline. - Run
go test ./....
Exit Criteria
The generic contracts can safely carry correction input and exact candidate material across every producer stage, no PromptKit type crosses the boundary, and existing runtime behavior is unchanged. This stage is one Terra prompt.
Stage 3 — Add Validation Policy Configuration And Resolution
Goal
Implement strict configuration, inheritance, identity, and provenance for terminal validation policy before the runner consumes it.
Work
- Add optional
validation_policyto a pipeline profile and to chunk, extract, merge, and normalize producer bindings. Use pointer-backed override fields so omission is distinguishable from an explicit value. - Reject an explicitly null policy object, null policy fields, duplicate or unknown fields, and values outside the settled enums. Keep the current file configuration version.
- Reject
validation_policyon input, output, and validator bindings. Reject a binding-level explicitproducer_structural_failurevalue on a deterministic producer; a pipeline-level structural default remains valid because a pipeline may contain LLM-backed producers. - Resolve each policy field independently in binding, pipeline, application default order. Store one detached concrete effective policy for chunk and for each lane's extract, merge, and normalize stage; do not leave runtime inheritance to the runner.
- Include both configured overrides and effective policies at their existing appropriate configuration/provenance boundaries. Include the effective values in resolved-pipeline digest and checkpoint identity. Preserve deterministic serialization and redaction.
- Update
examples/dnd-minimal/config.ymlandexamples/dnd-complete/config.ymlonly if their current shape must change to remain valid; do not add redundant explicit defaults merely to demonstrate the feature.
Verification
- Add focused parsing tests for omission versus null, unknown/duplicate fields, invalid placements, and invalid enums.
- Add resolution tests for field-by-field inheritance and mixed overrides.
- Add digest, clone, JSON/YAML round-trip, and redacted-effective-config tests.
- Run
go test ./internal/core/config ./internal/framework/pipeline ./internal/cli.
Exit Criteria
Every producer stage receives one immutable effective policy, invalid configuration fails before execution, and policy changes invalidate resolved identity. The runner still follows its old behavior until later stages. This stage is one Terra prompt.
Stage 4 — Declare And Validate Producer Correction Capability
Goal
Make correction support an explicit module contract and prepare the framework to reject impossible workflows.
Work
- Extend
pipeline.ModuleSpecwithCorrectionProtocol; normalize, clone, validate, and include it in relevant registry/spec fingerprints. - Permit
single_response_v1only for LLM-backed chunk, extract, merge, or normalize modules. Reject it for deterministic, input, output, or validator specs. Empty remains the default unsupported value. - Carry the selected protocol into resolved and prepared producer metadata and checkpoint fingerprints.
- Reject positive
retrieson deterministic validator bindings during resolution. LLM-backed validator retries remain valid and independent of producer capability.
Verification
- Add spec normalization/clone tests, invalid stage/class combinations, validator retry-class tests, and preparation tests for supported and unsupported producer workflows.
- Run
go test ./internal/framework/pipeline ./internal/core/config.
Exit Criteria
Capability is explicit, fingerprinted, and testable, while existing production pipelines remain executable pending their migrations. No dormant feature gate or unused preparation check is introduced. This stage is one Terra prompt.
Stage 5 — Adapt Corrections Through PromptKit
Goal
Map the transport-neutral correction contract onto PromptKit v0.9.0 without changing ordinary requests.
Work
- Add optional
SemanticCorrectiontoStructuredCompletionRequestand its debug-safe request representation. - In
PromptKitClient.CompleteStructured, validate and defensively copy the correction, then map it to exactly twopromptkit.RenderedMessagevalues inRunRequest.AppendedMessages, usingpromptkit.RoleAssistantfollowed bypromptkit.RoleUser. - Leave
AppendedMessagesnil for an ordinary request. Do not allow callers to choose other roles through the Notarius contract. - Ensure request formatting and default debug summaries expose only safe counts/digests. The existing explicitly requested detailed PromptKit trace may contain complete effective messages and must retain its existing sensitive-data treatment.
- Preserve the same prompt identity, inputs, variables, session, profile, execution overrides, repair attempts, scheduler, and prepared-execution path for corrected calls.
Verification
- Add focused adapter tests proving an ordinary request is unchanged and a corrected request has the ordinary rendered prefix plus exactly the two supplied messages in order with exact content.
- Test invalid/oversized correction rejection before preparation and absence of raw content from ordinary formatting/errors.
- Add one representative test showing PromptKit structural repair remains available on a request with appended messages; do not reproduce upstream's complete repair suite.
- Run
go test ./internal/framework/llm ./internal/framework/pipelineandgo test -race ./internal/framework/llm.
Exit Criteria
Notarius has one safe, tested adapter path for fresh corrected requests, and no stage invokes it yet. This stage is one Terra prompt.
Stage 6 — Migrate The Scene Chunker And Foundational Extractors
Goal
Make the scene chunker and the foundational direct D&D extractors satisfy
single_response_v1.
Work
- Update
dnd/scenes,dnd/npc-registry,dnd/item-registry,dnd/location-registry,dnd/scene-descriptions, anddnd/spellsproducer implementations to pass request correction to their structured completion and return an owned copy of the successful response's exact validated raw bytes asModelCandidate. - Declare
single_response_v1in each corresponding module spec. - Do not serialize normalized artifacts to fabricate candidate material. Keep artifact parsing, deterministic identity attachment, evidence validation, warnings, and durable schemas unchanged.
- Update shared D&D extraction helpers only where the behavior is genuinely common; keep domain prompt ownership in each module.
Verification
- Use representative package-level tests to prove exact-response propagation, correction forwarding, and input/result ownership. Update spec tests for the declared protocol without multiplying the same behavior test across all six modules.
- Run the affected module tests and
go test ./internal/modules/dnd/....
Exit Criteria
All named producers truthfully advertise and implement the exact-response protocol, with unchanged ordinary extraction behavior. This stage is one Terra prompt.
Stage 7 — Migrate Downstream D&D Extractors
Goal
Complete single_response_v1 support for the remaining direct D&D extraction
producers.
Work
- Migrate
dnd/npc-occurrences,dnd/item-occurrences,dnd/location-occurrences,dnd/combat-turns, anddnd/enemy-eventsusing the same contract as Stage 6. - Preserve their contextual-name and deterministic-ID rules. Correction messages must never introduce opaque registry IDs or ask the model to copy them.
- Preserve combat-scene gating, reference projections, and prompt-cache prefix ordering. The correction pair is appended after the complete existing request and never changes stable prompt assets.
Verification
- Add or adapt focused tests for correction forwarding and exact raw-response retention at the shared boundary, plus one registry-grounded extractor case proving no opaque IDs enter correction material.
- Run the affected module tests,
go test ./internal/modules/dnd/..., and the assembled D&D CLI contract tests.
Exit Criteria
Every direct production D&D LLM extractor implements the same correction protocol without changing artifact semantics. This stage is one Terra prompt.
Stage 8 — Migrate Semantic Reconciliation Normalizers
Goal
Support correction for the current single-proposal reconciliation path and activate capability enforcement for all eligible production producers.
Work
- Extend
semanticreconcile.Requestto acceptSemanticCorrectionand its result to expose the exact validated proposal response as ownedModelCandidatewhen an LLM call actually occurred. - Forward correction through the shared engine's ordinary structured completion request. Preserve request-local integer candidate handles, proposal validation, typed application policies, and fallback behavior.
- Update the NPC-, item-, and location-registry normalizers to carry the exact
proposal material through their typed result and declare
single_response_v1. - Mark deterministic skip/limit/fallback outcomes as having no model candidate. If such a candidate is later rejected, it is not correctable and must not consume a retry merely because its module execution class is LLM-backed.
- Audit all production LLM-backed producer specs. Add the preparation rule that
an LLM-backed producer with a non-empty validator chain and
retries > 0must declaresingle_response_v1. Any genuine multi-response producer remains unsupported and a configured validator-backed semantic retry for it must fail preparation.
Verification
- Add engine tests for initial and corrected requests, exact proposal bytes, deterministic no-call outcomes, invalid proposal outcomes, and ownership.
- Add representative normalizer and preparation tests for supported, unsupported, and no-validator/no-retry configurations.
- Run
go test ./internal/framework/semanticreconcile ./internal/framework/pipeline ./internal/modules/dnd/... ./internal/cli.
Exit Criteria
All eligible production LLM producers implement the declared protocol, unsupported workflows fail before source parsing, and ordinary operational retries remain allowed when semantic correction cannot occur. This stage is one Terra prompt.
Stage 9 — Build Complete Validator-Chain Execution
Goal
Replace first-result validation with one reusable, deterministic executor that aggregates the complete chain and isolates validator retries.
Work
- Introduce an internal immutable
validationReportmodel with one ordered invocation record per validator: approved, rejected, failed, or skipped; validator attempt count; safe reason/message metadata; warnings; and bounded correction guidance. - Implement one shared sequential executor around stage-specific invocation closures. It must continue after rejection and isolated execution failure, retry an LLM-backed failed validator against the same immutable candidate up to its binding budget, and stop retrying after a contract-valid approval or rejection.
- Reconstruct the same ordinary validator request for each validator retry. Do not append feedback to validator prompts and do not regenerate the producer candidate.
- Preserve only one final warning for an exhausted validator failure; retain individual attempt details for debug. Keep warnings from completed validators in configured order.
- Build aggregate correction guidance from all rejections using the settled ordering, deduplication, generic fallback, and bounds. Validator failures, skips, warnings, and operator messages must not enter it.
- Retain
skippedas a framework-owned outcome for an otherwise selected validator whose runtime prerequisites are unavailable. Module construction, type incompatibility, candidate cloning failure, cancellation, and debug persistence failure remain framework errors rather than skips.
Verification
- Add table-driven behavior tests for all approved, multiple rejections, rejection plus failure, failure only, skipped, retry success, retry exhaustion, immutable candidate reuse, deterministic ordering, duplicate guidance, and bounds.
- Prove deterministic validators cannot receive positive retry budgets and LLM validator retry counts do not consume producer attempts.
- Run
go test ./internal/framework/pipelineandgo test -race ./internal/framework/pipeline.
Exit Criteria
The shared executor returns a complete ordered report without deciding whether the producer candidate advances. Existing stage callers may still adapt the report through their old disposition path until the next stages. This stage is one Terra prompt.
Stage 10 — Implement The Generic Producer Attempt State Machine
Goal
Replace the boolean retry helper with one explicit domain-neutral state machine used by later stage integrations.
Work
- Refactor
runWithRetryinto an attempt engine that accepts producer and complete-validation closures and returns a terminal result plus ordered attempt provenance. Keep stage-specific artifact handling outside it. - Classify attempts as initial, operational-error retry, structural retry,
module-requested retry, or semantic correction. Preserve one total producer
budget of
retries + 1attempts. - On semantic rejection, correct only when another attempt exists and the
current candidate contains valid
single_response_v1material. Construct a freshSemanticCorrectionfrom that latest response and aggregate feedback. - Apply outcome precedence and effective terminal policy exactly as the
feature roadmap specifies. Ordinary producer errors remain framework errors
after retries. Only
ErrInvalidStructuredOutputuses the structural-failure policy;reject_outputrecords no usable artifact. - A deterministic or no-model candidate applies semantic terminal policy immediately without spending an ineffective retry.
- With rejection plus validator failure, use rejection guidance for a correction while recording incomplete validation. With failure only, apply validator-failure policy without regenerating the producer.
- Keep cancellation and debug-persistence failures immediately terminal. Abandoned-attempt warnings must not be promoted.
Verification
- Test the state machine through stable behavior with fake closures: budget accounting, fresh correction history, latest-response replacement, all terminal actions, failure precedence, deterministic candidates, cancellation, and warning promotion.
- Do not assert private helper call choreography or exact correction prose.
- Run
go test ./internal/framework/pipelineand its race-enabled suite.
Exit Criteria
One tested state machine owns attempt budgets and disposition, but no stage is partially migrated. This stage is one Terra prompt.
Stage 11 — Integrate Chunk Validation And Correction
Goal
Move chunk generation, validation, cache reuse, and retry disposition onto the new state machine.
Work
- Adapt chunk candidate materialization and both chunk/serialized validator targets to the complete-chain executor.
- Pass correction only to a generated chunker attempt and retain the exact raw chunker response associated with the materialized plan.
- Preserve chunk-plan cache identity and mode semantics. A rejected automatic cache hit is unusable for the invocation and falls through to generated attempt one without pretending the cache hit has model material. Publish a newly generated plan only after complete accepted validation.
- Apply chunk effective terminal policy. A terminal chunk rejection prevents lane execution; a framework error still prevents output encoding.
- Preserve annotation materialization, deterministic plan canonicalization, attempt debug paths, scheduler use, and checkpoint behavior.
Verification
- Add focused tests for corrected plan acceptance, multi-validator feedback, rejected cache hit regeneration, cache non-overwrite, incomplete-validation non-publication, policy outcomes, cancellation, and debug failure.
- Run
go test ./internal/framework/pipeline ./internal/modules/dnd/chunk/... ./internal/cliand the pipeline race tests.
Exit Criteria
Chunk is the first complete production stage using feedback-aware retries, with cache and terminal behavior matching the roadmap. This stage is one Terra prompt.
Stage 12 — Integrate Per-Chunk Extraction
Goal
Apply the state machine independently to every concurrent extraction job.
Work
- Adapt typed and serialized extract validation to the complete-chain executor and use the extract binding's effective policy.
- Keep correction state local to one lane/chunk job. Rebuild the same initial extraction request with correction attached only on a semantic retry.
- Preserve the run-wide worker pool, scheduled LLM client, chunk-first/lane- second public ordering, stable error selection, cancellation, and lane continuation rules.
- Ensure
reject_outputrecords the terminal chunk-scoped rejection without advancing it to merge.warn_continueis available only for validator failure with no semantic rejection and marks the accepted extract as validation-incomplete.
Verification
- Add representative extraction tests for correction success, exhausted rejection, validator failure policy, independent concurrent jobs, stable ordering, cancellation, and warning promotion.
- Add one assembled D&D pipeline test using fakes—not a live provider—to prove a rejected direct extraction is corrected using the exact prior response.
- Run
go test ./internal/framework/pipeline ./internal/modules/dnd/... ./internal/cliand relevant race tests.
Exit Criteria
Every extraction job has an isolated bounded correction conversation and continues to obey existing concurrency and ordering contracts. This stage is one Terra prompt.
Stage 13 — Integrate Merge And Normalize
Goal
Complete stage coverage and reconcile semantic correction with the existing normalizer fallback retry directive.
Work
- Adapt typed and serialized merge and normalize validation to the shared executor and their respective effective policies.
- Keep merge and normalize serial within a lane and reuse the exact same accepted upstream artifacts, references, source input, profile, and session on every attempt.
- Fold
NormalizeRetryinto the one producer attempt state machine. It consumes the same remaining stage budget, retains its validated safe fallback when exhausted, and never overrides a semantic rejection of that fallback or a later candidate. - Do not send correction to deterministic mergers/normalizers. A rejected deterministic candidate applies terminal policy immediately.
- Preserve typed erasure checks, codec boundaries, generated handoffs, and lane continuation/cancellation behavior.
Verification
- Add one representative merge path and normalize paths for correction
success, deterministic rejection,
NormalizeRetrysuccess/exhaustion, fallback rejection, validator failure, and terminal policies. - Run
go test ./internal/framework/pipeline ./internal/framework/semanticreconcile ./internal/modules/dnd/... ./internal/cliand relevant race tests.
Exit Criteria
Chunk, extract, merge, and normalize all use the same attempt and validation semantics, with no nested normalizer retry loop. This stage is one Terra prompt.
Stage 14 — Finalize Provenance, Checkpoints, Debug, And Durable Results
Goal
Make the new state machine auditable and safe without leaking correction content or allowing degraded output to become reusable state.
Work
- Extend internal attempt/debug records with attempt kind, PromptKit repair count and usage, ordered validator outcomes and attempt counts, aggregate reason codes, validation completeness, effective policy, and terminal decision.
- Keep raw assistant responses and complete correction messages only in the existing explicitly requested detailed trace. Default summaries contain counts, digests, identities, and bounded safe fields.
- Add one reusable durable validation summary with the JSON fields
status,rejecting_validators,reason_codes,incomplete_validators,producer_attempt_count, andterminal_action.statusis one ofcomplete,rejected, orincomplete; lists preserve configured order and omit duplicates after their first occurrence. Embed or project this summary at the manifest, rejection, and run-result boundaries that already expose the affected stage outcome. Preserve the existing singular rejection fields as the first configured rejection for downstream continuity; do not put raw guidance or responses in the summary. - Emit one genuine, bounded, deterministically ordered warning per validator
whose execution budget is exhausted under
warn_continue; do not emit a warning merely because a later correction succeeded. - Checkpoint only accepted and completely validated results. Never write or reuse rejected, structurally invalid, or validation-incomplete producer output. Ensure correction protocol and effective policy participate in fingerprints.
- Confirm cache, resume, generated-reference handoff, output encoding, and CLI result publication cannot treat a rejected or incomplete checkpoint as accepted.
Verification
- Add focused manifest, receipt, warning, debug, redaction, checkpoint reuse, generated-handoff, and sensitive-content tests at their canonical owners.
- Test that corrected success is auditable but quiet, warn-continue output is never checkpointed, and raw content is absent from ordinary durable files.
- Run
go test ./internal/framework/pipeline ./internal/framework/checkpoint ./internal/framework/artifacts ./internal/cliand relevant race tests.
Exit Criteria
Every terminal path has accurate bounded provenance, no sensitive correction content leaks by default, and only completely validated output is reusable. This stage is one Terra prompt.
Stage 15 — Update Canonical Documentation And Perform Final Verification
Goal
Document the implemented contracts in their canonical homes and prove the repository is ready for review.
Work
- Update
docs/policy/architecture.mdwith durable validator aggregation, retry-budget separation, producer correction capability, terminal-policy, checkpoint, and sensitive-data invariants. Link to ADR-0014 for rationale. - Update
docs/config.mdwith the exactvalidation_policyschema, enum values, defaults, inheritance, placement restrictions, retry-budget meanings, and invalid combinations. - Update
docs/operations.mdwith costs, terminal outcomes, warnings, debug sensitivity, retry exhaustion, resume/cache consequences, and recovery. - Update
docs/internal/pipeline.md,docs/internal/llm.md, anddocs/internal/modules.mdwith implemented mechanics and focused test routing. Keep public configuration definitions indocs/config.md. - Complete
docs/integrations/pkg-promptkit.mdfor v0.9.0 appended messages, supported roles, application-owned bounds, and the external catalog boundary. Update the run-result and affected output/subprocess integration contracts for any durable fields added in Stage 14. - Update maintained examples only to demonstrate implemented behavior. Use a minimal policy override in the complete D&D example if it materially aids operators; keep the minimal example minimal. Ensure all links point to canonical owners and remove stale v0.8.0 claims outside historical release notes.
- Do not create a release note until an actual release is prepared.
Verification
- Run
gofmton all changed Go files. - Run
go test ./...,go test -race ./...,go vet ./..., andgo build ./cmd/notarius. - Run the repository's example/config validation tests and documentation link checks. If no standalone link checker exists, verify changed relative links and record that manual check in the implementation report.
- Use targeted searches to confirm no current documentation still claims
PromptKit v0.8.0, no production correction-capable spec is missing its
protocol, no PromptKit type escaped
internal/framework/llm, and no raw correction content is serialized outside explicit detailed debug data. - Review
git diff --checkandgit status --short; do not include generated binaries, temporary files, or unrelated work.
Exit Criteria
All roadmap acceptance criteria are satisfied, canonical documentation matches the code, maintained examples validate, the complete ordinary and race-enabled test suites pass, and the worktree contains only intentional implementation changes. This stage is one Terra prompt.
Open Questions
None. PromptKit transport, producer representation, terminal-policy ownership, bounds, retry budgets, outcome precedence, and checkpoint treatment are all settled by the feature roadmap and this plan.