Files
notarius/docs/roadmap/implementation.md

743 lines
32 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Warning Signal And Presentation Implementation Plan
## Purpose
This document is the ordered implementation plan for the warning and diagnostic
reform defined by [the completed audit](audit.md). It is intended to be executed
stage by stage by a `gpt-5.6-terra` coding agent. Each numbered stage is one
implementation prompt and must leave the repository compiling, internally
coherent, and covered at the narrowest durable test boundaries relevant to that
stage.
The audit is the canonical source for findings, evidence, and target rationale.
This document is the canonical source for implementation order and task
breakdown.
## Governing Decisions
The following decisions are final for this work set.
1. **Warnings are process signals.** A warning means that the run completed
under policy despite process-level degradation or incomplete configured
work. Examples are an empty configured reference, an applicable validator
that could not complete under `warn_continue`, an unavailable required
upstream classification, or exhausted semantic-reconciliation fallback.
2. **Extraction-quality signals are not warnings.** LLM-judged uncertainty,
lexical source-relatedness findings, unresolved entity grounding, and other
accepted-artifact quality signals are advisories. They must never be
promoted to warnings merely because a model or heuristic expressed doubt.
3. **Routine successful transformations are observations.** Canonicalization,
sorting, source-reference cleanup, ID repair, and accepted duplicate
consolidation remain inspectable but do not require operator action.
4. **Ordinary successful runs have zero warnings.** A successful-but-degraded
run may have warnings when policy permits continuation. Advisory or
observation volume alone must not produce warning stderr or a nonzero
warning count.
5. **Errors and rejections remain separate.** This work must not change
validation approval, rejection, retry budgets, exit status, or error policy.
A corrected superseded attempt leaves no final warning. An exhausted
rejection remains a rejection or run failure according to existing policy.
6. **Modules own meaning; the framework owns context and presentation.** A
producer chooses disposition, category, reason code, scope, and safe message.
The framework attaches stage and pipeline origin, aggregates deterministically,
enforces bounds, and supplies the final collections to output and CLI code.
7. **Durable contracts are versioned.** The incompatible grouped warning file
is `notarius.warnings.v2`, the new diagnostic file is
`notarius.diagnostics.v1`, and the machine-readable run receipt becomes
`notarius.run-result.v2`. Do not silently redefine the v1 receipt or warning
payload.
8. **There is no first-release configuration surface for presentation policy.**
Classification, sample bounds, and aggregation are fixed application
policy. Do not add suppression, escalation, verbosity, or per-reason config
in this work set.
## Target Contract
Use the following model unless existing Go naming requires a narrowly scoped
variation. Any naming variation must preserve the specified fields and
semantics.
### Producer Diagnostic
Add a framework contract representing a locally grouped producer diagnostic:
- `disposition`: `warning`, `advisory`, or `observation`;
- `category`: one of `configuration`, `degradation`,
`validation_incomplete`, `fallback`, `data_quality`, or `normalization`;
- `reason_code`: stable, nonblank producer-owned identity;
- `occurrence_count`: exact positive number of represented occurrences;
- `samples`: deterministic bounded samples containing safe `scope` and
`message`; and
- `omitted_sample_count`: exactly `occurrence_count - len(samples)`.
Validate these combinations:
| Disposition | Allowed categories |
| --- | --- |
| `warning` | `configuration`, `degradation`, `validation_incomplete`, `fallback` |
| `advisory` | `data_quality` |
| `observation` | `normalization` |
This strict matrix is intentional. It makes the process-only warning rule a
contract invariant rather than a convention inferred from reason codes.
Use these fixed limits:
- reason code: 128 UTF-8 bytes;
- scope: 512 UTF-8 bytes;
- sample message: 4 KiB of valid UTF-8;
- retained distinct samples per group: 3; and
- groups returned by one producer or validator result: 64.
Blank or invalid required fields, invalid disposition/category combinations,
invalid UTF-8, inconsistent counts, excessive samples, or excessive local
groups make the producer result invalid. They must return a bounded contextual
error rather than be silently repaired. The three-sample and 64-group bounds
are public operational safeguards and may be asserted directly in their owning
contract tests.
The shared collector must count every occurrence and retain the first three
distinct samples in producer order. Repeated identical samples still increase
`occurrence_count`. The omission count is numeric metadata, never another
diagnostic record.
### Framework Origin And Aggregation
The framework adds this origin before final aggregation:
- stage: `references`, `chunk`, `extract`, `merge`, or `normalize`;
- step ID, when applicable;
- lane ID, when applicable;
- module key, when applicable; and
- validator key, for validator-produced diagnostics.
Chunk ID and zero-based chunk index belong on samples, not group origin,
because otherwise identical per-chunk findings cannot aggregate. Use an
optional integer representation that preserves chunk index zero.
The stable aggregation key is:
```text
disposition + category + reason_code
+ stage + step_id + lane_id + module_key + validator_key
```
Scope, message, chunk ID, and chunk index are excluded from the key. Groups and
samples retain first-occurrence order from the runner's existing canonical
ordering; completion timing must never affect the result. When groups merge,
sum exact occurrence counts and retain the first three distinct samples.
Use separate global bounds:
- at most 128 actionable warning groups; exceeding this limit is a framework
error because Notarius must not hide process degradation; and
- at most 256 advisory/observation groups. Additional non-warning groups are
omitted from representation while their occurrences contribute to an exact
`unrepresented_occurrence_count` and set `truncated: true`.
The non-warning envelope's group count means represented groups. Its occurrence
count includes represented and unrepresented occurrences. The warning
collection is never truncated, so both warning counts are exact.
### Classification Matrix
Migrate existing producers according to this matrix.
| Disposition and category | Existing conditions |
| --- | --- |
| Warning / `configuration` | `empty_reference` |
| Warning / `validation_incomplete` | `validator_execution_incomplete`, covering both exhausted failure and skip when `warn_continue` advances the candidate |
| Warning / `degradation` | `scene_classification_unavailable` from combat-turn and enemy-event extraction gates |
| Warning / `fallback` | `npc_semantic_reconciliation_exhausted`, `item_semantic_reconciliation_exhausted`, `location_semantic_reconciliation_exhausted` |
| Advisory / `data_quality` | All ten source-relatedness families; `spell_name_unresolved`; `item_occurrence_unknown_item_id`; `location_occurrence_unknown_location_id`; guarded invalid semantic-consolidation proposals |
| Observation / `normalization` | Field and whitespace normalization; durable ID recomputation; source-reference sorting/deduplication; canonical record ordering; exact or approved semantic duplicate consolidation |
Rename `item_occurrence_source_unrelated` to
`item_occurrence_not_near_source` when it moves into the new contract. Keep
other existing reason codes unless this plan explicitly changes them. Split the
item registry's internal retry reason from its accepted advisory: use
`item_semantic_retry_proposal_invalid` for the retry directive and retain
`item_semantic_proposal_invalid` only for the accepted data-quality advisory.
Local `*_warnings_omitted` reason codes disappear. Omission is represented by
group counts. Rejections, structural failures, provider failures, cancellation,
and successful corrected retries do not receive a diagnostic solely to mirror
their existing error, rejection, validation, or attempt-debug records.
### Durable And CLI Contracts
Always publish both companion files, including for empty collections:
- `warnings.json` with schema version `notarius.warnings.v2`, actionable groups,
exact `group_count`, and exact `occurrence_count`;
- `diagnostics.json` with schema version `notarius.diagnostics.v1`, advisory and
observation groups, represented `group_count`, total `occurrence_count`,
`truncated`, and `unrepresented_occurrence_count`; and
- `index.json` with both `warnings_file` and `diagnostics_file`.
The run receipt becomes `notarius.run-result.v2` and replaces ambiguous
`warning_count` with:
- `warning_group_count`;
- `warning_occurrence_count`;
- `diagnostic_group_count`;
- `diagnostic_occurrence_count`; and
- `diagnostics_truncated`.
The human CLI writes a warning summary to stderr only when
`warning_group_count > 0`. It reports group and occurrence counts plus the
durable warning path. Advisory and observation counts do not create a warning
line. Preserve the existing successful stdout summary and process exit rules.
Remove `OutputResult.Warnings`. An output encoder either returns its complete
logical files or returns an error. It cannot discover a warning after the
warning file has already been serialized.
## Stage 1 — Record The Decision And Add Core Diagnostic Primitives
### Goal
Create the durable decision record and the generic, independently tested
diagnostic model without changing current runtime behavior.
### Work
- Add `docs/adr/0015-separate-process-warnings-from-quality-diagnostics.md`
using the repository ADR format. Record the governing decisions, origin and
aggregation ownership, bounded samples, fresh/resume equivalence, versioned
public contracts, and removal of output-result warnings. Mark it accepted;
do not claim implementation is complete.
- Add the producer diagnostic, sample, disposition, category, origin, group,
and collection types under `internal/framework/contracts/`.
- Add contract validation with the exact allowed combinations and limits in
this plan.
- Add a small generic collector under `internal/framework/diagnostics/` that
groups producer-local occurrences by disposition, category, and reason code,
preserves exact counts, and retains three distinct samples.
- Keep `contracts.Warning` and existing result fields temporarily. Do not alter
current warning output in this stage.
### Tests
Use table-driven contract tests for valid classifications, invalid UTF-8,
blank/oversized fields, invalid counts, the disposition/category matrix, sample
bounds, exact repeated-occurrence counting, and deterministic distinct-sample
selection. Test behavior through exported package contracts rather than private
collector structure.
### Acceptance Criteria
- Existing application behavior and public JSON remain unchanged.
- The new primitives cannot represent an LLM-quality warning because
`warning` plus `data_quality` is rejected.
- `go test ./internal/framework/contracts ./internal/framework/diagnostics`
passes.
This stage is appropriately sized for one `gpt-5.6-terra` prompt.
## Stage 2 — Add Framework Diagnostic Transport And Transitional Projection
### Goal
Carry structured diagnostics through every framework result boundary and add
origin-aware aggregation while preserving the current public warning contract
temporarily.
### Work
- Add structured diagnostic fields alongside legacy warning fields in chunk
plan, typed extract, typed merge, typed normalize, normalize-retry fallback,
validation, run input/output, and output request contracts.
- Extend erased typed results, producer-attempt state, lane result collection,
ordered-step handoff, and run finalization to transport diagnostics.
- Preserve validator identity in validation reports instead of flattening new
diagnostics through a warning-only helper.
- Attach current stage, step, lane, module, validator, and sample-level chunk
context at promotion time.
- Add a run-level aggregator implementing the stable key, canonical
first-occurrence ordering, count merging, and bounds from this plan.
- Preserve terminal-attempt semantics: diagnostics from superseded attempts
remain debug-only; only terminal accepted or terminal rejected candidate
diagnostics are promoted.
- Add a private transitional projection from each structured diagnostic sample
to the existing legacy `Warning` collection so public surfaces remain
unchanged until Stage 8. Mark it explicitly temporary and ensure it does not
duplicate diagnostics already supplied through the legacy path.
### Tests
Extend focused framework tests for terminal attempt promotion, terminal
rejection, reversed concurrent completion, stage/lane/chunk origin, stable
grouping across chunks, and global bounds. Use controlled test dispositions and
origins; do not assert incidental diagnostic prose.
### Acceptance Criteria
- A producer may return either legacy warnings or new structured diagnostics
during migration, but not produce duplicate public records through both.
- Structured groups are deterministic under concurrent completion.
- Superseded attempt diagnostics never reach final groups.
- Existing public warning tests still pass through the transitional projection.
- Focused framework tests and `go test ./internal/framework/pipeline` pass.
This stage is appropriately sized for one high-reasoning
`gpt-5.6-terra` prompt. Do not combine it with D&D migration.
## Stage 3 — Migrate Framework Process Signals And Close The Output Boundary
### Goal
Move framework-owned warning conditions into the new process-only contract and
remove the output-encoder consistency defect.
### Work
- Convert `empty_reference` to warning/configuration with reference origin and
bounded, non-sensitive sample content.
- Convert incomplete validation to warning/validation-incomplete whenever
`warn_continue` advances a candidate after an applicable validator either
exhausts with failure or skips. Retain validator key and typed failure/skip
detail in validation summaries; do not copy provider errors or arbitrary
validator prose into diagnostic samples.
- Aggregate all incomplete-validator occurrences rather than generating
omission warning records.
- Validate module and validator diagnostic results before promotion. An invalid
diagnostic contract is a bounded contextual framework error.
- Remove `Warnings` from `contracts.OutputResult`, the output adapter, test
doubles, and runner append logic. Keep `OutputRequest` diagnostic input; an
encoder failure remains an error.
- Add a regression test proving no post-encoding result can make receipt/debug
warning state disagree with the already encoded logical files.
### Acceptance Criteria
- Failed and skipped validators are both visible when incomplete validation is
allowed to continue.
- A fail-run validation policy still fails rather than converting the failure
into a warning.
- Output encoders have no successful post-encoding warning capability.
- Existing validation decisions, retry budgets, and exit behavior are
unchanged.
- `go test ./internal/framework/... ./internal/modules/generic/output/json`
passes.
This stage is appropriately sized for one `gpt-5.6-terra` prompt.
## Stage 4 — Version Diagnostic-Bearing Cache And Checkpoint State
### Goal
Persist and replay structured producer diagnostics safely without allowing old
warning-only state to be interpreted as complete new state.
### Work
- Store producer-local structured diagnostics in chunk-plan and extract,
merge, and normalize checkpoint envelopes. Do not persist current pipeline
origin in source-keyed chunk-plan state.
- On reuse, attach the current run's origin at the same logical promotion point
used by fresh execution, then aggregate once.
- Bump the checkpoint workspace schema from v3 to
`notarius.workspace.v4` and the chunk-plan schema from v2 to
`notarius.chunk-plan.v3`.
- Treat earlier workspace and chunk-plan versions as incompatible cache misses,
never as corrupt fatal state and never as reusable diagnostic-complete state.
- Preserve the rule that validation-incomplete results and their dependents are
not reusable.
### Tests
Add or adapt behavioral tests for old-version invalidation, structured
diagnostic round trips, fresh/resume equality of groups and counts, exact-once
replay, and current-validator diagnostics on a reused chunk plan.
### Acceptance Criteria
- Fresh and resumed logical runs produce deeply equal diagnostic groups and
counts.
- Reuse provenance remains in checkpoint events and does not alter diagnostic
identity.
- Old cache data is safely bypassed.
- `go test ./internal/framework/checkpoint ./internal/framework/pipeline`
passes.
This stage is appropriately sized for one `gpt-5.6-terra` prompt.
## Stage 5 — Migrate All D&D Source-Relatedness Validators
### Goal
Move all ten heuristic relatedness families from warnings to bounded
data-quality advisories.
### Work
- Migrate combat turns, enemy events, item occurrences, item registry,
location occurrences, location registry, NPC occurrences, NPC registry,
scene descriptions, and spells source-relatedness validators.
- Use the shared generic collector so every family returns exact occurrence
counts and at most three distinct samples. Remove each relatedness omission
reason code.
- Bring NPC and spell validators under the same contract bounds as every other
family.
- Rename `item_occurrence_source_unrelated` to
`item_occurrence_not_near_source`.
- Preserve current approval behavior, validator chain registration and order,
lexical algorithms, scope selection, and messages except for changes needed
to satisfy generic safety bounds.
### Tests
Consolidate repetitive limiter tests where a shared collector contract already
owns the behavior. Retain family tests for realistic triggering and
non-triggering cases. Prove that a relatedness finding is advisory/data-quality
and cannot increment process warning groups.
### Acceptance Criteria
- All ten relatedness validators use the same structured advisory convention.
- No relatedness validator returns a legacy warning or an omission record.
- High-cardinality NPC and spell results remain bounded while reporting exact
occurrence counts.
- `go test ./internal/modules/dnd/validate/...` passes.
This repetitive but cohesive migration is appropriately sized for one
`gpt-5.6-terra` prompt. Do not combine it with normalizer migration.
## Stage 6 — Migrate D&D Registry Normalizers And Semantic Reconciliation
### Goal
Apply the new taxonomy to NPC, item, and location registry normalization and
semantic-reconciliation outcomes.
### Work
- Migrate deterministic field cleanup, ID recomputation, source-reference
normalization, ordering, and accepted duplicate consolidation to
observation/normalization groups.
- Migrate exhausted NPC, item, and location semantic reconciliation to
warning/fallback groups.
- Migrate guarded invalid item semantic proposals to advisory/data-quality.
- Split the item retry directive reason to
`item_semantic_retry_proposal_invalid`; keep correction control data separate
from the accepted advisory reason and from model-facing text.
- Replace local warning limiters and omission records with the shared
diagnostic collector while preserving artifact values, safe fallback, and
retry budgets.
### Tests
Protect exact artifact outcomes, classification, retry exhaustion, safe
currency behavior, occurrence counts, and bounded samples. Do not preserve
old warning slice lengths or omission prose.
### Acceptance Criteria
- Successful registry cleanup and consolidation produce observations only.
- A guarded model proposal produces an advisory, not a warning.
- Exhausted semantic reconciliation is the only registry-normalizer process
warning family.
- Semantic retry behavior and artifacts are unchanged.
- Registry normalizer and semantic-reconciliation tests pass.
This stage is appropriately sized for one high-reasoning
`gpt-5.6-terra` prompt.
## Stage 7 — Migrate Remaining D&D Producers
### Goal
Complete D&D diagnostic classification across spells, occurrences, combat,
enemy events, scenes, and extraction gates.
### Work
- Migrate spell, combat-turn, enemy-event, item-occurrence,
location-occurrence, NPC-occurrence, and scene-description normalizers.
- Classify deterministic cleanup, ordering, source-reference normalization,
and duplicate consolidation as observation/normalization.
- Classify unresolved spell, item, or location membership as
advisory/data-quality, never warning.
- Convert combat-turn and enemy-event `scene_classification_unavailable`
extraction gates to warning/degradation with module and chunk provenance.
- Remove all remaining D&D local warning omission reason codes and all uses of
`LimitWarnings`; retain text-safety helpers that still have value.
### Tests
Adapt family tests to assert artifact invariants and classification. Add one
assembled D&D test demonstrating that quality advisories and normalization
observations can be present while the warning group count remains zero.
### Acceptance Criteria
- No D&D producer returns a legacy `contracts.Warning`.
- No accepted-artifact uncertainty or unresolved grounding appears as a
process warning.
- Missing required scene classification remains an actionable process warning.
- All D&D package tests pass.
This stage is appropriately sized for one high-reasoning
`gpt-5.6-terra` prompt.
## Stage 8 — Remove The Legacy Warning Path And Finalize Aggregation
### Goal
Make the structured diagnostic collection the sole in-memory signal path.
### Work
- Migrate any remaining generic test modules and framework fixtures from
`contracts.Warning` to structured diagnostics.
- Remove `contracts.Warning`, legacy `Warnings` fields, clone helpers, warning
append helpers, transitional projections, and obsolete D&D warning limiters.
- Make `RunOutput` and `OutputRequest` carry the finalized warning and
non-warning grouped collections derived from one aggregator result.
- Enforce the 128-warning-group error bound and 256-non-warning-group truncation
behavior. Ensure occurrence counts remain exact and truncation metadata is
deterministic.
- Verify all producers are validated at their stable framework boundary.
### Tests
Add focused behavioral coverage for warning overflow failure, non-warning
truncation, exact occurrence totals, distinct sample retention, stable group
order, and no legacy double-counting. Remove tests whose only purpose was to
assert old flat-list caps or omission prose.
### Acceptance Criteria
- Repository search finds no production `contracts.Warning`, legacy warning
result field, or `*_warnings_omitted` reason code.
- One finalized collection supplies all later surfaces.
- An advisory-only successful run has zero warning groups.
- `go test ./internal/framework/... ./internal/modules/dnd/...` passes.
This stage is appropriately sized for one `gpt-5.6-terra` prompt.
## Stage 9 — Publish Versioned Warning And Diagnostic Files
### Goal
Replace the flat durable warning payload with the two target grouped contracts.
### Work
- Update the production JSON encoder to always emit grouped `warnings.json`
(`notarius.warnings.v2`) and `diagnostics.json`
(`notarius.diagnostics.v1`).
- Include exact counts and diagnostic truncation metadata specified above.
- Add `diagnostics_file` to `index.json`; retain `warnings_file`.
- Ensure warning groups appear only in `warnings.json` and advisory/observation
groups appear only in `diagnostics.json`.
- Update `docs/integrations/json-output.md` in the same stage. It owns file
names, envelope schemas, count meanings, group/sample fields, bounds,
truncation, and index discovery. Link rather than duplicate CLI behavior.
- Update maintained output examples or test fixtures only where they encode
implemented output contracts.
### Tests
Use encoder contract tests for empty and populated files, schema versions,
partitioning, counts, truncation, index paths, deterministic JSON ordering, and
newline/valid-JSON conventions. Update assembled pipeline tests to compare both
durable files with the finalized in-memory collections.
### Acceptance Criteria
- Every production JSON bundle contains both companion files and index paths.
- No diagnostic appears in both files.
- Published counts agree with the in-memory collection.
- Output encoder and maintained example contract tests pass.
This stage is appropriately sized for one `gpt-5.6-terra` prompt.
## Stage 10 — Introduce Run Result V2 And Quiet CLI Presentation
### Goal
Give subprocess consumers unambiguous counts and make an ordinary successful
run quiet on the warning stream.
### Work
- Change the emitted receipt to `notarius.run-result.v2` with the five fields
specified in the target contract. Remove v1 `warning_count`; do not emit two
competing count models.
- Derive receipt counts from the same finalized collection used by the JSON
encoder.
- Print a stderr warning summary only when actionable warning groups exist. It
must include group count, occurrence count, and the output-relative or
absolute details path consistent with current CLI path conventions.
- Do not mention advisory/observation counts as warnings. Preserve ordinary
stdout completion output and all exit classifications.
- Update `docs/cli.md`, `docs/integrations/run-result.md`,
`docs/consumers/subprocess.md`, and `docs/consumers/dnd-pipeline.md` in the
same stage. Each document must keep to its canonical scope and link to the
grouped JSON contract rather than duplicating schemas.
### Tests
Cover zero-warning approved success, advisory-only success, degraded success
with warnings, rejected output, JSON receipt delivery, and non-production
output modules. Assert semantic fields and stream choice, not complete prose.
### Acceptance Criteria
- Advisory-only and observation-only successful runs write no warning stderr.
- Degraded success writes one concise actionable warning summary.
- Receipt counts and durable files agree.
- Receipt, CLI command, and consumer contract tests pass.
This stage is appropriately sized for one `gpt-5.6-terra` prompt.
## Stage 11 — Align Debug, Manifest, And Resume Surfaces
### Goal
Ensure forensic and provenance surfaces reflect the new model without creating
another competing warning contract.
### Work
- Replace final flat warning debug summaries with final grouped warning and
non-warning diagnostic projections, using separate clearly named files or
one versioned diagnostic summary envelope consistently with existing debug
layout.
- Keep attempt-local candidate diagnostics in detailed debug records with
stage/attempt provenance. Preserve redaction and the rule that raw model
responses and correction messages appear only in detailed debug.
- Keep manifest validation and rejection summaries authoritative for validation
status; do not duplicate full diagnostic groups into the manifest.
- Ensure checkpoint events, not diagnostic origin, identify reuse.
- Update `docs/operations.md`, `docs/internal/state.md`, and the debug portions
of `docs/internal/pipeline.md` in the same stage.
### Tests
Cover successful debug capture, partial failure debug capture,
terminal-attempt-only final grouping, redaction, and fresh/resume equality.
Retain existing validation summary tests rather than duplicating every outcome
in diagnostic tests.
### Acceptance Criteria
- Debug consumers can distinguish process warnings, quality advisories, and
observations.
- Debug final counts match receipt and durable output when output is published.
- Attempt detail remains bounded and sensitive-data rules are preserved.
- Debug, state, and resume-focused tests pass.
This stage is appropriately sized for one `gpt-5.6-terra` prompt.
## Stage 12 — Update Architecture And Internal Producer Documentation
### Goal
Make the implemented ownership and classification rules durable for future
modules without duplicating public schemas.
### Work
- Update `docs/policy/architecture.md` to state the process-only warning
invariant, module semantic ownership, framework origin/aggregation ownership,
and ordinary-success-zero-warnings expectation. Link to ADR-0015 for
rationale.
- Update `docs/internal/pipeline.md` for terminal promotion, origin enrichment,
aggregation, bounds, concurrency ordering, checkpoint replay, and output
handoff.
- Update `docs/internal/modules.md` with the generic producer contract and the
prohibition on using warnings for extraction quality.
- Update `docs/internal/dnd.md` with the implemented classification matrix and
shared collector convention. Link to integration contracts for durable
schemas rather than restating them.
- Update `docs/internal/overview.md` only if the new generic diagnostics package
changes the component inventory.
- Remove stale terminology from current docs outside `docs/roadmap/`, but do
not write release history or document unimplemented configuration.
### Acceptance Criteria
- Each volatile fact has one canonical owner under the documentation policy.
- Future module authors can determine which disposition/category to use and
where origin and aggregation are attached.
- Current documentation contains no claim that heuristic quality doubt is a
warning.
- Relative Markdown links resolve.
This stage is appropriately sized for one `gpt-5.6-terra` prompt.
## Stage 13 — Final Behavioral Verification And Cleanup
### Goal
Verify the complete migration, remove transitional residue, and establish that
the audit findings are fully addressed.
### Work
- Run repository searches for legacy warning types, warning omission reason
codes, old run-result v1 fields, old flat warning JSON assumptions, and
`OutputResult.Warnings`.
- Review every producer inventoried in `audit.md` against the final
classification matrix.
- Run the maintained minimal and complete D&D workflows with offline fake LLM
clients. Do not require provider credentials or add live calls to the default
suite.
- Verify these relationships end to end:
- ordinary approved success and advisory-only success have zero warnings;
- successful process degradation has a warning;
- corrected retries retain only terminal diagnostics;
- rejected and failed runs retain their established semantics;
- warning and diagnostic group order is deterministic under concurrency;
- fresh and resumed runs are equivalent;
- global group and sample bounds hold; and
- receipt, stderr, durable files, index, and debug counts agree.
- Remove obsolete helpers and redundant tests made unnecessary by stronger
package-level collector or end-to-end contract tests.
### Validation Commands
Run at minimum:
```sh
go test ./...
go vet ./...
go build ./cmd/notarius
```
Also run any repository documentation or link checker discovered during the
stage. If none exists, perform a focused relative-link review for the documents
changed by Stages 912.
### Acceptance Criteria
- All three repository-wide Go commands pass offline.
- No live API key is required.
- No legacy flat-warning code path or public v1 warning-count claim remains in
current-behavior documentation.
- All seven audit findings are addressed without changing artifact values,
validator chain order, retry budgets, rejection policy, or exit semantics.
- The ordinary maintained successful workflow emits zero actionable warnings;
non-warning diagnostics remain available in `diagnostics.json`.
This final verification is appropriately sized for one `gpt-5.6-terra`
prompt.
## Deferred Work
The following are deliberately outside this implementation plan:
- provider-backed measurement of production advisory precision or frequency;
- configurable warning suppression, escalation, verbosity, or reason filters;
- converting heuristic relatedness checks into rejection rules;
- the planned LLM-backed D&D combat-scene validator;
- metrics or telemetry export beyond the specified durable and debug files;
- a two-phase output encoder protocol; and
- release preparation or release-note creation.
Provider-backed production data may inform later advisory tuning, but it is not
required to implement or validate this architecture.