# Warning Signal And Presentation Audit

## Executive Assessment

Notarius warning execution is mechanically stronger than its operator-facing
presentation. Terminal-attempt promotion, stable ordering after concurrent
work, checkpoint replay, validation summaries, and debug retention are all
substantially correct. The audit found no general duplicate-append defect in
the extract, merge, or normalize handoffs and no leakage of abandoned-attempt
warnings into a successful result.

The warning channel itself is not coherent. One flat `contracts.Warning` type
currently represents at least four materially different concepts:

- actionable degradation or incomplete validation;
- heuristic data-quality doubt;
- successful but potentially reviewable fallback; and
- routine canonicalization, ordering, and deduplication observations.

That conflation is the primary reason successful runs produce a count that is
large but operationally weak. The maintained complete D&D example demonstrates
the problem without a live provider: an approved run with no rejected outputs
published 12 warning records, all from three advisory relatedness checks. An
operator separately reported a successful complete D&D run with 10 outputs,
one rejection, and 85 warnings. The production bundle for that run was not
available in this environment, so its reason-code distribution could not be
measured.

The current implementation also has four correctness or robustness gaps:

1. warning records lose stage, step, lane, module, validator, and chunk
   provenance when promoted, which makes safe aggregation and diagnosis
   impossible from `warnings.json` alone;
2. there is no framework-level validation or aggregate bound, and the NPC- and
   spell-relatedness validators bypass the D&D warning limiter entirely;
3. warnings returned by an output encoder are added after `warnings.json` has
   already been encoded, so the receipt, stderr, debug bundle, and published
   warning file can disagree; and
4. a skipped validator contributes to `incomplete` validation but does not
   receive the warning generated for an exhausted validator failure.

The recommended end state is a structured diagnostic contract with explicit
disposition, category, origin, occurrence count, and bounded samples. Warnings
are reserved for process-level degradation or incompleteness. LLM-judged or
deterministically inferred extraction-quality signals are advisories, never
warnings, and routine normalization observations remain inspectable without
being reported as top-level warnings. An ordinary successful run in which all
configured work completes normally should therefore report zero warnings. This
is an architectural and durable-contract change, not merely revised CLI prose.

## Evidence And Limits

The audit used:

- a complete static search of production `contracts.Warning` constructors,
  reason-code constants, result fields, and promotion sites under `internal/`;
- call-path inspection through producer attempts, validators, lane
  coordination, chunk-plan reuse, checkpoints, output encoding, debug output,
  CLI presentation, and run-result construction;
- the maintained complete and minimal D&D examples with offline fake LLMs;
- focused deterministic tests for warning bounds, semantic-reconciliation
  fallback, `warn_continue`, semantic retries, terminal rejection, concurrency
  ordering, and checkpoint reuse; and
- the operator-provided observation of an 85-warning complete D&D run.

No provider-backed production run was attempted because this environment has
no API key. Consequently, the audit can establish warning mechanics, possible
multiplicity, synthetic volume, and obvious heuristic limitations, but cannot
estimate production frequency or the real false-positive rate of individual
D&D advisories. Those measurements are not required to choose the recommended
architecture; they are required before strengthening any heuristic advisory
into a rejection or setting a numerical production acceptance target.

## Complete Warning-Producer Inventory

### Framework And Generic Boundaries

| Producer | Reason code | Trigger and consequence | Multiplicity and bound | Current surfaces and coverage |
| --- | --- | --- | --- | --- |
| Reference materialization in `internal/framework/pipeline/references.go` | `empty_reference` | A bound external reference is a valid, accepted media type but contains zero bytes. The prompt may receive materially incomplete context. | One per empty bound file; finite by configuration but no shared run-level cap. | Enters `RunInput.Warnings`; reference tests protect contextual scope. |
| Producer-attempt policy in `internal/framework/pipeline/producer_attempts.go` | `validator_execution_incomplete` | An applicable validator exhausted its execution budget and `warn_continue` accepted the otherwise valid candidate. | One per failed validator on each terminal candidate. An extract chain can multiply this by chunks and lanes. There is no global cap. | Durable warning, receipt count, stderr, debug, and validation summary. `TestWarnContinueRecordsOneWarningForEachExhaustedValidator` covers failures. |
| Chunk, extract, merge, normalize, and output module result contracts | Module-defined | A module may return arbitrary warnings with its successful candidate. | No contract validation, message limit, per-result cap, or global cap. Current production modules are inventoried below. | Terminal-attempt filtering and concurrency ordering are well tested. |
| Production JSON output encoder | None | The encoder copies incoming warnings into `warnings.json`; it does not currently create warnings. | Same count as its input. | JSON encoder and assembled-pipeline tests compare the incoming run warnings with the published file. |
| Output encoder result contract | Module-defined | Any output encoder may return warnings discovered during encoding. | Unbounded by contract. No production encoder currently exercises this capability. | Appended to final `RunOutput` only after logical files were encoded; this is the cross-surface defect described in AUD-WARN-004. |

Input adapters and production mergers do not currently have independent
warning producers. Chunk-plan and checkpoint decisions are structured manifest
or debug provenance rather than warnings. Cancellation and hard persistence,
reference, parsing, serialization, and provider failures remain errors.

### D&D Extraction Gates

| Producer | Reason code | Trigger and consequence | Multiplicity and bound |
| --- | --- | --- | --- |
| `dnd/combat-turns` extractor | `scene_classification_unavailable` | The chunk has no exact matching scene-description classification. The extractor returns an empty accepted result and skips the LLM, so combat-turn output may be incomplete. | At most one per chunk for this lane. |
| `dnd/enemy-events` extractor | `scene_classification_unavailable` | The same missing or mismatched scene gate causes accepted empty enemy-event output. | At most one per chunk for this lane. |

An exact non-combat classification produces an intentional empty result without
a warning. An exact combat classification proceeds normally. The two producers
share a code and operator consequence but use different messages; their module
origins are not retained in the final warning record.

### D&D Source-Relatedness Validators

All ten relatedness validators are deterministic advisories: they approve the
candidate and warn when contextual prose or an entity name is not lexically
present in cited text. Shape and source-reference failures are deliberately
left to blocking validators earlier in the chain. The same relatedness
validator is registered in both the extract and normalize default chain for
each artifact family in `internal/modules/dnd/register/chains.go`.

| Artifact family | Warning reason | Per-record trigger | Local bound | Omission reason |
| --- | --- | --- | --- | --- |
| Combat turns | `combat_turn_not_near_source` | Actor token sequence absent | 20 per validator invocation | `combat_turn_relatedness_warnings_omitted` |
| Enemy events | `enemy_event_not_near_source` | Subject token sequence absent | 20 | `enemy_event_relatedness_warnings_omitted` |
| Item occurrences | `item_occurrence_source_unrelated` | Item name token sequence absent | 20 | `item_occurrence_relatedness_warnings_omitted` |
| Item registry | `item_not_near_source` | Item name token sequence absent | 20 | `item_relatedness_warnings_omitted` |
| Location occurrences | `location_occurrence_not_near_source` | Location name token sequence absent | 20 | `location_occurrence_relatedness_warnings_omitted` |
| Location registry | `location_not_near_source` | Location name token sequence absent | 20 | `location_relatedness_warnings_omitted` |
| NPC occurrences | `npc_occurrence_not_near_source` | NPC name token sequence absent | 20 | `npc_occurrence_relatedness_warnings_omitted` |
| NPC registry | `npc_not_near_source` | NPC name token sequence absent | **Unbounded** | None |
| Scene descriptions | `scene_description_not_near_source` | No significant title or summary token appears; up to two findings per scene | 20 | `scene_description_relatedness_warnings_omitted` |
| Spells | `spell_not_near_source` | Spell-name token sequence absent | **Unbounded** | None |

The eight limiter-generated omission records are presentation artifacts, not
new source-relatedness conditions. They occupy a warning slot and make list
length differ from the actual occurrence count.

### D&D Normalizers

Every production D&D normalizer bounds its returned warning slice to 20 through
`internal/modules/dnd/shared/diagnostics`, including a final omission record
when needed. Registry semantic retries reserve space for their fallback
warning. The following table is complete by semantically distinct condition;
codes listed together are parallel artifact-family variants.

| Condition | Reason codes | Result impact | Current classification assessment |
| --- | --- | --- | --- |
| Display or field whitespace/name canonicalization | `npc_fields_normalized`, `item_fields_normalized`, `location_fields_normalized`, `spell_name_canonicalized`, `combat_actor_canonicalized`, `enemy_event_name_canonicalized`, `item_occurrence_name_canonicalized`, `location_occurrence_name_canonicalized`, `scene_description_prose_normalized` | Deterministic successful mutation. The item-occurrence code can also describe `from`/`to` whitespace, not only the item name. | Routine observation. |
| Durable ID recomputation | `npc_id_recomputed`, `item_id_recomputed`, `location_id_recomputed` | Restores the deterministic name-derived ID. | Routine observation; invalid identity is separately rejected by default chains. |
| Source-reference sorting or deduplication | `source_references_normalized` | Sorts and removes exact duplicate references while deliberately preserving invalid references for their validators. | Routine observation. Shared code is useful but ambiguous without producer origin. |
| Canonical record ordering | `combat_turns_reordered`, `enemy_events_reordered`, `item_occurrences_reordered`, `location_occurrences_reordered`, `npc_occurrences_reordered`, `scene_description_order_normalized` | Deterministic order changes only. | Routine observation. |
| Exact or approved semantic duplicate consolidation | `duplicate_npc_collapsed`, `duplicate_item_collapsed`, `duplicate_location_collapsed`, `duplicate_spell_cast_collapsed`, `duplicate_combat_turn_collapsed`, `duplicate_enemy_event_collapsed`, `duplicate_item_occurrence_collapsed`, `duplicate_location_occurrence_collapsed`, `duplicate_npc_occurrence_collapsed`, `scene_description_duplicate_collapsed` | Removes duplicate records and preserves or combines canonical evidence according to the artifact policy. Registry codes cover both exact and accepted semantic consolidation. | Durable normalization observation; not normally operator-actionable. |
| Unresolved external membership | `spell_name_unresolved`, `item_occurrence_unknown_item_id`, `location_occurrence_unknown_location_id` | The value is preserved but is not grounded in the effective catalog or registry. Default chains normally reject the same condition before normalization; it remains reachable with validator overrides or defensive direct use. | Actionable data-quality warning. |
| Unsafe currency consolidation proposal | `item_semantic_proposal_invalid` | The proposed group is rejected and all records are preserved because denominations or currency/non-currency members are incompatible. The same code is also used internally as a retry reason. | Advisory about model proposal quality; no accepted-data loss. The control and diagnostic meanings should be separated. |
| Semantic reconciliation unavailable or exhausted | `npc_semantic_reconciliation_exhausted`, `item_semantic_reconciliation_exhausted`, `location_semantic_reconciliation_exhausted` | The safe deterministic result is accepted, but possible semantic duplicates remain. | Actionable fallback warning. |
| Local warning truncation | `npc_normalization_warnings_omitted`, `item_normalization_warnings_omitted`, `location_normalization_warnings_omitted`, `spell_normalization_warnings_omitted`, `combat_turn_normalization_warnings_omitted`, `enemy_event_normalization_warnings_omitted`, `item_occurrence_normalization_warnings_omitted`, `location_occurrence_normalization_warnings_omitted`, `npc_occurrence_normalization_warnings_omitted`, `scene_description_normalization_warnings_omitted` | Reports that individual records were omitted from presentation. | Group metadata, not an independent warning. |

No production D&D normalization warning exposes raw model responses,
correction guidance, or provider errors. Most dynamic names are quoted and
truncated by the shared helper. That local discipline is not enforced by the
generic warning contract, and the spell relatedness message does not use the
shared truncation helper.

## Warning Propagation And Surface Map

```text
external-reference warnings -----------------------------+
                                                           |
module candidate warnings -> validation chain warnings     |
             |                      |                       |
             +---- producer-attempt terminal policy -------+
                                  |                        |
                   accepted / terminal rejection only      |
                                  |                        |
              chunk or lane result in canonical order      |
                                  |                        |
       checkpoint record/replay and ordered step merge     |
                                  |                        |
                            RunOutput.Warnings <------------+
                                  |
                   OutputRequest -> output encoder
                                  |             |
                         warnings.json     OutputResult.Warnings
                                                |
                              appended to final RunOutput only
                                  |
                 receipt, stderr, final debug warning summary
```

### Attempts And Validation

- `runProducerAttempts` promotes only the terminal accepted or terminal
  rejected candidate's module and completed-validator warnings. Operational,
  structural, semantic, and module-directed attempts that are superseded are
  retained in attempt debug artifacts but not in the final collection.
- A module-directed semantic-reconciliation retry adds its fallback warning
  only when no retry remains. Earlier attempt warnings are discarded.
- On `warn_continue`, warnings from the otherwise accepted candidate and
  completed approved or rejected validators are retained. One fixed,
  non-sensitive `validator_execution_incomplete` warning is added for every
  failed validator. Skipped validators affect the validation summary and final
  `incomplete` status but do not receive such a warning.
- A semantic terminal rejection retains only warnings from that rejected
  attempt. Structural rejection after producer failure cannot retain a
  candidate warning because no valid candidate result exists.

These behaviors are protected by the producer-attempt, extract-handoff,
rejection-warning, normalize-retry, and attempt-debug tests. They are the right
foundation for the redesign and should not be replaced with early-exit or
all-attempt accumulation.

### Concurrency And Ordering

Extract jobs are dispatched chunk-first and lane-second. Results are stored by
chunk index, finalized in ascending chunk order, and lane continuations are
merged into a slice indexed by configured lane order. Pipeline steps run in
configured order. The resulting public order is therefore:

1. pre-run reference warnings;
2. chunk-stage warnings;
3. step order;
4. configured lane order within each step;
5. chunk order within extract;
6. merge warnings; then
7. normalize warnings; followed by any output-result warnings.

`TestRunnerBoundsExtractJobsAndStabilizesReverseCompletion` exercises warning
order under reversed completion. No completion-order leak was found.

### Chunk Plans And Checkpoints

- A reusable chunk plan stores producer warnings only. Current validators run
  again, and their current warnings are appended. Warnings from a cached plan
  candidate that fails current validation are discarded before regeneration.
- Accepted extract, merge, and normalize checkpoints store the terminal
  warnings for their stage. Reuse loads and appends those warnings once at the
  same logical handoff. Tests compare fresh and resumed warning collections and
  preserve their order.
- Validation-incomplete accepted outputs are not reusable, preventing a later
  run from silently treating incomplete validation as complete.
- Required accepted-normalize hydration replays that normalize checkpoint's
  warnings; checkpoint decisions separately expose that reuse occurred.

The recommended redesign should keep fresh and resumed logical diagnostics
equivalent. Whether a result was reused belongs in checkpoint provenance, not
in the diagnostic grouping key; adding a `reused` distinction would fragment
groups and make equivalent runs present differently.

### Terminal Surfaces

| Surface | Current content | Audience | Audit result |
| --- | --- | --- | --- |
| `RunOutput.Warnings` | Flat final slice | Framework and CLI | Canonical in-memory list, but lacks origin and bounds. |
| Published `warnings.json` | Object containing the warnings passed into the output encoder | Durable consumers | Exact for the production JSON encoder unless the encoder itself returns warnings. |
| `index.json` | Path to `warnings.json` | Durable consumers | Stable discovery path; no separate diagnostic-detail path. |
| Run-result v1 | `warning_count = len(final RunOutput.Warnings)` | Subprocess callers | Count only; no group/occurrence distinction. |
| Human stderr | `run completed with N warning(s)` | Operators | Count only and no direct detail path. Successful exit remains zero. |
| Manifest | Validation and rejection summaries, no warning collection | Durable provenance | Correctly avoids duplicating the flat list. |
| Debug summary `warnings.json` | Raw final warning array | Operators/developers | Includes final output-result warnings and can therefore differ from published `warnings.json`. |
| Debug run report | Final warning count | Operators/developers | Same final slice length as receipt and stderr. |
| Attempt/stage debug | Candidate-local warning detail and origin in path/envelope | Forensics | Sufficient to diagnose provenance, but debug capture is optional and is not a durable consumer contract. |

## Empirical Measurements

### Offline And Synthetic Runs

| Scenario | Result | What it establishes |
| --- | --- | --- |
| Maintained complete D&D config and transcript with the repository's offline fake LLM | Approved, 10 normalized outputs, 0 rejected outputs, 12 warnings; receipt, stderr, and published file all reported 12 | An ordinary structurally successful workflow can be noisy without fallback or incomplete validation. |
| Same complete run, grouped after publication | Three reason codes, seven exact `(reason, scope, message)` tuples, maximum exact-tuple repetition of three | The flat count materially overstates distinct operator conditions. Scope resets within chunks and does not identify origin. |
| Maintained focused scene-description workflow | Approved, one normalized output, 0 warnings | The warning channel can be quiet when synthetic model text is lexically grounded. |
| Generic warning publication contract | One warning reaches successful stderr, durable output, and debug summary | The ordinary pre-output path is consistent. |
| NPC semantic-reconciliation candidate-limit fallback | No LLM call, all records preserved, one exhaustion warning, total warnings no greater than 20 | Fallback is bounded and materially different from routine normalization. |
| `warn_continue` with two failed validators and one skipped validator | Validation status contains all three incomplete validators; warning slice contains two execution-incomplete records | Current warning count does not describe all incomplete validation. |
| Retrying extract candidate | Two producer attempts; only the accepted attempt's one warning is final | Retry does not amplify abandoned warnings. |
| Terminal semantic rejection | Only the final rejected attempt's operation and validator warnings are final | Rejection diagnostics are retained without retaining superseded warnings. |
| Fresh versus reused extract checkpoint | Warning collections are deeply equal | Checkpoint replay does not itself amplify warnings. |
| Spell normalizer with 21 unresolved entries | 20 records: 19 samples plus one omission record saying two additional warnings were omitted | `warning_count` is neither exact occurrence count nor distinct-condition count. |

The focused audit tests passed in `internal/cli`,
`internal/framework/pipeline`, the NPC-registry and spell normalizers, and all
D&D packages.

### Bounded Sample Review

The complete offline D&D run produced:

| Reason | Count | Sample | Review |
| --- | ---: | --- | --- |
| `location_not_near_source` | 4 | `Moon Gate` was absent from cited text | Correctly identifies deliberately unsupported fake output. Three records shared the same exact tuple because chunk and stage origin were lost. |
| `location_occurrence_not_near_source` | 4 | A `Moon Gate` visit was absent from cited text | Correctly identifies the same unsupported registry-driven occurrence, but repeats the same operator concern across extraction and normalization. |
| `scene_description_not_near_source` | 4 | A title or summary had no significant exact token in cited text | Mixed value. Generic `session scene` prose is ungrounded, while `Arrival` versus transcript `arrive` illustrates an expected lexical false positive. |

This fake workflow is an integration fixture, not a model-quality benchmark.
It nonetheless proves that the checks carry useful evidence while being too
imprecise and repetitive to serve as one-warning-per-record operator alerts.

### Production Evidence Still Needed

The reported 85-warning run establishes that high volume occurs in practice,
but the following remain unknown:

- dominant production reason codes and stage/lane sources;
- unique group count versus repeated occurrence count;
- false-positive rate for each relatedness family;
- how much volume comes from normalization observations versus advisories;
- whether fresh and resumed production runs remain equivalent; and
- a defensible numerical acceptance target.

If further data is worthwhile, the operator can supply the v1 receipt,
`warnings.json`, and manifest validation summaries without supplying transcript
or lane artifacts. An initial privacy-preserving report should group by reason
code and normalized scope family, count exact repeated tuples, and omit message
text. Reviewing heuristic precision requires a separately approved bounded
sample with its cited source context.

## Classification Of Current Conditions

| Target disposition | Current families | Result impact | Operator action | Durable placement |
| --- | --- | --- | --- | --- |
| **Warning** | Empty reference; validator failure or skip accepted under `warn_continue`; unavailable required scene classification; exhausted semantic reconciliation | A configured process completed under an allowed degraded or incomplete policy rather than completing normally | Correct reference/configuration, inspect provider/validator, or rerun | Actionable `warnings.json`, receipt summary, stderr summary, debug |
| **Advisory** | Source-relatedness heuristics; unresolved spell or registry membership; guarded invalid semantic proposal; any future LLM-judged uncertainty or extraction-quality signal | Uncertain data quality or poor model proposal, but accepted data is structurally valid and deterministic guards prevented unsafe mutation | Optional model/source review; no routine action for every record | Durable diagnostic detail and debug; never a top-level warning |
| **Observation** | Whitespace/name/ID/source-reference canonicalization; canonical ordering; exact and approved semantic duplicate consolidation | Successful intended normalization | None under normal operation | Durable bounded normalization diagnostics or debug; no stderr warning |
| **Not a diagnostic** | Rejection, invalid structure, cancellation, persistence error, provider failure under fail-run policy | Candidate or run did not complete according to policy | Inspect rejection/error and retry or correct input/configuration | Existing rejection, validation summary, error, and debug contracts |

Exact and semantic duplicate consolidation should remain distinguishable in
category or reason metadata even though both are observations. Semantic
reconciliation exhaustion remains a warning because a capability was not
applied; successful approved consolidation is an observation because it is the
normalizer's intended work.

## Ranked Findings

### AUD-WARN-001 — The Flat Warning Type Destroys Signal Quality

- **Priority:** High operator impact; high implementation leverage.
- **Evidence:** `contracts.Warning` has only scope, reason, and message. Routine
  normalizer changes, heuristic doubt, fallback, and incomplete validation all
  enter the same slice and the same CLI count. The offline complete run's 12
  records were all advisories; the operator observed 85 records in a successful
  real run.
- **Impact:** Operators cannot tell whether a warning requires a rerun, a
  configuration repair, optional review, or no action. Repeated routine output
  trains them to ignore the channel.
- **Recommendation:** Replace the flat semantic contract with explicit
  `warning`, `advisory`, and `observation` dispositions plus a small category
  vocabulary. Do not infer disposition from message text or require every
  downstream consumer to maintain a reason-code policy table.

### AUD-WARN-002 — Warning Records Lose The Origin Needed For Diagnosis And Aggregation

- **Priority:** High correctness and usability impact.
- **Evidence:** The runner knows stage, step, lane, module, validator, chunk ID,
  and chunk index at promotion time, but `terminalWarnings` flattens module and
  validator records into `[]contracts.Warning`. Per-chunk scopes such as
  `locations[0]` and `occurrences[0]` then repeat without identifying their
  chunk or producer. `source_references_normalized` is intentionally shared
  across families and is therefore especially ambiguous.
- **Impact:** `warnings.json` cannot answer which stage or module produced a
  record. Message- or scope-based deduplication would merge unrelated findings
  or retain accidental duplicates.
- **Recommendation:** Keep module findings free of framework context, then have
  the framework add a structured origin envelope before promotion. Validator
  findings must retain validator identity instead of passing through
  `validationReport.Warnings()` as a flat slice.

### AUD-WARN-003 — Warning Volume Is Not End-To-End Bounded Or Validated

- **Priority:** High robustness impact; medium immediate likelihood.
- **Evidence:** D&D's `LimitWarnings` caps most individual producers at 20, but
  NPC- and spell-relatedness return one warning per record without the helper.
  Every extract validator is invoked per chunk, all ten relatedness checks run
  again after normalization, and there is no run-level collector. The generic
  contract validates neither disposition nor reason/message size, UTF-8,
  blankness, or total records.
- **Impact:** Warning memory and output grow with chunks, lanes, configured
  validators, and record counts. Local omission records lose exact occurrence
  semantics while still incrementing `warning_count`.
- **Recommendation:** Add a generic bounded diagnostic collector that preserves
  exact occurrence counts and bounded samples. Validate all diagnostic fields
  at the module/framework boundary. Immediately bring NPC and spell
  relatedness under the existing cap if the full redesign is staged.

### AUD-WARN-004 — Output Encoder Warnings Make Durable Surfaces Disagree

- **Priority:** Medium current impact; high contract correctness risk.
- **Evidence:** `Runner.Run` passes existing warnings to `encoder.Encode`, then
  the production encoder serializes `warnings.json`. Only after encoding does
  the runner append `OutputResult.Warnings`. The receipt, stderr, debug summary,
  and debug run report see the final slice; the already-created published file
  cannot. No production encoder currently returns a warning, so ordinary JSON
  runs do not trigger the defect.
- **Impact:** A valid output-module implementation can violate the documented
  claim that `warning_count` describes the published warning collection.
- **Recommendation:** Remove successful output warnings from the output-module
  contract unless a demonstrated use case requires them; encoding failures
  should be errors and optional encoder observations should be debug data. A
  two-phase finalize API is the viable but more complex alternative.

### AUD-WARN-005 — Validation Skips Are Incomplete But Not Warned

- **Priority:** Medium operator/correctness impact.
- **Evidence:** `firstIncompleteValidation` treats failed and skipped validators
  alike, and validation summaries include both. `incompleteValidationWarnings`
  emits records only for `validationFailed`. The focused test demonstrates
  three incomplete validators but two warnings.
- **Impact:** A successful run can have `validation_status: incomplete` while
  its warning count understates or even omits the affected validators. A caller
  that checks only warnings receives a weaker signal than the manifest and
  receipt status.
- **Recommendation:** Produce one aggregated incomplete-validation warning
  group whose occurrences cover both failure and skip, while retaining typed
  outcome and safe reason metadata in the validation summary. Do not expose
  provider errors or arbitrary skip prose in model or operator messages.

### AUD-WARN-006 — Relatedness Checks Are Useful But Repetitive And Lexically Weak

- **Priority:** Medium operator impact; low acceptance-policy urgency.
- **Evidence:** Every family runs the advisory in both extract and normalize
  chains. The complete fixture contains exact repeated tuples, and the checks
  rely on exact normalized token sequences or a minimal significant-token
  overlap. Reason naming drifts between `*_not_near_source` and
  `*_source_unrelated`.
- **Impact:** The checks can catch unsupported entities, but aliases, pronouns,
  inflection, and generic scene prose create predictable false positives. Flat
  per-record presentation magnifies them.
- **Recommendation:** Retain the validators and their stage-local execution,
  but classify and aggregate them as advisories. Normalize reason-code naming
  when the diagnostic contract changes. Do not strengthen them into rejection
  rules without a human-reviewed production evaluation.

### AUD-WARN-007 — `warning_count` Has No Stable Operational Meaning

- **Priority:** High downstream-contract impact.
- **Evidence:** The receipt and CLI use `len(output.Warnings)`. One list element
  can be an omission summary representing several hidden occurrences; repeated
  records can represent the same condition; and skipped validators can be
  absent. A 21-occurrence spell test produces a list length of 20.
- **Impact:** The value is neither an exact occurrence count nor a distinct
  warning-group count. Consumers cannot set policy or present a trustworthy
  summary from it.
- **Recommendation:** Introduce explicit warning-group and warning-occurrence
  counts in a versioned receipt. Do not silently redefine the v1 field.

## Recommended Target Contract And Presentation Model

### Diagnostic Model

Use one validated internal diagnostic model with these concepts:

- **disposition:** `warning`, `advisory`, or `observation`;
- **category:** a small enum such as `configuration`, `degradation`,
  `validation_incomplete`, `data_quality`, `fallback`, or `normalization`;
- **reason code:** stable semantic identity owned by the producer;
- **origin:** framework-added phase/stage, step ID, lane ID, module key,
  validator name, chunk ID, and chunk index when applicable;
- **occurrence count:** exact number of matching findings;
- **samples:** a small deterministic list of bounded scope/message pairs; and
- **omitted sample count:** `occurrence_count - len(samples)`, represented as
  metadata rather than another diagnostic record.

Errors and rejected outputs must not become diagnostic dispositions. A warning
means that the run completed under policy despite a process-level degradation
or incomplete configured operation. Advisory and observation dispositions can
describe accepted artifact quality and transformation provenance, but no
LLM-judged extraction-quality signal may be promoted to a warning. Validation
status remains authoritative for approval, rejection, and incomplete
validation.

### Aggregation

The framework runner should own aggregation after it enriches findings with
origin and before public output construction. Modules and validators retain
semantic ownership of disposition, category, reason, scope, and message; they
must not own CLI or file presentation.

The default stable key should be:

```text
disposition + category + reason_code
+ phase/stage + step_id + lane_id + module_key + validator_name
```

Chunk, record scope, and message text belong in samples and must not be part of
the group key. This groups repeated per-chunk findings without merging the same
code across distinct producers or pipeline locations. Group order should be
the first occurrence in the runner's existing canonical order; sample order
should follow the same order. A final canonical sort by the complete origin key
is also viable, but completion timing must never choose either order.

Aggregation must be incremental and bounded. Producers should use a shared
collector that counts every occurrence while retaining only bounded samples;
the framework then merges producer groups without reconstructing counts from
omission prose. A global maximum group count is also required, with overflow
represented by structured aggregate metadata and with actionable groups given
priority over lower dispositions.

### Durable Files

Keep one canonical home for each class:

- `warnings.json` should contain versioned, grouped actionable warnings only;
- a new `diagnostics.json` should contain versioned advisory and observation
  groups only, avoiding duplication of warning groups;
- `index.json` should link both files;
- `rejected.json` and manifest validation summaries should retain their current
  separate responsibilities; and
- debug bundles should retain candidate-attempt detail plus the final grouped
  projections.

This is preferable to keeping all detail in `warnings.json` and filtering only
the CLI: downstream consumers would otherwise continue to receive a semantically
mixed warning contract, and routine observations would still dominate the
durable file.

### CLI And Receipt

For a successful human run with actionable warnings, print a concise summary
such as:

```text
notarius: run completed with 2 warning groups (7 occurrences); details=/.../warnings.json
```

Advisories and observations should not produce the warning line. Their durable
path remains discoverable through `index.json`; a concise non-warning count can
be added to the ordinary success line only if operator testing shows value. An
ordinary successful run with no process degradation should write nothing to
the warning stream even when it publishes quality advisories or normalization
observations.

Create `notarius.run-result.v2` rather than redefining v1. It should expose at
least:

- `warning_group_count`;
- `warning_occurrence_count`; and
- `diagnostic_group_count` for non-warning durable groups.

The receipt should continue to expose validation status, validation summaries,
and rejected-output count independently. Process exit behavior should not
change as part of warning presentation reform.

### Checkpoint Semantics

Store the structured terminal diagnostic groups with accepted checkpoints and
replay them exactly once at their logical stage. Fresh and reused runs should
produce the same public groups and counts. Checkpoint events and debug records,
not diagnostic identity, should disclose whether computation was reused.

### Output Encoder Boundary

Prefer removing `OutputResult.Warnings`. A successful output encoder should
either return the complete logical files or fail. If future encoders genuinely
need to produce durable post-encoding warnings, introduce an explicit
two-phase prepare/finalize contract so those warnings can be included in the
same published collection. Do not retain the current self-inconsistent
one-pass capability.

## Resolution Of Required Design Questions

| Question | Recommendation | Viable alternative and tradeoff |
| --- | --- | --- |
| Explicit severity/disposition or external reason mapping? | Put validated disposition and category in the contract. | A central reason-code registry avoids payload fields but makes new modules depend on a second synchronized policy table and leaves downstream meaning implicit. |
| Keep all detail in `warnings.json` or separate it? | Separate grouped actionable warnings from grouped advisories/observations in `diagnostics.json`. | Keep the flat durable list and aggregate only CLI output; simpler migration, but it preserves the noisy downstream contract and ambiguous count. |
| Who owns aggregation? | Framework runner/coordinator after origin enrichment. | Output module aggregation keeps framework types smaller but duplicates policy across encoders and cannot repair missing validator origin. |
| Stable aggregation key? | Disposition, category, reason code, and full producer origin; exclude chunk/scope/message. | Explicit producer-supplied grouping keys offer flexibility but add another identity that can drift from reason codes. Message-template grouping is brittle and unsafe. |
| Samples and omissions? | Exact occurrence count plus deterministic bounded samples and numeric omitted-sample count. | Omission warning records preserve the current representation but inflate group counts and require prose parsing. |
| `warning_count` semantics? | Version receipt and replace ambiguity with group and occurrence counts. | Keep v1 count as published record length and add optional fields; compatible, but two competing warning counts remain easy to misuse. |
| Which normalization changes remain warnings? | Only exhausted process fallback. Unresolved membership is a data-quality advisory; successful canonicalization, reordering, ID repair, source-ref dedupe, and duplicate consolidation are observations. | Treat unresolved membership or semantic duplicate consolidation as warnings because they affect grounding or cardinality; more conservative, but it violates the process-only warning rule and reports accepted artifact quality as an operational failure. |
| Source-relatedness disposition? | Grouped advisory by default; preserve current approve behavior. | Retain warning disposition or make rejection configurable. Rejection requires production precision evidence; current lexical rules are not strong enough. |
| Checkpoint-loaded warnings? | Present the same logical groups as fresh execution and use checkpoint events for reuse provenance. | Mark groups as replayed; aids forensics but fragments aggregation and makes semantically equivalent runs differ. |
| ADR and schema versions? | Add an ADR and version the run receipt and diagnostic files. | Treat the work as CLI-only presentation and avoid an ADR; insufficient because module contracts, output files, checkpoint payloads, and downstream fields change. |

## Compatibility, Documentation, And ADR Implications

The target alters public and internal contracts enough to require a new ADR.
It should record:

- the distinction among warnings, advisories, observations, rejections, and
  errors;
- the invariant that warnings are process-level signals, LLM-judged extraction
  quality is never a warning, and ordinary non-degraded success has zero
  warnings;
- module semantic ownership versus framework origin/aggregation ownership;
- bounded group and sample semantics;
- fresh/checkpoint equivalence; and
- the output-encoder decision.

Implementation should introduce `notarius.run-result.v2`. The grouped warning
and diagnostic envelopes should each carry their own schema version. Because
the content of `warnings.json` changes incompatibly from a flat array wrapper
to groups, release notes and the published JSON integration contract must call
out the migration. `index.json` gains the diagnostic file path.

Canonical documentation updates belong in:

- `docs/cli.md` for stderr presentation only;
- `docs/operations.md` for operator review and debug workflow;
- `docs/integrations/json-output.md` for warning and diagnostic file schemas;
- `docs/integrations/run-result.md` for v2 fields and compatibility;
- `docs/consumers/subprocess.md` and `docs/consumers/dnd-pipeline.md` for
  downstream policy checks;
- `docs/internal/pipeline.md` for promotion, aggregation, retry, and checkpoint
  mechanics;
- `docs/internal/modules.md` and `docs/internal/dnd.md` for producer rules and
  the D&D classification matrix; and
- `docs/policy/architecture.md` for the durable ownership invariant after the
  ADR is accepted and implemented.

No configuration knob is required for the first implementation. A fixed,
well-documented taxonomy is easier to reason about than per-reason display
overrides. Configurable escalation or suppression can be considered only after
production review demonstrates a concrete operator need.

## Test Coverage Assessment

Existing coverage worth preserving includes:

- accepted-attempt and terminal-rejection warning promotion;
- validator failure retry exhaustion and `warn_continue`;
- module semantic retry fallback;
- deterministic warning order under concurrent lane completion;
- chunk-plan invalidation and discarded-cache warning behavior;
- fresh/checkpoint warning equivalence;
- local D&D warning caps and safe dynamic-message quoting;
- JSON warning-file publication; and
- CLI stderr, debug, and receipt counts.

Material gaps are:

- no bound test for NPC- or spell-relatedness warnings;
- no generic warning-field or result-size validation;
- no test for output encoder warnings versus published `warnings.json`;
- no operator-level aggregation or bounded-sample tests;
- no fresh/resume test for grouped counts because groups do not yet exist; and
- no production evaluation of advisory precision.

Tests should protect the semantic relationships: exact occurrence counts,
bounded samples, deterministic group order, actionable-only warning
presentation, and cross-surface equality. They should not assert one exact
warning count for every complete D&D run or treat message wording as a public
API unless the wording itself enforces a security boundary.

## Audit Conclusion

The application is in a good position for warning reform. Its retry,
validation, checkpoint, and concurrency mechanics provide reliable points at
which to attach structured diagnostics. The most valuable change is not to
suppress individual reason codes; it is to replace the semantically flat,
origin-free collection with bounded typed groups and to reserve the word
“warning” for conditions that merit operator attention.

Provider-backed runs would improve prioritization and help tune the D&D
advisories, but they are not necessary to conclude that routine normalization
and heuristic doubt should not dominate stderr or the durable warning
contract. They should be gathered before changing heuristic acceptance policy
or adopting a numerical production warning-volume target.
