# NWS AFD Section Parsing Resilience

## Status

Implemented.

## Implemented Behavior

The NWS Area Forecast Discussion parser recognizes these heading families for
all structurally valid section identities:

```text
.<SECTION>...<optional qualifier>
.<SECTION> /<qualifier>/...
```

Discovery, qualifier extraction, and recognized-section boundary detection use
one provider-specific parser. Slash delimiters are removed from qualifiers,
legacy qualifier text is preserved, and aviation remains a boundary rather than
a canonical field.

The parser separates:

1. generic structural heading recognition;
2. section boundary scanning;
3. canonical section-role selection; and
4. section-preamble and presentation cleanup.

Structurally valid topic names act as safe boundaries without becoming new
canonical fields.

## Implemented Details

### Generic heading recognition

One NWS-specific heading parser recognizes structurally valid uppercase AFD
topic lines independently of the canonical model. It supports:

```text
.<IDENTITY>...<optional qualifier>
.<IDENTITY> /<qualifier>/...
```

The identity may contain uppercase ASCII letters, digits, horizontal whitespace,
`/`, `&`, apostrophes, and hyphens. Outer whitespace and whitespace immediately
before the ellipsis are ignored, and internal identity whitespace is collapsed
to one space for role lookup. Slash-qualified headings require a nonempty
qualifier but may contain slash characters inside that qualifier. Lowercase
prose, ordinary dotted lines, and malformed delimiters remain non-headings.

Every structurally valid heading terminates the preceding section. This applies
to known boundary-only sections such as aviation and to future or office-specific
topic identities that weatherfeeder does not map.

### Canonical role selection

A single provider-local role registry maps only these identities into existing
parsed fields:

- `KEY MESSAGES` to key messages;
- `SHORT TERM` to the short-term section; and
- `LONG TERM` to the long-term section.

All other identities are boundary-only. The first occurrence of each mapped
identity wins, preserving current behavior if an unusual bulletin repeats a
section. Heading syntax and canonical roles do not duplicate identity lists.

### Section scanning

The discussion text is scanned once into ordered section blocks. A new heading,
`&&`, `$$`, or the existing watch/advisory terminator ends the active block.
Original body lines are preserved until provider presentation cleanup is
applied. Preamble text before the first heading and post-`$$` signatures are not
treated as section content.

### Preamble and presentation handling

For short- and long-term sections:

- a qualifier on the heading line remains authoritative;
- when the heading has no qualifier, a standalone parenthesized first content
  line becomes the qualifier and retains its parentheses;
- an optional case-insensitive `Issued at` line after the qualifier is parsed
  into the existing section issue time; and
- exact NWS change-presentation marker lines are removed without removing
  arbitrary dashed prose.

For key messages, exact change-presentation markers and one leading `Issued at`
or `Updated at` metadata line are removed before bullet parsing. Those metadata
lines never become key messages.

### Representative coverage

Tests include small structural tables and maintained local HTML fixtures for at
least two real NWS formatting families: slash-qualified same-line headings and
ellipsis-first headings with multiline qualifiers or change-presentation
markers. Tests remain deterministic and never contact live services.

## Compatibility and Contracts

This remains a provider-parsing compatibility improvement. It does not change:

- event kinds or raw and canonical schema identifiers;
- canonical models or JSON field names;
- source configuration, URLs, or polling behavior;
- event envelope or effective-time behavior;
- Postgres tables or event-to-row mapping; or
- downstream sink and consumer responsibilities.

A single `DISCUSSION` section and other currently unmapped identities are
recognized as boundaries but are not forced into short- or long-term fields.
Exposing such content would require a separate canonical schema decision.

## Verified Coverage

Automated tests demonstrate that:

- every previously accepted heading and canonical result remains compatible;
- generic structurally valid identities terminate preceding content without
  becoming canonical fields;
- malformed heading-like lines and lowercase prose remain body content;
- identities containing `/` and slash qualifiers containing `/` are parsed
  without ambiguity;
- next-line parenthesized qualifiers and following `Issued at` lines populate
  the existing short- and long-term metadata correctly;
- exact NWS change markers and leading key-message timestamps do not leak into
  canonical prose or messages;
- repeated mapped sections preserve first-occurrence behavior;
- genuine cross-office fixture styles propagate correctly through the NWS
  normalizer with unchanged wire shape; and
- focused tests, the full repository suite, and static analysis pass.

## Non-Goals

This follow-up does not:

- add canonical `discussion`, aviation, marine, hydrology, climate, fire
  weather, update, or arbitrary-section fields;
- infer short- or long-term semantics from an unknown heading;
- accept free-form or lowercase prose as a heading;
- introduce heuristic summarization;
- preserve complete raw AFD documents in canonical payloads;
- change schemas, persistence contracts, configuration, or downstream APIs; or
- fetch live NWS data during tests.

Additional canonical support for AFD section identities requires a separate
consumer requirement and schema roadmap.