NWS AFD Section Heading Variants

Status

Implemented.

Problem

The NWS Area Forecast Discussion parser recognizes supported section headings only when the section name is followed immediately by an ellipsis, for example:

.SHORT TERM...  (Through Monday)
.LONG TERM...  (Tuesday through Sunday)
.AVIATION...  (For the 18z TAFs through 18z Monday)

NWS offices also publish slash-qualified headings:

.SHORT TERM /THROUGH MONDAY/...
.LONG TERM /MONDAY NIGHT THROUGH SUNDAY/...
.AVIATION /18Z TAFS THROUGH 18Z MONDAY/...

The current exact-prefix discovery logic does not find slash-qualified short- or long-term sections. Its boundary detection also does not recognize a slash-qualified aviation heading, so preceding section text can absorb aviation content. The forecast discussion can still normalize successfully, leaving downstream consumers with missing or incorrectly bounded structured prose.

Feature Objective

Make supported AFD section recognition consistent across section discovery, section-boundary detection, and qualifier extraction while preserving the existing canonical forecast-discussion contract.

Targeted End State

One NWS-specific heading parser classifies a trimmed AFD line as either a supported heading or a non-heading. For a supported heading it provides:

the canonical section identity;
the qualifier, if present; and
enough information for discovery and boundary detection to use the same recognition result.

The supported section identities remain:

KEY MESSAGES;
SHORT TERM;
LONG TERM; and
AVIATION.

KEY MESSAGES, SHORT TERM, and LONG TERM participate in the existing canonical extraction behavior. AVIATION remains a recognized boundary only; it does not become a canonical payload field.

For each supported identity, the parser accepts both heading families:

.<SECTION>...<optional qualifier>
.<SECTION> /<qualifier>/...

Qualifier handling is deterministic:

leading and trailing whitespace is removed;
the enclosing slash pair used by the slash-qualified form is removed;
qualifier content otherwise retains its published text, including parentheses in the existing ellipsis-first form; and
a heading without a qualifier yields an empty qualifier.

The same classification rules govern discovery and termination. A recognized heading ends the preceding section without becoming part of its text. Unknown section names, unrelated dotted lines, and malformed slash-qualified lines are not treated as supported headings.

Scope of Work

The feature includes:

consolidating supported-heading recognition and qualifier extraction in internal/providers/nws;
applying that recognition consistently to section lookup and section boundaries;
preserving key-message parsing and short- and long-term section mapping;
covering legacy, slash-qualified, mixed-format, and malformed headings with focused provider-parser tests; and
verifying that parsed short- and long-term sections continue through the NWS normalizer into the existing canonical payload.

Existing ellipsis-first fixtures and behavior remain regression coverage. New fixtures should be minimal and representative rather than copies of complete upstream bulletins unless a full bulletin is needed to prove an interaction.

Compatibility and Contracts

This is a provider-parsing compatibility improvement. It does not change:

event kinds or raw and canonical schema identifiers;
canonical models or JSON field names;
source configuration or polling behavior;
event envelope or effective-time behavior;
Postgres tables or event-to-row mapping; or
downstream sink and consumer responsibilities.

Current-behavior documentation should be updated only if implementation reveals an externally observable contract change beyond the scope defined here.

Acceptance Criteria

The feature is complete when automated tests demonstrate that:

all currently accepted ellipsis-first headings produce unchanged results;
slash-qualified short- and long-term headings are discovered and their qualifiers exclude slash delimiters;
slash-qualified recognized headings correctly terminate a preceding section, including an aviation heading following a long-term section;
documents mixing the two heading families are parsed correctly;
headings without qualifiers retain existing behavior;
unknown or malformed heading-like lines do not create supported sections or prematurely terminate one;
the NWS forecast-discussion normalizer emits populated canonical short- and long-term fields for representative slash-qualified input without adding an aviation field; and
the full repository test suite passes.

Non-Goals

This feature does not:

add new canonical forecast-discussion sections;
add aviation prose to the canonical model;
recognize arbitrary or previously unsupported NWS section families;
introduce heuristic section-name matching or AFD summarization;
preserve complete raw AFD documents in canonical payloads; or
change schemas, persistence contracts, configuration, or downstream APIs.

Support for additional section identities or broader AFD structure should be driven by a separate consumer requirement and roadmap.

5.2 KiB Raw Blame History