142 lines
5.2 KiB
Markdown
142 lines
5.2 KiB
Markdown
# NWS AFD Section Heading Variants
|
|
|
|
## Status
|
|
|
|
Implemented.
|
|
|
|
## Problem
|
|
|
|
The NWS Area Forecast Discussion parser recognizes supported section headings
|
|
only when the section name is followed immediately by an ellipsis, for example:
|
|
|
|
```text
|
|
.SHORT TERM... (Through Monday)
|
|
.LONG TERM... (Tuesday through Sunday)
|
|
.AVIATION... (For the 18z TAFs through 18z Monday)
|
|
```
|
|
|
|
NWS offices also publish slash-qualified headings:
|
|
|
|
```text
|
|
.SHORT TERM /THROUGH MONDAY/...
|
|
.LONG TERM /MONDAY NIGHT THROUGH SUNDAY/...
|
|
.AVIATION /18Z TAFS THROUGH 18Z MONDAY/...
|
|
```
|
|
|
|
The current exact-prefix discovery logic does not find slash-qualified short-
|
|
or long-term sections. Its boundary detection also does not recognize a
|
|
slash-qualified aviation heading, so preceding section text can absorb aviation
|
|
content. The forecast discussion can still normalize successfully, leaving
|
|
downstream consumers with missing or incorrectly bounded structured prose.
|
|
|
|
## Feature Objective
|
|
|
|
Make supported AFD section recognition consistent across section discovery,
|
|
section-boundary detection, and qualifier extraction while preserving the
|
|
existing canonical forecast-discussion contract.
|
|
|
|
## Targeted End State
|
|
|
|
One NWS-specific heading parser classifies a trimmed AFD line as either a
|
|
supported heading or a non-heading. For a supported heading it provides:
|
|
|
|
- the canonical section identity;
|
|
- the qualifier, if present; and
|
|
- enough information for discovery and boundary detection to use the same
|
|
recognition result.
|
|
|
|
The supported section identities remain:
|
|
|
|
- `KEY MESSAGES`;
|
|
- `SHORT TERM`;
|
|
- `LONG TERM`; and
|
|
- `AVIATION`.
|
|
|
|
`KEY MESSAGES`, `SHORT TERM`, and `LONG TERM` participate in the existing
|
|
canonical extraction behavior. `AVIATION` remains a recognized boundary only;
|
|
it does not become a canonical payload field.
|
|
|
|
For each supported identity, the parser accepts both heading families:
|
|
|
|
```text
|
|
.<SECTION>...<optional qualifier>
|
|
.<SECTION> /<qualifier>/...
|
|
```
|
|
|
|
Qualifier handling is deterministic:
|
|
|
|
- leading and trailing whitespace is removed;
|
|
- the enclosing slash pair used by the slash-qualified form is removed;
|
|
- qualifier content otherwise retains its published text, including
|
|
parentheses in the existing ellipsis-first form; and
|
|
- a heading without a qualifier yields an empty qualifier.
|
|
|
|
The same classification rules govern discovery and termination. A recognized
|
|
heading ends the preceding section without becoming part of its text. Unknown
|
|
section names, unrelated dotted lines, and malformed slash-qualified lines are
|
|
not treated as supported headings.
|
|
|
|
## Scope of Work
|
|
|
|
The feature includes:
|
|
|
|
- consolidating supported-heading recognition and qualifier extraction in
|
|
`internal/providers/nws`;
|
|
- applying that recognition consistently to section lookup and section
|
|
boundaries;
|
|
- preserving key-message parsing and short- and long-term section mapping;
|
|
- covering legacy, slash-qualified, mixed-format, and malformed headings with
|
|
focused provider-parser tests; and
|
|
- verifying that parsed short- and long-term sections continue through the NWS
|
|
normalizer into the existing canonical payload.
|
|
|
|
Existing ellipsis-first fixtures and behavior remain regression coverage. New
|
|
fixtures should be minimal and representative rather than copies of complete
|
|
upstream bulletins unless a full bulletin is needed to prove an interaction.
|
|
|
|
## Compatibility and Contracts
|
|
|
|
This is a provider-parsing compatibility improvement. It does not change:
|
|
|
|
- event kinds or raw and canonical schema identifiers;
|
|
- canonical models or JSON field names;
|
|
- source configuration or polling behavior;
|
|
- event envelope or effective-time behavior;
|
|
- Postgres tables or event-to-row mapping; or
|
|
- downstream sink and consumer responsibilities.
|
|
|
|
Current-behavior documentation should be updated only if implementation reveals
|
|
an externally observable contract change beyond the scope defined here.
|
|
|
|
## Acceptance Criteria
|
|
|
|
The feature is complete when automated tests demonstrate that:
|
|
|
|
- all currently accepted ellipsis-first headings produce unchanged results;
|
|
- slash-qualified short- and long-term headings are discovered and their
|
|
qualifiers exclude slash delimiters;
|
|
- slash-qualified recognized headings correctly terminate a preceding section,
|
|
including an aviation heading following a long-term section;
|
|
- documents mixing the two heading families are parsed correctly;
|
|
- headings without qualifiers retain existing behavior;
|
|
- unknown or malformed heading-like lines do not create supported sections or
|
|
prematurely terminate one;
|
|
- the NWS forecast-discussion normalizer emits populated canonical short- and
|
|
long-term fields for representative slash-qualified input without adding an
|
|
aviation field; and
|
|
- the full repository test suite passes.
|
|
|
|
## Non-Goals
|
|
|
|
This feature does not:
|
|
|
|
- add new canonical forecast-discussion sections;
|
|
- add aviation prose to the canonical model;
|
|
- recognize arbitrary or previously unsupported NWS section families;
|
|
- introduce heuristic section-name matching or AFD summarization;
|
|
- preserve complete raw AFD documents in canonical payloads; or
|
|
- change schemas, persistence contracts, configuration, or downstream APIs.
|
|
|
|
Support for additional section identities or broader AFD structure should be
|
|
driven by a separate consumer requirement and roadmap.
|