Summer WAS*IS 2006

National Weather Service National Weather Service Verification Program OverviewVerification Program Overview

Brenton MacAloney II

National Weather Service Headquarters

Silver Spring, MD

July 19th, 2006

Summer WAS*IS 2006

Verification ProgramsVerification Programs

The National Weather Service (NWS) Performance Branch collects warning and forecast products, as well as observations to generate verification scores.

These verification scores are displayed on the Performance Management website accessible only to NWS employees.

Summer WAS*IS 2006

Verification ProgramsVerification Programs

Verification data is available via the Stats on Demand program.

Summer WAS*IS 2006

Verification ProgramsVerification Programs

Verification statistics are calculated on two types of products.

WarningsForecasts

Summer WAS*IS 2006

Forecast VerificationForecast Verification

Aviation– Terminal Aerodrome Forecast (TAF)

• Ceiling, Visibility, Significant Weather, Wind Speed, Direction, and Gust, and Flight Category

Hydrology– River Stage Forecast – Quantitative Precipitation Forecasts (QPF)

Summer WAS*IS 2006

Forecast VerificationForecast VerificationMarine

– Coastal and Offshore Forecast• Wave Height, Wind Speed, Wind Direction

Fire Weather– National Fire Danger Rating System Forecasts (NFDRS)

• State of the Weather, Temperature, Humidity, Wind Speed

Public– Point Forecast Matrices (PFM) Forecasts

• Max/Min Temperature, Probability of Precipitation (PoP)

Summer WAS*IS 2006

Warning VerificationWarning Verification

• Tornado (TOR)• Severe Thunderstorm (SVR)• Flash Flood (FFW)• Special Marine (SMW)

• Winter Storm (WSW)• High Wind (NPW)• Coastal Flood (CFW)

Short Duration Warnings Long Duration Warnings*

* Verification of long duration warnings is currently done by hand. Starting in October 2006, the verification of long duration warnings will be automated.

Summer WAS*IS 2006

Warning VerificationWarning Verification

As far back as 1986, warnings have been verified using counties warned in the Universal Geographic Code (UGC) line of a warning product.

Verifiable events are collected, entered into storm data at the forecast offices (WFO), and sent to NWS Headquarters monthly.

Summer WAS*IS 2006

Warning VerificationWarning Verification

Any event occurring during the valid time of warned county, verifies the warning. The event is also considered “warned”.

Summer WAS*IS 2006

Warning VerificationWarning VerificationThe NWS is changing how they issue short duration warnings. In the next few years, warnings will be issued where the “polygon” is the warning area, not the county.

This “polygon” information is currently available on all SVR, TOR, FFW, and SMW products.

Summer WAS*IS 2006

Brent’s Dream WorldBrent’s Dream World

• Due to a limited budget, the majority of the money the Performance Branch receives is spent on the development of new verification programs.

• Having verification information is great, but it does not tell the whole story.

• I would love to see other areas of “Performance Evaluation” explored.

Summer WAS*IS 2006

Questions and IssuesQuestions and Issues

• Performance evaluation questions I would like to find answers to:

– What is an acceptable False Alarm Ratio to the public? to Emergency Managers? to businesses? to others who depend on our warnings?

Summer WAS*IS 2006

Questions and IssuesQuestions and Issues

• Other questions:

– Are there any new or existing metrics that we are not using and might be useful in showing a better picture of our performance?

Summer WAS*IS 2006

Questions and IssuesQuestions and Issues

• Other questions:

– How can we get the forecasters to better understand the current metrics so that they are able to improve their forecasts and warnings?

Summer WAS*IS 2006

Questions and IssuesQuestions and Issues

• Other questions:

– Are there any ways for the WFOs to easily do their own service assessments? Does anyone know of any software that might help? What types of relationships can be set up within the local community that can help channel feedback to the WFO on their performance?

Summer WAS*IS 2006

Contact InformationContact Information

Brenton MacAloney II

Brent.MacAloney@noaa.gov

Phone: 301-713-0090 x135