Forecast Skill and Major Forecast Failures over the Northeastern

Pacific and Western North America

Lynn McMurdie

and Cliff Mass

University of Washington

Why the North Pacific Ocean?

• The Pacific is one of the largest areas of sparse insitu observations in the world

• Uncertainty over the Pacific has a large impact on predictability over the North American continent and beyond

• There are often large initialization errors and short-term forecasts over the northern Pacific ocean.

• One symptom of such problems is that short-term forecast skill on the western side of North America is worse than in other areas.

An example of a short-term forecast error

Eta 24-h

03 March 00UT 1999

Eta 48-h

03 March 00UT 1999

And the public and media have noticed these failures….

Seattle Times Eugene Register Guard

Some applicable research in the literature

• Langland et al. (2002) – poor forecast of Jan 2000 storm on East Coast due in part to sensitivity over the Pacific.

• Bosart et al. (2002) – lack of convection over the midwest not represented in forecasts due to poorly initialized trough along west coast

• Schultz et al. (2005) – 70% of troughs arriving on the West Coast were underforecast, a portion of which continued to effect short-term forecasts across the North American continent.

• McMurdie and Mass (2004) – documented forecast failures over the eastern Pacific.

This talk will …

• Demonstrate that large initial condition and short-term forecast errors still occur over the eastern Pacific and downstream

• Present a feature-based approach to monitoring errors in this region

• Discuss implications for THORPEX

How frequent are large numerical forecast errors?

• Approach: compare buoys/coastal observations of sea level pressure (SLP) to NCEP’s Eta and GFS 00, 24, 36 and 48 hr forecasts.

• Error = Forecast SLP – Observed SLP• At each station, calculated average and

absolute error and the standard deviation using winter (Oct – Mar) data.

• Large Error = |Error| > (average error + 2 * SD)

Station Locations

Tatoosh Is.

Cape Arago

24 h

Large Errors

Tatoosh Is., WA

Cape Arago, OR

Inter-annual variability

48-h Errors

48h errors much larger and more frequent than 24-h errors

GFS vs. Eta 24-h errors

NCEP GFS better than Eta on average

48-h errors

GFS over forecastsEta under forecasts

GFS has more accurate SLP initializations and forecasts than Eta over the Northeast Pacific

• For 00-h forecasts (initial conditions), GFS has smaller mean absolute error (MAE) and standard deviation (SD) than Eta at all 17 stations

• For 24- and 36-h forecasts, GFS has smaller MAE and SD than Eta at 13/17 buoy and coastal stations

• For 48-h forecasts, GFS has smaller MAE and SD than Eta at 12/17 stations.

Forecast Verification: The Need for Feature-Based Evaluation

• Examining statistics at observing sites is not sufficient for understanding the problems.

• Must also track features to gain an understanding of the deficiencies.

• Case studies of major failures should reveal important information.

GFS

What are these large forecast errors associated with?

How frequently do large forecast errors of synoptic events occur?

Number of Events/Season associated with Lows/Troughs/Highs

Season Total Low Trough High

1999 – 2000 21 12 7 2

2000 – 2001 19 12 4 2

2001 – 2002 16 14 2 0

2002 – 2003 16 11 5 0

(from McMurdie and Mass 2004)

Event = large error at 2 or more adjacent stations for 2 or more forecasts periods

Data shown for Eta model only

Of the forecast failures associated with lows, what are the central pressure and cyclone position errors?

Ave SLP error = 3.4 mbSD = 8.7 mbAbsolute error = 7.5 mb

Ave position error = 453.8 kmSD = 260 km

Recent examples of major forecast errors

• February 2002

• October 2003

• February 2004

• November 2004

• April 2005

• May 2005

An example of a recent high-impact, poorly forecast storm

•Power outages, large trees uprooted in Eugene, OR

• Powerful, rapidly developing storm with strong winds (70 kts)

• Very poor short-term numerical guidance

L1008

L1004

L996

L1010

7 – 8 February 2002 Cyclone

48-hr Forecasts Valid 00 UTC 8 February 2002

AVN

UKMO

ETA

NOGAPS

24-Hr Forecasts Valid 00 UTC 8 February 2002

AVN

UKMO

ETA

NOGAPS

Difference between UKMO and Eta 850 mb Temperature K

Valid 00 UTC 7 February 2002

Solid = UKMO, Dashed = ETA, Shades, blue = differences

L1010

20 Oct 03

Flood of 20 October 2003

00hr GFS 24hr GFS 48hr GFS 00hr + 48hr GFS

GFS Forecasts of 12-hr Precipitation

12h Forecast24h Forecast36h Forecast48h Forecast

February 04

00UTC 16 Feb 04 GFS 00-hr Forecast

00 UTC 16 Feb 04 24-hr GFS

00 UTC 16 Feb 04 48 hr GFS

Apr 05

24-hr forecast GFS Position error ~ 420km

Large Short Term Forecast Errors Still Occur

• Number of slp errors > 10 mb continues to be 10 – 15 per winter (despite the ridge this year)

• Vast majority of large errors due to mispositioned or under (or over) forecast low centers (see McMurdie and Mass, 2004)

• For Feb 02 case, forecast errors were likely due to initial condition errors (McMurdie and Mass, 2004)

Some Unanswered Questions• What are the origins of these short-term forecast

errors – initial condition/data assimilation errors, model errors?

• Are there particular flow patterns (or regimes) where short-term (or longer term) forecasts are less accurate (e.g., E-T transitions)?

• How do model sensitivity structures compare for major forecast failure cases? How do they project on obvious initialization problems? How do adjoint-based and ensemble-based sensitivities compare for such cases?

Unanswered Questions continued

• What are the downstream implications for medium to long- range forecasts when initial condition errors are large over the Pacific? To what degree are downstream errors mitigated by greater data density over North America?

Implications for THORPEX• Major forecast failures still occur, even at the short-

time ranges. So there is still work to be done!• It is important to monitor the quality of model

initializations and forecasts to know how well we are doing and where the failures are.

• Both statistical and feature-based approaches are needed to gain a full understanding of model failures.

• Case studies can provide important insights into forecast failures

The END

Recent trends in forecast accuracy

From Simmons and Hollingsworth (2002) Hemispheric r.m.s. error of SLP

Increased skill of 3-5 Day forecast skill of SLP (and 500 hghts) especially last 10 years.

Unable to discern forecast skill of storm systems in particular locations from these statistics

Brief Outline

• Show statistics of short term errors along West Coast

• Highlight several examples of major forecast failures

• Briefly discuss the effect of uncertainty over the Pacific on longer term forecasts

Adjoint Sensitivity wrt 850 mb Temperature

Area of forecast error projected onto sensitivities

Courtesy of Brian Ancell

2004 17 Nov

2004 17 Nov

6-hr forecast Eta

6-hr Forecast GFS