Performance of NOAA-EPA Performance of NOAA-EPA air quality predictions, 2007 - 2009air quality predictions, 2007 - 2009

Jerry Gorline and Pius LeeJerry Gorline and Pius Lee

• Overview of the National Air Quality Overview of the National Air Quality Forecasting Capability with recent examples.Forecasting Capability with recent examples.

• Evaluation of Evaluation of experimental ozoneexperimental ozone predictions predictions for summer 2007, 2008, 2009.for summer 2007, 2008, 2009.

• Evaluation of Evaluation of developmental aerosol developmental aerosol predictions for January 2008 to August 2009.predictions for January 2008 to August 2009.

Issues for multi-year evaluation:Issues for multi-year evaluation:• Unusually cool weather in the East in 2009.Unusually cool weather in the East in 2009.• CB05 replaced CBIV.CB05 replaced CBIV.• AERO4 introduced.AERO4 introduced.• NAM updates.NAM updates.

National Centers for Environmental Prediction (NCEP)National Centers for Environmental Prediction (NCEP) - Model predictions (8-h avg ozone, 1-h avg aerosols):- Model predictions (8-h avg ozone, 1-h avg aerosols):

Jeff McQueen, Youhua Tang, Marina TsidulkoJeff McQueen, Youhua Tang, Marina TsidulkoEnvironmental Protection Agency (EPA)Environmental Protection Agency (EPA) - Point observations: (ozone: ~1300, aerosols: ~600):- Point observations: (ozone: ~1300, aerosols: ~600):

Brad Johns, Phil Dickerson, Scott JacksonBrad Johns, Phil Dickerson, Scott JacksonAir Resources Laboratory (ARL)Air Resources Laboratory (ARL)

Daewon Byun, Pius LeeDaewon Byun, Pius LeeMeteorological Development Laboratory (MDL):Meteorological Development Laboratory (MDL):

Jerry Gorline et al.Jerry Gorline et al.

National Air Quality National Air Quality Forecast Capability (NAQFC) Forecast Capability (NAQFC)

Program Manager:Program Manager: Paula Davidson Paula Davidson

3

National Air Quality Forecast CapabilityNational Air Quality Forecast CapabilityCurrent and Planned Capabilities, 10/09Current and Planned Capabilities, 10/09 • Improving the basis for AQ alertsImproving the basis for AQ alerts• Providing AQ information for people at risk Providing AQ information for people at risk

Near-term Operational Targets:Near-term Operational Targets:• Ozone, smoke coverage extended Nationwide• Higher resolution prediction (4km)

Longer range:Longer range:• Quantitative PMQuantitative PM2.52.5 prediction prediction• Extend air quality forecast range to 48-72 hoursExtend air quality forecast range to 48-72 hours• Include broader range of significant pollutantsInclude broader range of significant pollutants

FY09 Prediction Capabilities: FY09 Prediction Capabilities: • Operations: Operations:

Ozone, expanded from EUS to CONUS, 9/07Ozone, expanded from EUS to CONUS, 9/07Smoke implemented over CONUS, 3/07 and AK, 9/09Smoke implemented over CONUS, 3/07 and AK, 9/09

• ExperimentalExperimental testing: testing: - Ozone upgrades- Ozone upgradesSmoke predictions over HI in 2009Smoke predictions over HI in 2009

• DevelopmentalDevelopmental testing:testing: Ozone over AK and HIOzone over AK and HI- - Components for particulate matter (PM) forecastsComponents for particulate matter (PM) forecasts

2005: O2005: O3320072007: : OO3,3,& smoke& smoke

66

2010: smoke

2010: O3 AK,HI2009: smoke

Other products: Smoke Other products: Smoke PredictionsPredictions

Station Fire, California Aug-Sept Station Fire, California Aug-Sept 20092009

Intense wild-land fire north of LA • 2 fatalities

• Smoke impacts closed roads, national parks in CA; extended throughout Southwest US

• 8/26- 9/12 (still active; 84% contained) • > 160,000 acres burned

2x2 contingency table definitions2x2 contingency table definitions

Observedabove threshold? Yes No

Predicted Yes a byy yn

No c dny nn

Categorical performance metrics:Categorical performance metrics: FC = (a + d)/(a + b + c + d)FC = (a + d)/(a + b + c + d) TS = a/(a + b + c) TS = a/(a + b + c) Thresholds used:Thresholds used: POD = a/(a + c) ozone: 76 ppbPOD = a/(a + c) ozone: 76 ppb FAR = b/(a+b) aerosols: 35 ug/mFAR = b/(a+b) aerosols: 35 ug/m33

Real-time Testing, Summer 2009:Real-time Testing, Summer 2009:

Experimental vs Operational OExperimental vs Operational O33 at 76 ppb at 76 ppb

FC lower for experimental predictionsFC lower for experimental predictions

OPNL Predictions Fraction Correct, from 4/09: 5X 8-hr avg CONUS 76 ppb THRESHOLD

0.972 0.9680.9540.994

0.992

0.8

0.85

0.9

0.95

1

4/1/09 4/16/09 5/1/09 5/16/09 5/31/09 6/15/09 6/30/09 7/15/09 7/30/09 8/14/09 8/29/09

Fraction Correct 75ppb

Monthly Cum 75-Threshold

Target

OperationalOperational

CBIV-basedCBIV-based

ExperimentalExperimental

CB05-basedCB05-based

Fraction Correct, Experimental Ozone Predictions, 1200 UTC Daily Maximum of 8-h avg, Full 5X Domain, Th=76 ppb

0.80

0.85

0.90

0.95

1.00

11-May 25-May 8-Jun 22-Jun 6-Jul 20-Jul 3-Aug 17-Aug 31-Aug

Fraction Correct

Target

20082008ozone activityozone activity

aa + + cc

bbccaa

Less active Less active in 2009,in 2009,

less under-less under-predictionprediction

in June 2009in June 2009

20092009ozone activityozone activity

aa + + cc

bbccaa

outbreakoutbreak

no outbreaksno outbreaks

8-h ozone bias, three-year comparison8-h ozone bias, three-year comparisonAugust 2007, 2008, 2009August 2007, 2008, 2009

CONUS, 1200 UTC cycleCONUS, 1200 UTC cycle

experimental:experimental:slightly higher slightly higher

bias in 2009bias in 2009

2007: CBIV2007: CBIV2008: CB052008: CB052009: CB052009: CB05

Daily max, 8-h ozone, July 18, Daily max, 8-h ozone, July 18, 20082008

1200 UTC experimental1200 UTC experimental

FC=0.773TS=0.362POD=0.766FAR=0.593

predicted in dark blueobserved as red dots

Four-day Four-day outbreak of outbreak of 2008, day 3,2008, day 3,

188 observed 188 observed cases abovecases above

76 ppb76 ppb

Daily max, 8-h ozone, July 1, Daily max, 8-h ozone, July 1, 20092009

1200 UTC experimental1200 UTC experimental

FC=0.898TS=0.341POD=0.838FAR=0.635

predicted in dark blueobserved as red dots

Most active Most active day of 2009,day of 2009,68 observed, 68 observed, Performance Performance

similar to similar to July 18, 2008July 18, 2008

Number of observed values higher than Number of observed values higher than 76 ppb threshold, 1200 UTC experimental76 ppb threshold, 1200 UTC experimental

8-h ozone, by region8-h ozone, by region

Fewer cases in Fewer cases in the East in 2009,the East in 2009,

compared to compared to 2007 and 2008 2007 and 2008

Threat Score (TS)Threat Score (TS)8-h ozone, by region 8-h ozone, by region

2007 to 2009, Th=76 ppb2007 to 2009, Th=76 ppb

Lower TS in the Lower TS in the East in 2009, the East in 2009, the unusually cool unusually cool summer, fewer summer, fewer

cases above cases above threshold, may threshold, may have played a have played a

role here.role here.

Seasonal variation of activitySeasonal variation of activity8-h ozone vs. 1-h aerosols8-h ozone vs. 1-h aerosolsJune 2008 to August 2009June 2008 to August 2009

Theshold: 76 ppb (ozone) 35 ug/mTheshold: 76 ppb (ozone) 35 ug/m3 3 (aerosols)(aerosols)

Aerosols are Aerosols are active in both active in both

winter and winter and summersummer

Regional monthly bias of aerosolsRegional monthly bias of aerosolsJan 2008 to Aug 2009, 35 ug/mJan 2008 to Aug 2009, 35 ug/m33

lower bias inlower bias inwinter 2009 winter 2009 compared to compared to winter 2008,winter 2008,

except PC regionexcept PC region

CB05CB05AERO-4AERO-4

July 4, 2009July 4, 2009Aerosols from fireworks,Aerosols from fireworks,

not in emissions inventorynot in emissions inventory

FC=0.600TS=0.024POD=0.024FAR=0.167 under-predictionunder-prediction

Summary: ozone and aerosolsSummary: ozone and aerosols

2008 ozone performance similar to 2007, lower 2008 ozone performance similar to 2007, lower TS in 2009 in the East with fewer observed cases TS in 2009 in the East with fewer observed cases above threshold (unusually cool summer, above threshold (unusually cool summer, possibly emission issues, economy?).possibly emission issues, economy?). Slightly higher and more consistent ozone bias Slightly higher and more consistent ozone bias in 2009 (CB05) compared to 2008 (CBIV).in 2009 (CB05) compared to 2008 (CBIV). Aerosol over-prediction in cool season (Oct-Aerosol over-prediction in cool season (Oct-Mar), under-prediction in warm season (Apr-Sep).Mar), under-prediction in warm season (Apr-Sep). Seasonal aerosol biases smaller in 2009 Seasonal aerosol biases smaller in 2009 than in 2008.than in 2008. Plan to include verification of AK and HI Plan to include verification of AK and HI in 2010.in 2010.

Acknowledgments:Acknowledgments: AQF Implementation Team Members AQF Implementation Team Members

OSTOST Paula Davidson Paula Davidson Program manager Program manager

OCWWSOCWWS Jannie Ferrell Jannie Ferrell Outreach, FeedbackOutreach, Feedback

OCIO OCIO Cindy Cromwell, Dan Starosta, Bob BungeCindy Cromwell, Dan Starosta, Bob Bunge Data CommunicationsData Communications

OST/MDL OST/MDL Jerry Gorline Jerry Gorline Dev. VerificationDev. Verification

OST/MDLOST/MDL Marc Saccucci, Tim Boyer, Dave RuthMarc Saccucci, Tim Boyer, Dave Ruth NDGD Product DevelopmentNDGD Product Development

OSTOST Ken Carey, Ivanka StajnerKen Carey, Ivanka Stajner Program SupportProgram Support

NESDIS/NCDCNESDIS/NCDC Alan HallAlan Hall Product ArchivingProduct Archiving

NOAA/OARNOAA/OAR Jim MeagherJim Meagher NOAA AQ Matrix ManagerNOAA AQ Matrix Manager

NCEPNCEP

Jeff McQueen, Youhua Tang, Marina Tsidulko, Jeff McQueen, Youhua Tang, Marina Tsidulko, EMC AQF model interface development, testing and integration EMC AQF model interface development, testing and integration

Jianping Huang, Dongchul Kim Jianping Huang, Dongchul Kim

*Sarah Lu , Ho-Chun Huang *Sarah Lu , Ho-Chun Huang Global data assimilation and feedback testing (NASA, NESDIS)Global data assimilation and feedback testing (NASA, NESDIS)

*Brad Ferrier, *Dan Johnson, *Eric Rogers, *Hui-Ya Chuang*Brad Ferrier, *Dan Johnson, *Eric Rogers, *Hui-Ya Chuang WRF/NAM coordinationWRF/NAM coordination

Geoff ManikinGeoff Manikin Smoke Product testing and integrationSmoke Product testing and integration

John Ward, Brent Gordon, Chris MageeJohn Ward, Brent Gordon, Chris Magee NCO transition and systems testingNCO transition and systems testing

Robert Kelly, Bob Bodner, Andrew OrrisonRobert Kelly, Bob Bodner, Andrew Orrison HPC coordination and AQF webdrawerHPC coordination and AQF webdrawer

* * Guest ContributorsGuest Contributors

NOAA/OARNOAA/OAR

Daewon Byun, Pius Lee, Rick Saylor, Hsin-Mu Lin, Daewon Byun, Pius Lee, Rick Saylor, Hsin-Mu Lin, CMAQ development, adaptation of AQ simulations for AQFCMAQ development, adaptation of AQ simulations for AQF

Daniel Tong, Tianfeng Chai, Fantine NganDaniel Tong, Tianfeng Chai, Fantine Ngan

Roland Draxler, Glenn Rolph, Ariel SteinRoland Draxler, Glenn Rolph, Ariel Stein HYSPLIT adaptationsHYSPLIT adaptations

NOAA/NESDISNOAA/NESDIS

Shobha Kondragunta, Jian ZengShobha Kondragunta, Jian Zeng Smoke Verification product developmentSmoke Verification product development

Matt Seybold, Mark RuminskiMatt Seybold, Mark Ruminski HMS product integration with smoke forecast toolHMS product integration with smoke forecast tool

EPA/OAQPSEPA/OAQPS

Chet Wayland, Phil Dickerson, Scott Jackson, Brad JohnsChet Wayland, Phil Dickerson, Scott Jackson, Brad Johns AIRNow development, coordination with NAQFCAIRNow development, coordination with NAQFC

EPA/ORDEPA/ORD

Rohit Mathur, Ken Schere, Jon Pleim, Tanya Otte, Rohit Mathur, Ken Schere, Jon Pleim, Tanya Otte, CMAQ development, adaptation of AQ simulations for AQFCMAQ development, adaptation of AQ simulations for AQF

Jeff Young, George Pouliot, Brian Eder, David Wong Jeff Young, George Pouliot, Brian Eder, David Wong

Background slidesBackground slideshttp://wwwt.emc.ncep.noaa.gov/mmb/aq/AQChangelogE.htmlhttp://wwwt.emc.ncep.noaa.gov/mmb/aq/AQChangelogE.html

Meteorological Development Lab / OST / National Weather Service

Fraction Correct (FC), Threat Score (TS)Fraction Correct (FC), Threat Score (TS)summer 2008 (85 vs. 76 ppb) 1200 UTC cyclesummer 2008 (85 vs. 76 ppb) 1200 UTC cycle

Ozone: Ozone: higher TS with lower thresholdhigher TS with lower threshold

RedRed: 85 ppb: 85 ppbBlueBlue: 76 ppb: 76 ppb

Meteorological Development Lab / OST / National Weather Service

Probability Of Detection (POD) Probability Of Detection (POD) 8-h ozone, 1200 UTC experimental8-h ozone, 1200 UTC experimental

by region, 76 ppb, 2007 to 2009 by region, 76 ppb, 2007 to 2009Higher Higher

uncertainty in uncertainty in 2009, too few 2009, too few

cases for reliable cases for reliable

95% CI95% CI

higher uncertainty in 2009higher uncertainty in 2009not enough cases > thresholdnot enough cases > threshold

Meteorological Development Lab / OST / National Weather Service

Number of observed Number of observed 1-h aerosol cases above 35 ug/m1-h aerosol cases above 35 ug/m33

by region, monthly averageby region, monthly average

PC region PC region more active inmore active in

winter 2009winter 2009compared tocompared towinter 2008winter 2008

NCEP Air Quality Forecast 2009 Verification NCEP Air Quality Forecast 2009 Verification (1 hr Max ozone vs NAM 2 m Max Temperature)(1 hr Max ozone vs NAM 2 m Max Temperature)

BIAS

West U.S.

NAM 2m Max Temperature1 hr daily Max Ozone

Max temp bias improves slightly by August but ozone errors increased in East U.S.

East U.S.