Bulk Statistics on Ensemble Model Forecasts for MDSS Demo 2003
description
Transcript of Bulk Statistics on Ensemble Model Forecasts for MDSS Demo 2003
Bulk Statistics on Ensemble Model Forecasts for MDSS
Demo 2003
Paul Schultz
NOAA Forecast Systems Laboratory
June 17, 2003
The MDSS ensemble modeling component
• What is it?– Several computer model forecasts to supplement the NWS model
forecast services
• Why are we doing this?– Better forecasts. Just seeing if you’re paying attention.
• How does it work?– By combining multiple (imperfect) forecasts of the (imperfectly
observed) atmosphere, we can make a single ensemble forecast that is better than any of the forecasts that went into it.
2
Ensemble modeling
• Did it work during the 2003 Demo?– Not as well as it can. It shows promise. It can be improved.
3
The ensemble for Demo 2003
• Three models, two LBC source models, total of six ensemble members– models: MM5, RAMS, WRF– LBC sources (from NCEP): AVN, Eta– 6-hour cycle– 27-hour forecasts– 12-km grid
5
Bulk statisticsState variables, 12-hr forecasts
Feb 1 – Apr 8, 2003
6
Temperature (K) Wind speed (m/s) Dewpoint (K)
MM5-AVN 3.1 -0.7 2.5 +0.8 5.6 +1.5
MM5-Eta 3.0 -0.5 2.5 +0.8 5.5 +1.6
RAMS-AVN 5.8 -1.1 2.6 +1.6 6.5 -0.9
RAMS-Eta 5.9 -1.1 2.6 +1.7 6.9 -1.0
WRF-AVN 3.1 -0.4 2.4 +1.1 5.7 +1.4
WRF-Eta 3.1 -0.4 2.4 +1.0 5.7 +1.3
Precipitation verification
MDSS 0-3 h QPF Equitable Skill Score 133 runs from 1 Feb - 26 Mar 2003
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.01 0.1 0.25 0.5
Precip Threshold (in)
Eq
uit
able
Ski
ll S
core
MM5
RAMS
WRF
7
MDSS 0-3 h QPF Bias Score 133 runs from 1 Feb - 26 Mar 2003
0
0.5
1
1.5
2
2.5
3
3.5
4
4.5
0.01 0.1 0.25 0.5
Precip Threshold (in)
Fre
qu
ency
Bia
s
MM5
RAMS
WRF
A closer look
9 pm model runs, verifying only Iowa stations, entire expt
Improving the ensemble
• Remove unhelpful members– If we can’t fix RAMS problems, it’s gone
– Different LBC models don’t seem to help (?????)
8
Unhelpful members
MM5+Avn
WRF+Avn
MM5+Eta
WRF+Eta
The LBC models don’t add enough
diversity
Improving the ensemble
• Add good models– FSL/RUC a very good candidate for Demo 2004
• Change model configurations– WRF cloud/precip physics– Model cycle frequency, lead times, etc.
• Optimize use of available computing resources• Take advantage of what regional models do best
• Improve post-processing– Better PoP (probability of precip) estimates -- FSL– Better tuning procedures -- NCAR– Hope for “better” weather during tuning period
Reliability9
Percentage of expected FSL model runs
0
10
20
30
40
50
60
70
80
90
100
2/3 2/10 2/17 2/24 3/3 3/10 3/17 3/24 3/31 4/7
NCEP data problems Giant
snowstorm in Boulder
Planned power
outage at FSL
Reliability
• MM5 shows good reliability
• Others will improve with better scripting
10
Percentage of expected FSL model runs
0
10
20
30
40
50
60
70
80
90
100
MM5-AVN MM5-ETA RAMS-AVN RAMS-ETA WRF-AVN WRF-ETA
Photos from MDSS field trip
Bob Stradley and Ron Simmons
Downward-pointed radiometer mounted on rear-view mirror of
Jim Van Sickle’s truck
RWIS tower, I-35 south of Ames