JAN | Feb-Mar-Apr Mar-Apr-May Apr-May-Jun May-Jun-Jul IRI’s monthly issued probability forecasts...
-
Upload
hubert-wilkinson -
Category
Documents
-
view
221 -
download
0
Transcript of JAN | Feb-Mar-Apr Mar-Apr-May Apr-May-Jun May-Jun-Jul IRI’s monthly issued probability forecasts...
JAN | Feb-Mar-Apr Mar-Apr-May
Apr-May-Jun May-Jun-Jul
IRI’s monthly issued probability forecasts of seasonal global precipitation and temperature
We issue forecasts at four lead times. For example:
Forecast models are run 7 months into future. Observed dataare available through the end of the previous month (end ofDecember in example above). Probabilities are given for thethree tercile-based categories of the climatological distribution.
“Now”| Forecasts made for:
0 10 20 30 40 50 60 70 80 90 100 110 120 130 140 150 160 170 180 190 200 | || ||| ||||.| || | | || | | | . | | | | | | | | | |
Rainfall Amount (mm)
Below| Near | Below| Near | Above Below| Near |
The tercile category system:Below, near, and above normal
(30 years of historical data for a particular location & season)
Forecasts of the climate
Data:
33% 33% 33%Probability:
0 10 20 30 40 50 60 70 80 90 100 110 120 130 140 150 160 170 180 190 200
Rainfall Amount (mm)
Below| Near | Below| Near | Above Below| Near |
20% 35% 45%
Example of a typical climate forecastfor a particular location & season
Probability:
0 10 20 30 40 50 60 70 80 90 100 110 120 130 140 150 160 170 180 190 200
Rainfall Amount (mm)
Below| Near | Below| Near | Above Below| Near |
5% 25% 70%
Example of a STRONG climate forecastfor a particular location & season
Probability:
Below Above
Historically, the probabilities of above and below are 0.33. Shifting the mean by half a standard-deviation and reducing the variance by 20% (red curve) changes the probability of below to 0.15 and of above to 0.53.
Historical distribution Forecast distribution
The probabilistic nature of climate forecasts
30
12
30
24
12
24
24
10
24
10
FORECAST SST
TROP. PACIFIC: THREE (multi-models, dynamical and statistical)
TROP. ATL, INDIAN (ONE statistical)
EXTRATROPICAL (damped persistence)
GLOBAL ATMOSPHERIC
MODELS
ECPC(Scripps)
ECHAM4.5(MPI)
CCM3.6(NCAR)
NCEP(MRF9)
NSIPP(NASA)
COLA2
GFDL
ForecastSST
Ensembles3/6 Mo. lead
PersistedSST
Ensembles3 Mo. lead
IRI DYNAMICAL CLIMATE FORECAST SYSTEM
POSTPROCESSING
MULTIMODELENSEMBLING
PERSISTED
GLOBAL
SST
ANOMALY
2-tiered OCEAN ATMOSPHERE
30
GFDL has 10 to PSST
FORECAST SST
TROP. PACIFIC: THREE scenarios: 1) CFS prediction 2) LDEO prediction 3) Constructed Analog prediction TROP. ATL, and INDIAN oceans CCA, or slowly damped persistence EXTRATROPICAL damped persistence
MULTIPLEGLOBAL
ATMOSPHERICMODELS
ECPC(Scripps)
ECHAM4.5(MPI)
CCM3.6(NCAR)
NCEP(MRF9)
NSIPP(NASA)
COLA2
GFDL
IRI DYNAMICAL CLIMATE FORECAST SYSTEM
PERSISTED
GLOBAL
SST
ANOMALY
2-tiered OCEAN ATMOSPHERE
Atmospheric General Circulation Models Used in the IRI's Seasonal Forecasts, for Superensembles
Name Where Model Was Developed Where Model Is Run
NCEP MRF-9 NCEP, Washington, DC QDNR, Queensland, Australia
ECHAM 4.5 MPI, Hamburg, Germany IRI, Palisades, New York
NSIPP NASA/GSFC, Greenbelt, MD NASA/GSFC, Greenbelt, MD
COLA COLA, Calverton, MD COLA, Calverton, MD
ECPC SIO, La Jolla, CA SIO, La Jolla, CA
CCM3.6 NCAR, Boulder, CO IRI, Palisades, New York
GFDL GFDL, Princeton, NJ GFDL, Princeton, NJ
Collaboration on Input to Forecast Production
Sources of the Global Sea Surface Temperature Forecasts
Tropical Pacific Tropical Atlantic Indian Ocean Extratropical Oceans
NCEP Coupled CPTEC Statistical IRI Statistical Damped PersistenceLDEO Coupled Constr Analogue
----- -----
Much of our predictability comes about due to
Strong trade winds
Westward currents, upwelling
Cold east, warm west
Convection, rising motion in west
Weak trade winds
Eastward currents, suppressed upwelling
Warm west and east
Enhanced convection, eastward displacement
NDJ Precipitation Probabilities associated with El Nino
Empirical Approach (a good baseline by which to judge the more advanced methods “honest”)
NDJ
Combining the Predictions of Seasonal Climate
by Several Atmospheric General Circulation
Models (AGCMs) into a Single Prediction.
What is Done at IRI
Goals in Combining
To combine the probability forecasts of severalmodels, with relative weights based on the past performance of the individual models
To assign appropriate forecast probability distribution: e.g. damp overconfident forecasts
toward climatology
One Choice: Multiple Regression
Multiple regression uses the predictions of 2 or more models,along with the corresponding observations, to derive a set ofmodel weights that minimizes squared errors of the weightedpredictions.
Good feature: The actual skills of each model are taken into account, andalso the overlap in sources of predictability. So, if two models have high skillin the same way or for the same reason, or if two good models are nearly identical, both of them do not get the same high weight.
Bad features: (1) Random variations in the training sample are taken asseriously as the predictable part of the variability. This leads to overfitting,and poor results in real forecasts. The more predictors compared to thenumber of independent cases, the more severe this problem is. (2) Forecastsfor conditions outside the range of the training sample may be dangerous.
Note: A model’s weight does not necessarily describe its skill.In fact, negative weights can be seen for skillful models.
Another Choice: Simple Skill Weighting
Here, the weights given to 2 or more models are determinedby each model’s individual skill in forecasting observations. Theskill may be a correlation or other measure. The weights arenormalized so that their sum is 1.
Good features: (1) The actual skills of each model are takeninto account. (2) Overfitting is not as serious a problem hereas it is in multiple regression. (3) Final forecast is always withinthe range of the individual forecasts.
Bad feature: Duplication in the sources of skill is not checked.(Colinearity ignored.) So, if two good models are almost identical,both will be weighted strongly, even though the second one doesnot add much skill. A medium-skill model that has a uniquesource of skill will not influence the forecast enough.
Note: A model’s weight is easily interpreted as being its skill.
Combine climatology forecast (“prior”) and an AGCM forecast, with its
evidence of historical skill, to produce weighted (“posterior”) forecast probabilities, by maximizing the
historical likelihood score.
Methods Used at IRI:(1) Bayesian Combination (2) Canonical Variate Combination
Bayesian Combination
Six GCM Precip. Forecasts, JAS 2000
RPSS Skill of Individual Models: JAS 1950-97
Probabilities and Uncertainty
Pkt (x) Pk (x)
Pkt (y) mkt
m
1/3
1/3
1/3
6/24
8/24
10/24
Above Normal
Near-Normal
Below Normal
ClimatologicalProbabilities
GCMProbabilities
x2
y2
x1
y1
Tercile boundaries are identified for the models’ own climatology, by aggregating all years and ensemble members. This corrects overall bias.
k = tercile numbert = forecast time
m = no. ens members
Bayesian Model Combination
Combine climatology forecast (“prior”) and an AGCM forecast, with its
evidence of historical skill, to produce weighted (“posterior”) forecast probabilities, by maximizing the
historical likelihood score.
The multi-year product of the probabilities that were hindcast for the category
that was observed.
(Could maximize other scores, such as RPSS)
Prescribed, observed SST used to force AGCMs.Such simulations used in absence of ones usingtruly forecasted SST for at least half of AGCMs.
1
( ) log PN
ktt
L w
Aim to maximize thelikelihood score
k=tercile category t=year number
(.333)c j jktjkt
c j
w w PP
w w
1. Calibration of each model, individually, against climatology
2. Calibration of the weighted model combination against climatol
k=tercile category (1,2, or 3)t=year number
j=model number (1 to 7) w=weight for climo (c) or for model j
where wMM uses wj proportional to results of the first step above
PMMkt= weighted linearcomb of Pjkt over all j,normalized by Σ(wj)
Optimizelikelihood
score
Optimizelikelihood
scoreMMc
MMktMMckt
final
ww
PwwP
*
)333(.*
Algorithm used to maximize the designated score:Feasible Sequential Quadratic Programming (FSQP)
“Nonmonotone line search for minimax problems”
C M : TOTAL NUMBER OF CONSTRAINTS.C ME : NUMBER OF EQUALITY CONSTRAINTS.C MMAX : ROW DIMENSION OF A. MMAX MUST BE AT LEAST ONE AND GREATERC THAN M.C N : NUMBER OF VARIABLES.C NMAX : ROW DIMENSION OF C. NMAX MUST BE GREATER OR EQUAL TO N.C MNN : MUST BE EQUAL TO M + N + N.C C(NMAX,NMAX): OBJECTIVE FUNCTION MATRIX WHICH SHOULD BE SYMMETRIC ANDC POSITIVE DEFINITE. IF IWAR(1) = 0, C IS SUPPOSED TO BE THEC CHOLESKEY-FACTOR OF ANOTHER MATRIX, I.E. C IS UPPERC TRIANGULAR.C D(NMAX) : CONTAINS THE CONSTANT VECTOR OF THE OBJECTIVE FUNCTION.C A(MMAX,NMAX): CONTAINS THE DATA MATRIX OF THE LINEAR CONSTRAINTS.C B(MMAX) : CONTAINS THE CONSTANT DATA OF THE LINEAR CONSTRAINTS.C XL(N),XU(N): CONTAIN THE LOWER AND UPPER BOUNDS FOR THE VARIABLES.C X(N) : ON RETURN, X CONTAINS THE OPTIMAL SOLUTION VECTOR.C U(MNN) : ON RETURN, U CONTAINS THE LAGRANGE MULTIPLIERS. THE FIRSTC M POSITIONS ARE RESERVED FOR THE MULTIPLIERS OF THE MC LINEAR CONSTRAINTS AND THE SUBSEQUENT ONES FOR THEC MULTIPLIERS OF THE LOWER AND UPPER BOUNDS. ON SUCCESSFULC TERMINATION, ALL VALUES OF U WITH RESPECT TO INEQUALITIESC AND BOUNDS SHOULD BE GREATER OR EQUAL TO ZERO.
Circumventing the effects of sampling variability
• Sampling variability appears to be an issue: noisy weight distribution with large number of zero weights and some unity weights
• Bootstrap the optimization, omitting contiguous 6-year blocks of the 48-yr time series– yields 43 samples of 42 years
– shows the sampling variability of the likelihood over subsets of years
– We average the weights across the samples
Example
• Six GCMs’ Jul-Aug-Sep precipitation simulations
• Training period: 1950–97
• Ensembles of between 9 and 24 members
Model Weights – initially, by individual model
Climatological Weights – Multi-model
Model Weights – after second (damping) step
Model Weights – step 2, and Averaged over Subsamples
For more spatially smooth results, theweighting of each grid point is averaged
with that of its 8 neighbors, using binomial weighting.
X X XX X XX X X
Climatological Weights
Combination Forecasts of July-Sept PrecipitationAfter first stage only
After second (damping) stage
After spatial smoothingAfter sampling subperiods
ReliabilityJAS Precip., 30S-30N
Above-Normal Below-Normal
Forecast probabilityForecast probability
Obse
rved r
ela
tive
Freq.
Obse
rved r
ela
tive
Freq.
Bayesian
Pooled
(3-model)from Goddard et al. 2003
IndividualAGCM
RPSS Precip.
from Roberson et al. (2004):Mon. Wea. Rev., 132, 2732-2744
RPSS 2-m
Temp.from Roberson et al. (2004):
Mon. Wea. Rev., 132, 2732-2744
Conclusions - Bayesian
• The “climatological” (equal-odds) forecast provides a useful prior for combining multiple ensemble forecasts
• Sampling problems become severe when attempting to combine many models from a short training period (“noisy weights”)
• A two-stage process combines the models together according to a pre-assessment of each against climatology
• Smoothing of the weights across data sub-samples and spatially appears beneficial
IRI’s forecasts use also a second consolidationscheme, whose result is averaged with
the result of the Bayesian scheme.
1. Bayesian scheme
2. Canonical Variate scheme
Canonical Variate Analysis (CVA)
A number of statistical techniques involve calculating linear combinations (weighted sums) of variables. The weights are defined to achieve specific objectives:
• PCA – weighted sums maximize variance
• CCA – weighted sums maximize correlation
• CVA – weighted sums maximize discrimination
Canonical Variate Analysis
Let X be a set of centered explanatory variables, with variance-covariance matrix Sxx. Let Y be a set of (non-centered) indicator variables that define the group membership of a set of observations, with cross-products matrix Syy. In our case, membership is with respect to tercile. Solve the eigenproblem:
1 1 2xx xy yy xy r S S S S I a 0,
which is identical to the CCA problem, but with Y as the set of indicator variables rather than continuous values on the dependent variables. The loadings, a, maximize ratio of between-group variance to total variance, represented by the canonical correlation, r. A discrimination is maximized w.r.t. tercile group membership.
Canonical Variate Analysis
The canonical variates are defined to maximize the ratio of the between-category (separation between the crosses) to the within-category (separation of dots from like-colored crosses) variance.
Conclusion
IRI is presently using a 2-tiered prediction system.
It is interested in using fully coupled systems also,and is looking into incorporating those.
Within its 2-tiered system it uses 4 SST predictionscenarios, and combines the predictions of 7
AGCMs.
The merging of 7 predictions into a single one uses
two multi-model ensemble systems: Bayesian andcanonical variate. These give somewhat differing solutions, and are presently given equal weight.
Plan for Improvement of the Basic Forecast Product
IRI plans to issue forecasts as probabilitydensity functions (pdfs) instead of just forthe 3 tercile-based categories.
From the pdfs, users can construct proba-bilities for any categories desired.
Skill results for IRI real-timeclimate forecasts from 1997-2001
Real-timeForecastSkillGoddard etal. 2003,Bull. Amer.Meteorol. Soc.,84, 1973-1796.
Real-timeForecastSkillGoddard etal. 2003,Bull. Amer.Meteorol. Soc.,84, 1973-1796.
Real-timeForecastSkillGoddard etal. 2003,Bull. Amer.Meteorol. Soc.,84, 1973-1796.
Real-timeForecastSkillGoddard etal. 2003,Bull. Amer.Meteorol. Soc.,84, 1973-1796.
Perfe
ct re
liabil
ity
RELIABILITY DIAGRAM
Climate Information System: A USER centric perspective
Issues aboutrelating toour users
Some Common Customer Requests
1. More accurate climate forecasts (tighter pdfs) (we can at least deliver most reliable ones)
2. Forecasts of full pdf in physical units
3. Forecasts of the weather within climate
4. Forecasts for subseasonal periods
5. Climate change nowcast