Deterministic vsProbabilistic Forecast

J.P. Céron – Météo-France

The Predictability

« a Thunderstorm will be observed next Sunday over the Toulouse « Météopole » between 15h and 16h » Irrealistic, the confidence that one can

have in this forecast is very low

« a rainy system will cross the Toulouse region Sunday afternoon  »

realistic, one can be quite confident in this forecast

The Predictability

The predictability depends on : The scale of the forecasted phenomenum

(Thunderstorm, Easterly Wave, Blocking situation, ENSO, …)

The Range of the forecast (NowCasting, Short , Medium , Seasonnal , Climatic)

Deterministic formulation error or lost of informations

Probabilistic formulation more possibilities in the forecast but interpretation problems

The different Forecasts How do you play to Horse races?

Informations about the form of horses, trainers, jockey, race conditions (type of soil, weather, …), predictions, …

Synthesis then making a bet on horses (in a more or less subjective choice)

Use of seasonnal forecasts. Informations about the states of atmosphere, continental

surfaces, oceanic system and its evolutions, the different forecasts, …

Synthesis and decision/action (i.e. make a bet on the real solution in a more or less objective way)

The different Forecasts The horse n°5 will win the race.

It will rain 650 mm at Niamey during the next rainy season.

Consequences : gain or lost depending of the success or not of the forecast.

The different ForecastsDeterministic forecast

Deterministic forecast

The different Forecasts

The different ForecastsDeterministic forecast

Limits of Numerical Forecasting

The forecast errors can come from different part of the forexast system :

ANALYSIS- errors (or uncertainty) from measurements, assimilation. - lack of observations over some regions

But also ...

Network density of surface observationsOver the whole globe

Limits of Numerical Forecasting

MODELS LIMITS- mesh, number of vertical levels (resolution of the model) - Equations, parameterisations - small scale phenomena (under the mesh or badly know)

But also ...

- L 'é q u a tio n v e c to r ie lle d u m o u v e m e n t :

S u r l'h o r iz o n ta le :

d V h

d t

1

h p - 2 V h + F h=

V

S u r la v e rtic a le : é q u a tio n d e l'h y d ro s ta tiq u e d 'a p rè s le s a p p ro x im a tio n s

0 =1

p

Zg

- L 'é q u a tio n d e la T h e rm o d y n a m iq u e :

d t

d (c pT )=

d t

d pR T

p+ Q

etc...

Equations are generally “simplifyed”and one calibrate “parametrisations”in the model, that is to say that one

use data computed in an approximateform (even sometime as “constant”).

ANALYSIS- errors (or uncertainty) from measurements, assimilation. - lack of observations over some regions

Limits of Numerical Forecasting

INTERPRETATION- Models generally provide “raw data” andconsequently the interpretation is often difficult)

And finally ...

MODELS LIMITS- mesh, number of vertical levels (resolution of the model) - Equations, parameterisations - small scale phenomena (under the mesh or badly know)

ANALYSIS- errors (or uncertainty) from measurements, assimilation. - lack of observations over some regions

Limits of Numerical Forecasting

COMMUNICATION- misfitted vocabulary - misreading of users’ needs - dissemination without feedbacks from the user

MODELS LIMITS- mesh, number of vertical levels (resolution of the model) - Equations, parameterisations - small scale phenomena (under the mesh or badly know)

ANALYSIS- errors (or uncertainty) from measurements, assimilation. - lack of observations over some regions

INTERPRETATION- Models generally provide “raw data” andconsequently the interpretation is often difficult)

Limits of Numerical Forecasting Natural variability of the Ocean/Atmosphere system and

the Atmospheric respons to external forcing

Limits of Numerical Forecasting Uncertainties of the initial state of the Climatic system Modelisation Error (both Oceanic and Atmosphéric) Natural varibility of Atmosphere and its respons to

external forcing Interpretation of the forecast

To sample the initial state uncertainty disturbances of the analysis

To sample the model uncertainty distubances of the model

To sample in the forecast all the possible solutions of the Ocean/Atmosphere system

Limits of Numerical Forecasting

The numerical forecast using "ARPEGE«  model at short range is a « déterministic »  forecast. It uses the initial state description of the climate

system and, using model’s equation, allow to perform the time evolution of the atmospheric state.

Major errors of this type of forecasts mainly come from initial state errors intoduced inside the model. The uncertainty spread increase with the range of

the forecast.

error at time t0

(initial uncertainty)

error at time t0 + range

Limits of Numerical ForecastingTo take into account the probability of the deterministic evolution, one

perform several forecasts starting from the same initial time but using slightly modified values of the parameters of the simulation

(namely inside the probable range of errors introduced at the initial time).

Deterministic forecast

At this range:Strong probability

region

Forecast range

valu

e o

f th

e p

aram

eter

Initial time

Limits of Numerical ForecastingBecause of computer ressources, one use a larger mesh model (to limit the computation time) and one perform, typically, around thirty

forecast using different conditions for the model.

850 hPa Temperature plumes (Toulouse)base 21 september 1999 at 12h UTC

125 - 50%

6 - 25%

0 - 6%

50 - 75%

75 - 100% Deterministic model

verification

Limits of Numerical ForecastingOne can look at the parameter’s dispersion as a function of the time

intergration and describe the encountered value distribution rath rather than the values themselves. One can also give a confidence

indice with a more or less high value (the larger the dispersion of the forecasts, the lower the indice that is to say the lower the confidence).

Temperature dispersion plumes as a function of the time integration

850 hPa Temperature plumes (Toulouse)base 21 september 1999 at 12h UTC

125 - 50%

6 - 25%

0 - 6%

50 - 75%

75 - 100% Deterministic model verification

This method is named “Ensemble forecast”

Limits of Numerical Forecasting

The ensemble forecast (Resume) To sample the initial state uncertainty disturbances

of the analysis To sample the model uncertainty distubances of the

model To sample in the forecast all the possible solutions of

the Ocean/Atmosphere system

Several forecasts – trying to get the distribution of the possible solutions instead of a single value

Several models – How can we merge the informations coming from many different models – empirical, AGCM, COAGCM ? Multimodel approach.

Limites of Statistical Forecasting Uncertainty already included inside the statistical tools

The different Forecasts The horse n°5 will win the race. It will rain 650 mm at Niamey during the next

rainy season. The horse n°5 has good chances to be in the

firsts in this race. The situation of the Ocean/Atmosphere

system and its probable evolutions indicate that the next rainy season in Niger has a good probability to be « above the Normal ».

Consequences : gain or lost depending of the success or not of the forecast

The different ForecastsProbabilistic Forecast

The different ForecastsProbabilistic Forecast

The different ForecastsProbabilistic Forecast

The different ForecastsProbabilistic Forecast

The different Forecasts Analogue technics

Probabilistic Forecast : formulation 1 model et n members p models et n members Gaussian : mean + standard deviation Analogues Statistical Methods (Discriminant Analysis,

Multiple Regression, Probabilistic Regression, …)

Catégories ForecastProbabilistic Forecast

Catégories Forecast How to define the categories?

Number Categories Limits Needs of user?

How to evaluate the forecasted probabilities for each category? Frequency/Probability , Climatological Probabilities,

Conditionnal probabilities, Confidence Indice Statistical Models Numerical Models

How to transform the forecast in « readable and comprehensive » form for the user?

Quadratic Scores (Brier, RPS, …) Relative Operating Characteristic (FA vs ND) Cost/Lost ratio approach

Deux categories: + dry / + wet et Ratio c/L=0.5

C1= averaged cost using a climatological forecastC2 = averaged cost using a perfect forecast C3= averaged cost using a the model forecast

Probabilistic Forecast: verification

obs non obs

prev c c

non prev

L 0

obs non obs

prev n11 n12

non prev

n21 n22

C2C1C3C1100V

Brier Scores :

BrSc = 1/N (pi – oi)2

pi probabilité prévue pour l’événement oi variable indicatrice de l’observation de l’événement 1

BrSc= o(p) 1-p2 + 1-o(p) p2 g(p) dp 0

g(p) Fréquence relative avec laquelle l’événement est prévu avec une probabilité comprise entre p et p+dp 1 1 1

BrSc= f 1- f + p1-o(p) 2- f-o(p) 2 g(p) dp 0 0 0

Uncertainty Reliability Resolution

Pour un système qui aurait toujours prévu la probabilité climatique d’occurrence ( pflt ) LCBrSc= f 1- f + f 1- flt

[L/S]CBrSkSc = 1 – BrSC / [L/S]CBrSc Brier Skill Score

BSSREL = 1 – BSREL // [L/S]CBrSc Brier Reliability SC

BSSRSL = BSRSL / Uncertainty Brier Resolution SC

Probabilistic Forecast: verification Reliability Diagrams

Probabilistic Forecast

Cost/Lost ratio approach

Probabilistic Forecast Differents users

Probabilistic Forecast Comparison between Deterministic and

Probabilistic formulation

Probabiliste

Déterministe

Probabilistic Forecast

Resume