Meteocast: a real time nowcasting system
-
Upload
alessandro-staniscia -
Category
Education
-
view
1.135 -
download
1
description
Transcript of Meteocast: a real time nowcasting system
Geostationary MultispectralImagery Using Neural Models For
Meteorological Applications
Dr. Michele de Rosa1,2,Prof. Frank S. Marzano1,Dr. Antonio Eleuteri4, Dr. Giancarlo Rivolta3
1 Sapienza University of Rome, via Eudossiana, 18 - 00184 Rome – Italy2 T.R.S. S.p.A., via della Bufalotta, 378 – 00139 Rome – Italy3 Logica UK at the European Space Agency (ESA) - ESRIN (EOP-GTR), Po-Box 64 - 00044 Frascati (RM) - Italy4 Royal Liverpool & Broadgreen University Hospitals NHS Trust (RLBUHT) and the University of Liverpool, Prescot Street, L7 8XP, Liverpool, UK
RMets Conference 2011 - Exeter, 2011/06/28
RMets Conference 2011, Exeter UK
Summary
� Introduction� The problem� The starting point� The model� The case studies� The rainfall estimation� The future� The near future
RMets Conference 2011, Exeter UK
Introduction
� Precipitation is a key factor in regulating equilibrium and life on Earth. It is a crucial geophysical parameter and one of the main actors in the global water cycle.
� A relevant part of environmental risk can be ascribed to meteorological severe events with high precipitation rate.
� Heavy precipitation associated to severe weather may cause serious damages in terms of economic losses and, in extreme cases, of human life losses.
� Managing the environmental risk due to precipitation is strictly linked to monitoring and understanding the storms that produces hazards such as flash floods.
RMets Conference 2011, Exeter UK
Introduction: the global hydrological cycle
RMets Conference 2011, Exeter UK
Introduction: The rainfall remote sensing
� The remote sensing provides an indirect measurements of rainfall.
� It is done through measurements of the radiative properties of the hydrometeors (i.e. inferring cloud/rain structures by measuring their radiative properties), both in a passive way (i.e. measuring the radiation spontaneously emitted by the hydrometeors and sensed by a radiometer) as in an active way (i.e. inferring the rain/cloud structure by measuring the reflected portion of the radiation emitted by a radar towards the precipitating cloud).
� Visible to IR estimates of rainfall are only indirect because they try to infer the underlying cloud structure from the top-of-the-cloud appearance
RMets Conference 2011, Exeter UK
Introduction: The MSG (Meteosat Second Generation)�
� Geostationary satellite� First mission 1977 (Meteosat 1)�� 12 Channels (3 Vis,8 IR,1 HRis)�� Vis and IR resolution 3712x3712� HRis resolution 11136x5568� 15 minutes observation period� About 3 x 3 Km of resolution
RMets Conference 2011, Exeter UK
Introduction: The MSG channels
Channel Spectral Band (µm) Main application
λ cen λ min λ max1 VIS0.6 0,635 0,56 0,71 Surface, clouds, wind fields 2 VIS0.8 0,81 0,74 0,88 Surface, clouds, wind fields3 NIR1.6 1,64 1,5 1,78 Surface, cloud phase4 IR3.9 3,9 3,48 4,36 Surface, clouds, wind fields5 WV6.2 6,25 5,35 7,15 Water vapor, high level clouds, atmospheric instability6 WV7.3 7,35 6,85 7,85 Water vapor, atmospheric instability7 IR8.7 8,7 8,3 9,1 Surface, clouds, atmospheric instability8 IR9.7 9,66 9,38 9,94 Ozone9 IR10.8 10,8 9,8 11,8 Surface, clouds, wind fields, atmospheric instability10 IR12.0 12 11 13 Surface, clouds, atmospheric instability11 IR13.4 13,4 12,4 14,4 Cirrus cloud height, atmospheric instability12 HRV Surface, clouds
Spectral Band (µm)
Broadband (about 0.4 – 1.1
RMets Conference 2011, Exeter UK
The problem
� Develop a model based on the MSG frames to make nowcasts (from 30 MINs to 60 MINs) about the rainfield.� The model would predict the MSG IR
channels in order to predict the rainfield.� The model would be flexible, accurate and
quick.
RMets Conference 2011, Exeter UK
The starting point
� The NeuCAST (Marzano et al.)� Meteosat 7's images application� IR channel (10.8 µm) nowcast (30 mins)�� Rain estimation from MW and IR sources, using
the IR channel nowcast� Model for IR-MW mapping (Neural net)�
RMets Conference 2011, Exeter UK
The model: the multichannel approach
� MSG's images application� IR channels (4,5,6,7,8,9,10,11) nowcast (30 min-1
Hr)� Rain estimation from MW and IR sources, using
the IR channels nowcast� Bayesian approach to train the model� GLM nowcast model� Model for IR-to-Rain Rate mapping
RMets Conference 2011, Exeter UK
The model: The multichannel model tools
� Cao’s method to find the optimal temporal window
� PCA (Principal Component Analysis) to reduce the number of information sources: the 8 IR channels are replaced by a linear combination of them.
� Bayesian model to nowcast the next frame� The Dynamically Averaging Network (DAN)
Ensemble
RMets Conference 2011, Exeter UK
The model: the Bayesian approach
� The bayesian framework was developed by David J. C. Mackay in the context of the neural networks.� The framework implements the Occam Razor
in order to penalize complex models vs. simple models.� The framework applies the evidence
approach to penalize the complex models.� The framework is general.
RMets Conference 2011, Exeter UK
The model: the Ensemble of models
� An ensemble is a composition of different models.� In general, the ensemble is used to average
between different models.� The output of an ensemble minimizes the
average error with respect to the ensemble’s components (see Bishop C. M.).
RMets Conference 2011, Exeter UK
The model: the different kinds of
Ensemble
� The GEM (General Ensemble Model)
∑=
=n
iiiGEM xff
1
)(α
� The BEM (Basic Ensemble Model)
∑=
=n
iiBEM xf
nf
1
)(1
RMets Conference 2011, Exeter UK
The model: the DAN Ensemble
� Let y be the output of a neural net and let be the probability associated to y.
� Let the DAN (Dynamically Averaging Networks) ensemble be defined as:
∑=
=n
iiiDAN ywf
1
where:
∑=
= n
jy
yi
j
i
pc
pcw
1
)(
)(
−=
y
y
y p
ppc
1)(and
5.0≥yp
otherwise
yp
Certainty
RMets Conference 2011, Exeter UK
The model: the probability computation
� If is the estimated error bar related to the prediction y of the pixel (i,j) of the ensemble’s component n then:
)(,
yErrnji
∑=
= n
i
nji
njin
yErr
yErryp
ji
1,
,
)(
)()(
,
so that:
1)(0 , ≤≤ ypnji and ∑
=
=n
i
nji yp
1, 1)(
RMets Conference 2011, Exeter UK
The model: the multichannel approach layout
RMets Conference 2011, Exeter UK
The case-studies
� The area analyzed is East longitude ranging from 7°to 18 °and North Latitude ranging from 36.5°to 48 °� 2006-07-24 � 2006-08-13� 2006-09-14� 2007-03-20 (for generalization test)� Each frame consists of 275x344 pixels
RMets Conference 2011, Exeter UK
The case studies
RMets Conference 2011, Exeter UK
The case studies: the performance indexes
( ) ( ) ( )[ ]∑ − kibkiest
bpoints
kεt,PTt,PT
N=tm
1� BIAS (K)
( ) ( ) ( )[ ] 21
1 2
−∑ kibki
estb
pointskε
t,PTt,PTN
=ts� RMSE (K)
( ) ( )[ ] ( ) ( )[ ]
( ) ( )[ ] ( ) ( )[ ] 21
)(
22
−−
−−
∑∑
∑
kbkibkest
bkiest
b
kbkibkest
bkiest
b
kε
tTt,PTtTt,PT
tTt,PTtTt,PT=tr� Correlation
index (%)
RMets Conference 2011, Exeter UK
The case studies: the benchmarks
� The Persistence � The Steady State Displacement
(SSD)
ttt FF =∆+
vFF ttt
r+=∆+
RMets Conference 2011, Exeter UK
The case studies: Ensemble setup
� 3 GLMs for each case-study: one GLM for the lower correlation frame, one for the higher correlation frame and one for the median correlation frame (like the worst, best and mean case in computer science).� 3 PCA channels� Each bayesian GLM consists of 726 inputs
(nc=5, embed=6), 1 output.� 9 components and 27 GLMs
RMets Conference 2011, Exeter UK
The case studies:
30 mins ahead mean values
-0.20
0.20.40.60.8
11.21.4
DAN SSD Persistence
BIAS
Ch. 10Ch. 11Ch. 4Ch. 5Ch. 6Ch. 7Ch. 8Ch. 9
K
RMets Conference 2011, Exeter UK
The case studies :
30 mins ahead mean values
0
2
4
6
8
10
12
DAN SSD Persistence
RMSE
Ch. 10Ch. 11Ch. 4Ch. 5Ch. 6Ch. 7Ch. 8Ch. 9
K
RMets Conference 2011, Exeter UK
The case studies:
30 mins ahead mean values
80
82
84
86
88
90
92
94
DAN SSD Persistence
Correlation
Ch. 10Ch. 11Ch. 4Ch. 5Ch. 6Ch. 7Ch. 8Ch. 9
%
RMets Conference 2011, Exeter UK
The case studies:
60 mins ahead mean values
0
0.5
1
1.5
2
DAN SSD Persistence
BIAS
Ch. 10Ch. 11Ch. 4Ch. 5Ch. 6Ch. 7Ch. 8Ch. 9
K
RMets Conference 2011, Exeter UK
The case studies :
60 mins ahead mean values
0
5
10
15
20
DAN SSD Persistence
RMSE
Ch. 10Ch. 11Ch. 4Ch. 5Ch. 6Ch. 7Ch. 8Ch. 9
K
RMets Conference 2011, Exeter UK
The case studies:
60 mins ahead mean values
0
20
40
60
80
100
DAN SSD Persistence
Correlation
Ch. 10Ch. 11Ch. 4Ch. 5Ch. 6Ch. 7Ch. 8Ch. 9
%
RMets Conference 2011, Exeter UK
The case studies:
case-study 13:30 2007/03/20 UTC
30 mins ahead
-0.2
0
0.2
0.4
0.6
0.8
1
1.2
DAN SSD Persistence
BIAS
Ch. 10Ch. 11Ch. 4Ch. 5Ch. 6Ch. 7Ch. 8Ch. 9
K
RMets Conference 2011, Exeter UK
The case studies :
case-study 13:30 2007/03/20 UTC
30 mins ahead
0
2
4
6
8
10
12
14
DAN SSD Persistence
RMSE
Ch. 10Ch. 11Ch. 4Ch. 5Ch. 6Ch. 7Ch. 8Ch. 9
K
RMets Conference 2011, Exeter UK
The case studies:
case-study 13:30 2007/03/20 UTC
30 mins ahead
0
20
40
60
80
100
DAN SSD Persistence
Correlation
Ch. 10Ch. 11Ch. 4Ch. 5Ch. 6Ch. 7Ch. 8Ch. 9
%
RMets Conference 2011, Exeter UK
The case studies:
case-study 13:30 2007/03/20 UTC
60 mins ahead
-0.5
0
0.5
1
1.5
2
DAN SSD Persistence
BIAS
Ch. 10Ch. 11Ch. 4Ch. 5Ch. 6Ch. 7Ch. 8Ch. 9
K
RMets Conference 2011, Exeter UK
The case studies :
case-study 13:30 2007/03/20 UTC
60 mins ahead
02468
10121416
DAN SSD Persistence
RMSE
Ch. 10Ch. 11Ch. 4Ch. 5Ch. 6Ch. 7Ch. 8Ch. 9
K
RMets Conference 2011, Exeter UK
The case studies:
case-study 13:30 2007/03/20 UTC
60 mins ahead
0
20
40
60
80
100
DAN SSD Persistence
Correlation
Ch. 10Ch. 11Ch. 4Ch. 5Ch. 6Ch. 7Ch. 8Ch. 9
%
RMets Conference 2011, Exeter UK
The case studies: real statistics
01/03/2010 - 31/03/2010� Corr(Model)>Corr(Persistence) = 90,47%� Corr(Model)>Corr(SSD) = 85,53%� Corr(Model)>Corr(Persistence) = 92,23%
(Cloud pixels)� Corr(Model)>Corr(SSD) = 87,65% (Cloud
pixels)� Corr(Model vs meanTB) = 99,37%� Computation time about 800 secs (13 mins).
RMets Conference 2011, Exeter UK
Conclusions (1)
� The model is flexible.� The ensemble nowcast performances are
very good.� The model seems to generalize very well.� A procedure, to find the optimal frame size in
order to reduce the prediction error, has been found.
RMets Conference 2011, Exeter UK
The rainfall estimation: the components
� The Eumetsat Multi-sensor Precipitation Estimate (product used to validate the model)
� The GLM Cloud Mask product used to filter the “no rain” pixels. This model uses the 4,9,10 MSG channels.
� A Land Surface Temperature (LST) estimator. The estimator uses the 9,10,11 MSG channels.
� A MLP Neural Net rain classifier. The classifier uses the 9,10,11 MSG channels.
� A MLP Neural Net rain estimator. The estimator uses the 4,5,9,10 MSG channels.
RMets Conference 2011, Exeter UK
The rainfall estimation: the model layout
RMets Conference 2011, Exeter UK
The rainfall estimation: the classes
� Class 1. Light rain: 0 < RRMax ≤ 2 mm/h� Class 2. Moderate rain: 2 mm/h < RRMax ≤
10 mm/h� Class 3. Heavy rain: 10 < RRMax ≤ 50 mm/h� Class 4. Violent rain: RRMax > 50 mm/h
RMets Conference 2011, Exeter UK
The rainfall estimation: a case study
10:15 2010/01/26 UTC: thunderstorm over Central Italy
RMets Conference 2011, Exeter UK
A case study: 10:15 2010/01/26 UTC -
the rainfall classification 30 Mins ahead.
91.03%Classification Rate
54.88%388262948Violent
58.80%11989573435Heavy
5.58%707591661978Moderate
99.06%1126111240551Light
PODViolentHeavyModerateLightPredicted/True
RMets Conference 2011, Exeter UK
A case study: 10:15 2010/01/26 UTC -
the rainfall estimation 30 Mins ahead.
%76.27Correlation
mm/h10.29RMSE
mm/h2.56BIAS
Performance Indexes 30 Min
RMets Conference 2011, Exeter UK
A case study : 10:15 2010/01/26 UTC -
the rainfall classification 60 Mins ahead.
89.07%Classification Rate
9.62%6832525289Violent
8.66%0132331359Heavy
1.15%0138342795Moderate
99.75%0802441082Light
PODViolentHeavyModerateLightPredicted/True
RMets Conference 2011, Exeter UK
A case study: 10:15 2010/01/26 UTC -
the rainfall estimation 60 Mins ahead.
%68.47Correlation
mm/h9.05RMSE
mm/h1.33BIAS
Performance Indexes 60 Min
RMets Conference 2011, Exeter UK
Conclusions (2)
� The rainfall classifier is very sensitive to the prediction error.� The rainfall estimator works better on
“violent” events.� The estimator performs poor on “Light” and
“Moderate” events (due to the model structure).� It should be possible to generate a lot of
meteorological product using the ensemble model.
RMets Conference 2011, Exeter UK
The future
� Characterize better the rainfall estimator in order to perform better.
� In order to nowcast the rainfield, it should be possible to correlate the MSG data with Meteorological Radar.
� Apply the multichannel to a real-time system.� Try to apply the frame prediction in order to nowcast
other meteorological entities (for example using the SAF suite).
� Continue the TRS collaboration in order to enrich the Weather Products functionalities and to develop new products.
RMets Conference 2011, Exeter UK
The near future
� Work in progress: the development of a software to convert the forecasts into kml files in order to load the forecast with Google Earth.
� Our forecasts, in kml format, will be published on our web sites www.mondometeo.org (italian) and www.kwos.org (english).
� Work in progress: the development of a software, named MeteoCast and running on Android platform, to have weather information (Rain, Thunderstorm etc.) about the area where the user is.
RMets Conference 2011, Exeter UK
The near future : A KML example
RMets Conference 2011, Exeter UK
Collaborations
All people and/or organizations, interested in our work, are welcome.
RMets Conference 2011, Exeter UK
Bibliography
� Imran Maqsood et al., An ensemble of neural networks for weather forecasting, Neural Comput & Applic (2004) 13: 112–122
� Yinyin Liu et al., OPTIMIZING NUMBER OF HIDDEN NEURONS IN NEURAL NETWORKS, Proceedings of the 25th IASTED Internation multiconference February 12-14, 2007, Innsbruck, Austria
� George Dahl et al., PARALLELIZING NEURAL NETWORK TRAINING FOR CLUSTER SYSTEMS, Proceedings of the 25th IASTED Internation multiconference February 12-14, 2007, Innsbruck, Austria
� Frank S. Marzano et al., Rainfall Nowcast from Multi-Satellite Passive Rainfall Nowcast from Multi-Satellite Passive
� Cao, L., Pratical Method for Determining the Minimum Embedding Dimension of a Scalar Time Series.
� Jollife I. T., Principal Component Analysis, New York: Springer-Verlag� Kohonen T., Self-Organized formation of topology correct feature maps, Biological
Cybernetics 43, 59-69.� MacKay, D. J. C., A Practical Bayesian Framework for Backpropagation Networks,
Neural Computation 1992 vol.4 n°.3 pags. 448-472.� MacKay, D. J. C., The Evidence Framework Applied to Classification Networks, Neural
Computation 1992 vol.4 n°.5 pags. 698-714.� P.M. Granitto, P.F. Verdes, H.A. Ceccatto, Neural Networks Ensemble: Evaluation of
Aggregation Algorithms, Elsevier Science 2005.� Bishop C. M., Neural Networks for Pattern recognition, Oxford Press 1995, ISBN 0-19-
853864-2
RMets Conference 2011, Exeter UK
Thanks for your [email protected]