Arctic sea ice predictions and how to evaluate them
Transcript of Arctic sea ice predictions and how to evaluate them
![Page 1: Arctic sea ice predictions and how to evaluate them](https://reader031.fdocuments.in/reader031/viewer/2022012507/61831e251e24206acc1a499e/html5/thumbnails/1.jpg)
Arctic sea ice predictionsand how to evaluate them
François Massonnet
@Fmassonnet
www.climate.be/u/fmasson
PARAMOUR Seminar Series
14 February 2019 - UCLouvain
![Page 2: Arctic sea ice predictions and how to evaluate them](https://reader031.fdocuments.in/reader031/viewer/2022012507/61831e251e24206acc1a499e/html5/thumbnails/2.jpg)
![Page 3: Arctic sea ice predictions and how to evaluate them](https://reader031.fdocuments.in/reader031/viewer/2022012507/61831e251e24206acc1a499e/html5/thumbnails/3.jpg)
Number of results from Google Scholar query
« Arctic sea ice prediction » per year of publication
Arctic sea ice prediction:
an emerging area of researchN
um
ber
of
pub
licatio
ns
2007
![Page 4: Arctic sea ice predictions and how to evaluate them](https://reader031.fdocuments.in/reader031/viewer/2022012507/61831e251e24206acc1a499e/html5/thumbnails/4.jpg)
Black Swan
![Page 5: Arctic sea ice predictions and how to evaluate them](https://reader031.fdocuments.in/reader031/viewer/2022012507/61831e251e24206acc1a499e/html5/thumbnails/5.jpg)
September 2007: the
Arctic black swan
Mill
ion k
m²
![Page 6: Arctic sea ice predictions and how to evaluate them](https://reader031.fdocuments.in/reader031/viewer/2022012507/61831e251e24206acc1a499e/html5/thumbnails/6.jpg)
Four suggested references
on Arctic sea ice
predictability and prediction
Chapter. 10; sea ice
(Chevallier,
Massonnet,
Guemas, Goessling
and Jung)
![Page 7: Arctic sea ice predictions and how to evaluate them](https://reader031.fdocuments.in/reader031/viewer/2022012507/61831e251e24206acc1a499e/html5/thumbnails/7.jpg)
2. Important considerations regarding the
evaluation of upcoming PARAMOUR predictions
1. Predictability and prediction of Arctic
sea ice from days to centuries
![Page 8: Arctic sea ice predictions and how to evaluate them](https://reader031.fdocuments.in/reader031/viewer/2022012507/61831e251e24206acc1a499e/html5/thumbnails/8.jpg)
2. Important considerations regarding the
evaluation of upcoming PARAMOUR predictions
1. Predictability and prediction of Arctic
sea ice from days to centuries
![Page 9: Arctic sea ice predictions and how to evaluate them](https://reader031.fdocuments.in/reader031/viewer/2022012507/61831e251e24206acc1a499e/html5/thumbnails/9.jpg)
Persistence
Autocorrelation of 1979-2015 sea ice
thickness (model output, one grid point)
Lag [days]
1/e
![Page 10: Arctic sea ice predictions and how to evaluate them](https://reader031.fdocuments.in/reader031/viewer/2022012507/61831e251e24206acc1a499e/html5/thumbnails/10.jpg)
Sea ice speed (one point)
Total sea ice kinetic energy
Sea ice concentration (one point)
Total sea ice extent
Total snow on sea ice volume
Total sea ice area
Snow on sea ice depth (one point)
Sea ice thickness (one point)
Total sea ice volume
Persistence of anomalies [days]
Data: satellite (NSIDC) + reanalysis (PIOMAS) + ocean-sea ice global simulations
Lag [days]
Autocorrelation of 1979-2015 sea ice
thickness (model output, one grid point)
1/e
Persistence: a primary source of sea ice
predictability on a spectrum of time scales
S2S book - Chapter. 10: sea ice (Chevallier, Massonnet, Guemas, Goessling and Jung)
![Page 11: Arctic sea ice predictions and how to evaluate them](https://reader031.fdocuments.in/reader031/viewer/2022012507/61831e251e24206acc1a499e/html5/thumbnails/11.jpg)
![Page 12: Arctic sea ice predictions and how to evaluate them](https://reader031.fdocuments.in/reader031/viewer/2022012507/61831e251e24206acc1a499e/html5/thumbnails/12.jpg)
June 20th – July 12th 2015, LANCE-MODIS, 2 images per day
https://forum.arctic-sea-ice.net/index.php?action=dlattach;topic=176.0;attach=18238;image
![Page 13: Arctic sea ice predictions and how to evaluate them](https://reader031.fdocuments.in/reader031/viewer/2022012507/61831e251e24206acc1a499e/html5/thumbnails/13.jpg)
June 20th – July 12th 2015, LANCE-MODIS, 2 images per day
https://forum.arctic-sea-ice.net/index.php?action=dlattach;topic=176.0;attach=18238;image
![Page 14: Arctic sea ice predictions and how to evaluate them](https://reader031.fdocuments.in/reader031/viewer/2022012507/61831e251e24206acc1a499e/html5/thumbnails/14.jpg)
June 20th – July 12th 2015, LANCE-MODIS, 2 images per day
https://forum.arctic-sea-ice.net/index.php?action=dlattach;topic=176.0;attach=18238;image
![Page 15: Arctic sea ice predictions and how to evaluate them](https://reader031.fdocuments.in/reader031/viewer/2022012507/61831e251e24206acc1a499e/html5/thumbnails/15.jpg)
Sources of
predictability
-Persistence
June 20th – July 12th 2015, LANCE-MODIS, 2 images per day
https://forum.arctic-sea-ice.net/index.php?action=dlattach;topic=176.0;attach=18238;image
![Page 16: Arctic sea ice predictions and how to evaluate them](https://reader031.fdocuments.in/reader031/viewer/2022012507/61831e251e24206acc1a499e/html5/thumbnails/16.jpg)
Sources of
predictability
-Persistence
-Mechanical forcing
by wind
-Current ice state
(deformation, age,
thickness ,
compactness)
June 20th – July 12th 2010, LANCE-MODIS, 2 images per day
https://forum.arctic-sea-ice.net/index.php?action=dlattach;topic=176.0;attach=18238;image
![Page 17: Arctic sea ice predictions and how to evaluate them](https://reader031.fdocuments.in/reader031/viewer/2022012507/61831e251e24206acc1a499e/html5/thumbnails/17.jpg)
![Page 18: Arctic sea ice predictions and how to evaluate them](https://reader031.fdocuments.in/reader031/viewer/2022012507/61831e251e24206acc1a499e/html5/thumbnails/18.jpg)
July August Sept.
Data
: N
SID
C
Weekly sea ice extent predictability stems from
persistence
![Page 19: Arctic sea ice predictions and how to evaluate them](https://reader031.fdocuments.in/reader031/viewer/2022012507/61831e251e24206acc1a499e/html5/thumbnails/19.jpg)
[Simmonds and Rudeva, Geophys. Res. Lett, 2012, Zhang et al., Geophys. Res. Lett., 2013; Parkinson and Comiso, Geophys. Res. Lett., 2013 ]
Great Arctic
Cyclone
July August Sept.
Weekly sea ice extent predictability stems from
persistence but can be affected by synoptic events
Sea Level Pressure 6th Aug
2012 1800 UTC (NCEP-CFSR)
Data
: N
SID
C
![Page 20: Arctic sea ice predictions and how to evaluate them](https://reader031.fdocuments.in/reader031/viewer/2022012507/61831e251e24206acc1a499e/html5/thumbnails/20.jpg)
![Page 21: Arctic sea ice predictions and how to evaluate them](https://reader031.fdocuments.in/reader031/viewer/2022012507/61831e251e24206acc1a499e/html5/thumbnails/21.jpg)
Data
: N
SID
C
Persistence
Reemergence through
ice area-SST coupling
Blanchard-Wrigglesworth et al., J. Clim., 2011; Chevallier et al., J. Clim., 2012; Day et al., J. Clim., 2014, Stammerjohn et al., Geophys. Res. Lett., 2012
Correlation date of ice retreat vs
date of ice advance (1979-2010)
Example of reemergence: melt to freeze up
Lag [months]
![Page 22: Arctic sea ice predictions and how to evaluate them](https://reader031.fdocuments.in/reader031/viewer/2022012507/61831e251e24206acc1a499e/html5/thumbnails/22.jpg)
Chevallier et al., J. Clim., 2013; Msadek et al., Geophys. Res. Lett., 2014; Sigmond et al., Geophys. Res. Lett., 2013; Peterson et al., Clim. Dyn.,
2015; Massonnet et al., Ocean Model., 2015; Merryfield et al., Geophys. Res. Lett., 2013; Bushuk et al., Geophys. Res. Lett. 2017
![Page 23: Arctic sea ice predictions and how to evaluate them](https://reader031.fdocuments.in/reader031/viewer/2022012507/61831e251e24206acc1a499e/html5/thumbnails/23.jpg)
Hamilton and Stroeve, Polar Geography, 2016
FORECASTS
OBS
![Page 24: Arctic sea ice predictions and how to evaluate them](https://reader031.fdocuments.in/reader031/viewer/2022012507/61831e251e24206acc1a499e/html5/thumbnails/24.jpg)
Predictions are unfortunately not
skillful in « operational » mode.
Possible reasons:
• Technical issues (e.g., fields not
available at time of forecast) imply that
groups cannot perform as well as on
retrospective predictions
• Predicting sea ice is tougher today
than it used to be
Hamilton and Stroeve, Polar Geography, 2016
FORECASTS
OBS
![Page 25: Arctic sea ice predictions and how to evaluate them](https://reader031.fdocuments.in/reader031/viewer/2022012507/61831e251e24206acc1a499e/html5/thumbnails/25.jpg)
OBS
OBS
FORECAST
Sea ice thickness initialized from
observations (CryoSat-2)
Sea ice thickness not initialized
FORECAST
Blockley and Peterson., Cryosphere, 2018
Forecasting September 2012
(coupled model)
Massonnet et al., Ocean Model., 2015
Sea ice data assimilation yields encouraging
results for seasonal predictions
Forecasting September 2007 (ocean-sea ice model)
![Page 26: Arctic sea ice predictions and how to evaluate them](https://reader031.fdocuments.in/reader031/viewer/2022012507/61831e251e24206acc1a499e/html5/thumbnails/26.jpg)
![Page 27: Arctic sea ice predictions and how to evaluate them](https://reader031.fdocuments.in/reader031/viewer/2022012507/61831e251e24206acc1a499e/html5/thumbnails/27.jpg)
Tietsche et al., Geophys. Res. Lett., 2014
Ensemble spread of total sea
ice volume from 4 GCMs
Interannual time scales: « grey zone »
of sea ice predictability
![Page 28: Arctic sea ice predictions and how to evaluate them](https://reader031.fdocuments.in/reader031/viewer/2022012507/61831e251e24206acc1a499e/html5/thumbnails/28.jpg)
Interannual time scales: « grey
zone » of sea ice predictability
Truth (model)
Tietsche et al., Clim. Dyn., 2013
Climatology
« Perfect »
prediction
Data
assimilation
Damped
persistence
![Page 29: Arctic sea ice predictions and how to evaluate them](https://reader031.fdocuments.in/reader031/viewer/2022012507/61831e251e24206acc1a499e/html5/thumbnails/29.jpg)
Interannual time scales: « grey
zone » of sea ice predictability
Truth (model)
Tietsche et al., Clim. Dyn., 2013
Climatology
Minute
perturbations
to initial state
Data
assimilation
Damped
persistence
![Page 30: Arctic sea ice predictions and how to evaluate them](https://reader031.fdocuments.in/reader031/viewer/2022012507/61831e251e24206acc1a499e/html5/thumbnails/30.jpg)
Interannual time scales: « grey
zone » of sea ice predictability
Swart et al., Nature Clim. Change, 2014
Distribution of all possible 7-yr trends
(1979-2013) in September sea ice extent
![Page 31: Arctic sea ice predictions and how to evaluate them](https://reader031.fdocuments.in/reader031/viewer/2022012507/61831e251e24206acc1a499e/html5/thumbnails/31.jpg)
![Page 32: Arctic sea ice predictions and how to evaluate them](https://reader031.fdocuments.in/reader031/viewer/2022012507/61831e251e24206acc1a499e/html5/thumbnails/32.jpg)
Yeager et al., Geophys. Res. Lett., 2015; Årthun et al., Nat. Comm., 2017
Decadal predictions are
mostly skillful
-In winter
-In the Atlantic Sector
Skill stems from poleward
oceanic heat transport
and from radiative forcing
(trend)
Forecast
Observed
Forecast
Obs
![Page 33: Arctic sea ice predictions and how to evaluate them](https://reader031.fdocuments.in/reader031/viewer/2022012507/61831e251e24206acc1a499e/html5/thumbnails/33.jpg)
![Page 34: Arctic sea ice predictions and how to evaluate them](https://reader031.fdocuments.in/reader031/viewer/2022012507/61831e251e24206acc1a499e/html5/thumbnails/34.jpg)
Notz and Stroeve, Science, 2016
Arctic sea ice area is slaved
to the forcing
…
![Page 35: Arctic sea ice predictions and how to evaluate them](https://reader031.fdocuments.in/reader031/viewer/2022012507/61831e251e24206acc1a499e/html5/thumbnails/35.jpg)
Notz and Stroeve, Science, 2016; Bitz and Roe, J. Clim., 2004; Massonnet et al., Nature Clim. Change, 2018
Arctic sea ice area is slaved
to the forcing
…
but thinning rate depends
on initial thickness
![Page 36: Arctic sea ice predictions and how to evaluate them](https://reader031.fdocuments.in/reader031/viewer/2022012507/61831e251e24206acc1a499e/html5/thumbnails/36.jpg)
Has Arctic sea ice
passed a tipping point?
Anomalies of Arctic sea ice extent
relative to the 1979-2018 seasonal cycle
Data: NSIDC sea ice index
High mean,
low variance
Low mean,
high variance
2007
![Page 37: Arctic sea ice predictions and how to evaluate them](https://reader031.fdocuments.in/reader031/viewer/2022012507/61831e251e24206acc1a499e/html5/thumbnails/37.jpg)
Simple, conceptual models can exhibit multiple equilibria,
but this is a parameter-dependent feature
Eisenman, J. Geophys. Res., 2012
![Page 38: Arctic sea ice predictions and how to evaluate them](https://reader031.fdocuments.in/reader031/viewer/2022012507/61831e251e24206acc1a499e/html5/thumbnails/38.jpg)
No evidence for Arctic sea ice
irreversibility from comprehensive models
Li et al., J. Climate, 2013
![Page 39: Arctic sea ice predictions and how to evaluate them](https://reader031.fdocuments.in/reader031/viewer/2022012507/61831e251e24206acc1a499e/html5/thumbnails/39.jpg)
• There is in general predictability beyond
persistence, but predictive capacity depends on
- Time scale considered
- Season considered
- Region considered
- Parameter considered
• Knowledge of baseline sea ice+ocean state is key
to perform skillful predictions
• There is a « grey zone » of Arctic sea ice
predictability at interannual-to-decadal time scales
Arctic sea ice predictions: conclusions
![Page 40: Arctic sea ice predictions and how to evaluate them](https://reader031.fdocuments.in/reader031/viewer/2022012507/61831e251e24206acc1a499e/html5/thumbnails/40.jpg)
2. Important considerations regarding the
evaluation of upcoming PARAMOUR predictions
![Page 41: Arctic sea ice predictions and how to evaluate them](https://reader031.fdocuments.in/reader031/viewer/2022012507/61831e251e24206acc1a499e/html5/thumbnails/41.jpg)
Why don’t models and
observations match each other?
model
obs
![Page 42: Arctic sea ice predictions and how to evaluate them](https://reader031.fdocuments.in/reader031/viewer/2022012507/61831e251e24206acc1a499e/html5/thumbnails/42.jpg)
model
obs
It’s the modelers fault-Physical equations are wrong
-Equations are discretized
-Forcing is not correct
-Initial conditions are not correct
-Processes are parameterized
-There are computational errors
Why don’t models and
observations match each other?
[Orrell et al., Nonlin. Proc. Geophys., 2001]
![Page 43: Arctic sea ice predictions and how to evaluate them](https://reader031.fdocuments.in/reader031/viewer/2022012507/61831e251e24206acc1a499e/html5/thumbnails/43.jpg)
model
obs
[Orrell et al., Nonlin. Proc. Geophys., 2001]
Why don’t models and
observations match each other?
It’s the modelers fault-Physical equations are wrong
-Equations are discretized
-Forcing is not correct
-Initial conditions are not correct
-Processes are parameterized
-There are computational errors
![Page 44: Arctic sea ice predictions and how to evaluate them](https://reader031.fdocuments.in/reader031/viewer/2022012507/61831e251e24206acc1a499e/html5/thumbnails/44.jpg)
It’s the observers fault
model
obs
Instrumental errors
Algorithm errors
Assumptions (e.g. hydrostatic)
Sampling errors
[Ivanova et al., Cryosphere, 2014; Zygmuntowska et al., Cryosphere, 2014; Worby et al., J. Geophys. Res., 2008]
It’s the modellers fault
Why don’t models and
observations match each other?
![Page 45: Arctic sea ice predictions and how to evaluate them](https://reader031.fdocuments.in/reader031/viewer/2022012507/61831e251e24206acc1a499e/html5/thumbnails/45.jpg)
model
obs
Why don’t models and
observations match each other?
It’s the modellers fault
[Ivanova et al., Cryosphere, 2014; Zygmuntowska et al., Cryosphere, 2014; Worby et al., J. Geophys. Res., 2008]
It’s the observers fault
Instrumental errors
Algorithm errors
Assumptions (e.g. hydrostatic)
Sampling errors
![Page 46: Arctic sea ice predictions and how to evaluate them](https://reader031.fdocuments.in/reader031/viewer/2022012507/61831e251e24206acc1a499e/html5/thumbnails/46.jpg)
model
obs
It’s my faultNo scale-awareness
No definition-awareness
[Kay et al., J. Geophys. Res., 2016]
Why don’t models and
observations match each other?
It’s the modellers fault It’s the observers fault
![Page 47: Arctic sea ice predictions and how to evaluate them](https://reader031.fdocuments.in/reader031/viewer/2022012507/61831e251e24206acc1a499e/html5/thumbnails/47.jpg)
model
obs
It’s my faultNo scale-awareness
No definition-awareness
[Kay et al., J. Geophys. Res., 2016]
It’s the modellers fault
Why don’t models and
observations match each other?
It’s the observers fault
![Page 48: Arctic sea ice predictions and how to evaluate them](https://reader031.fdocuments.in/reader031/viewer/2022012507/61831e251e24206acc1a499e/html5/thumbnails/48.jpg)
model
obs
It’s my faultNo scale-awareness
No definition-awareness
It’s the modellers fault
Why don’t models and
observations match each other?
It’s no one’s faultInternal variability
[Notz, Phil. Trans. Roy. Soc., 2015]
It’s the observers fault
![Page 49: Arctic sea ice predictions and how to evaluate them](https://reader031.fdocuments.in/reader031/viewer/2022012507/61831e251e24206acc1a499e/html5/thumbnails/49.jpg)
Bunzel et al., Geophys. Res. Lett., 2016Massonnet et al., Science, 2016
Seasonal sea ice prediction skill is significantly affected
by the choice of the verification product
![Page 50: Arctic sea ice predictions and how to evaluate them](https://reader031.fdocuments.in/reader031/viewer/2022012507/61831e251e24206acc1a499e/html5/thumbnails/50.jpg)
Why do models score better for the two
most advanced and recent products?
Models simulate directly sea ice concentration and
output it as a physical variable; observations don’t.
Models can be really good references in that case!
Observations have deficiencies that models
don’t have e.g. concentration of thin ice
Truth
ModelObservation
Sea ice thickness [m]
Sea
ice
conce
ntr
atio
n [%
]
Satellite estimation of sea ice concentration
where it’s known to be 100%
![Page 51: Arctic sea ice predictions and how to evaluate them](https://reader031.fdocuments.in/reader031/viewer/2022012507/61831e251e24206acc1a499e/html5/thumbnails/51.jpg)
Challenges related to ensemble size,
statistical power and statistical testing
![Page 52: Arctic sea ice predictions and how to evaluate them](https://reader031.fdocuments.in/reader031/viewer/2022012507/61831e251e24206acc1a499e/html5/thumbnails/52.jpg)
Imagine 1000 forecast systems trying to predict the Arctic sea ice.
Assume that 20% (200) are good enough to be skillful (actual correlation
with obs is positive)
20 % of forecast systems are skillful
![Page 53: Arctic sea ice predictions and how to evaluate them](https://reader031.fdocuments.in/reader031/viewer/2022012507/61831e251e24206acc1a499e/html5/thumbnails/53.jpg)
Due to limited statistical power (say, 50%), not all of the skillful systems will be
recognized as skillful (100).
All positive results will be published (100), while only few of the negative results (10%)
will be published (10).
Correctly reported
as skillfulIncorrectly reported
as unskillfulNot reported
![Page 54: Arctic sea ice predictions and how to evaluate them](https://reader031.fdocuments.in/reader031/viewer/2022012507/61831e251e24206acc1a499e/html5/thumbnails/54.jpg)
In addition, 5% of the nonskillful forecast systems (40) will produce skillful
results just by chance, and all will be published. Again, only a limited
number (10%) of negative results will be published (80)
Incorrectly reported
as skillfulCorrectly reported
as unskillfulNot reported
![Page 55: Arctic sea ice predictions and how to evaluate them](https://reader031.fdocuments.in/reader031/viewer/2022012507/61831e251e24206acc1a499e/html5/thumbnails/55.jpg)
Conclusion: From 230 published studies, 140 (61%) will report skillful results
even though only 20% are actually skillful.
Due to low statistical power and unreported negative results, climate
predictions are probably less often skillful than suggested by the literature.
![Page 56: Arctic sea ice predictions and how to evaluate them](https://reader031.fdocuments.in/reader031/viewer/2022012507/61831e251e24206acc1a499e/html5/thumbnails/56.jpg)
Final remarks: data
• At UCLouvain we are hosting much sea ice and related data:• Observational references
• Reanalyses
• Model output (CMIP5, now CMIP6)
• We are keen to provide support or expertise when PARAMOUR people outside UCLouvainwould like to use them