Post on 25-Oct-2020
Overview Data Analyzing Time Series Quantiles and Extremes Conclusions
Using Stochastic Techniques to analyze
AIRS data
Sergio De-Souza Machado, Andrew TangbornLarrabee Strow, Philip Sura⇤
Department of Physics, JCETUniversity of Maryland Baltimore County (UMBC)
⇤ Florida State University, Tallahasee, FL
AIRS Science Team MeetingOctober 2017Greenbelt, MD
1
Overview
Overview Data Analyzing Time Series Quantiles and Extremes Conclusions
Outline
AIRS has given us 15+ years of high qualityTop-Of-Atmosphere radiance data
retrievals (L2, L3), assimilation,climate studies (radiance trends, L2/L3 trends)
We use data to study variability via PDFs of observations
Talk summarizes JAMC May 2017 paper
Newer work
2
Overview Data Analyzing Time Series Quantiles and Extremes Conclusions
Motivation
(1) The high resolution AIRS spectra allow us to probedifferent regions of the atmosphere (eg surface, strat T(z),trop T(z), trop WV(z), UT WV(z), stratospheric ozone)
Climate studies with AIRS data now feasible eg trace gas rates,T(z) and WV(z) rates
(2) Progress in speed/accuracy of scattering RTAs allow us tocompare AIRS observational data with GCM model fields; firstmoment (biases) and second moment (standard deviations)give primary indications of NWP and/or GCM accuracy
(3) Variability of observational and model data can be furtherstudied using higher order PDF moments (third = skewness,fourth = kurtosis)Gaussian : skewness = 0, kurtosis = 3
3
Overview Data Analyzing Time Series Quantiles and Extremes Conclusions
PDF Moments
Sharply peaked distribution (less in the tails) : Kurtosis > 3Wider distribution (more in the tails) : Kurtosis < 3"More stuff on the left" or "tail extending to right : Skewness > 0"More stuff on the right" or "tail extending to left : Skewness < 0
4
Overview Data Analyzing Time Series Quantiles and Extremes Conclusions
Earlier stochastic analysis of atmospheric/ocean data
Use stats from "microphysical" locations (eg over multiplegridboxes) to look for "macroscopic" relationshipSST, sea level heights, 300 mb vorticity shows that
K � 3/2S
2 � r
power law behavior in tails pdf (x) = x
�↵ for large x
5
Overview Data Analyzing Time Series Quantiles and Extremes Conclusions
Advances in stochastic modeling
This can be modeled!Take dynamics (forcing, linear terms, nonlinear terms) equationsand separate out into slow and fast scales; the nonlinearinteraction of fast scales leads to a SDE
Multiplicative noise in stochastically forced models reproducesnon-Gaussian statistics and power law behavior in PDF tails
dx
dt
= a(x(t))+ b(x(t))⌘(t)
where a = deterministic slow processes, while b⌘ represents statedependent multiplicative noise [as opposed to state independentadditive noise [a(x(t))+ ⌘(t)]; ⌘(t) is Gaussian white noise
6
Overview Data Analyzing Time Series Quantiles and Extremes Conclusions
Advances in stochastic modeling
Linearizing the equation x = <x> + y (y is the anomaly) we get
dy
dt
= �Ay(t)+ [G + Ey(t)]⌘(t)
where the same noise ⌘(t) multiplies the additive noise G and themultiplicative noise Ey(t) hence Correlated and AdditiveMultiplicative noise (CAM)
Time dependent probability distribution function can be derivedfrom SDE, from which the K � 3
2S
2 + B relationship and power lawtails pdf (x) = x
�↵ for large x can be derived
7
Data
Overview Data Analyzing Time Series Quantiles and Extremes Conclusions
Applications to AIRS data
Radiative transfer for any AIRS channel is a convolution overmultidimensional phase space (includes T (z),WV(z), othertrace gases, surface temp, clouds etc)
Allsky PDFS are extremely non-Gaussian, evidence ofdeviations from Gaussian in the tails (cold tail = clouds)
So limit to clear sky PDFs
Do the obs/cal show K � 3/2S
2 � r? where r is an offsetarising from reducing the dynamical equations to a scalar
8
Overview Data Analyzing Time Series Quantiles and Extremes Conclusions
AIRS data and SARTA calcs
CLEAR SKY
use AIRXBCAL data , filtered for clear scenes
co-locate ERA geophysical, ran off SARTA clear forOcean/Night/Season
collect 10+ years of data into 4�bins, make PDFs for handfulof channels strat/trop T (662,754 cm�1), ozone (1024 cm�1),window (1231 cm�1) and trop/strat WV (1344,1420 cm�1)
compute S,K, look for extremes
Reduce effects of seasonal cycle by concentrating on DJF andremoving mean
9
Overview Data Analyzing Time Series Quantiles and Extremes Conclusions
Clear Sky : 662 and 754 cm�1
(a) 662 cm�1 (b) 754 cm�1
(c) 662 cm�1 (d) 754 cm�1
(e) 662 cm�1 (f) 754 cm�1
10
Overview Data Analyzing Time Series Quantiles and Extremes Conclusions
Clear Sky : 1024 and 1231 cm�1
(a) 1024 cm�1 (b) 1231 cm�1
(c) 1024 cm�1 (d) 1231 cm�1
(e) 1024 cm�1 (f) 1231 cm�1
11
Overview Data Analyzing Time Series Quantiles and Extremes Conclusions
Clear Sky : 1344 and 1420 cm�1
(a) 1344 cm�1 (b) 1420 cm�1
(c) 1344 cm�1 (d) 1420 cm�1
(e) 1344 cm�1 (f) 1420 cm�1
12
Overview Data Analyzing Time Series Quantiles and Extremes Conclusions
Global maps 1420 cm�1 Skewness
Night time DJF, boxed less than 500 observations are removed
OBS skew
Longitude [deg]-150 -100 -50 0 50 100 150
Latit
ude [deg]
-80
-60
-40
-20
0
20
40
60
80
-0.5
0
0.5
1
1.5
2
ERA skew
Longitude [deg]-150 -100 -50 0 50 100 150
Latit
ude [deg]
-80
-60
-40
-20
0
20
40
60
80
-0.5
0
0.5
1
1.5
2
13
Overview Data Analyzing Time Series Quantiles and Extremes Conclusions
Global maps 1420 cm�1 Kurtosis
Night time DJF, boxed less than 500 observations are removed
OBS exkurt
Longitude [deg]-150 -100 -50 0 50 100 150
Latit
ude [deg]
-80
-60
-40
-20
0
20
40
60
80
-2
-1.5
-1
-0.5
0
0.5
1
1.5
2ERA exkurt
Longitude [deg]-150 -100 -50 0 50 100 150
Latit
ude [deg]
-80
-60
-40
-20
0
20
40
60
80
-2
-1.5
-1
-0.5
0
0.5
1
1.5
2
14
Overview Data Analyzing Time Series Quantiles and Extremes Conclusions
Some observations
The 6 different channels probe different regions/constituentsof atmosphere
They show K vs S
2 behavior that can be modeled bystochastic CAM theory
Negative skewness in obs =) ?? cloud contamination; butalso have negative skewness in SARTA clear sky calcs
Offset of curves from zero is indication of correlations (ie wereduced fluid eqns to a single scalar)
Strat T(z) channels have skew/exK between -1 and +1(quiescent); MERRA/ERA stats similar
1231 cm�1 channel has most of its data above K vs 1.5 S
2 sostrong CAM forcing
Power law tails in some grid boxes (not shown in this talk)
15
Analyzing Time Series
Overview Data Analyzing Time Series Quantiles and Extremes Conclusions
Clear sky stochastic time series
Weather is short time scales, Seasonal Cycles (driven by periodicsolar insolation), Climate is much longer time scales.Other climate processes (eg El Nino) have in between time scales
We are already familiar with auto-regressive processesTaking this to a bigger picture, some climate phenomena/timeseries can be regarded as having a “memory” ie daily fluctuationsimpact seasonal fluctuations impact long term climate phenomena
Use equations that embrace a wide range of temporal scalesslow/fast Stochastic Eqns!
16
Overview Data Analyzing Time Series Quantiles and Extremes Conclusions
Clear sky stochastic time series
“A unified nonlinear stochastic time series analysis for climatescience”, Moon and Wettlaufer, 2017, Nature Scientific Reports
dx(t)dt
= a(t)x(t)+N(t)⌘(t)+ F(⌧)
a(t) is seasonal (slow freq), N(t) ⌘(t) is noise ⇥ Weiner process, F isslow forcingNext page shows preliminary analysis of 14 years of area weightedBT1231 clear sky observations, yielding damping and noise
17
Overview Data Analyzing Time Series Quantiles and Extremes Conclusions
Clear sky stochastic time series analysis
Units on left panel are in /year, right panel are in KelvinRemember this is clear sky, so have problems at polar regionsAlso have ability to generate error estimatesPlan to adapt this to CAM forcing (tricky! tricky!)
18
Quantiles and Extremes
Overview Data Analyzing Time Series Quantiles and Extremes Conclusions
Quantiles
Quantiles from 14 years of tropical AIRS observations
0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
Quantile
-0.2
-0.15
-0.1
-0.05
0
0.05
0.1
0.15
0.2
K/y
ear
180
200
220
240
260
280
300
320
BT
1231 o
bs
(K)
quant K/yr
BT1231 obs
UMBCERAAIRS
Black curve (right axis) shows the mean observed BT1231 quantileGreen curve = d/dt(obsBT1231) quantile (K/year)Red curve = d/dt(calcBT1231) quantile (K/year) (SARTA 2Slab,ERA)Quantiles ⇠ 1 : extreme hot events +ve so getting hotterQuantiles ⇠ 0 : thick, high clouds +ve so are cloud tops movinglower? Less thick? 19
Overview Data Analyzing Time Series Quantiles and Extremes Conclusions
Extremes
Pick off 100 hottest points daily, look at the extreme distributionsBlack curve is the (GEV) extreme distribution for a typical yearThe other curves are the � between extremes from(2015/09-2016/08) and (2002/09-2003/08) for OBS, ERA andUMBC retrievalsHottest points are getting hotter
295 300 305 310
BT1231 obs
-6
-4
-2
0
2
4
6
pd
f
10 -3
0
0.005
0.01
0.015
0.02
0.025
0.03
0.035
0.04
0.045
0.05
Ob
s G
EV
pd
f
ObsUMBCERA
Different channels show different GEV distributions(shapes/parameters)
20
Conclusions
Overview Data Analyzing Time Series Quantiles and Extremes Conclusions
Conclusions
Hyperspectral sounder channels show evidence of stochasticforcing, which can be explained using a CAM model
K � 3/2S
2 � r
power law behavior in tails pdf (x) = x
�↵ for large x
JAMC paper establishing this published May 2017“Non-Gaussian Analysis of Observations from the AtmosphericInfrared Sounder Compared with ERA and MERRA Reanalyses”J.Appl.Met. and. Clim, 2017https://doi.org/10.1175/JAMC-D-16-0278.1
Plan to continue work on more climate related studiesSpectral trends (see Larrabee’s talks)Time series analysis : damping and forcing constantExtremesNeural Net for cloud height estimations
21