EVSC 495/EVAT 795 Data Analysis & Climate

Change

Class hours: TuTh 2:00-3:15 pm

Instructor: Michael E. Mann

EVSC 495/EVAT 795 WEBPAGE

http://holocene.evsc.virginia.edu/~mann/COURSES/495homeFall04.html

•SYLLABUS•LECTURES

•COURSE INFORMATION•PROBLEM SETS

Although there is no ideal textbook for the class, the following book is helpful as supplementary material (two copies are available on reserve in the Science & Engineering library):

Statistical Methods in the Atmospheric Sciences, D. Wilks, 1995 (Academic Press)

INTRODUCTION Hypothesis testing; Probability; Distributions

LECTURE 1

Supplementary Readings: Wilks, chapters 1-4

•Characterizing spatial and temporal patterns of variation

Applications of statistics to the study of climate variability and climate change

•Comparing theoretical predictions and observations

•Detecting statistically significant trends

Applications of statistics to the study of climate variability and climate change

Combined global land air and sea surface temperatures 1860-1997 (relative to 1961-1990 average)

Global Temperature TrendsThe instrumental surface temperature record is not perfect.

The instrumental surface temperature record is not perfect.

Applications of statistics to the study of climate variability and climate change

Grayshade: 1902-1993

Checkerboard: 1854-1993

Note how much sparser the data is prior to prior to the 20th century...

Applications of statistics to the study of climate variability and climate change

Global Temperature TrendsThe instrumental surface temperature record is not perfect.

Note how much sparser the data is prior to the early 19th century...

Applications of statistics to the study of climate variability and climate change

Global Temperature Trends

Statistical methods can be used to estimate the associated uncertainty

Recent history of ENSO

phenomenon

Multivariate ENSO Index

(“MEI”)

Applications of statistics to the study of climate variability and climate change

“Explains” enhanced warming in certain regions of Northern Hemisphere in past couple decades

For the hemisphere on the whole, the warming or cooling due to the NAO is probably a zero-sum game (note that cooling is expected cooling over Greenland and most of Arctic sea, where no data is available

Applications of statistics to the study of climate variability and climate change

Null Hypothesis (H0)

Test Statistic

Alternative Hypothesis (HA)

Hypothesis Testing

Hypothesis Testing

Statistical Model

[Observed Data] = [Signal] + [Noise]

“noise” has to satisfy certain properties!

If not, we must iterate on this process...

Bayesian/Subjectivist vs Frequentist Approach

Frequency Approach

NasENa 0|}Pr{/|

(ie, fraction occurances/opportunities of an event converges to the probability of the event)

Bayesian Approach

Conditional Probability

21|Pr EE

Bayesian/Subjectivist vs Frequentist Approach

(Probability of E1 given the occurrence of E2)

Consider Mutually Exclusive and Collectively Exhaustive (MECE) set of events {Ei} and an event A

I

i iiEEAA

1}Pr{}|Pr{Pr

Bayes’ Theorem

A

iE

iEA

AEi Pr

Pr|Pr|Pr

Bayesian Approach

Bayesian/Subjectivist vs Frequentist Approach

I

i iiEEAA

1}Pr{}|Pr{Pr

Jj j

Ej

EAiE

iEA

AEi

1}Pr{}|Pr{

Pr|Pr|Pr

Bayes’ Theorem

PriorLikelihoodsPosterior Distribution

Bayesian Approach

Bayes’ Theorem

Prior

Jj j

Ej

EAiE

iEA

AEi

1}Pr{}|Pr{

Pr|Pr|Pr

Bayesian Approach

LikelihoodsPosterior Distribution

The central equation of Bayesian statistics combines the prior distribution and the likelihood function to reach the posterior distribution:

Coin Flipping Example

Binomial Distribution

nNpnpn

NniP

)1()()!(!

!nNn

N

n

N

(Probability Of Heads)

What is “p”?

Consider probability of obtaining seven heads in ten flips:

N=10; n=7 P(7)=0.12

Coin Flipping Example

Binomial Distribution

nNpnpn

NniP

)1()()!(!

!nNn

N

n

N

(Probability Of Heads)

What if the coin is weighted?

How does the frequentist deal with this issue?

Consider probability of obtaining seven heads in ten flips:

Coin Flipping Example

Binomial Distribution

nNpnpn

NniP

)1()()!(!

!nNn

N

n

N

(Probability Of Heads)

Weighted coin

1)1(1)()()()(

qxpxqpqpx

iP nNpnp

n

NpiP

)1()(

Coin Flipping Example

(Probability Of Heads)

Weighted coin

1)1(1)()()()(

qxpxqpqpx

iP

Beta Distribution

Coin Flipping Example

(Probability Of Heads)

Probability density function

nNpnpn

NpiP

)1()(

p

Bayesian analysis of a set of coin flips. The prior density was calculated assuming 20 heads from 40 tosses for a perfect coin (p = 0.5). The likelihood or data density was calculated assuming 7 heads from 10 tosses. The resulting posterior density is also plotted

Coin Flipping Example

(Probability Of Heads)

p

The posterior distribution for Pr(heads) peaks just above 0.5 because of the observed data of 7 heads from 10 tosses. The extent of the shift from the data value (0.7) is incorporated into the analysis by the form of the prior distribution. In any case, as the number of observations, n, increases, the resulting distribution, becomes more concentrated at the observed ratio of heads to tosses.

Probability Distributions and PDFs

•Binomial distribution

•Poisson distribution

•Gaussian distribution

•Chi-squared distribution

•Lognormal distribution

•Gamma Distribution

•Beta Distribution

Binomial Distribution

Describes the probability distribution of multiple independent events characterized

by a fixed rate of occurrence

•Coin Flipping

•Dice Rolling

nNpnpn

NniP

)1()()!(!

!nNn

N

n

N

•Precipitation Occurrence?

Binomial Distribution

nNpnpn

NniP

)1()()!(!

!nNn

N

n

N

Now consider the limit where p<<1 and N>>1

enn

niP

!1)(

Under these circumstances, we have the approximation:

pN(occurrence rate)

“Poisson” Distribution

enn

niP

!1)(

(mean occurrence rate)

pN

)(VAR

HistogramEstimate of PDF

Revisit the Binomial Distribution…

nNpnpn

NniP

)1()()!(!

!nNn

N

n

N

)!2/()!2/(!

snsnN

n

N

Let s=n-N/2

])!2/log[(])!2/log[(!lnln sNsNNn

N

Revisit the Binomial Distribution…

nNpnpn

NniP

)1()()!(!

!nNn

N

n

N

Now consider the limit where N>>1 (but p finite)

We now can make use of Stirling’s approximation:

)exp(2/1)2(! mmmmm

mmmm ln)2/1()2ln(2/1!ln

Revisit the Binomial Distribution…

])!2/ln[(])!2/ln[(!lnln sNsNNn

N

NsNNn

N/222ln)/2ln(2/1ln

)exp(2/1)2(! mmmmm

mmmm ln)2/1()2ln(2/1!ln

)/22exp(22/1)/2( NsNNn

N

Revisit the Binomial Distribution…

]/2)2/(2exp[22/1)/2( NNnNNn

N

])!2/ln[(])!2/ln[(!lnln sNsNNn

N

NsNNn

N/222ln)/2ln(2/1ln

)/22exp(22/1)/2( NsNNn

N

]/)(2exp[22/1)/2( 22

xNNx

N

nNpnpn

NniP

)1()()!(!

!nNn

N

n

N

2

21exp

21)(

xx

iP

Gaussian or “Normal” Distribution

]/)(2exp[22/1)/2( 22

xNNx

N

Gaussian or “Normal” Distribution

2

21exp

21)(

xx

iP

Z

Gaussian or “Normal” Distribution

El Nino, La Nina, and ‘La Nada’

1998 GLOBAL TEMPERATURE PATTERN

Heavily influenced by a huge El Nino

NINO3 (90-150W, 5S-5N)

NINO3 (90-150W, 5S-5N)

Variance

2s

Mean

Ni ix

Nx

11

Ni

xxN

si1

2

11

Standard Deviation

Histogram of Monthly Nino3 Index

Goodness of fit?

Topic of next lecture…