Introduction to Time Series Analysis. Lecture 1.Peter Bartlett

1. Organizational issues.

2. Objectives of time series analysis. Examples.

3. Overview of the course.

4. Time series models.

5. Time series modelling: Chasing stationarity.

1

Organizational Issues

• Peter Bartlett. bartlett@stat. Office hours: Tue 11-12, Thu10-11

(Evans 399).

• Joe Neeman. jneeman@stat. Office hours: Wed 1:30–2:30, Fri 2-3

(Evans ???).

• http://www.stat.berkeley.edu/∼bartlett/courses/153-fall2010/

Check it for announcements, assignments, slides, ...

• Text: Time Series Analysis and its Applications. With R Examples,

Shumway and Stoffer. 2nd Edition. 2006.

2

Organizational Issues

Classroom and Computer Lab Section: Friday 9–11, in 344 Evans.

Starting tomorrow, August 27:

Sign up for computer accounts. Introduction to R.

Assessment:Lab/Homework Assignments (25%): posted on the website.

These involve a mix of pen-and-paper and computer exercises. You may use

any programming language you choose (R, Splus, Matlab, python).

Midterm Exams (30%): scheduled for October 7 and November 9,at the

lecture.

Project (10%): Analysis of a data set that you choose.

Final Exam (35%): scheduled for Friday, December 17.

3

A Time Series

0 1000 2000 3000 4000 5000 6000 70000

50

100

150

200

250

300

350

400

4

A Time Series

1960 1965 1970 1975 1980 1985 19900

50

100

150

200

250

300

350

400

year

5

A Time Series

1960 1965 1970 1975 1980 1985 19900

50

100

150

200

250

300

350

400

year

$

6

A Time Series

1960 1965 1970 1975 1980 1985 19900

50

100

150

200

250

300

350

400

year

$

SP500: 1960−1990

7

A Time Series

1987 1987.05 1987.1 1987.15 1987.2 1987.25 1987.3 1987.35 1987.4 1987.45 1987.5220

240

260

280

300

320

340

year

$

SP500: Jan−Jun 1987

8

A Time Series

240 250 260 270 280 290 300 3100

5

10

15

20

25

30

$

SP500 Jan−Jun 1987. Histogram

9

A Time Series

0 20 40 60 80 100 120220

240

260

280

300

320

340

$

SP500: Jan−Jun 1987. Permuted.

10

Objectives of Time Series Analysis

1. Compact description of data.

2. Interpretation.

3. Forecasting.

4. Control.

5. Hypothesis testing.

6. Simulation.

11

Classical decomposition: An example

Monthly sales for a souvenir shop at a beach resort town in Queensland.

(Makridakis, Wheelwright and Hyndman, 1998)

0 10 20 30 40 50 60 70 80 900

2

4

6

8

10

12x 10

4

12

Transformed data

0 10 20 30 40 50 60 70 80 907

7.5

8

8.5

9

9.5

10

10.5

11

11.5

12

13

Trend

0 10 20 30 40 50 60 70 80 907

7.5

8

8.5

9

9.5

10

10.5

11

11.5

12

14

Residuals

0 10 20 30 40 50 60 70 80 90−1

−0.5

0

0.5

1

1.5

15

Trend and seasonal variation

0 10 20 30 40 50 60 70 80 907

7.5

8

8.5

9

9.5

10

10.5

11

11.5

12

16

Objectives of Time Series Analysis

1. Compact description of data.

Example: Classical decomposition: Xt = Tt + St + Yt.

2. Interpretation. Example: Seasonal adjustment.

3. Forecasting. Example: Predict sales.

4. Control.

5. Hypothesis testing.

6. Simulation.

17

Unemployment data

Monthly number of unemployed people in Australia.(Hipel and McLeod, 1994)

1983 1984 1985 1986 1987 1988 1989 19904

4.5

5

5.5

6

6.5

7

7.5

8x 10

5

18

Trend

1983 1984 1985 1986 1987 1988 1989 19904

4.5

5

5.5

6

6.5

7

7.5

8x 10

5

19

Trend plus seasonal variation

1983 1984 1985 1986 1987 1988 1989 19904

4.5

5

5.5

6

6.5

7

7.5

8x 10

5

20

Residuals

1983 1984 1985 1986 1987 1988 1989 1990−6

−4

−2

0

2

4

6

8x 10

4

21

Predictions based on a (simulated) variable

1983 1984 1985 1986 1987 1988 1989 19904

4.5

5

5.5

6

6.5

7

7.5

8x 10

5

22

Objectives of Time Series Analysis

1. Compact description of data:

Xt = Tt + St + f(Yt) + Wt.

2. Interpretation. Example: Seasonal adjustment.

3. Forecasting. Example: Predict unemployment.

4. Control. Example: Impact of monetary policy on unemployment.

5. Hypothesis testing. Example: Global warming.

6. Simulation. Example: Estimate probability of catastrophic events.

23

Overview of the Course

1. Time series models

2. Time domain methods

3. Spectral analysis

4. State space models(?)

24

Overview of the Course

1. Time series models

(a) Stationarity.

(b) Autocorrelation function.

(c) Transforming to stationarity.

2. Time domain methods

3. Spectral analysis

4. State space models(?)

25

Overview of the Course

1. Time series models

2. Time domain methods

(a) AR/MA/ARMA models.

(b) ACF and partial autocorrelation function.

(c) Forecasting

(d) Parameter estimation

(e) ARIMA models/seasonal ARIMA models

3. Spectral analysis

4. State space models(?)

26

Overview of the Course

1. Time series models

2. Time domain methods

3. Spectral analysis

(a) Spectral density

(b) Periodogram

(c) Spectral estimation

4. State space models(?)

27

Overview of the Course

1. Time series models

2. Time domain methods

3. Spectral analysis

4. State space models(?)

(a) ARMAX models.

(b) Forecasting, Kalman filter.

(c) Parameter estimation.

28

Time Series Models

A time series model specifies the joint distribution of the se-

quence{Xt} of random variables.

For example:

P [X1 ≤ x1, . . . , Xt ≤ xt] for all t andx1, . . . , xt.

Notation:

X1, X2, . . . is a stochastic process.

x1, x2, . . . is a single realization.

We’ll mostly restrict our attention tosecond-order properties only:

EXt, E(Xt1 , Xt2).

29

Time Series Models

Example: White noise:Xt ∼ WN(0, σ2).

i.e.,{Xt} uncorrelated, EXt = 0, VarXt = σ2.

Example: i.i.d. noise:{Xt} independent and identically distributed.

P [X1 ≤ x1, . . . , Xt ≤ xt] = P [X1 ≤ x1] · · ·P [Xt ≤ xt].

Not interesting for forecasting:

P [Xt ≤ xt|X1, . . . , Xt−1] = P [Xt ≤ xt].

30

Gaussian white noise

P [Xt ≤ xt] = Φ(xt) =1√2π

∫ xt

−∞

e−x2/2dx.

0 5 10 15 20 25 30 35 40 45 50−2.5

−2

−1.5

−1

−0.5

0

0.5

1

1.5

2

2.5

31

Gaussian white noise

0 5 10 15 20 25 30 35 40 45 50−2.5

−2

−1.5

−1

−0.5

0

0.5

1

1.5

2

2.5

32

Time Series Models

Example: Binary i.i.d. P [Xt = 1] = P [Xt = −1] = 1/2.

0 5 10 15 20 25 30 35 40 45 50

−1

−0.8

−0.6

−0.4

−0.2

0

0.2

0.4

0.6

0.8

1

33

Random walk

St =∑t

i=1Xi. Differences:∇St = St − St−1 = Xt.

0 5 10 15 20 25 30 35 40 45 50−4

−2

0

2

4

6

8

34

Random walk

ESt? VarSt?

0 5 10 15 20 25 30 35 40 45 50−15

−10

−5

0

5

10

35

Random Walk

Recall S&P500 data. (Notice that it’s smooth)

1987 1987.05 1987.1 1987.15 1987.2 1987.25 1987.3 1987.35 1987.4 1987.45 1987.5220

240

260

280

300

320

340

year

$

SP500: Jan−Jun 1987

36

Random Walk

Differences: ∇St = St − St−1 = Xt.

1987 1987.05 1987.1 1987.15 1987.2 1987.25 1987.3 1987.35 1987.4 1987.45 1987.5−10

−8

−6

−4

−2

0

2

4

6

8

10

year

$

SP500, Jan−Jun 1987. first differences

37

Trend and Seasonal Models

Xt = Tt + St + Et = β0 + β1t +∑

i (βi cos(λit) + γi sin(λit)) + Et

0 50 100 150 200 2502.5

3

3.5

4

4.5

5

5.5

6

38

Trend and Seasonal Models

Xt = Tt + Et = β0 + β1t + Et

0 50 100 150 200 2502.5

3

3.5

4

4.5

5

5.5

6

39

Trend and Seasonal Models

Xt = Tt + St + Et = β0 + β1t +∑

i (βi cos(λit) + γi sin(λit)) + Et

0 50 100 150 200 2502.5

3

3.5

4

4.5

5

5.5

6

40

Trend and Seasonal Models: Residuals

0 50 100 150 200 250−0.5

−0.4

−0.3

−0.2

−0.1

0

0.1

0.2

0.3

0.4

0.5

41

Time Series Modelling

1. Plot the time series.

Look for trends, seasonal components, step changes, outliers.

2. Transform data so that residuals arestationary.

(a) Estimate and subtractTt, St.

(b) Differencing.

(c) Nonlinear transformations (log,√·).

3. Fit model to residuals.

42

Nonlinear transformations

Recall: Monthly sales.(Makridakis, Wheelwright and Hyndman, 1998)

0 10 20 30 40 50 60 70 80 900

2

4

6

8

10

12x 10

4

0 10 20 30 40 50 60 70 80 907

7.5

8

8.5

9

9.5

10

10.5

11

11.5

12

43

Time Series Modelling

1. Plot the time series.

Look for trends, seasonal components, step changes, outliers.

2. Transform data so that residuals arestationary.

(a) Estimate and subtractTt, St.

(b) Differencing.

(c) Nonlinear transformations (log,√·).

3. Fit model to residuals.

44

Differencing

Recall: S&P 500 data.

1987 1987.05 1987.1 1987.15 1987.2 1987.25 1987.3 1987.35 1987.4 1987.45 1987.5220

240

260

280

300

320

340

year

$

SP500: Jan−Jun 1987

1987 1987.05 1987.1 1987.15 1987.2 1987.25 1987.3 1987.35 1987.4 1987.45 1987.5−10

−8

−6

−4

−2

0

2

4

6

8

10

year

$

SP500, Jan−Jun 1987. first differences

45

Differencing and Trend

Define the lag-1difference operator, (think ‘first derivative’)

∇Xt = Xt − Xt−1 = (1 − B)Xt,

whereB is thebackshift operator,BXt = Xt−1.

• If Xt = β0 + β1t + Yt, then

∇Xt = β1 + ∇Yt.

• If Xt =∑k

i=0βit

i + Yt, then

∇kXt = k!βk + ∇kYt,

where∇kXt = ∇(∇k−1Xt) and∇1Xt = ∇Xt.

46

Differencing and Seasonal Variation

Define the lag-sdifference operator,

∇sXt = Xt − Xt−s = (1 − Bs)Xt,

whereBs is the backshift operator applieds times,BsXt = B(Bs−1Xt)

andB1Xt = BXt.

If Xt = Tt + St + Yt, andSt has periods (that is,St = St−s for all t), then

∇sXt = Tt − Tt−s + ∇sYt.

47

Time Series Modelling

1. Plot the time series.

Look for trends, seasonal components, step changes, outliers.

2. Transform data so that residuals arestationary.

(a) Estimate and subtractTt, St.

(b) Differencing.

(c) Nonlinear transformations (log,√·).

3. Fit model to residuals.

48

Outline

1. Objectives of time series analysis. Examples.

2. Overview of the course.

3. Time series models.

4. Time series modelling: Chasing stationarity.

49