Introduction to Time Series Analysishalweb.uc3m.es/.../personas/kaiser/esp/doctorado/docto1b.pdf ·...
Transcript of Introduction to Time Series Analysishalweb.uc3m.es/.../personas/kaiser/esp/doctorado/docto1b.pdf ·...
Introduction to time series (2008 ) 1
Introduction to Time Series Analysis
Regina KaiserOffice. 10.1.17E-mail:[email protected]ágina web de la asignatura
Introduction to time series (2008 ) 2
Outline
Chapter 1. Univariate ARIMA models.Chapter 2. Model fitting and checking.Chapter 3. Prediction and model selection.Chapter 4. Outliers and influential observations.Chapter 5. Heterocedastic models.Chapter 6. The TRAMO software.
Introduction to time series (2008 ) 3
Textbooks Abraham, B. and Ledolter, J. (1983) . Statistical Methods for
Forecasting. Wiley, New York. Anderson, T. W. (1971). The Statistical Analysis of Time Series.
Springer Verlag. New York. Box, G.E.P., Jenkins, G.M. and Reinsel, G. (1994). Time Series
Analysis: Forecasting and Control, 3rd. Ed. Prentice-Hall, Englewood Cliffs, NJ.
Brockwell, P.J. and Davis, R.A. (1996). Time Series: Theory and Methods. Springer-Verlag. New York.
Introduction to time series (2008 ) 4
Grading
• Final exam (60%).• Empirical project (40%).
Introduction to time series (2008 ) 5
Web resources• Time series data library.(http://www-personal.buseco.monash.edu.au/~hyndman/TSDL/)• Macroeconomic time serieshttp://www.fgn.unisg.ch/eurmacro/macrodata/index.html• Time series applets 1.http://www.aranya.com/resources/java/• Time series applets 2http://www.ubmail.ubalt.edu/~harsham/zero/scientificCal.htm• Time series applets 3.https://stat.tamu.edu/~jhardin/applets/index.html
Introduction to time series (2008 ) 6
Motivation
• Time series: sequence of observations takenat regular intervals of time
• Data in bussines, engineering, enviroment, medicine, etc.
Introduction to time series (2008 ) 7
Motivation.
Introduction to time series (2008 ) 8
Motivation.
Introduction to time series (2008 ) 9
Motivation.
Introduction to time series (2008 ) 10
Motivation.
Introduction to time series (2008 ) 11
Motivation.
Introduction to time series (2008 ) 12
Motivation.World,Oecd crude oil price(Q)
0
10
20
30
40
50
60
1 16 31 46 61 76 91 106 121 136 151 166 181
Introduction to time series (2008 ) 13
Motivation.
World, food price (Q)
0
0,5
1
1,5
2
1 10 19 28 37 46 55 64 73 82 91 100 109
Introduction to time series (2008 ) 14
Motivation.World, minerals price (Q)
0
0,5
1
1,5
2
1 10 19 28 37 46 55 64 73 82 91 100 109
Introduction to time series (2008 ) 15
Motivation.
World, agricultural raw price (Q)
00,20,40,60,8
11,21,41,6
1 10 19 28 37 46 55 64 73 82 91 100 109
Introduction to time series (2008 ) 16
Motivation.World, tropical drinks price (q)
0
0,5
1
1,5
2
2,5
3
1 10 19 28 37 46 55 64 73 82 91 100 109
Introduction to time series (2008 ) 17
Motivation.
0
0,5
1
1,5
2
2,5
3
1 15 29 43 57 71 85 99
Tropical drinks
Agricultural rawmat
Minerals, oreand metals
Food
Introduction to time series (2008 ) 18
Motivation.
Introduction to time series (2008 ) 19
Motivation.
Introduction to time series (2008 ) 20
Motivation.
• Knowledge of the dynamic structure willhelp to produce accurate forecasts of futureobservations and design optimal control schemes.
Introduction to time series (2008 ) 21
Motivation
Main objectives
• Understanding the dynamic structure of theobservations in a single series.
• Ascertain the leading, lagging and feedback relationships among several variables.
Introduction to time series (2008 ) 22
Chapter 1
Univariate ARIMA Models
Recommended readings: chapters 1 to 7 of Peña (2005)
Introduction to time series (2008 ) 23
Chapter 1. Contents.
1.1. Introduction. Definitions, examples,
1.2. Properties of univariate ARMA.ARMA weights.Stationarity and covariance structure.Autocorrelation function.Partial autocorrelación function.
1.3. Model especification.1.4. Examples.
Introduction to time series (2008 ) 24
Introduction
• Stochastic Process. Real-valued randomvariable that follows a distribution.
• The T-dimensional variable will have a joint distribution that dependson
tz )( tt zf
),...,,( 21 tTtt zzz
),...,,( 21 Tttt
Introduction to time series (2008 ) 25
Introduction
• Time Series will denote a particular realization of the stochasticprocess .
• To simplify notation
],...,,[ 21 tTtt zzz
),...,,( 21 Tzzz
Introduction to time series (2008 ) 26
IntroductionAssumptions
1. The process is stationary
2. The joint distribution of isa multivariate normal distribution
),...,,( 21 Tzzz
Introduction to time series (2008 ) 27
IntroductionImplications of stationarity• The joint distribution remains constant
over time
• It is then true that
),...,,(),...,,( 2121 kTkkT zzzfzzzf
;ztEz zt VVz
Introduction to time series (2008 ) 28
Introduction
• Stationarity implies a constant mean andbounded deviations from it.
• Strong requirement, few actual economicseries will satisfy it.
Introduction to time series (2008 ) 29
Introduction
Introduction to time series (2008 ) 30
Introduction
Introduction to time series (2008 ) 31
Introduction
Introduction to time series (2008 ) 32
Introduction
Transformations to achieve stationarity
– Constant variance: log/level plus outliercorrection.
– Stationary in mean: differencing.
Introduction to time series (2008 ) 33
Introduction
Differencing• Let B be the backward operator
• We shall use the operators:
– Regular difference– Seasonal difference
jttj zzB
B 1s
s B 1
Introduction to time series (2008 ) 34
Introduction
If is a deterministic trend
Where
• In general, will reduce a polynomial of degreed to a constant.
tz )( tt baz
;0
;2
t
t
z
bz
).(2tt zz
d
Introduction to time series (2008 ) 35
Introduction
• Example: will cancel a constant, but it will also cancel otherdeterministic periodic functions.
44 ttt zzz
Introduction to time series (2008 ) 36
Introduction
• Homogeneous difference equation
• Characteristic equation:• Solution is given by (four roots of
the unit circle)
– Two real roots and two complex conjugateswith modulus 1and frequency
0)1( 44
4 tttt zzzBz
irirrr 4321 ,,1,1
014 r4 1r
2/
Introduction to time series (2008 ) 37
Introduction
Therefore,
1. One in the zero frequency (trend)2. One in the twice-a-year seasonality3. Associated with once-a-year seasonality
)1)(1)(1( 24 BBB
2/
Introduction to time series (2008 ) 38
Univariate ARMA models
“Building block” is the white noise process
Implications:
),0( 2Niidat
0,...),,/()( 321 ttttt aaaaEaE
0),(),( jttjtt aaCovaaE2
21 ,...),/(),( atttjtt aaaVaraaVar
Introduction to time series (2008 ) 39
Introduction
General ARMA(p,q) processes:
• is the observable time series• is sequence of white noise• is the constant term
tt aBczB )()(
}{ ta}{ tz
c
Introduction to time series (2008 ) 40
Introduction
• Autoregressive polynomial
• Moving-average polynomial
• The AR and MA polynomials are assumedto have no common factor
ppBBBB ...1)( 2
21
qqBBBB ...1)( 2
21
Introduction to time series (2008 ) 41
Introduction
• Stationarity implies that all zeros ofare restricted to lie outside the unit circleand in this case:
Where is the mean of the series.
• Overall behavior of the series remains thesame over time.
)(B
)....1( 1 pc
Introduction to time series (2008 ) 42
Introduction
• In real world, however, time series data often exhibits a drifting behavior. Nonsta-stionarity can be modeled allowing some ofthe zeroes in to be equal to one. Thus,
• This is known as the ARIMA (p,d,q) model.
)(B
ttd aBczBB )()1)((
Introduction to time series (2008 ) 43
Introduction
• Some special cases of ARIMA(p,d,q):
– AR(1)– MA(1)– ARMA(1,1)– IMA(1,1)– ARIMA(0,1,1)(0,1,1)
Introduction to time series (2008 ) 44
Introduction
• AR(1) to MA( ) by recursive substitution.
ttt azz 1
ttttttt aazaazz 122
12 )(
tttktk
ktk
t aaaazz
122
11 ...
Introduction to time series (2008 ) 45
Introduction
• If , then
Which is a MA( )
1
0jjt
jt az
Introduction to time series (2008 ) 46
Introduction
• Some considerations:
• ARMA models are not unique (variety ofpossible representations.
• Parsimony is always a desirable property.• AR representations are easiest to estimate
(OLS) and to be interpretated.
Introduction to time series (2008 ) 47
Properties
• For simplicity, we will assume zero mean and an starting point m. The ARIMA (p,d,q)
• Can be rewritten as:
• Where,
tt aBzB )()(
aDzD
)',....,( tm zzz )',...,( tm aaa
Introduction to time series (2008 ) 48
Properties
1..................
...............
...1
1
1
p
pD
1..................
...............
...1
1
1
q
qD
)1)(1( mtmt
Introduction to time series (2008 ) 49
Properties
• ARMA weights
Where,
aDDz 1
1......
1
1
11
mt
DD
Introduction to time series (2008 ) 50
Properties
• The -weights can be obtained by equatingcoefficients of powers of B from therelations:
• Where
)()()( BBB
...1)( 221 BBB
Introduction to time series (2008 ) 51
Properties
• The ARIMA (p,d,q) model can then be rewritten in the MA form as:
mt
hhthtt aaz
1
Introduction to time series (2008 ) 52
Properties
• In the same way,
• Where,
azDD
1
1......
1
1
11
mt
DD
Introduction to time series (2008 ) 53
Properties
• With
• The AR form of the model is then,
).()()( BBB
t
mt
hhtht azz
1
Introduction to time series (2008 ) 54
Properties
• These two expressions are of fundamental importance in understanding the nature ofthe model.
• The MA form with the weights, showshow the observation is affected by current and past shocks or innovations.
• The AR form with the weigths, indicateshow the observation is related to its ownpast values.
tz
Introduction to time series (2008 ) 55
Properties
Examples : AR(1) model
• MA representation:
• AR representation:
....22
11 tttt aaaz
ttt azz 1
Introduction to time series (2008 ) 56
Properties
Examples : MA(1) model
• MA representation:
• AR representation:
tjtj
tt azzz .......1
1 ttt aaz
Introduction to time series (2008 ) 57
Properties
Examples : IMA(1,1) model
• MA representation:
• AR representation:
ttttt azzzz ...)1()1()1( 32
21
....))(1( 21 tttt aaaz
Introduction to time series (2008 ) 58
Stationarity condition
• Consider,
• Since the innovations are assumed normallydistributed, it follows that the observationsare also normally distributed.
• It is seen that, if in the characteristicequation:
mt
hhthtt aaz
1
kpp
kk rArA 0011 ...
Introduction to time series (2008 ) 59
Stationarity condition
• (Where the A’s denote polinomials in k andthe r’s are the distinct zeroes of )
• Then as we have that
)(B
,1jr poj ,...,1
,mt
,0)( tzE
0
2),cov(h
khhaktt zz
Introduction to time series (2008 ) 60
Stationarity condition
• So that will be stationary in thisasymptotic sense.
• This is the stationarity condition of anARMA model, and is equivalent to requirethat the zeroes of are lying outside theunit circle.
tz
)(B
Introduction to time series (2008 ) 61
Lag k autocovariance
• Let us denote,
• For an alternative expressions in terms ofthe ARMA parameters,
)(),cov(),cov()( kzzzzk kttktt
)...( 11 ptpttkt zzzz )...( 11 qtqttkt aaaz
Introduction to time series (2008 ) 62
Lag k autocovariance
• By taking expectations on both sides andusing the MA form, we obtain, for
• Where,
k
p
hh ghkk
1)()(
qk
kgkq
hkhha
k0
,...2,1,00
2
,0k
Introduction to time series (2008 ) 63
Autocorrelation function
• The autocorrelation function is defined as
• By substitution of , we obtain that
,...2,1,0,)0()()( kkk
)(k
0)0(
)()(1
kghkkp
h
kh
Introduction to time series (2008 ) 64
Autocorrelation function
• When , in a MA(q) model, then,
• That is, the autocorrelation function cuts offafter lag q.
0)( B
qkqkk qq
,0,)...1()(
1221
Introduction to time series (2008 ) 65
Partial autocorrelation function
• Intuition
• AR(1):
• AR(2):
tttt zzzz 123
tttt zzzz 123
Introduction to time series (2008 ) 66
Partial autocorrelation function
• The autocorrelation function only takes intoaccount that and are related in bothcases.
• But if we want to measure the directrelationship (without the intermediate ), we found that it is zero for the AR(1) anddifferent from zero for the AR(2) model
tz 2tz
1tz
Introduction to time series (2008 ) 67
Partial autocorrelation function
• The partial autocorrelation function is, therefore, a measure of the linear relationamong observations k-periods apart, independently of the intermediate values.
Introduction to time series (2008 ) 68
Partial autocorrelation function
• Consider first an stationary AR(p) model. The autoregressive coefficients are relatedto the autocorrelations by the Yule-Walkerequations,
• Where, is the matrix,
pppG
pG )( pxp
Introduction to time series (2008 ) 69
Partial autocorrelation function
1)1(...)2()1()1(1)2(
)2(1)1()1()2(...)1(1
ppp
ppp
Gp
Introduction to time series (2008 ) 70
Partial autocorrelation function
• Regarding this as system of p equations andp unknowns (the coefficients), thesolution is, for p>1, the ratio of twodeterminants
ppp GH /
Introduction to time series (2008 ) 71
Partial autocorrelation function
)()1(...)2()1()1(1)2(
)2(1)1()1()2(...)1(1
ppppp
p
Hp
where
Introduction to time series (2008 ) 72
Partial autocorrelation function
•A pxp matrix, and for p=1. This leads to define, for any stationary model
•Which is known as the Partial Autocorrelation Function.
)1( p
1/1)(
kGHkk
kkk
Introduction to time series (2008 ) 73
Partial autocorrelation function
It has the property that, for a stationary AR(p) model,
In other words, vanishes for k>p when the model is AR(p).
pkpkk
k 0
k
Introduction to time series (2008 ) 74
properties
Slow decay towards zero
Slow decay towards zero
ARMA(p,q)
slow decay towards zero
q different from zero
MA(q)
p different from zero
slow decay towards zero
AR(P)
PAFSAF
Introduction to time series (2008 ) 75
Examples
• AR(1) model
• Model:
• Variance:
ttt azz 1
2222azz
2
22
1
az
Introduction to time series (2008 ) 76
Examples
• Autocovariance function:
1)1(
11
0
)( 2
2
2
kk
k
k
k a
z
Introduction to time series (2008 ) 77
Examples
• Autocorrelation function:
• Partial autocorrelation:
1101
)(kkk
kk
)1()1(
Introduction to time series (2008 ) 78
Examples
• AR(2) model
• Model:
• Variance:
tttt azzz 2211
21
22
2
2
22
)1(11
az
Introduction to time series (2008 ) 79
Examples
• Autocorrelation function:
2
1
11
)(
2
21
2
2
1
k
kk
22211 kkkk
Introduction to time series (2008 ) 80
Examples
• MA(1) model
• Model:
• Variance:
1 ttt aaz
2222aaz
)1( 222 az
Introduction to time series (2008 ) 81
Examples
• Autocovariance function:
1010
)( 2
2
kkk
k a
z
Introduction to time series (2008 ) 82
Examples
• Autocorrelation function:
10
11
01)( 2
k
kk
k
Introduction to time series (2008 ) 83
Specification
• The class of ARMA(p,q) models isextensive.
• Guidelines are needed in selecting a member of the class to represent the time series data at hand
Introduction to time series (2008 ) 84
Specification
• Box and Jenkins (1976) proposed aniterative model building strategy.
– Tentative specification or identification of a model.
– Efficient estimation of model parameters.– Diagnostic checking of fitted model for further
improvement.
Introduction to time series (2008 ) 85
Specification
• Tentative specification. The aim is toemploy statistics that:
– 1. Can be readily calculated from thedata.
– 2. Allow the user to tentatively select a model .
Introduction to time series (2008 ) 86
Specification
• Three methods:
– The Sample Autocorrelation Function(SAF)
– The Sample Partial AutocorrelationFunction (SPAF)
– Use of diagnostic tools (AIC,BIC) to findthe best model (TRAMO-automatic).
Introduction to time series (2008 ) 87
Specification
• Sample Autocorrelation Function. TheSACF of are defined as
With
and the sample mean of the n availableobservations
tz
,...2,1/ˆ 0 kCCkk
))((1
zzzzC jt
jn
ttj
z
Introduction to time series (2008 ) 88
Specification
• Properties:
• 1.For stationary models,
• 2. When there exists a unit root in the AR polynomial
nkk ̂
1ˆ p
k
Introduction to time series (2008 ) 89
Specification
• If the SACF of the original series ispersistently close to 1 as k increases, oneforms the first difference , , and studiesits SACF to determine whether furtherdifferencing is called for.
tz
Introduction to time series (2008 ) 90
Specification
• Once stationary is achieved, a cutting offpattern after, say a lag “q”, in the SACF willthen lead to tentative specification of a MA(q) model.
Introduction to time series (2008 ) 91
Specification
• The Sample Partial AutocorrelationFunction:
• Are obtained by replacing the in
• by their sample estimates
,...2,1ˆ kkk
1/1)(
kGHkk
kkk
k̂
Introduction to time series (2008 ) 92
Specification
Properties:• 1.For stationary models,
• 2. The are asymptotically normallydistributed
• 3. For a stationary AR(p) model
nkk ̂
k̂
pknVar k 1)ˆ(
Introduction to time series (2008 ) 93
Specification
• Properties 1, 2 and 3 make SPACF a convenient tool for specifying the order or a stationary AR model (cutting off patternafter lag p)
• Not valid for non-stationary models.
Introduction to time series (2008 ) 94
Specification
Weakness of the SACF and SPACF.
1. Subjective judgment is often required todecide on the order of differencing.
2. For stationary ARMA models, both SACF and SPACF tend to exhibit a gradual “tapering off” behavior, making thespecification very difficult.
Introduction to time series (2008 ) 95
Specification
Diagnostic Tools
• The program (TRAMO), in an automaticway, specifies a set of possible models, estimate them and select the best one basedon AIC and BIC criteria.
• IMPOSSIBLE 10-15 years before