ECONOMETRICS
ECONOMETRICS
Sponsored by a Grant TÁMOP-4.1.2-08/2/A/KMR-2009-0041 Course Material Developed by Department of Economics,
Faculty of Social Sciences, Eötvös Loránd University Budapest (ELTE) Department of Economics, Eötvös Loránd University Budapest
Institute of Economics, Hungarian Academy of Sciences Balassi Kiadó, Budapest
ECONOMETRICS
Authors: Péter Elek, Anikó Bíró Supervised by: Péter Elek
June 2010
ELTE Faculty of Social Sciences, Department of Economics
ECONOMETRICS
Week 10.
Univariate time series analysis II.
Péter Elek, Anikó Bíró
Plan
AR, MA, ARMA and ARIMA processes
Box–Jenkins methodology, estimation and goodness of fit test of ARMA models
Forecasting from ARMA models
Material: M 13.4–13.6
AR(1) process
2
2 2
1 2
0
1
0 1 2
2 1
1 / )
Var(
) Var(
) Var(
1 / )
(
repr.) )
(M A(
1
/
: and ,
stationary is
model then the
1
|
| If
1 1
...
) ,
0 (
~
,
t t
t t
i
i t i t
n
i
i t i n
t n n
t t
t t
t t
t t
X X
X X E
X
X X X
IN X
X
then the model is stationary, and:
If
(MA(∞) – repr.)
ACF, PACF in AR(1) models
1 k
if , 0
1 k
if , 1
) ,
cov(
) ,
cov(
2
1 1
k
k k
k k
k k
t t
t k
t t
k
X X X X
if if
AR(p) process
p
t tp t
p p
t p p t
t k k
t
t p
t p t
t t
L L
L
L L
L X
x x
X L L
L
X L X
X X
X c
X
1 2
1
1 1
2 2 1
t i
1 1 p
i
2 2 1
2 2
1 1
1 ...
1 1
...
1
and stationary is
X then 1,
|
| satisfy 0
...
of roots all
If
. ...
1
operator, lag
the using and
0 c Assuming
...
Assuming c = 0 and using the lag operator,
If all roots of satisfy then is stationary and
Properties of stationary AR(p) processes
. if
0 : PACF
recursion :
), (
equations of
system :
...
...
, ...
cov ,
cov
equations Walker
- Yule :
ACF
...
1 /
k - k
2 2 1
1
2 2 1
1
1 1 2 1
p k
p k
p k
X X
X c
X X
c X
E
k
p k p k
k k
p k p k
k
k t t p t p t
k t t k
p t
ACF: Yule–Walker equations
: system of equations : recursion
PACF
Example: AR(1) process
X
t= 0,7X
t–1+
t(ACF is easy to calculate)
-3 -2 -1 0 1 2 3 4
25 50 75 100
Example: AR(2) process
X
t= 0,4X
t–1+ 0,5X
t–2+
t-3 -2 -1 0 1 2 3 4 5
25 50 75 100
MA(1) process
. every for
stationary :
X
0 to decays :
1
if 0
1 /
1
if 0
1 Var
t
2 1
2 1
2 2
0
1
k k k
t t
t t
t
k k X
c X
E c X
decays to 0
stationary for every
MA(q) process
i t
0 0 2
2 2
1 2
0
1 1
every for
stationary :
X
zero to
decay s :
/
. if
0,
0 if ,
...
1 Var
...
k
k k
k q
i
k i i k
q t
t
q t q t
t t
q k
q k
X
c X
E c X
decays to zero
stationary for every
Example: MA(1) process
X
t=
t+ 0,7
t–1(ACF is easy to calculate)
-3 -2 -1 0 1 2 3 4
25 50 75 100
ARMA(p,q) process
Xt = c + α1Xt–1 +…+ αpXt–p+
t + β1
t–1 +…+ βq
t–qStationary if its AR(p) component is stationary (all roots of the characteristic equation…)
Neither ACF nor PACF is 0, but both tend to zero at an exponential rate.
Remark: ACF PACF
AR(p) decays to 0 0 for k>q MA(q) 0 for k>q decays to 0
ARIMA(p,d,q) process
Xt is an ARIMA(p,1,q) process if Xt is a stationary ARMA(p,q) process.
Similarly, Xt is an ARIMA(p,d,q) process if Xt is an ARIMA(p,d–1,q) process.
Order of integration of ARIMA(p,d,q) is I(d).
Examples:
ARIMA(0,1,0): Xt – Xt-1 = t is the random walk.
ARIMA(1,1,0): Xt – Xt–1 = (Xt–1 – Xt–2) + t, where ||<1.
So Xt = (1 + )Xt–1 – Xt–2 + t is a nonstationary AR(2) process.
Example: ARMA(1,1) process
-6 -4 -2 0 2 4 6
25 50 75 100
Estimation of ACF
Estimation:
Only makes sense in the stationary case, and in this case it is consistent (i.e. for large T it
estimates
k with a small variance).
T
t
t k
T
t
k t t
k
X X
X X
X X
1
2
ˆ 1
Box-Jenkins methodology
Taking differences
The series is differentiated until it becomes stationary
Identification
Conjecture of the orders p, q of the ARMA model based on the ACF
Estimation
Examining the goodness of fit
Estimation of ARMA models
Simple in the case of AR models
OLS (minimising the sum of squares of the estimated innovations (t))
Consistent and asymptotically normal in the stationary case
In the case of MA or ARMA models
Full maximum likelihood or Searching methods
Choosing the starting innovations as zero,
the subsequent innovations can be calculated as
a function of the parameters, and their sum of squares can be minimised
Model selection criteria in ARMA models
These criteria control for the fact that using more
parameters in the model may only apparently give a better fit
Minimising a criterion yields to the optimal size of the model.
Examples:
Akaike information criterion
AIC = n·log(RSS/(n – s)) + 2s
Bayes (Schwartz) information criterion BIC = n·log(RSS/(n – s)) + s · logn Where
s: number of estimated parameters RSS: sum of squares of innovations n: sample size
Testing autocorrelation in the residuals
sample) large
, H (under
~
... ˆ ˆ
ˆ ˆ
s innovation on the
regression :
test -
Godfrey -
Breusch
sample) large
, H (under
ˆ ~ 2
: test -
Box -
Ljung
0
residuals the
of ation autocorrel
k lag :
0 2
2
2 2 1
1
0 2
1 2 2
1 0
m
t m
t m t
t t
s m m
k
k LB
m k
NR
u b
b b
n-k n r
n Q
r ...
r : r
H r
lag k autocorrelation of the residuals
Ljung–Box-test
Breusch–Godfrey-test: regression on the innovations (under H0, largesample)
(under H0, largesample)
Example: white noise test for S&P logarithmic returns
0 200 400 600 800 1000 1200 1400 1600
1000 2000 3000 4000 5000
-.25 -.20 -.15 -.10 -.05 .00 .05 .10
1000 2000 3000 4000 5000
White noise test (cont.)
low, perhaps significant autocorrelation
but: one should be careful when drawing conclusions because of heteroscedasticity (changing variance)
Forecasting from ARIMA models
t t
t
t t t
t t
t t
t t
t t
t t
t t
t t
t t
t t
t
t t t
t t
t t
t t
X X
I X
X E
I X
E X
X X
I X
X E
I X
E X
X X I
X X
X
ˆ ˆ ˆ ˆ
ˆ ˆ
| ˆ |
ˆ ˆ ˆ ˆ
ˆ ˆ ˆ
| ˆ |
,...) ,
( in t set n informatio
g Forecastin
ˆ ˆ ˆ ˆ
ˆ ˆ ˆ
ˆ
ˆ 0 , ˆ 0
s innovation Estimated
2 2
1 1
2 1
1 2
2 1
1 2
1
1 2 1
1 2
1
1 2 1
1 1
2 1
1 1
1
2 2 1
1 2
2 1
1 1
0
Estimated innovations
Forecasting
information set in t
Types of forecasts and evaluation of their performance
Forecasts
In sample
Out of sample
Performance evaluation: root mean squared error (RMSE), mean absolute error (MAE)
Estimation on interval [1,T], evaluation on interval [T+1,T+m]
m X X
RMSE
m T
T t
t
t
1
ˆ 2
m X X
MAE
m T
T t
t
t
1
ˆ |
|
Seminar
Univariate time series II.
Exercises I
Simulation of AR(1), MA(1), AR(2) and MA(2) time series
Graphical representation of the ACF and PACF
Determination of the ACF and PACF by the Yule-Walker equations
Evaluation of the stationarity of the AR(2)
model by the roots of the characteristic
equation
Exercises: series of company bond data I.
Evaluation of stationarity by visual inspection of the ACF
Estimation of ARMA models on a subsample
Goodness of fit test and model selection
Significance of parameters
Uncorrelatedness of residuals (Ljung–Box and Breusch–Godfrey-tests)
Model selection based on AIC and BIC
Exercises: series of company bond data II
Static (multi-period) forecast based on the best performing model
Dynamic (one-period) forecast
Comparison of its RMSE with that of the naive forecast
