Authors: Péter Elek, Anikó Bíró Supervised by: Péter Elek

ECONOMETRICS

ECONOMETRICS

Sponsored by a Grant TÁMOP-4.1.2-08/2/A/KMR-2009-0041 Course Material Developed by Department of Economics,

Faculty of Social Sciences, Eötvös Loránd University Budapest (ELTE) Department of Economics, Eötvös Loránd University Budapest

Institute of Economics, Hungarian Academy of Sciences Balassi Kiadó, Budapest

ECONOMETRICS

Authors: Péter Elek, Anikó Bíró Supervised by: Péter Elek

June 2010

ELTE Faculty of Social Sciences, Department of Economics

ECONOMETRICS

Week 7.

Summary of estimation methods and large sample theory

Péter Elek, Anikó Bíró

Regression model

y_i = α + β₁x_1i + β₂x_2i +…+ β_kx_ki + u_i, i = 1…n

Assumptions 1. E(u_i) = 0

2. u_i, u_j independent for all i≠j

3. x_i, u_j independent for all i, j (exogeneity) 4. No perfect collinearity

5. Var(u_i) = σ² for all i

6. u_i has normal distribution

1–5.: Gauss–Markov conditions

1–6.: Conditions of classical linear model

Assumptions differently (for large sample theory – stochastic explanatory variables)

1. Population model: y = α + β₁x₁ + β₂x₂ +…+ β_kx_k + u.

2. {(x_1i,x_2i,…,x_ki,y_i), i = 1,…,n} random independent sample of the model.

3. None of the regressors is constant, no perfect collinearity among the regressors.

4. Exogeneity: E(u|x₁,…,x_k) = 0

5. Homoscedasticity: Var(u|x₁,…,x_k) = σ²

6. u independent of the regressors, normally distributed.

1–5.: Gauss–Markov conditions

1–6.: Conditions of classical linear model

Multivariate regression model

Estimation: method of moments or OLS (also ML estimation if error term is normal)

Matrix

2 2

2 1

ˆ 1

ˆ,

( ˆ ˆ ˆ ... ˆ )

min

k











 ^



) ' ( ) '

ˆ ( X X

X y β

u

Xβ

y   

Simple regression

i i

i i

i

i i

xx xy

x y

y y

u

x y

x y

S S









































ˆ ˆ ˆ

ˆ

ˆ ˆ ˆ

ˆ ˆ ˆ

 

  

 

2 2 2

y n y

y y

S

y x n y

x y

y x x

S

x n x

x x

S

i i

yy

i i i

i xy

i i

xx

































 



Interpretation of multivariate model

Interpretation of coefficients

Partial effect (“ceteris paribus”): effect of a given regressor on the dependent variable, holding

the other regressors fixed

Coefficient of determination: R

RSS = S

(1 – R

)

Small sample properties of estimation

If assumptions 1–4 hold: OLS unbiased

If assumptions 1–5 (Gauss-Markov) hold: the estimation is BLUE, and the common formula of variance is correct:

If assumptions 1–6 (classical linear model) hold:

the t- and F-statistic have t- and F-distribution, respectively (any sample size).

i

i RSS

Var

2

ˆ )

(  

Multivariate regression, t-test

Two sided test: pl. H₀: β_i = 0, H₁: β_i ≠ 0 One sided test: pl. H₀: β_i = 0, H₁: β_i > 0

In case of normal error term:

~ 1

ˆ ) ( ˆ







k n i

i

i t

SE 





i

i RSS

Var

2

ˆ )

(  

ˆ

1



 

k n

 RSS

Simple regression





2

2 2

~ /

/ ˆ 1

ˆ

~ ˆ /

ˆ











n xx

n xx

t S

x n

t S













Multivariate regression, F-test

Testing nested hypotheses Testing multiple restrictions

) 1 ( 2 ,

2 2

2 1

0

2 2

) 1 ( 2 ,

2 2

) ~ 1 /(

) 1

( /

0

0 ...

: H

: used be

cannot Regression

) 1

(

) 1

(

) ~ 1 /(

) 1

(

/ ) (

) 1 /(

/ ) (













 























 





 

k n k U

U R

k

U yy

R yy

k n r U

R U

k F n R

k F R

R

R S

URSS R

S RRSS

k F n R

r R

R k

n URSS

r URSS F RRSS







Regression cannot be used:

Analysis of variance

Source of

variance

Sum of squares

Degree of freedom

Mean sum of squares

F Regr. R²S_yy k R²S_yy/k = MS₁ F =

= MS₁/MS₂

Residual (1 – R²)S_yy n – k – 1 (1 – R²)S_yy/(n – k – 1) =

= MS₂

Total Syy n – 1

Large sample properties I:

consistency

If assumptions 1–4 hold: OLS is consistent. Proof for simple regression







 









 

 





) (

) , (

) (

) ,

ˆ ( plim

ˆ

x Var

u x Cov

x Var

u x

x Cov Var(x)

Cov(x,y) S

S

xx xy

Large sample properties II:

asymptotic normality

If assumptions 1–5 (Gauss–Markov) hold: OLS estimator is asymptotically normal:

Thus the standard deviation goes to zero in order n^1/2. The common estimator of σ² is consistent, therefore the common t-test is asymptotically valid (even if assumption 6 (normality) does not hold)!

  ^{ }

) 1

( )

1 ) (

( ˆ

, 0 ˆ ~

2 2

2 2

2

i x

i i

i

asympt i

i

R R

n TSS Var

n c

c N

n

i 

 















 





Large sample properties III:

F-test and others

If assumptions 1-5 hold (assumption 6 (normality) not needed):

F-test is asymptotically valid.

Other large sample tests (only asymptotically valid):

Wald-test: n(RRSS-URSS)/URSS ~ χ_r²

regression cannot be used: nR²/(1-R²)~ χ_k²

Lagrange-multiplicator (LM) test: n(RRSS-URSS)/RRSS ~ χ_r² regression cannot be used: nR² ~ χ_k²

Model selection

Adjusted R²

Nested hypotheses: t- and F-test

Non-nested hypotheses, same dependent

variable: adjusted R², information criteria (AIC, BIC – based on log-likelihood)

) 1

1(

1 ² 1 R²

k n

R n 





 



Omitting relevant variables

If omitted variable is correlated with included regressors: biased estimation (endogeneity) Simple regression

True model: y = β

Estimated model: y = γ

Bias: Corr(x

)>0 Corr(x

)<0

β₂

>0 +

β₂

<0

+

Including irrelevant variables

True model: y = β₁x₁ + β₂x₂ + u

Estimated model: y= β₁x₁ + β₂x₂ + β₃x₃+ u, β₃ = 0

Does not affect unbiasedness (no endogeneity) Variance increases

i

i RSS

2

)

Var(  

Other topics

Forecasting Outliers

Alternative functional forms Tests of stability

Dummy regressors

Quadratic terms, interactions

Heteroscedasticity, etc.

Seminar

First exam