Appendix 2A. Simple regression and multiple

regression

ByCheng Few LeeJoseph Finnerty

John LeeAlice C Lee

Donald Wort

Appendix 2A. Simple regression and multiple regression

2. A.1 INTRODUCTION

2. A.2 SIMPLE REGRESSION

Variance of

Multiple Regressionb

2

Appendix 2A. Simple regression and multiple regression

(2.A.1a)

(2.A.1b)

(2.A.2a)

(2.A.2b)

1t t tY a bX

1log logt t tY a b X

1

1 1 1 2cov , 2cov , 2cov , ,

t t t

t t t t t t

Var Y Var a bX

Var a Var bX Var a bX a bX

21t t tVar Y b Var X Var

3

Appendix 2A. Simple regression and multiple regression

(2.A.3)

(2.A.4)

(2.A.5a)

(2.A.5b)

2

21

Variation explained by the explanatory variable

Total variation in the dependent variable

t

t

R

b Var X

Var Y

2 2

11 1

ˆˆ ˆn n

t t t tt t

ESS Y Y Y a bX

11

( ) ˆˆ2 0n

t tt

ESSY a bX

a

1 11

( ) ˆˆ2 0n

t t tt

ESSX Y a bX

b

4

Appendix 2A. Simple regression and multiple regression

(2.A.6a)

(2.A.6b)

11 1

ˆˆn n

t tt t

an b X Y

5

Appendix 2A. Simple regression and multiple regression

(2.A.7)

(2.A.7a)

1

1 1 1 11 1 1 1 1

2 21 11

1 11

21 1

1 1

( ) ( )ˆ

( )

n

tt

n n n n n

t t t t t t tt t t t t

n nn

t ttt tt

n n

t tt t

n Y

X X Y n X Y X Yb

n X Xn X

X X

1

1

[ , ]ˆ[ ]

t t

t

Cov X Yb

Var X

6

Appendix 2A. Simple regression and multiple regression

(2.A.8)

(2.A.8a)

12

2 1 1 11 1 1 1 1 1

2 211 1

1 121 1

2 21 1 1 1 1 1

1 1 1 1 1 1

( )( ) ( )( )ˆ

( )

( )[ ( ) ( ) ] ( )[ ( ) ( )( )]

t n n n n

t t t t tt t t t t t t

n nt

t tt t

t t

n n n n n n

t t t t t t t tt t t t t t t

Y XY X X X YX Y X

an X n X XX X

Y n n X X X n n X Y X X

1

2 21 1

1 1( )

n

n n

t tt t

n X X

ˆa Y Xb

7

Variance of

Equation (2.A.7a) implies that:

(2.A.7b)

Where

b

1121 111

( )ˆ n nt t

t tnt ttt

x yb W y

x

1 1

11 2

11

t t

t t

tt n

tt

x X X

y Y Y

xW

x

8

Variance of

(2.A.7c)

(2.A.9)

b

1 1 11 1

ˆ n n

t t t tt t

b W bx W

2

21 1 1

1 1

21 1 1

1 1

21 1 1

1 1

ˆ ˆ( ) ( )

( )

[( 1) ]

( ) , since 1.

n n

t t t tt t

n n

t t t tt t

n n

t t t tt t

Var b E b b

E W bX W b

E W x b W

E W W x

2 20 1 0 1 1 2 1 2

ˆ( ) [( ) 2( ) ( ) ]Var b E W W W W

9

Variance of b

2 20 1 1 2

2 2 2 20 1 1 2

ˆ( ) ( ) ( )

( ) ( )

Var b E W E W

W E W E

2 21

1

221

1

ˆ( ) ( )

n

t tt

n

tt

Var b W E

W

2112

1 2 221 1 11 1

1

( )

nntt

t n nt t tt t

xW

x x

10

Variance of b

(2.A.10)

(2.A.11)

(2.A.12)

2

211

ˆ( )n

tt

Var bx

2112

211

ˆ( )n

ttn

tt

xVar a

n x

22

11

ˆˆ( , )n

tt

XCov a b

x

11

Multiple Regression

(2.A.13a)

The error sum of squares can be defined as:

Where

1, 1 2, 1t t t tY a bX cX

2 2ˆˆ ( )t t tESS Y Y

1, 1 2, 1ˆˆ ˆ ˆt t t tY a bX cX

12

Multiple Regression

(2.A.14a)

(2.A.14b)

(2.A.14c)

1, 1 2, 10 or t t t

ESSY na b X c X

a

21, 1 1, 1 1, 1 1, 1 2, 10 or t t t t t t

ESSX Y a X b X c X X

b

22, 1 2, 1 1, 1 2, 1 2, 10 or t t t t t t

ESSX Y a X b X X c X

c

13

Multiple Regression

0 = na + b(0) + c(0),

(2.A.15a)

(2.A.15b)

(2.A.15c)

21, 1 1, 1 1, 1 2, 1(0)t t t t tx y a b x c x x

22, 1 1, 1 2, 1 2, 1(0)t t t t tx x a b x x c x

14

Multiple Regression

(2.A.16a)

(2.A.16b)

(2.A.17)

21,21,1

21,2

21,1

1,21,11,22

1,21,1

)())((

)(ˆ

tttt

ttttttt

xxxx

xxyxxyxb

22, 1 1, 1 1, 1 1, 1 2, 1

2 2 21, 1 2, 1 1, 1 2, 1

( )ˆ

( )( ) ( )t t t t t t t

t t t t

x y x x y x xc

x x x x

1 2ˆˆˆ ˆa Y bX cX

15

Multiple Regression

(2.A.13b)

(2.A.18)

(2.A.19)

1, 1 2, 10.2837 0.7564 0.2990 (0.4323) (0.3288) (0.2240)

t t tY X X

ˆ 1.7071(0.7564)(1.8448) (0.2990)(1.6904 ) 0.2837.a

ˆ ˆ( ) ( ) ( )t t t t t tY Y Y Y Y Y

1, 1 2, 1ˆˆ ˆ ˆt t tY a bX cX

16

Multiple Regression

(2.A.20)

where

TSS = Total sum of squares;

ESS = Residual sum of squares; and

RSS = Regression sum of squares.

2 2 2ˆ ˆ( ) ( ) ( ) , TSS ESS RSS

t t t t t tY Y Y Y Y Y

17

Multiple Regression

(2.A.21)

(2.A.22)

Where

and k = the number of independent variables.

2 22

2 2

ˆ ˆ( )RSS1

TSS ( ) ( )t t t

t t t t

Y YR

Y Y Y Y

2

2 ˆ1

( )t

t

RVar Y

22 ˆ

ˆ( ) ttVar

n k

2( )

( )1

tt

Y YVar Y

n

18

Multiple Regression

(2.A.23)

where F(k-1, n-k) represents F-statistic with

k－ 1 and n－ k degrees of freedom.

2 2 11 (1 )

nR R

n k

2

2( 1, )

1 1

R n kF k n k

R k

19

Appendix 2B.

Instrumental variables and two-stage least squares

ByCheng Few LeeJoseph Finnerty

John LeeAlice C Lee

Donald Wort

Appendix 2B. Instrumental variables and two-stage least squares

2. B.1 ERRORS-IN-VARIABLE PROBLEM2. B.2 INSTRUMENTAL VARIABLES2. B.3 TWO-STAGE, LEAST-SQUARE

21

2. B.1 ERRORS-IN-VARIABLE PROBLEM

(2.B.1)

(2.B.2)

(2.B.3)

ttmjjtj RBAR ,,

*, ,m t m t tR R V

* 2 2, ,( ) ( )m t m t t m VVar R Var R V

22

2. B.1 ERRORS-IN-VARIABLE PROBLEM

(2.B.4)

(2.B.5)

ttmjjtj RBAR *,

*,

*, , ,

*, ,

,

2 2,

( , ) ( , )ˆ( ) ( ) ( )

( , , ) ( , )

( ) ( ) 1 /

m t jt m t t j j m t tj

m t m t t

j m t m t t t j

m t t V M

Cov R R Cov R V B RB

Var R Var R Var V

B Cov R R Cov V B

Var R Var V

23

2. B.2 INSTRUMENTAL VARIABLES

(2.B.6)

(2.B.7)

(2.B.8a)

(2.B.8b)

( , ) ( , ) ( , )j j mCov R Z B Cov R Z Cov Z

*

( , ) ( , )ˆ( , ) ( , )

j jj

m m

Cov R Z Cov R ZB

Cov R Z Cov R Z

12101 EYAAY

2121102 EZBYBBY

24

2. B.2 INSTRUMENTAL VARIABLES

(2.B.9a)

(2.B.9b)

(2.B.10a)

(2.B.10b)

1222101 EZAYAAY

2121102 EZBYBBY

1 0 1 2 2 2 3 3 1Y A AY A Z A Z E Y

2121102 EZBYBBY

25

2.B.3 TWO-STAGE, LEAST-SQUARE

(2.B.11a)

(2.B.11b)

(2.B.10′a)

(2.B.10′b)

133221101 EZCZCZCCY

233221102 EZDZDZDDY

133222101ˆ EZAZAYAAY

2121102ˆ EZBYBBY

26

2.B.3 TWO-STAGE, LEAST-SQUARE

(2.B.12a)

(2.B.12b)

21 1 10.2399 0.8198 1.9004 , 0.3449,

(0.1012) (0.2802) (1.245)Y Z Z R

22 1 20.0746 0.1133 0.7849 , 0.4240,

(0.0195) (0.0541) (0.2405)Y Z Z R

27