Multiple Regression

PSYC 6130, PROF. J. ELDER 2

Multiple Regression

• Multiple regression extends linear regression to allow for 2 or more independent variables.

• There is still only one dependent (criterion) variable.

• We can think of the independent variables as ‘predictors’ of the dependent variable.

• The main complication in multiple regression arises when the predictors are not statistically independent.

PSYC 6130, PROF. J. ELDER 3

Example 1: Predicting Income

Age

Hours Worked

MultipleRegression

Income

PSYC 6130, PROF. J. ELDER 4

Example 2: Predicting Final Exam Grades

Assignments

Midterm

MultipleRegression

Final

PSYC 6130, PROF. J. ELDER 5

Coefficient of Multiple Determination

• The proportion of variance explained by all of the independent variables together is called the coefficient of multiple determination (R2).

• R is called the multiple correlation coefficient.

• R measures the correlation between the predictions and the actual values of the dependent variable.

• The correlation riY of predictor i with the criterion (dependent variable) Y is called the validity of predictor i.

PSYC 6130, PROF. J. ELDER 6

Uncorrelated Predictors

21 Yr 2

2Yr

Total variance

Variance explained by assignments Variance explained by midterm

2 2 2 2 21 2=Total proportion of variance explained = Y Y Y YR r r

PSYC 6130, PROF. J. ELDER 7

Uncorrelated Predictors

• Recall the regression formula for a single predictor:

• If the predictors were not correlated, we could easily generalize this formula:

Y Xz rz

1 1 2 2Y Y Yz r z r z

PSYC 6130, PROF. J. ELDER 8

Example 1. Predicting Income

Correlations

1 .040* .229**

.012 .000

3975 3975 3975

.040* 1 .187**

.012 .000

3975 3975 3975

.229** .187** 1

.000 .000

3975 3975 3975

Pearson Correlation

Sig. (2-tailed)

N

Pearson Correlation

Sig. (2-tailed)

N

Pearson Correlation

Sig. (2-tailed)

N

AGE

HOURS WORKEDFOR PAY OR INSELF-EMPLOYMENT- in Reference Week

TOTAL INCOME

AGE

HOURSWORKEDFOR PAY

OR INSELF-

EMPLOYMENT - inReference Week

TOTALINCOME

Correlation is significant at the 0.05 level (2-tailed).*.

Correlation is significant at the 0.01 level (2-tailed).**.

PSYC 6130, PROF. J. ELDER 9

Correlated Predictors

21 Yr 2

2Yr

Total variance

Variance explained by assignments Variance explained by midterm

2 2 21 2=Total proportion of variance explained < Y YR r r

PSYC 6130, PROF. J. ELDER 10

Correlated Predictors

• Due to the correlation in the predictors, the optimal regression weights must be reduced:

1 1 2 2Yz z z

1 2 12 2 1 121 22 2

12 12

where

and 1 1

Y Y Y Yr r r r r r

r r

1 2 beta weights

(standardized partial re

and

gres

are calle

sion coeffi

d th

c

s)

e

ient

2 22 1 2 1 2 12

1 1 2 2 212

2

1Y Y Y Y

Y Y

r r r r rR r r

r

PSYC 6130, PROF. J. ELDER 11

Raw-Score Formulas

0 1 1 2 2Y B B X B X

1 2

1 1 2 2

0 1 1 2 2

where

and

and

Y Y

X X

s sB B

s s

B Y B X B X

PSYC 6130, PROF. J. ELDER 12

Example 1. Predicting Income

Correlations

1 .040* .229**

.012 .000

3975 3975 3975

.040* 1 .187**

.012 .000

3975 3975 3975

.229** .187** 1

.000 .000

3975 3975 3975

Pearson Correlation

Sig. (2-tailed)

N

Pearson Correlation

Sig. (2-tailed)

N

Pearson Correlation

Sig. (2-tailed)

N

AGE

HOURS WORKEDFOR PAY OR INSELF-EMPLOYMENT- in Reference Week

TOTAL INCOME

AGE

HOURSWORKEDFOR PAY

OR INSELF-

EMPLOYMENT - inReference Week

TOTALINCOME

Correlation is significant at the 0.05 level (2-tailed).*.

Correlation is significant at the 0.01 level (2-tailed).**.

1 1 2 2Yz z z

1 2 12 2 1 121 22 2

12 12

where

and 1 1

Y Y Y Yr r r r r r

r r

2 22 1 2 1 2 12

1 1 2 2 212

2

1Y Y Y Y

Y Y

r r r r rR r r

r

PSYC 6130, PROF. J. ELDER 13

Example 1. Predicting Income

020

4060

80

0

20

40

60

800

1

2

3

4

5

6

7

x 104

Age (years)Hours worked per week (hours)

An

nu

al I

nco

me

(C

AD

)

PSYC 6130, PROF. J. ELDER 14

Degrees of freedom

1 where

sample size

number of predictors

df n k

n

k

PSYC 6130, PROF. J. ELDER 15

Semipartial (Part) Correlations

• The semipartial correlations measure the correlation between each predictor and the criterion when all other predictors are held fixed.

• In this way, the effects of correlations between predictors are eliminated.

• In general, the semipartial correlations are smaller than the validities.

PSYC 6130, PROF. J. ELDER 16

Calculating Semipartial Correlations

• One way to calculate the semipartial correlation for a predictor (say Predictor 1) is to partial out the effects of all other predictors on Predictor 1and then calculate the correlation between the residual of Predictor 1 and the criterion.

• For example, we could partial out the effects of age on hours worked, and then measure the correlation between income and the residual hours worked.

PSYC 6130, PROF. J. ELDER 17

Calculating Semipartial Correlations

• A more straightforward method:

1 2 12(1.2) 2

121Y Y

Y

r r rr

r

(1.2)where is the semipartial correlation between Predictor 1 and Yr Y

i.e., the correlation between and Predictor 1

after partialling out the effects of Predictor 2 on Predictor 1.

Y

PSYC 6130, PROF. J. ELDER 18

Example 2: Predicting Final Exam Grades

Assignments

Midterm

MultipleRegression

Final

PSYC 6130, PROF. J. ELDER 19

Example 2. Predicting Final Exam Grades (PSYC 6130A, 2005-2006)

Correlations

1 .356 .127

.233 .680

13 13 13

.356 1 .615*

.233 .025

13 13 13

.127 .615* 1

.680 .025

13 13 13

Pearson Correlation

Sig. (2-tailed)

N

Pearson Correlation

Sig. (2-tailed)

N

Pearson Correlation

Sig. (2-tailed)

N

Assignments

Midterm

Final

Assignments Midterm Final

Correlation is significant at the 0.05 level (2-tailed).*.

212 120.356 0.127r r 2

1 120.127 0.016Yr r 22 20.615 0.378Y Yr r

PSYC 6130, PROF. J. ELDER 20

Example 2. Predicting Final Exam Grades (PSYC 6130A, 2005-2006)

212 120.356 0.127r r 2

1 120.127 0.016Yr r 22 20.615 0.378Y Yr r

1 1 2 2Yz z z

1 2 12 2 1 121 22 2

12 12

where

and 1 1

Y Y Y Yr r r r r r

r r

2 22 1 2 1 2 12

1 1 2 2 212

2

1Y Y Y Y

Y Y

r r r r rR r r

r

PSYC 6130, PROF. J. ELDER 21

Example 2. Predicting Final Exam Grades

0 1 1 2 2Y B B X B X

1 2

1 1 2 2

0 1 1 2 2

where

and

and

Y Y

X X

s sB B

s s

B Y B X B X

PSYC 6130, PROF. J. ELDER 22

Example 2. Predicting Final Exam Grades

7080

90100

20

40

60

800

50

100

150

Assignment grade (%)Midterm grade (%)

Fin

al g

rad

e (

%)

PSYC 6130, PROF. J. ELDER 23

SPSS Output

PSYC 6130, PROF. J. ELDER 24

Example 3. 2006-07 6130 Grades

• Try doing the calculations on this dataset for practice.