Equations in Simple Regression Analysis
description
Transcript of Equations in Simple Regression Analysis
![Page 1: Equations in Simple Regression Analysis](https://reader034.fdocuments.in/reader034/viewer/2022051402/568157f3550346895dc572d2/html5/thumbnails/1.jpg)
Equations in Simple Regression Analysis
![Page 2: Equations in Simple Regression Analysis](https://reader034.fdocuments.in/reader034/viewer/2022051402/568157f3550346895dc572d2/html5/thumbnails/2.jpg)
The Variance
sx
nx2
2
1
![Page 3: Equations in Simple Regression Analysis](https://reader034.fdocuments.in/reader034/viewer/2022051402/568157f3550346895dc572d2/html5/thumbnails/3.jpg)
The standard deviation
S sx x 2
![Page 4: Equations in Simple Regression Analysis](https://reader034.fdocuments.in/reader034/viewer/2022051402/568157f3550346895dc572d2/html5/thumbnails/4.jpg)
The covariance
sxy
nxy 1
![Page 5: Equations in Simple Regression Analysis](https://reader034.fdocuments.in/reader034/viewer/2022051402/568157f3550346895dc572d2/html5/thumbnails/5.jpg)
The Pearson product moment correlation
rs
s sxyxy
x y
![Page 6: Equations in Simple Regression Analysis](https://reader034.fdocuments.in/reader034/viewer/2022051402/568157f3550346895dc572d2/html5/thumbnails/6.jpg)
The normal equations (for the regressions of y on x)
bxyx
s
s
xy
xy
x
2
2
a Y b Xyx -
![Page 7: Equations in Simple Regression Analysis](https://reader034.fdocuments.in/reader034/viewer/2022051402/568157f3550346895dc572d2/html5/thumbnails/7.jpg)
The structural model (for an observation on individual i)
Y a b X ei yx i i
![Page 8: Equations in Simple Regression Analysis](https://reader034.fdocuments.in/reader034/viewer/2022051402/568157f3550346895dc572d2/html5/thumbnails/8.jpg)
The regression equation
( )
( )
Y a b X
Y b X b X
Y b X X
Y b X
yx
yx yx
yx
yx
![Page 9: Equations in Simple Regression Analysis](https://reader034.fdocuments.in/reader034/viewer/2022051402/568157f3550346895dc572d2/html5/thumbnails/9.jpg)
Partitioning a deviation score, y
y Y Y
Y Y Y Y Y Y
Y Y Y Y
( ) ( )
( ) ( )
![Page 10: Equations in Simple Regression Analysis](https://reader034.fdocuments.in/reader034/viewer/2022051402/568157f3550346895dc572d2/html5/thumbnails/10.jpg)
Partitioning the sum of squared deviations (sum of squares,
SSy)
y Y Y
Y Y Y Y
Y Y Y Y
SS SS
2 2
2
2 2
( )
[( ) ( )]
( ) ( )
reg res
![Page 11: Equations in Simple Regression Analysis](https://reader034.fdocuments.in/reader034/viewer/2022051402/568157f3550346895dc572d2/html5/thumbnails/11.jpg)
Calculation of proportions of sums of squares due to regression and due to
error (or residual)
y
y
SS
y
SS
y
SS
y
SS
y
2
2
21
reg
2
res
2
reg
2
res
![Page 12: Equations in Simple Regression Analysis](https://reader034.fdocuments.in/reader034/viewer/2022051402/568157f3550346895dc572d2/html5/thumbnails/12.jpg)
Alternative formulas for computing the sums of squares due to regression
SS Y Y
Y bx Y
bx
b x
xy
xx
xy
x
xy
xxy
b xy
reg
( )
( )
( )
( )
( )
( )
2
2
2
2 2
2
2 22
2
2
2
![Page 13: Equations in Simple Regression Analysis](https://reader034.fdocuments.in/reader034/viewer/2022051402/568157f3550346895dc572d2/html5/thumbnails/13.jpg)
Test of the regression coefficient, byx, (i.e. test the null hypothesis that byx =
0)First compute the variance of estimate
s est
Y Y
N kSS
N k
y x y
2 2
2
1
1
( )
( )
res
![Page 14: Equations in Simple Regression Analysis](https://reader034.fdocuments.in/reader034/viewer/2022051402/568157f3550346895dc572d2/html5/thumbnails/14.jpg)
Test of the regression coefficient, byx, (i.e. test the null hypothesis that byx =
0)Then obtain the standard error of estimate
Then compute the standard error of the regression coefficient, Sb
s sy x y x 2
ss
x n
s
x Nb
y x y x
2
2 21 1( ) / ( ) ( ) / ( )
![Page 15: Equations in Simple Regression Analysis](https://reader034.fdocuments.in/reader034/viewer/2022051402/568157f3550346895dc572d2/html5/thumbnails/15.jpg)
The test of significance of the regression coefficient (byx)
The significance of the regression coefficient is tested using a t test with (N-k-1) degrees of freedom:
tb
s
b
S
S n
yx
b
yx
y x
x
1
![Page 16: Equations in Simple Regression Analysis](https://reader034.fdocuments.in/reader034/viewer/2022051402/568157f3550346895dc572d2/html5/thumbnails/16.jpg)
Computing regression using correlations
The correlation, in the population, is given by
The population correlation coefficient, ρxy, is estimated
by the sample correlation coefficient, rxy
xy
x yN
rz z
Ns
s s
xy
x y
xyx y
xy
x y
2 2
![Page 17: Equations in Simple Regression Analysis](https://reader034.fdocuments.in/reader034/viewer/2022051402/568157f3550346895dc572d2/html5/thumbnails/17.jpg)
Sums of squares, regression (SSreg)
Recalling that r2 gives the proportion of variance of Y accounted for (or explained) by X, we can obtain
or, in other words, SSreg is that portion of SSy predicted or explained by the regression of Y on X.
SS r y
SS r y
reg
res
2 2
2 21( )
![Page 18: Equations in Simple Regression Analysis](https://reader034.fdocuments.in/reader034/viewer/2022051402/568157f3550346895dc572d2/html5/thumbnails/18.jpg)
Standard error of estimate
From SSres we can compute the variance of estimate and standard error of estimate as
(Note alternative formulas were given earlier.)
sr y
N ks s
y x
y x y x
22 21
1( )
![Page 19: Equations in Simple Regression Analysis](https://reader034.fdocuments.in/reader034/viewer/2022051402/568157f3550346895dc572d2/html5/thumbnails/19.jpg)
Testing the Significance of r
The significance of a correlation coefficient, r, is tested using a t test:
With N-2 degrees of freedom.
21
2
r
Nrt
![Page 20: Equations in Simple Regression Analysis](https://reader034.fdocuments.in/reader034/viewer/2022051402/568157f3550346895dc572d2/html5/thumbnails/20.jpg)
Testing the difference between two correlations
To test the difference between two Pearson correlation coefficients, use the “Comparing two correlation coefficients” calculator on my web site.
![Page 21: Equations in Simple Regression Analysis](https://reader034.fdocuments.in/reader034/viewer/2022051402/568157f3550346895dc572d2/html5/thumbnails/21.jpg)
Testing the difference between two regression coefficients
This, also, is a t test:
Where
was given earlier. When the variances, , are unequal, used the pooled estimate given on page 258 of our textbook.
22
21
21 bb SS
bbt
2bS
2bS
![Page 22: Equations in Simple Regression Analysis](https://reader034.fdocuments.in/reader034/viewer/2022051402/568157f3550346895dc572d2/html5/thumbnails/22.jpg)
Other measures of correlation
Chapter 10 in the text gives several alternative measures of correlation:Point-biserial correlationPhi correlationBiserial correlationTetrachoric correlationSpearman correlation
![Page 23: Equations in Simple Regression Analysis](https://reader034.fdocuments.in/reader034/viewer/2022051402/568157f3550346895dc572d2/html5/thumbnails/23.jpg)
Point-biserial and Phi correlation
These are both Pearson Product-moment correlationsThe Point-biserial correlation is used when
on variable is a scale variable and the other represents a true dichotomy.For instance, the correlation between an
performance on an item—the dichotomous variable—and the total score on a test—the scaled variable.
![Page 24: Equations in Simple Regression Analysis](https://reader034.fdocuments.in/reader034/viewer/2022051402/568157f3550346895dc572d2/html5/thumbnails/24.jpg)
Point-biserial and Phi correlation
The Phi correlation is used when both variables represent a true dichotomy.For instance, the correlation between two
test items.
![Page 25: Equations in Simple Regression Analysis](https://reader034.fdocuments.in/reader034/viewer/2022051402/568157f3550346895dc572d2/html5/thumbnails/25.jpg)
Biserial and Tetrachoric correlation
These are non-Pearson correlations.Both are rarely used anymore.The biserial correlation is used when
one variable is truly a scaled variable and the other represents an artificial dichotomy.
The Tetrachoric correlation is used when both variables represent an artificial dichotomy.
![Page 26: Equations in Simple Regression Analysis](https://reader034.fdocuments.in/reader034/viewer/2022051402/568157f3550346895dc572d2/html5/thumbnails/26.jpg)
Spearman’s Rho Coefficient and Kendall’s Tau Coefficient
Spearman’s rho is used to compute the correlation between two ordinal (or ranked) variables.It is the correlation between two sets of
ranks.
Kendall’s tau (see pages 286-288 in the text) is also a measure of the relationship between two sets of ranked data.