Measures of relationship

Dr. Omar Al Jadaan

Agenda

• Correlation– Need – meaning,

• simple linear regression– analysis – prediction

Correlation

• Correlation is a statistical measurement of the relationship between two variables.

• Possible correlations range from +1 to –1. • A zero correlation indicates that there is no relationship

between the variables. • A correlation of –1 indicates a perfect negative correlation,

meaning that as one variable goes up, the other goes down.

• A correlation of +1 indicates a perfect positive correlation, meaning that both variables move in the same direction together.

Correlation

• Why we need correlation?To discover the interaction patterns between

the dependent variable and infer the mathematical model of the relation.

Linear regression

• simple linear regression is the least squares estimator of a linear regression model with a single predictor variable. In other words, simple linear regression fits a straight line through the set of n points in such a way that makes the sum of squared residuals of the model (that is, vertical distances between the points of the data set and the fitted line) as small as possible.

• The adjective simple refers to the fact that this regression is one of the simplest in statistics. The fitted line has the slope equal to the correlation between y and x corrected by the ratio of standard deviations of these variables. The intercept of the fitted line is such that it passes through the center of mass (x, y) of the data points.

• The purpose of least-squares method is to find the equation of the straight line that fits the data in the sense of least squares.

• Assumption of regression:– Normality of errors (with zero mean of each value)– variation around the line of regression is constant for all

the values of x (this means that the errors vary by the same amount for small x as for large x.

– The errors are independent for all values of x.– The relationship between x, y is postulated to be linear.

• The linear model y=β0+β1x+ε

• to calculate the estimate of β0, β1 we have to calculate

xyss

ss

yxxxss

xxss

xx

xy

xy

xx

101

2

,  

))((

)(

Example (study hours and score) studied hours x scored on test observed y x-mean(x) squar y-mean(y) (x-mean(x))*(y-mean(y)) predicted y y-predicted y squar

10 78 -0.8 0.64 -0.9 0.72 77.59091 0.4090913 0.167356

15 83 4.2 17.64 4.1 17.22 85.77273 -2.772727 7.688014

8 75 -2.8 7.84 -3.9 10.92 74.31818 0.6818186 0.464877

7 77 -3.8 14.44 -1.9 7.22 72.68182 4.3181822 18.6467

13 80 2.2 4.84 1.1 2.42 82.5 -2.5 6.249998

15 85 4.2 17.64 6.1 25.62 85.77273 -0.772727 0.597107

20 95 9.2 84.64 16.1 148.12 93.95455 1.045455 1.092976

10 83 -0.8 0.64 4.1 -3.28 77.59091 5.4090913 29.25827

5 65 -5.8 33.64 -13.9 80.62 69.40909 -4.40909 19.44008

5 68 -5.8 33.64 -10.9 63.22 69.40909 -1.40909 1.985536

mean 10.8 78.9 4.312E-06 85.59091 SSE

SSxx 215.6this might be

zero

this is the minimum value you

can get

Ssxy 352.8

b1 1.636363636

b0 61.22727273

Exercise

• The following table shows the prominent product sales in millions for the years 1998-2003. Assuming the trend continues in 2004, predict the sales in 2004

Year Year coded Sales in millions

1998 1 71.3

1999 2 59.5

2000 3 51.9

2001 4 41.1

2002 5 24.9

2003 6 17.5

Solution

• Sales = 82.7-11.0 year coded• Sales= 82.7-11.0 (7) = 5.7 million• From the line equation we can conclude that

each year we pass the sales decreases 11 million.

Inference about the slope of the regression line

• Purpose of the test is to determine whether the given value is reasonable for the slope of the population regression line (H0: β1=c).

• The test H0: β1=0 is a test to determine whether a straight line should be fit to data. If he null hypothesis is not rejected then the straight line does not model the relationship between x and y.

• Assumptions: • The regression model is y=β0+β1x+ε and β1 is

the slope of the model. • To test the null hypothesis that β1 equals

some value, say c , we divide the difference (β1 - c ) by the standard error of β1

• The following test H0: β1=0 , Ha: β1 0

• Student test t with n-2 degree of freedom, where SE(β1) is the standard error of β1

)( 1

1

SE

ct

Example

• The following table shows the systolic blood pressure readings with weights for 10 newly diagnosed patients with high blood pressure.

Patient Systolic (y) weight(x) (pound)

1 145 2102 155 2453 160 2604 155 2305 130 1756 140 1857 135 2308 165 2499 150 200

10 130 190

We would like to test that the systolic blood pressure increases one point for each pound that the patient increases

Solution

  Coefficients Standard Error t Stat P-value

Intercept 72.56893125 19.4598404 3.729163743 0.005794692

X Variable 1 0.340069313 0.088777605 3.830575445 0.005013816

The regression equation is systolic = 72.56893 +0340069 weight

The statistic is computed as follow. C=1, β1= 0.34007 and the standard error of β1= 0.08878

42.7089.0

0.134.0

t

• We calculate

• At α=0.05, the t values with 8 degree of freedom are 2.306.

• The data would refute the null hypothesis, Each additional pound would increase the systolic blood pressure by less than 1.

• The T value (3.83) shown in the table along with the two-tailed p-value (0.005) is for the null hypothesis H0: β1=0 , Ha: β1 0

42.7089.0

0.134.0

t

The coefficient of Correlation

• The coefficient correlation is used to measure the strength of the linear relationship between two random variables.

• A measure very much related to the slope of regression line is the Pearson correlation coefficient.

xyxx

xy

SSSS

SSr

• The value of r will be in the range of -1 to +1• If the point fall on the straight line with a

positive slope then r=+1, • If the point fall on the straight line with a

negative slope then r=-1, • If the point from the shotgun pattern, r=0.

Example

• The correlations for the systolic blood pressure with weight is as follow

• As you can see the linear relation is negative

  Systolic Weight

Systolic 1

Weight 0.804464 1

The coefficient of determination

• The coefficient of determination is used to measure the strength of the linear relationship between dependent variables.

• Assumptions : the analysis of variance (ANOVA) for simple linear regression may be represented as follows

Source d.f. Sum of squares

Mean of squares

F-value

Explained variation

1 SSR MSR=SSR/1 F=MSR/MSE

Unexplained variation

N-2 SSE MSE=SSE/(n-2)

Total N-1 SS(total)

• The symbol r2 is used to represent the ration SSR/SS(total) and is called the coefficient of determination. Which measures the proportion of variation in y that explained by x.

• coefficient of determination can be called explained variation, regression variation, unexplained variation, residual variation.

Example

• The following table explains the contraceptive prevalence (x) and the fertility rate (y)

Country Contraceptive (x) Fertility (y)

Thailand 69 2.3

Costa Rica 71 3.5

Turkey 62 3.4

Mexico 55 4

Zimbabwe 46 5.4

Jordan 35 5.5

Gana 14 6

Pakistan 13 5

Sudan 10 4.8

Nigeria 7 5.7

SUMMARY OUTPUT

Regression Statistics

Multiple R 0.810919794

R Square 0.657590913

Adjusted R Square 0.614789777

Standard Error 0.748909931

Observations 10

ANOVA

  df SS MS F Significance F

Regression 1 8.617071323 8.617071323 15.36386592 0.004420408

Residual 8 4.486928677 0.560866085

Total 9 13.104

  Coefficients Standard Error t Stat P-value Lower 95% Upper 95% Lower 95.0%Upper 95.0%

Intercept 6.015740866 0.440476374 13.65735195 7.95434E-07 5.000000527 7.031481204 5.000000527 7.031481

contraceptive (x) -0.0381084 0.009722332 -3.919676762 0.004420408 -0.060528138 -0.015688661 -0.060528138 -0.01569

• The coefficient of determination is shown as R Square= 65.8%

• It may computed as

• The interpretation is that about 65.8% of the variation in fertility rates is explained by the variation in contraceptive prevalence

%8.651001040.13

6167.8

)(2

totalSS

SSRr

Using the model for estimation and prediction

• The estimated regression equation y=β0+β1x+ε can be used to predict the value of y for some value of x. also the same equation can be used to estimate the mean values of ys .

• Example – Suppose you would like to know the estimate

systolic blood pressure of a patient weighted 250 pound. Simply substitute the of the weight in the equation systolic = 72.56893 +0340069*(250)=157.6

• We would expect the prediction interval to be wider than the confidence interval, that is the interval estimate of the expected value of y will be narrower that the prediction interval for the same value of x and confidence interval.

• A (1-α)100% prediction interval for an individual new value of y at x=x0 is

• Where y^=b0+b1x, the t value is based on (n-2) degree of freedom and is referred as the estimated standard error of the regression model , n is the sample size, x0 is the fixed value of x ,

2n

SSEs

2)( xxSSxx

xxSS

xx

nsty

20

2

^ )(11

• A (1-α)100% confidence interval for the mean value of y at x=x0 is

•

xxSS

xx

nsty

20

2

^ )(1

Example

• The following table shows the results of an experiment conducted on 15 diabetic patients, the independent variable x was hemoglobin A1C value, taken after 3 months of taking the fasting blood glucose value each morning of the three months period and averaging the values. The later values was the dependent value y.

• We wish to set a 95% prediction interval for average glucose reading of a diabetic who has hemoglobin A1C value of 7.0 as well as 95% confidence interval for all diabetics with hemoglobin A1C value of 7.0.

Patient x, Hemoglobiny, average fasting blood sugar over 3

month period

1 6.1 120

2 6.8 146

3 6.5 125

4 7.1 135

5 7.4 140

6 5.8 115

7 8 145

8 8.3 147

9 8 150

10 5.5 110

11 10 160

12 7.7 145

13 9 155

14 11 170

15 5.5 118

SUMMARY OUTPUT

Regression Statistics

Multiple R 0.94747

R Square 0.897699

Adjusted R Square 0.88983

Standard Error 5.8809

Observations 15

ANOVA

  df SS MS F Significance F

Regression 1 3945.328514 3945.328514 114.0763366 8.33726E-08

Residual 13 449.6048192 34.58498609

Total 14 4394.933333

 Coefficient

s Standard Error t Stat P-value Lower 95% Upper 95% Lower 95.0% Upper 95.0%

Intercept 60.55238 7.475701499 8.099892991 1.95158E-06 44.402111 76.70265334 44.402111 76.70265334

X Variable 1 10.40563 0.974250214 10.68065244 8.33726E-08 8.300888305 12.51036755 8.300888305 12.51036755

• Where y^=60.6+10.4(7.0)=133.39

• The 95% confidence interval is

• The prediction interval is

604.1425.36

)7.5133330.7(

15

1881.539.133 

2

FitSE

)85.136,93.129(425.36

)7.5133330.7(

15

1881.5*160.239.133%95

2

CI

)56.146,23.120(425.36

)7.5133330.7(

15

11881.5*160.239.133%95

2

PI

• We are 95% confident that a diabetic with a hemoglobin A1C value of 7 had a fasting blood sugar over the past 3 months that average between 120.23 and 146.56.

• We are 95% confident that diabetics with hemoglobin A1A value of 7.0 had an average fasting blood sugar over the past 3 months between 129.93 and 136.85

Exerscises

1. Give the deterministic equation for the line passing through the following pair of points:

a) (1,1.5) and (3,8.5)b) (0,1) and (2,-3)c) (0,3.1) and (1,4.8)

Reference

• This lecture prepared from• Advanced statistics demystified “MCGrawHill”

Dr. Larry J. Stephens