Regression Analysis

Regression Analysis

1. To comprehend the nature of correlation analysis.

2. To understand bivariate regression analysis.

3. To become aware of the coefficient of determination

Bivariate Analysis DefinedThe degree of association between two variablesBivariate techniques

Statistical methods of analyzing the relationship between two variables.

Multivariate TechniquesWhen more than two variables are involved

Independent variableAffects the value of the dependent variable

Dependent variableexplained or caused by the independent variable

Bivariate Analysis of Association

Types of Bivariate Procedures

• Two group t-tests

• chi-square analysis of cross-tabulation or contingency tables

• ANOVA (analysis of variance) for two groups

• Bivariate regression

• Pearson product moment correlation

Bivariate Analysis of Association

Bivariate Regression DefinedAnalyzing the strength of the linear relationship between the dependent variable and the independent variable.

Nature of the Relationship

• Plot in a scatter diagram

• Dependent variable

Y is plotted on the vertical axis

• Independent variable

X is plotted on the horizontal axis

• Nonlinear relationship

Bivariate Regression

Y

XA - Strong Positive Linear Relationship

Types of Relationships Found in Scatter DiagramsBivariate Regression Example

Bivariate Regression

Y

XB - Positive Linear Relationship

Types of Relationships Found in Scatter Diagrams

Bivariate Regression

Y

XC - Perfect Negative Linear Relationship

Types of Relationships Found in Scatter Diagrams

Bivariate Regression

XD - Perfect Parabolic Relationship

Types of Relationships Found in Scatter Diagrams

Bivariate Regression

Y

XE - Negative Curvilinear Relationship

Types of Relationships Found in Scatter Diagrams

Bivariate Regression

Y

XF - No Relationship between X and Y

Types of Relationships Found in Scatter Diagrams

Bivariate Regression

Least Squares Estimation ProcedureResults in a straight line that fits the actual observations better than any other line that could be fitted to the observations.

where

Y = dependent variable

X = independent variable

e = error

b = estimated slope of the regression line

a = estimated Y intercept

Bivariate Regression

Y = a + bX + e

Values for a and b can be calculated as follows:

XiYi - nXYb =

X2i - n(X)2

n = sample size

a = Y - bX

X = mean of value X

Y = mean of value y

Bivariate Regression

y= β0 + β1 + Є

β1 = Sxy /Sxx

β0 = y - β1x

Bivariate Regression

Strength of Association: R2

Coefficient of Determination, R2: The measure of the strength of the linear relationship between X and Y.

The Regression LinePredicted values for Y, based on calculated values.

Bivariate Regression

R2 =explained variance

total variance

explained variance =total variance - unexplained variance

R2 =total variance - unexplained variance

total variance

= 1 -unexplained variance

total variance

Bivariate Regression

R2 = 1 -unexplained variance

total variance

= 1 - (Yi - Yi)2n

I = 1

(Yi - Y)2n

I = 1

Bivariate Regression

Predicted response

Statistical Significance of Regression Results

Total variation =

Explained variation + Unexplained variation

To become aware of the coefficient of determination, R2.

The total variation is a measure of variation of the observed Y values around their mean.

It measures the variation of the Y values without any consideration of the X values.

Bivariate Regression

Total variation: Sum of squares (SST)

SST = (Yi - Y)2n

i = 1

Yi 2n

i = 1=

Yi 2n

i = 1

n

Bivariate Regression

Sum of squares due to regression (SSR)

SSR = (Yi - Y)2n

i = 1

Yi

n

i = 1= a

Yi

n

i = 1

nb Xi Yi

n

i = 1+

2

Bivariate Regression

Error sums of squares (SSE)

SSE = (Yi - Y)2n

i = 1

Y2i

n

i = 1= a Yi

n

i = 1 b XiYi

n

i = 1

Bivariate Regression

Hypotheses Concerning the Overall Regression

Null Hypothesis Ho:There is no linear relationship between X and Y.

Alternative Hypothesis Ha:There is a linear relationship between X and Y.

Bivariate Regression

Hypotheses about the Regression Coefficient

Null Hypothesis Ho:b = 0

Alternative Hypothesis Ha:b 0

The appropriate test is the t-test.

Bivariate Regression

0 XXiX

(X, Y)

a

Y

Total Variation

Explained variation

Y

Unexplained variation

Measures of Variation in a Regression

Yi =a + bXi

Correlation for Metric Data - Pearson’s Product Moment Correlation

Correlation analysisAnalysis of the degree to which changes in one variable are associated with changes in another variable.

Pearson’s product moment correlationCorrelation analysis technique for use with metric data

Correlation Analysis

To become aware of the coefficient of determination, R2.

R = +- R2√

R can be computed directly from the data:

R = n XY - ( X) - ( Y)

[n X2 - ( X) 2] [n Y2 - Y)2]√

To become aware of the coefficient of determination, R2.

Correlation Analysis

SUMMARY

• Bivariate Analysis of Association

• Bivariate Regression

• Correlation Analysis