Multiple RegressionMultiple Regression

Dr. Rohit Vishal KumarDr. Rohit Vishal KumarReader, Department of MarketingReader, Department of MarketingXavier Institute of Social ServiceXavier Institute of Social Service

PO Box No 7, Purulia RoadPO Box No 7, Purulia RoadRanchi – 834001, Jharkhand, IndiaRanchi – 834001, Jharkhand, India

Email: Email: rohitvishalkumar@gmail.comrohitvishalkumar@gmail.com

Types of Regression ModelsTypes of Regression Models

RegressionModels

LinearNon-

Linear

2+ ExplanatoryVariables

Simple

Non-Linear

Multiple

Linear

1 ExplanatoryVariable

RegressionModels

LinearNon-

Linear

2+ ExplanatoryVariables

Simple

Non-Linear

Multiple

Linear

1 ExplanatoryVariable

Regression Modeling Steps Regression Modeling Steps

1. Specify the model and estimate all unknown parameters

2. Evaluate Model

3. Use Model for Prediction & Estimation

• Decide on the dependent variableDecide on the dependent variable

• List all potential Independent List all potential Independent variablesvariables

Model SpecificationModel Specification

Linear Multiple Regression ModelLinear Multiple Regression Model1.Relationship between 1 dependent

& 2 or more independent variables is a linear function

Y X X Xi i i k ki i 0 1 1 2 2 Y X X Xi i i k ki i 0 1 1 2 2

Dependent Dependent (response) (response) variablevariable

Independent Independent (explanatory) (explanatory) variablesvariables

Population Population slopesslopes

Population Population Y-interceptY-intercept

Random Random errorerror

Linear Regression AssumptionsLinear Regression Assumptions• Mean of Distribution of Error Is 0

• Distribution of Error Has Constant Variance

• Distribution of Error is Normal

• Errors Are Independent

Extremely

ImportantExtremely

Important

Parameter EstimationParameter Estimation

• Step 1:– Gather Data for all the Independent

and Dependent Variables

• Step 2:– Estimate the Parameters using the

Least Square Method

Estimating the ParameterEstimating the Parameter

• Do it manually:– Requires knowledge of Matrix

Manipulation of Huge Sizes– B = (X’X)-1X’Y

• Use a Software– MS Excel Can handle 15 independent

Variables– No Limit on Statistical Software

Interpretation of Estimated CoefficientsInterpretation of Estimated Coefficients

1. Slope (k)

– Estimated average change in Y by k for 1 Unit Increase in Xk Holding All Other Variables Constant

– Example:•If 1 = 0.13, then Y is expected to

Increase by 0.13 for Each 1 unit increase in X1 Given X2 X3 X4… Xn are held constant

^̂

Interpretation of Estimated CoefficientsInterpretation of Estimated Coefficients

• 2. Constant (B0)– The value of Y when all other Variables

are = 0

– Also Know As the “Autonomous Value” of Y

Evaluating Multiple Regression ModelsEvaluating Multiple Regression Models

• Examine Variation Measures

• Test Significance of Overall Model, portions of overall model and Individual Coefficients

• Other Things that needs to be Checked:– Check conditions of a multiple linear regression model

using Residuals– Assess Multi-co linearity among independent variables

Variation Measures 1Variation Measures 1• Coefficient of Multiple Determination

• Proportion of Variation in Y ‘Explained’ by All X Variables Taken Together

yyyy

yy

SSSSE

SS

SSESSR

1

variationTotal variationExplained2

• Adjusted R2

• R2 Never Decreases When New X Variable Is Added to Model (Disadvantage When Comparing Models)

• Solution: Adjusted R2

– Each additional variable reduces adjusted R2, unless SSE goes up enough to compensate

22 1

11

1 RSSyySSE

SSSSE

knn

Ryy

a

Variation Measures 2Variation Measures 2

Testing Overall SignificanceTesting Overall Significance1. Tests if there is a Linear Relationship Between All X

Variables Together & Y

2. Hypotheses– H0: 1 = 2 = ... = k = 0

• No Linear Relationship

– Ha: At Least One Coefficient Is Not 0 • At Least One X Variable linearly Affects Y

3. Uses F test statistic

0

2

2

, 1

( ) //( ) /

/( 1) / 1 1 / 1

~

yy

H

k n k

SS SSE kSSR k R kF

SSE n k SSE n k R n k

F