Multiple Regression and Model Building

Purposes of multiple regressionApplicationsModel and OLS criterionInferencesModel building - variable selectionData considerationsLINE assumptions

Purposes of multiple regression

Prediction - y hat To predict values of Y, the response

variable, for given levels of X, the vector of predictor variables

Estimation - beta hat To estimate the effect of individual

predictor variables on the response variable Y

Applications

Relating portfolio return to market return Negotiating professional sports salariesExamining implications of a nation’s

education policy for infant mortalityAssessing the effect of training on

employee performancePricing models for residential real estateCompensation models for Title VII

compliance

Model and OLS criterion

y = 0 + 1x1 +…+ kxk + page 528j is the expected change in y associated with a

unit change in xj, all other variables remaining unchanged

Minimize (Yi - Ŷihat)2 =Yi - XiB) 2

where X is the matrix of x values with an initial column of ones

and B is the vector of OLS estimates of the beta vector

B = [X’X]-1X’Y

Inferences

Hypothesis of model usefulness (utility) page 535

Tests of hypotheses H0: j = 0 page 539F drop tests of hypotheses page 554

H0: g = g+1 = …= k = 0Interval estimates of coefficientsInterval estimates of conditional meansPrediction intervals for individual values

Partitioning sum of squared error

(yi-ybar)2 = (yihat-ybar)2+ (yi-yihat)2

Complete model TSS = SS(Model)c+ SSEc

Reduced model TSS = SS(Model)r+ SSEr SSEc < SSEr

Testing nested models: the F drop test

Is SSEc significantly less than SSEr ? H0:g = g+1 = … = 0

HA: k > 0 or <0 for some k Test Statistic: [(SSEr - SSEc)/# of variables dropped]/[SSEc/error df in

C]

= F #of variables dropped, error df in complete model

Model Building - Blocks

Observed variablesHigher order terms

Powers (e.g., squared or cubed variables)

Interactions (products of variables)Qualitative variables introduced with

indicator or dummy (0, 1) variablesTransformations (eg. Ln[y])

Quantitative variables with higher order terms

Interactions X1* X2 Age * Height Height * Yrspro

Powers (X1)^2 Age Squared - i.e. salary increases with

age up to a point beyond which it declines

Models Indicator (dummy) Variables

Parallel lines - one quantitative X, one dummy variable D

Nonparallel lines - X, D, and X*DEquidistant parabolas - X. X2, and DNon-equidistant parabolas - X, X2, D,

and X*D

Multiplicative models and transformations

Y = E(Y) *Ln Y = 0 + 1x1 +…+ kxk +

where = ln