Analisa Regresi Week 7 The Multiple Linear Regression Model Key ideas from case study Model...
-
Upload
ashley-lang -
Category
Documents
-
view
224 -
download
0
Transcript of Analisa Regresi Week 7 The Multiple Linear Regression Model Key ideas from case study Model...
![Page 1: Analisa Regresi Week 7 The Multiple Linear Regression Model Key ideas from case study Model Description and assumption The general linear model and the.](https://reader030.fdocuments.in/reader030/viewer/2022033103/56649f1e5503460f94c368c0/html5/thumbnails/1.jpg)
Analisa Regresi Week 7
The Multiple Linear Regression ModelKey ideas from case study Model Description and assumptionThe general linear model and the
least square procedureInference for multiple regression
![Page 2: Analisa Regresi Week 7 The Multiple Linear Regression Model Key ideas from case study Model Description and assumption The general linear model and the.](https://reader030.fdocuments.in/reader030/viewer/2022033103/56649f1e5503460f94c368c0/html5/thumbnails/2.jpg)
Key ideas from case study (2)The relationship between a response variable
and an explanatory variable depends on what other explanatory variables are in the model
A variable can be a significant (P<.05) predictor alone and not significant (P>0.5) when other Xs are in the model
![Page 3: Analisa Regresi Week 7 The Multiple Linear Regression Model Key ideas from case study Model Description and assumption The general linear model and the.](https://reader030.fdocuments.in/reader030/viewer/2022033103/56649f1e5503460f94c368c0/html5/thumbnails/3.jpg)
Key ideas from case study (1)
First, look at graphical and numerical summaries for one variable at a time
Then, look at relationships between pairs of variables with graphical and numerical summaries.
Use plots and correlations
![Page 4: Analisa Regresi Week 7 The Multiple Linear Regression Model Key ideas from case study Model Description and assumption The general linear model and the.](https://reader030.fdocuments.in/reader030/viewer/2022033103/56649f1e5503460f94c368c0/html5/thumbnails/4.jpg)
Key ideas from case study (3)Regression coefficients, standard errors and
the results of significance tests depend on what other explanatory variables are in the model
![Page 5: Analisa Regresi Week 7 The Multiple Linear Regression Model Key ideas from case study Model Description and assumption The general linear model and the.](https://reader030.fdocuments.in/reader030/viewer/2022033103/56649f1e5503460f94c368c0/html5/thumbnails/5.jpg)
Key ideas from case study (4)Significance tests (P values) do not tell the
whole storySquared multiple correlations give the
proportion of variation in the response variable explained by the explanatory variables) can give a different view
We often express R2 as a percent
![Page 6: Analisa Regresi Week 7 The Multiple Linear Regression Model Key ideas from case study Model Description and assumption The general linear model and the.](https://reader030.fdocuments.in/reader030/viewer/2022033103/56649f1e5503460f94c368c0/html5/thumbnails/6.jpg)
Key ideas from case study (5)You can fully understand the theory in terms of
Y = Xβ + εTo effectively use this methodology in practice
you need to understand how the data were collected, the nature of the variables, and how they relate to each other
![Page 7: Analisa Regresi Week 7 The Multiple Linear Regression Model Key ideas from case study Model Description and assumption The general linear model and the.](https://reader030.fdocuments.in/reader030/viewer/2022033103/56649f1e5503460f94c368c0/html5/thumbnails/7.jpg)
Model Description and AssumptionsConsider an experiment in which data are generated of the type
knnnn
k
k
k
xxxy
xxxy
xxxy
xxxy
21
222122
121111
21
![Page 8: Analisa Regresi Week 7 The Multiple Linear Regression Model Key ideas from case study Model Description and assumption The general linear model and the.](https://reader030.fdocuments.in/reader030/viewer/2022033103/56649f1e5503460f94c368c0/html5/thumbnails/8.jpg)
Model Description and Assumptions (2)If the experimenter is willing to assume that in the region of the x’s defined by the data, yi is related approximately to the regressor variables, then the model formulation:(1)whereYi is the response variable for the ith caseXi1, Xi2, … , Xik are k explanatory variables for cases i = 1 to n and n ≥ k + 1ε i is a model error β0 is the interceptβ1, β2, … , βk are the regression coefficients for the explanatory variables
ikikiioi XXXY 2211
![Page 9: Analisa Regresi Week 7 The Multiple Linear Regression Model Key ideas from case study Model Description and assumption The general linear model and the.](https://reader030.fdocuments.in/reader030/viewer/2022033103/56649f1e5503460f94c368c0/html5/thumbnails/9.jpg)
Model Description and Assumptions (3)What is a Linear Model?A linear model is defined as a model that is
linear in the parameter, i.e., linear in the coefficients, the β’s in the eq (1)
For example : A model quadratic in X
A linear model with interactioniiioi XXY 2
21
iiiiioi XXXXY 21122211
![Page 10: Analisa Regresi Week 7 The Multiple Linear Regression Model Key ideas from case study Model Description and assumption The general linear model and the.](https://reader030.fdocuments.in/reader030/viewer/2022033103/56649f1e5503460f94c368c0/html5/thumbnails/10.jpg)
Model Description and Assumptions (4)What is the meaning of Regression Coefficients? βo is the Y intercept of the regression plane. If the scope of the model includes Xi1 = 0, … , Xik = 0, it gives the mean response at Xi1 = 0, … , Xik = 0.The parameter β1 indicates the change in the mean response E(Y) per unit change in X1 when other X’s held constant.The β’s are often caled partial regression coefficients because they reflect the partial effect on one independent variable when the other variables are included in the model and are held constant.
![Page 11: Analisa Regresi Week 7 The Multiple Linear Regression Model Key ideas from case study Model Description and assumption The general linear model and the.](https://reader030.fdocuments.in/reader030/viewer/2022033103/56649f1e5503460f94c368c0/html5/thumbnails/11.jpg)
Model Description and Assumptions (5)Assumptionsε i are independent normally distributed
random errors with mean 0 and variance σ2
Xji are not random and are measured with negligible error
![Page 12: Analisa Regresi Week 7 The Multiple Linear Regression Model Key ideas from case study Model Description and assumption The general linear model and the.](https://reader030.fdocuments.in/reader030/viewer/2022033103/56649f1e5503460f94c368c0/html5/thumbnails/12.jpg)
The GLM and the Least Square Procedure
In matrix term:
Where:Y is a vector of responsesβ is a vector of parametersX is a matrix of constantsε is a vector of model errors which
ε ~ N(0,σ2I)Consequently, Y ~ N(Xβ,σ2I)
111 nxpxnxpnx
XY
![Page 13: Analisa Regresi Week 7 The Multiple Linear Regression Model Key ideas from case study Model Description and assumption The general linear model and the.](https://reader030.fdocuments.in/reader030/viewer/2022033103/56649f1e5503460f94c368c0/html5/thumbnails/13.jpg)
The GLM and the Least Square Procedure (2)
Least Square procedure:
minimize e’e = (Y-Xb)’(Y-Xb)
The least square normal equations:(X’X)b = X’y
Assuming X is of full column rank, b =(X’X)-1X’y
111 nxpxnxpnx
XY
0'
XbYXbYb
![Page 14: Analisa Regresi Week 7 The Multiple Linear Regression Model Key ideas from case study Model Description and assumption The general linear model and the.](https://reader030.fdocuments.in/reader030/viewer/2022033103/56649f1e5503460f94c368c0/html5/thumbnails/14.jpg)
The GLM and the Least Square Procedure (3)
The fitted predicted values:
And the residual terms by
Or
The variance-covariance matrix of the residuals is
Which is estimated by s2{e}= MSE(I-H)
HYYX'X)X(X'Xb 1^
Y
XbYYYe ^
YHIe )(
)(}{ 22 HIe
![Page 15: Analisa Regresi Week 7 The Multiple Linear Regression Model Key ideas from case study Model Description and assumption The general linear model and the.](https://reader030.fdocuments.in/reader030/viewer/2022033103/56649f1e5503460f94c368c0/html5/thumbnails/15.jpg)
![Page 16: Analisa Regresi Week 7 The Multiple Linear Regression Model Key ideas from case study Model Description and assumption The general linear model and the.](https://reader030.fdocuments.in/reader030/viewer/2022033103/56649f1e5503460f94c368c0/html5/thumbnails/16.jpg)
![Page 17: Analisa Regresi Week 7 The Multiple Linear Regression Model Key ideas from case study Model Description and assumption The general linear model and the.](https://reader030.fdocuments.in/reader030/viewer/2022033103/56649f1e5503460f94c368c0/html5/thumbnails/17.jpg)
ANOVA TableTo organize arithmeticSources of variation are Model (SAS) or Regression (NKNW)Error (SAS, NKNW) or ResidualTotal
SS and df addSSM + SSE =SSTdfM + dfE = dfT
![Page 18: Analisa Regresi Week 7 The Multiple Linear Regression Model Key ideas from case study Model Description and assumption The general linear model and the.](https://reader030.fdocuments.in/reader030/viewer/2022033103/56649f1e5503460f94c368c0/html5/thumbnails/18.jpg)
![Page 19: Analisa Regresi Week 7 The Multiple Linear Regression Model Key ideas from case study Model Description and assumption The general linear model and the.](https://reader030.fdocuments.in/reader030/viewer/2022033103/56649f1e5503460f94c368c0/html5/thumbnails/19.jpg)
SS (2)
The sum squares for ANOVA in matrix terms are :
Yn
1''
n
1 - Y'
JIYJYYYSST
Y' Xb)-(YXb)'-(Y e' HIYeSSE
Yn
1''
n
1 - ''
JHYJYYYXbSSM
![Page 20: Analisa Regresi Week 7 The Multiple Linear Regression Model Key ideas from case study Model Description and assumption The general linear model and the.](https://reader030.fdocuments.in/reader030/viewer/2022033103/56649f1e5503460f94c368c0/html5/thumbnails/20.jpg)
![Page 21: Analisa Regresi Week 7 The Multiple Linear Regression Model Key ideas from case study Model Description and assumption The general linear model and the.](https://reader030.fdocuments.in/reader030/viewer/2022033103/56649f1e5503460f94c368c0/html5/thumbnails/21.jpg)
![Page 22: Analisa Regresi Week 7 The Multiple Linear Regression Model Key ideas from case study Model Description and assumption The general linear model and the.](https://reader030.fdocuments.in/reader030/viewer/2022033103/56649f1e5503460f94c368c0/html5/thumbnails/22.jpg)
![Page 23: Analisa Regresi Week 7 The Multiple Linear Regression Model Key ideas from case study Model Description and assumption The general linear model and the.](https://reader030.fdocuments.in/reader030/viewer/2022033103/56649f1e5503460f94c368c0/html5/thumbnails/23.jpg)
Source SS df MS Model SSM dfM MSMError SSE dfE MSE Total SST dfT (MST)
F = MSM/MSE
![Page 24: Analisa Regresi Week 7 The Multiple Linear Regression Model Key ideas from case study Model Description and assumption The general linear model and the.](https://reader030.fdocuments.in/reader030/viewer/2022033103/56649f1e5503460f94c368c0/html5/thumbnails/24.jpg)
ANOVA F testH0: β1 = β2 = … βp-1 = 0
Ha: βk neq 0, for at least one k=1, … , p-1Another form of the null hypothesis is H0: β1 = 0, and β2 = 0, … , and βp-1 = 0
Under H0, F ~ F(p-1,n-p)
Reject H0 if F is large, use P value
![Page 25: Analisa Regresi Week 7 The Multiple Linear Regression Model Key ideas from case study Model Description and assumption The general linear model and the.](https://reader030.fdocuments.in/reader030/viewer/2022033103/56649f1e5503460f94c368c0/html5/thumbnails/25.jpg)
Example NKNW p 249The Zartan company sells a special skin
cream through fashion stores exclusively in 15 districts
Y is salesX1 is target population
X2 is per capita discretionary incomen = 15 districts
![Page 26: Analisa Regresi Week 7 The Multiple Linear Regression Model Key ideas from case study Model Description and assumption The general linear model and the.](https://reader030.fdocuments.in/reader030/viewer/2022033103/56649f1e5503460f94c368c0/html5/thumbnails/26.jpg)
Check the data
obs yi x1i x2i obs yi x1i x2i
1 162 274 2450 9 116 195 2137
2 120 180 3254 10 55 53 2560
3 223 375 3802 11 252 430 4020
4 131 205 2838 12 232 372 4427
5 67 86 2347 13 144 236 2660
6 169 265 3782 14 103 157 2088
7 81 98 3008 15 212 370 2605
8 192 330 2450
![Page 27: Analisa Regresi Week 7 The Multiple Linear Regression Model Key ideas from case study Model Description and assumption The general linear model and the.](https://reader030.fdocuments.in/reader030/viewer/2022033103/56649f1e5503460f94c368c0/html5/thumbnails/27.jpg)
•H0: β1 = β2 = … βp-1 = 0 •F = MSM/MSE•Reject H0 if the P value is leq .05
![Page 28: Analisa Regresi Week 7 The Multiple Linear Regression Model Key ideas from case study Model Description and assumption The general linear model and the.](https://reader030.fdocuments.in/reader030/viewer/2022033103/56649f1e5503460f94c368c0/html5/thumbnails/28.jpg)
ANOVAb
53844,716 2 26922,358 5679,466 ,000a
56,884 12 4,740
53901,600 14
Regression
Residual
Total
Model1
Sum ofSquares df Mean Square F Sig.
Predictors: (Constant), INCOME, POPa.
Dependent Variable: SALESb.
What do we conclude?
![Page 29: Analisa Regresi Week 7 The Multiple Linear Regression Model Key ideas from case study Model Description and assumption The general linear model and the.](https://reader030.fdocuments.in/reader030/viewer/2022033103/56649f1e5503460f94c368c0/html5/thumbnails/29.jpg)
R2
The squared multiple regression correlation (R2) gives the proportion of variation in the response variable explained by the explanatory variables included in the model
It is usually expressed as a percentIt is sometimes called the coefficient of multiple
determination (NKNW p 230)
![Page 30: Analisa Regresi Week 7 The Multiple Linear Regression Model Key ideas from case study Model Description and assumption The general linear model and the.](https://reader030.fdocuments.in/reader030/viewer/2022033103/56649f1e5503460f94c368c0/html5/thumbnails/30.jpg)
R2 (2)
R2 = SSM/SST, the proportion of variation explained
R2 = 1 – (SSE/SST), 1 – the proportion of variation not explained
H0: β1 = β2 = … βp-1 = 0 is equivalent to H0: the population R2 is zero
F = [ (R2)/(p-1) ] / [ (1- R2)/(n-p) ]
![Page 31: Analisa Regresi Week 7 The Multiple Linear Regression Model Key ideas from case study Model Description and assumption The general linear model and the.](https://reader030.fdocuments.in/reader030/viewer/2022033103/56649f1e5503460f94c368c0/html5/thumbnails/31.jpg)
What and WhyAt this point we have examined the
distribution of the explanatory variables (and the response variable if that is appropriate) and we have taken remedial measures where needed
We have looked at plots and numerical summaries
![Page 32: Analisa Regresi Week 7 The Multiple Linear Regression Model Key ideas from case study Model Description and assumption The general linear model and the.](https://reader030.fdocuments.in/reader030/viewer/2022033103/56649f1e5503460f94c368c0/html5/thumbnails/32.jpg)
What and Why (2)The P-value for the F significance test tells us
one of the following:there is no evidence to conclude that any of
our explanatory variables can help us to model the response variable using this kind of model (P gt .05)
one or more of the explanatory variables in our model is potentially useful for predicting the response variable in a linear model (P leq .05)
![Page 33: Analisa Regresi Week 7 The Multiple Linear Regression Model Key ideas from case study Model Description and assumption The general linear model and the.](https://reader030.fdocuments.in/reader030/viewer/2022033103/56649f1e5503460f94c368c0/html5/thumbnails/33.jpg)
R-Sq = 0.999Adj R-Sq = 0.999
Coeff Var = 6.0
![Page 34: Analisa Regresi Week 7 The Multiple Linear Regression Model Key ideas from case study Model Description and assumption The general linear model and the.](https://reader030.fdocuments.in/reader030/viewer/2022033103/56649f1e5503460f94c368c0/html5/thumbnails/34.jpg)
Inference for individual regression coefficientsb ~ N(β, (s(bi, bk)))
s(bi, bi) = s2(bi)
CI: bi ± t*s(bi)
Significance test for H0i: βi, = 0 uses the test statistic t =bi/s(bi), df=dfE=n-p, and the P-value computed from the t(n-p) distribution
![Page 35: Analisa Regresi Week 7 The Multiple Linear Regression Model Key ideas from case study Model Description and assumption The general linear model and the.](https://reader030.fdocuments.in/reader030/viewer/2022033103/56649f1e5503460f94c368c0/html5/thumbnails/35.jpg)
Par StVar Est Err t PInt 3.453 2.431 1.420 0.181Pop 0.496 0.006 81.924 <.0001income 0.009 0.001 9.502 <.0001
![Page 36: Analisa Regresi Week 7 The Multiple Linear Regression Model Key ideas from case study Model Description and assumption The general linear model and the.](https://reader030.fdocuments.in/reader030/viewer/2022033103/56649f1e5503460f94c368c0/html5/thumbnails/36.jpg)
Estimation by Doolittle General Format:
X’X | X’Y | I
By using row transformation we’ll have
I | b | (X’X)-1
Example : see worksheet 1
![Page 37: Analisa Regresi Week 7 The Multiple Linear Regression Model Key ideas from case study Model Description and assumption The general linear model and the.](https://reader030.fdocuments.in/reader030/viewer/2022033103/56649f1e5503460f94c368c0/html5/thumbnails/37.jpg)
Estimation of E(Yh)
Xh is now a vector
(1, Xh1, Xh2, … , Xh1)’We want an point estimate and a
confidence interval for the subpopulation mean corresponding to Xh
![Page 38: Analisa Regresi Week 7 The Multiple Linear Regression Model Key ideas from case study Model Description and assumption The general linear model and the.](https://reader030.fdocuments.in/reader030/viewer/2022033103/56649f1e5503460f94c368c0/html5/thumbnails/38.jpg)
The mean response to be estimated is
The estimated mean response corresponding to Xh, is
This estimated is unbiased and its variance is
bX'h^
hY
hX')( hYE
)E(Y X')( hh
^
hYE
h1-
h2
^2 XX)(X'X')( hY
![Page 39: Analisa Regresi Week 7 The Multiple Linear Regression Model Key ideas from case study Model Description and assumption The general linear model and the.](https://reader030.fdocuments.in/reader030/viewer/2022033103/56649f1e5503460f94c368c0/html5/thumbnails/39.jpg)
The estimated variance s2(Ŷh) is given by
The 1 – α CI for E(Yh) are
Example (in the class)
h2
hh1-
h
^2 X)(X')XX)(X'X'()( bsMSEYs h
)();2/1(^^
hh YspntY
![Page 40: Analisa Regresi Week 7 The Multiple Linear Regression Model Key ideas from case study Model Description and assumption The general linear model and the.](https://reader030.fdocuments.in/reader030/viewer/2022033103/56649f1e5503460f94c368c0/html5/thumbnails/40.jpg)
It is used when a data set requires repeat observation
Repeat observation in multiple regression are replicate observation on Y coresponding to level of each of the X variables that are constant from trial to trial.
Whit two independent variables repeat observations require that X1 and X2 each remain at given levels from trial to trial.
![Page 41: Analisa Regresi Week 7 The Multiple Linear Regression Model Key ideas from case study Model Description and assumption The general linear model and the.](https://reader030.fdocuments.in/reader030/viewer/2022033103/56649f1e5503460f94c368c0/html5/thumbnails/41.jpg)
SSE is decomposed into pure error and lack of fit component
SSE = SSPE + SSLF
The pure error sum of square SSPE is obtained by first calculating for each replicate group the sum of squared deviations of the Y observation around the group mean, where a replicate group has the same values for each of the X variables.
2^
22^
)()()(
ijjjijijij YYYYYY
![Page 42: Analisa Regresi Week 7 The Multiple Linear Regression Model Key ideas from case study Model Description and assumption The general linear model and the.](https://reader030.fdocuments.in/reader030/viewer/2022033103/56649f1e5503460f94c368c0/html5/thumbnails/42.jpg)
If the linear regression function is appropriate, then the means Ȳj will be near the fitted values Ŷij calculated from the esyimated linear regression function and SSLF will be small. See the illistration on NWK page 138.
Df(SSPE) = n – c and df(SSLF) = (n-p)-(n-c)=c-p where c is is the number of replicate group
![Page 43: Analisa Regresi Week 7 The Multiple Linear Regression Model Key ideas from case study Model Description and assumption The general linear model and the.](https://reader030.fdocuments.in/reader030/viewer/2022033103/56649f1e5503460f94c368c0/html5/thumbnails/43.jpg)
The hypotheses statements are:
The approriate test statistic is
And the appropriate decision rule is conclude Ho if
122111
12211
}{:
}{:
pko
pko
XXXYEH
XXXYEHo
MSPE
MSLF
cn
SSPE
pc
SSLFFo
*
),;1(* cnpcFFo
![Page 44: Analisa Regresi Week 7 The Multiple Linear Regression Model Key ideas from case study Model Description and assumption The general linear model and the.](https://reader030.fdocuments.in/reader030/viewer/2022033103/56649f1e5503460f94c368c0/html5/thumbnails/44.jpg)
Prediction of new observation Yh(new)
Xh is now a vector
(1, Xh1, Xh2, … , Xh1)’
We want a prediction for Yh with an interval that expresses the uncertainty in our prediction
![Page 45: Analisa Regresi Week 7 The Multiple Linear Regression Model Key ideas from case study Model Description and assumption The general linear model and the.](https://reader030.fdocuments.in/reader030/viewer/2022033103/56649f1e5503460f94c368c0/html5/thumbnails/45.jpg)
The prediction of new observation Yh(new) corresponding to Xh has 1 – α CI of
Where
Example (in the class)
)XX)(X'X'1()( h1-
h)(2 MSEYs newh
)();2/1( )(
^
newhh YspntY
![Page 46: Analisa Regresi Week 7 The Multiple Linear Regression Model Key ideas from case study Model Description and assumption The general linear model and the.](https://reader030.fdocuments.in/reader030/viewer/2022033103/56649f1e5503460f94c368c0/html5/thumbnails/46.jpg)
When m new observation at Xh and their
mean Ȳh(new) is to be predicted, the 1- corresponding to has 1 – α CI of
Where
Example (in the class)
)XX)(X'X'1
()( h1-
h)(
_2
mMSEYs newh
)();2/1( )(
^
newhh YspntY
![Page 47: Analisa Regresi Week 7 The Multiple Linear Regression Model Key ideas from case study Model Description and assumption The general linear model and the.](https://reader030.fdocuments.in/reader030/viewer/2022033103/56649f1e5503460f94c368c0/html5/thumbnails/47.jpg)
Last slideReading NKMW 7.1 to 7.8Exercise NKMW page 264 no 7.8-7.11Homework NKMW page 264-267 n0 7.12-
7.19