1 Chapter 3 Multiple Linear Regression. 2 3.1 Multiple Regression Models Suppose that the yield in...
-
Upload
sophia-ellis -
Category
Documents
-
view
214 -
download
0
Transcript of 1 Chapter 3 Multiple Linear Regression. 2 3.1 Multiple Regression Models Suppose that the yield in...
![Page 1: 1 Chapter 3 Multiple Linear Regression. 2 3.1 Multiple Regression Models Suppose that the yield in pounds of conversion in a chemical process depends.](https://reader036.fdocuments.in/reader036/viewer/2022062517/56649ec15503460f94bcdbbe/html5/thumbnails/1.jpg)
1
Chapter 3Multiple Linear Regression
![Page 2: 1 Chapter 3 Multiple Linear Regression. 2 3.1 Multiple Regression Models Suppose that the yield in pounds of conversion in a chemical process depends.](https://reader036.fdocuments.in/reader036/viewer/2022062517/56649ec15503460f94bcdbbe/html5/thumbnails/2.jpg)
2
3.1 Multiple Regression Models
• Suppose that the yield in pounds of conversion in a chemical process depends on temperature and the catalyst concentration. A multiple regression model that might describe this relationship is
• This is a multiple linear regression model in two variables.
![Page 3: 1 Chapter 3 Multiple Linear Regression. 2 3.1 Multiple Regression Models Suppose that the yield in pounds of conversion in a chemical process depends.](https://reader036.fdocuments.in/reader036/viewer/2022062517/56649ec15503460f94bcdbbe/html5/thumbnails/3.jpg)
3
3.1 Multiple Regression Models
Figure 3.1 (a) The regression plane for the model E(y)= 50+10x1+7x2 . (b) The contour plot.
![Page 4: 1 Chapter 3 Multiple Linear Regression. 2 3.1 Multiple Regression Models Suppose that the yield in pounds of conversion in a chemical process depends.](https://reader036.fdocuments.in/reader036/viewer/2022062517/56649ec15503460f94bcdbbe/html5/thumbnails/4.jpg)
4
3.1 Multiple Regression Models
In general, the multiple linear regression model with k regressors is
![Page 5: 1 Chapter 3 Multiple Linear Regression. 2 3.1 Multiple Regression Models Suppose that the yield in pounds of conversion in a chemical process depends.](https://reader036.fdocuments.in/reader036/viewer/2022062517/56649ec15503460f94bcdbbe/html5/thumbnails/5.jpg)
5
3.1 Multiple Regression Models
![Page 6: 1 Chapter 3 Multiple Linear Regression. 2 3.1 Multiple Regression Models Suppose that the yield in pounds of conversion in a chemical process depends.](https://reader036.fdocuments.in/reader036/viewer/2022062517/56649ec15503460f94bcdbbe/html5/thumbnails/6.jpg)
6
3.1 Multiple Regression Models
Linear regression models may also contain interaction effects:
If we let x3 = x1x2 and 3 = 12, then the model can be written in the form
![Page 7: 1 Chapter 3 Multiple Linear Regression. 2 3.1 Multiple Regression Models Suppose that the yield in pounds of conversion in a chemical process depends.](https://reader036.fdocuments.in/reader036/viewer/2022062517/56649ec15503460f94bcdbbe/html5/thumbnails/7.jpg)
7
3.1 Multiple Regression Models
![Page 8: 1 Chapter 3 Multiple Linear Regression. 2 3.1 Multiple Regression Models Suppose that the yield in pounds of conversion in a chemical process depends.](https://reader036.fdocuments.in/reader036/viewer/2022062517/56649ec15503460f94bcdbbe/html5/thumbnails/8.jpg)
8
![Page 9: 1 Chapter 3 Multiple Linear Regression. 2 3.1 Multiple Regression Models Suppose that the yield in pounds of conversion in a chemical process depends.](https://reader036.fdocuments.in/reader036/viewer/2022062517/56649ec15503460f94bcdbbe/html5/thumbnails/9.jpg)
9
3.2 Estimation of the Model Parameters
3.2.1 Least Squares Estimation of the Regression Coefficients
Notation
n – number of observations available
k – number of regressor variables, p-- k+1( number of regression coefficients)
y – response or dependent variable
xij – ith observation on jth regressor j.
![Page 10: 1 Chapter 3 Multiple Linear Regression. 2 3.1 Multiple Regression Models Suppose that the yield in pounds of conversion in a chemical process depends.](https://reader036.fdocuments.in/reader036/viewer/2022062517/56649ec15503460f94bcdbbe/html5/thumbnails/10.jpg)
10
3.2.1 Least Squares Estimation of Regression Coefficients
![Page 11: 1 Chapter 3 Multiple Linear Regression. 2 3.1 Multiple Regression Models Suppose that the yield in pounds of conversion in a chemical process depends.](https://reader036.fdocuments.in/reader036/viewer/2022062517/56649ec15503460f94bcdbbe/html5/thumbnails/11.jpg)
11
3.2.1 Least Squares Estimation of the Regression Coefficients
The sample regression model can be written as
![Page 12: 1 Chapter 3 Multiple Linear Regression. 2 3.1 Multiple Regression Models Suppose that the yield in pounds of conversion in a chemical process depends.](https://reader036.fdocuments.in/reader036/viewer/2022062517/56649ec15503460f94bcdbbe/html5/thumbnails/12.jpg)
12
3.2.1 Least Squares Estimation of the Regression Coefficients
The least squares function is
The function S must be minimized with respect to the coefficients.
![Page 13: 1 Chapter 3 Multiple Linear Regression. 2 3.1 Multiple Regression Models Suppose that the yield in pounds of conversion in a chemical process depends.](https://reader036.fdocuments.in/reader036/viewer/2022062517/56649ec15503460f94bcdbbe/html5/thumbnails/13.jpg)
13
3.2.1 Least Squares Estimation of the Regression Coefficients
The least squares estimates of the coefficients must satisfy
![Page 14: 1 Chapter 3 Multiple Linear Regression. 2 3.1 Multiple Regression Models Suppose that the yield in pounds of conversion in a chemical process depends.](https://reader036.fdocuments.in/reader036/viewer/2022062517/56649ec15503460f94bcdbbe/html5/thumbnails/14.jpg)
14
3.2.1 Least Squares Estimation of the Regression Coefficients
Simplifying, we obtain the least squares normal equations:
The ordinary least squares estimators are the solutions to the normal equations.
![Page 15: 1 Chapter 3 Multiple Linear Regression. 2 3.1 Multiple Regression Models Suppose that the yield in pounds of conversion in a chemical process depends.](https://reader036.fdocuments.in/reader036/viewer/2022062517/56649ec15503460f94bcdbbe/html5/thumbnails/15.jpg)
15
3.2.1 Least Squares Estimation of the Regression Coefficients
Matrix notation is more convenient to find the estimiates
Let
where
![Page 16: 1 Chapter 3 Multiple Linear Regression. 2 3.1 Multiple Regression Models Suppose that the yield in pounds of conversion in a chemical process depends.](https://reader036.fdocuments.in/reader036/viewer/2022062517/56649ec15503460f94bcdbbe/html5/thumbnails/16.jpg)
16
3.2.1 Least Squares Estimation of the Regression Coefficients
![Page 17: 1 Chapter 3 Multiple Linear Regression. 2 3.1 Multiple Regression Models Suppose that the yield in pounds of conversion in a chemical process depends.](https://reader036.fdocuments.in/reader036/viewer/2022062517/56649ec15503460f94bcdbbe/html5/thumbnails/17.jpg)
17
3.2.1 Least Squares Estimation of the Regression Coefficients
These are the least-squares normal equations. The solution is
![Page 18: 1 Chapter 3 Multiple Linear Regression. 2 3.1 Multiple Regression Models Suppose that the yield in pounds of conversion in a chemical process depends.](https://reader036.fdocuments.in/reader036/viewer/2022062517/56649ec15503460f94bcdbbe/html5/thumbnails/18.jpg)
Linear Regression Analysis 5E Montgomery, Peck & Vining
18
3.2.1 Least Squares Estimation of the Regression Coefficients
![Page 19: 1 Chapter 3 Multiple Linear Regression. 2 3.1 Multiple Regression Models Suppose that the yield in pounds of conversion in a chemical process depends.](https://reader036.fdocuments.in/reader036/viewer/2022062517/56649ec15503460f94bcdbbe/html5/thumbnails/19.jpg)
19
3.2.1 Least Squares Estimation of the Regression Coefficients
The n residuals can be written in matrix form as
There will be some situations where an alternative form will prove useful
Where H is called hat matrix
![Page 20: 1 Chapter 3 Multiple Linear Regression. 2 3.1 Multiple Regression Models Suppose that the yield in pounds of conversion in a chemical process depends.](https://reader036.fdocuments.in/reader036/viewer/2022062517/56649ec15503460f94bcdbbe/html5/thumbnails/20.jpg)
20
Example 3-1. The Delivery Time Data
The model of interest is
y = 0 + 1x1+ 2x2 +
![Page 21: 1 Chapter 3 Multiple Linear Regression. 2 3.1 Multiple Regression Models Suppose that the yield in pounds of conversion in a chemical process depends.](https://reader036.fdocuments.in/reader036/viewer/2022062517/56649ec15503460f94bcdbbe/html5/thumbnails/21.jpg)
21
Example 3-1. The Delivery Time Data
Figure 3.4 Scatterplot matrix for the delivery time data from Example 3.1.
R codes for the figure in “Chapter_3_nulti_reg.txt”
![Page 22: 1 Chapter 3 Multiple Linear Regression. 2 3.1 Multiple Regression Models Suppose that the yield in pounds of conversion in a chemical process depends.](https://reader036.fdocuments.in/reader036/viewer/2022062517/56649ec15503460f94bcdbbe/html5/thumbnails/22.jpg)
22
Example 3-1 The Delivery Time Data
Figure 3.5 Three-dimensional scatterplot of the delivery time data from Example 3.1.
![Page 23: 1 Chapter 3 Multiple Linear Regression. 2 3.1 Multiple Regression Models Suppose that the yield in pounds of conversion in a chemical process depends.](https://reader036.fdocuments.in/reader036/viewer/2022062517/56649ec15503460f94bcdbbe/html5/thumbnails/23.jpg)
23
Example 3-1 The Delivery Time Data
![Page 24: 1 Chapter 3 Multiple Linear Regression. 2 3.1 Multiple Regression Models Suppose that the yield in pounds of conversion in a chemical process depends.](https://reader036.fdocuments.in/reader036/viewer/2022062517/56649ec15503460f94bcdbbe/html5/thumbnails/24.jpg)
24
Example 3-1 The Delivery Time Data
![Page 25: 1 Chapter 3 Multiple Linear Regression. 2 3.1 Multiple Regression Models Suppose that the yield in pounds of conversion in a chemical process depends.](https://reader036.fdocuments.in/reader036/viewer/2022062517/56649ec15503460f94bcdbbe/html5/thumbnails/25.jpg)
25
Example 3-1 The Delivery Time Data
![Page 26: 1 Chapter 3 Multiple Linear Regression. 2 3.1 Multiple Regression Models Suppose that the yield in pounds of conversion in a chemical process depends.](https://reader036.fdocuments.in/reader036/viewer/2022062517/56649ec15503460f94bcdbbe/html5/thumbnails/26.jpg)
26
![Page 27: 1 Chapter 3 Multiple Linear Regression. 2 3.1 Multiple Regression Models Suppose that the yield in pounds of conversion in a chemical process depends.](https://reader036.fdocuments.in/reader036/viewer/2022062517/56649ec15503460f94bcdbbe/html5/thumbnails/27.jpg)
27
R Output
![Page 28: 1 Chapter 3 Multiple Linear Regression. 2 3.1 Multiple Regression Models Suppose that the yield in pounds of conversion in a chemical process depends.](https://reader036.fdocuments.in/reader036/viewer/2022062517/56649ec15503460f94bcdbbe/html5/thumbnails/28.jpg)
28
3.2.3 Properties of Least-Squares Estimators
• Statistical Properties
• Variances/Covariances
12 ')ˆ(
)ˆ(
XXβ
ββ
Cov
E
ijji
jjj
C)ˆ,ˆ(Cov
C)ˆ(Var
2
2
β
p×p matrixDiagonal entities Cjj are variances,And the remaining Cij are covariance of two regression coefficients
![Page 29: 1 Chapter 3 Multiple Linear Regression. 2 3.1 Multiple Regression Models Suppose that the yield in pounds of conversion in a chemical process depends.](https://reader036.fdocuments.in/reader036/viewer/2022062517/56649ec15503460f94bcdbbe/html5/thumbnails/29.jpg)
Linear Regression Analysis 5E Montgomery, Peck & Vining
29
3.2.4 Estimation of 2
• The residual sum of squares can be shown to be:
• The residual mean square for the model with p parameters is:
ResˆSS y'y β'X'y
2ReRe ̂
pn
SSMS s
s
![Page 30: 1 Chapter 3 Multiple Linear Regression. 2 3.1 Multiple Regression Models Suppose that the yield in pounds of conversion in a chemical process depends.](https://reader036.fdocuments.in/reader036/viewer/2022062517/56649ec15503460f94bcdbbe/html5/thumbnails/30.jpg)
Linear Regression Analysis 5E Montgomery, Peck & Vining
30
3.2.4 Estimation of 2
• Recall that the estimator of 2 is model dependent - that is, change the form of the model and the estimate of 2 will invariably change.– Note that the variance estimate is a function of
the errors; “unexplained noise about the fitted regression line”
![Page 31: 1 Chapter 3 Multiple Linear Regression. 2 3.1 Multiple Regression Models Suppose that the yield in pounds of conversion in a chemical process depends.](https://reader036.fdocuments.in/reader036/viewer/2022062517/56649ec15503460f94bcdbbe/html5/thumbnails/31.jpg)
Which model is better?• Let’s calculate the variance of errors of different models
Model 1; consider two reggressors ( case and distance)
Model 2; only consider reggressor “case”
We would usually prefer a model with a small residual mean square (estimated variance of error).
31
![Page 32: 1 Chapter 3 Multiple Linear Regression. 2 3.1 Multiple Regression Models Suppose that the yield in pounds of conversion in a chemical process depends.](https://reader036.fdocuments.in/reader036/viewer/2022062517/56649ec15503460f94bcdbbe/html5/thumbnails/32.jpg)
Linear Regression Analysis 5E Montgomery, Peck & Vining
32
Example 3.2 Delivery Time Data
![Page 33: 1 Chapter 3 Multiple Linear Regression. 2 3.1 Multiple Regression Models Suppose that the yield in pounds of conversion in a chemical process depends.](https://reader036.fdocuments.in/reader036/viewer/2022062517/56649ec15503460f94bcdbbe/html5/thumbnails/33.jpg)
Linear Regression Analysis 5E Montgomery, Peck & Vining
33
Example 3.2 Delivery Time Data
![Page 34: 1 Chapter 3 Multiple Linear Regression. 2 3.1 Multiple Regression Models Suppose that the yield in pounds of conversion in a chemical process depends.](https://reader036.fdocuments.in/reader036/viewer/2022062517/56649ec15503460f94bcdbbe/html5/thumbnails/34.jpg)
Linear Regression Analysis 5E Montgomery, Peck & Vining
34
3.2.5 Inadequacy of Scatter Diagrams in Multiple Regression
• Scatter diagrams of the regressor variable(s) against the response may be of little value in multiple regression.– These plots can actually be misleading– If there is an interdependency between two or
more regressor variables, the true relationship between xi and y may be masked.
![Page 35: 1 Chapter 3 Multiple Linear Regression. 2 3.1 Multiple Regression Models Suppose that the yield in pounds of conversion in a chemical process depends.](https://reader036.fdocuments.in/reader036/viewer/2022062517/56649ec15503460f94bcdbbe/html5/thumbnails/35.jpg)
35
Illustration of the Inadequacy of Scatter Diagrams in Multiple Regression
![Page 36: 1 Chapter 3 Multiple Linear Regression. 2 3.1 Multiple Regression Models Suppose that the yield in pounds of conversion in a chemical process depends.](https://reader036.fdocuments.in/reader036/viewer/2022062517/56649ec15503460f94bcdbbe/html5/thumbnails/36.jpg)
• There is only one (of few) dominate reggressor
• The regressors operate nearly independent
• !!! Scartterplot could be misleading when several important regressors are related. ( we will discuss the analytical methods for sorting out the relationships between regressors in later chapter.
Scatterplot is useful if…
36
![Page 37: 1 Chapter 3 Multiple Linear Regression. 2 3.1 Multiple Regression Models Suppose that the yield in pounds of conversion in a chemical process depends.](https://reader036.fdocuments.in/reader036/viewer/2022062517/56649ec15503460f94bcdbbe/html5/thumbnails/37.jpg)
37
3.3 Hypothesis Testing in Multiple Linear Regression
Once we have estimated the parameters in the model, we face two immediate questions:
1. What is the overall adequacy of the model?
2. Which specific regressors seem important?
![Page 38: 1 Chapter 3 Multiple Linear Regression. 2 3.1 Multiple Regression Models Suppose that the yield in pounds of conversion in a chemical process depends.](https://reader036.fdocuments.in/reader036/viewer/2022062517/56649ec15503460f94bcdbbe/html5/thumbnails/38.jpg)
Linear Regression Analysis 5E Montgomery, Peck & Vining
38
3.3 Hypothesis Testing in Multiple Linear Regression
• Test for Significance of Regression (sometimes called the global test of model adequacy)
• Tests on Individual Regression Coefficients (or groups of coefficients)
Next we will consider:
![Page 39: 1 Chapter 3 Multiple Linear Regression. 2 3.1 Multiple Regression Models Suppose that the yield in pounds of conversion in a chemical process depends.](https://reader036.fdocuments.in/reader036/viewer/2022062517/56649ec15503460f94bcdbbe/html5/thumbnails/39.jpg)
Linear Regression Analysis 5E Montgomery, Peck & Vining
39
3.3.1 Test for Significance of Regression
• The test for significance is a test to determine if there is a linear relationship between the response and any of the regressor variables
• The hypotheses are H0: 1 = 2 = …= k = 0
H1: j 0 for at least one j
![Page 40: 1 Chapter 3 Multiple Linear Regression. 2 3.1 Multiple Regression Models Suppose that the yield in pounds of conversion in a chemical process depends.](https://reader036.fdocuments.in/reader036/viewer/2022062517/56649ec15503460f94bcdbbe/html5/thumbnails/40.jpg)
40
3.3.1 Test for Significance of Regression
• As in Chapter 2, the total sum of squares can be partitioned in two parts:
SST = SSR + SSRes
• This leads to an ANOVA procedure with the test (F) statistic
s
R
s
R
MS
MS
pnSS
pSSF
ReRe0 )/(
/
![Page 41: 1 Chapter 3 Multiple Linear Regression. 2 3.1 Multiple Regression Models Suppose that the yield in pounds of conversion in a chemical process depends.](https://reader036.fdocuments.in/reader036/viewer/2022062517/56649ec15503460f94bcdbbe/html5/thumbnails/41.jpg)
41
3.3.1 Test for Significance of Regression
• The standard ANOVA is conducted with
![Page 42: 1 Chapter 3 Multiple Linear Regression. 2 3.1 Multiple Regression Models Suppose that the yield in pounds of conversion in a chemical process depends.](https://reader036.fdocuments.in/reader036/viewer/2022062517/56649ec15503460f94bcdbbe/html5/thumbnails/42.jpg)
Linear Regression Analysis 5E Montgomery, Peck & Vining
42
3.3.1 Test for Significance of Regression
ANOVA Table:
Reject H0 if
or p-1or n-p
![Page 43: 1 Chapter 3 Multiple Linear Regression. 2 3.1 Multiple Regression Models Suppose that the yield in pounds of conversion in a chemical process depends.](https://reader036.fdocuments.in/reader036/viewer/2022062517/56649ec15503460f94bcdbbe/html5/thumbnails/43.jpg)
43
3.3.1 Test for Significance of Regression
• R2
– R2 is calculated exactly as in simple linear regression
– R2 can be inflated simply by adding more terms to the model (even insignificant terms)
• Adjusted R2
– Penalizes you for added terms to the model that are not significant
)1/(
)/(1 Re2
nSS
pnSSR
T
sadj
T
s
T
R
SS
SS
SS
SSR Re2 1
![Page 44: 1 Chapter 3 Multiple Linear Regression. 2 3.1 Multiple Regression Models Suppose that the yield in pounds of conversion in a chemical process depends.](https://reader036.fdocuments.in/reader036/viewer/2022062517/56649ec15503460f94bcdbbe/html5/thumbnails/44.jpg)
Linear Regression Analysis 5E Montgomery, Peck & Vining
44
Example 3.3 Delivery Time Data
![Page 45: 1 Chapter 3 Multiple Linear Regression. 2 3.1 Multiple Regression Models Suppose that the yield in pounds of conversion in a chemical process depends.](https://reader036.fdocuments.in/reader036/viewer/2022062517/56649ec15503460f94bcdbbe/html5/thumbnails/45.jpg)
Linear Regression Analysis 5E Montgomery, Peck & Vining
45
Example 3.3 Delivery Time Data
To test H0: 1 = 2 = 0, we calculate the F–statistic:
![Page 46: 1 Chapter 3 Multiple Linear Regression. 2 3.1 Multiple Regression Models Suppose that the yield in pounds of conversion in a chemical process depends.](https://reader036.fdocuments.in/reader036/viewer/2022062517/56649ec15503460f94bcdbbe/html5/thumbnails/46.jpg)
46
Example 3.3 Delivery Time Data
R2 = 0.9596
Adjusted R2 = 0.9559
To look at the overall significance of regression: p-value of F test R2
Adjusted R2
![Page 47: 1 Chapter 3 Multiple Linear Regression. 2 3.1 Multiple Regression Models Suppose that the yield in pounds of conversion in a chemical process depends.](https://reader036.fdocuments.in/reader036/viewer/2022062517/56649ec15503460f94bcdbbe/html5/thumbnails/47.jpg)
Adding a variable will always result in increase of R –squared.
Our goal is to only add necessary regressors that will reduce the residual variability..
But we do not want over-fitting( add un necessary variables ( will learn variable selection procedure in later chapters).
47
![Page 48: 1 Chapter 3 Multiple Linear Regression. 2 3.1 Multiple Regression Models Suppose that the yield in pounds of conversion in a chemical process depends.](https://reader036.fdocuments.in/reader036/viewer/2022062517/56649ec15503460f94bcdbbe/html5/thumbnails/48.jpg)
48
3.3.2 Tests on Individual Regression Coefficients
• Hypothesis test on any single regression coefficient:
• Test Statistic:
– Reject H0 if |t0| >
– This is a partial or marginal test!
0:
0:
1
0
j
j
H
H
)ˆ(
ˆ
ˆ
ˆ
20
j
j
jj
j
seCt
1,2/ knt
![Page 49: 1 Chapter 3 Multiple Linear Regression. 2 3.1 Multiple Regression Models Suppose that the yield in pounds of conversion in a chemical process depends.](https://reader036.fdocuments.in/reader036/viewer/2022062517/56649ec15503460f94bcdbbe/html5/thumbnails/49.jpg)
Linear Regression Analysis 5E Montgomery, Peck & Vining
49
The Extra Sum of Squares method can also be used to test hypotheses on individual model parameters or groups of parameters
Full model
![Page 50: 1 Chapter 3 Multiple Linear Regression. 2 3.1 Multiple Regression Models Suppose that the yield in pounds of conversion in a chemical process depends.](https://reader036.fdocuments.in/reader036/viewer/2022062517/56649ec15503460f94bcdbbe/html5/thumbnails/50.jpg)
Linear Regression Analysis 5E Montgomery, Peck & Vining
50
![Page 51: 1 Chapter 3 Multiple Linear Regression. 2 3.1 Multiple Regression Models Suppose that the yield in pounds of conversion in a chemical process depends.](https://reader036.fdocuments.in/reader036/viewer/2022062517/56649ec15503460f94bcdbbe/html5/thumbnails/51.jpg)
Linear Regression Analysis 5E Montgomery, Peck & Vining
51
![Page 52: 1 Chapter 3 Multiple Linear Regression. 2 3.1 Multiple Regression Models Suppose that the yield in pounds of conversion in a chemical process depends.](https://reader036.fdocuments.in/reader036/viewer/2022062517/56649ec15503460f94bcdbbe/html5/thumbnails/52.jpg)
Linear Regression Analysis 5E Montgomery, Peck & Vining
52
3.3.3 Special Case of Orthogonal Columns in X
• If the columns X1 are orthogonal to the columns in X2, the sum of squares due to 2
that is free of any dependence on the the regressors in X1.
![Page 53: 1 Chapter 3 Multiple Linear Regression. 2 3.1 Multiple Regression Models Suppose that the yield in pounds of conversion in a chemical process depends.](https://reader036.fdocuments.in/reader036/viewer/2022062517/56649ec15503460f94bcdbbe/html5/thumbnails/53.jpg)
Linear Regression Analysis 5E Montgomery, Peck & Vining
53
Example
• Consider a dataset with four regressor variables and a single response.
• Fit the equation with all regressors and find that: y = - 19.9 + 0.0123x1 + 27.3x2 - 0.0655x3 - 0.196x4
• Looking at the t-tests, suppose that x3 is insignificant. So it is removed. What is the equation now?
• Generally, it is not
y = - 19.9 + 0.0123x1 + 27.3x2 - 0.196x4
![Page 54: 1 Chapter 3 Multiple Linear Regression. 2 3.1 Multiple Regression Models Suppose that the yield in pounds of conversion in a chemical process depends.](https://reader036.fdocuments.in/reader036/viewer/2022062517/56649ec15503460f94bcdbbe/html5/thumbnails/54.jpg)
Linear Regression Analysis 5E Montgomery, Peck & Vining
54
Example
• The model must be refit with the insignificant regressors left out of the model.
• The regression equation is
y = - 24.9 + 0.0117x1 + 31.0x2 - 0.217x4
• The refitting must be done since the coefficient estimates for an individual regressor depend on all of the regressors, xj
![Page 55: 1 Chapter 3 Multiple Linear Regression. 2 3.1 Multiple Regression Models Suppose that the yield in pounds of conversion in a chemical process depends.](https://reader036.fdocuments.in/reader036/viewer/2022062517/56649ec15503460f94bcdbbe/html5/thumbnails/55.jpg)
Linear Regression Analysis 5E Montgomery, Peck & Vining
55
Example
• However, if the columns are orthogonal to each other, then there is no need to refit.
• Can you think of some situations where we would have orthogonal columns?
![Page 56: 1 Chapter 3 Multiple Linear Regression. 2 3.1 Multiple Regression Models Suppose that the yield in pounds of conversion in a chemical process depends.](https://reader036.fdocuments.in/reader036/viewer/2022062517/56649ec15503460f94bcdbbe/html5/thumbnails/56.jpg)
Linear Regression Analysis 5E Montgomery, Peck & Vining
56
3.4.1. Confidence Intervals on the Regression Coefficients
/ 2, / 2,ˆ ˆ ˆ ˆ( ) ( )j n p j j j n p jt se t se
A 100(1-) percent C.I. for the regression coefficient, j is:
2 2/ 2, / 2,
ˆ ˆˆ ˆj n p jj j j n p jjt C t C
Or,
![Page 57: 1 Chapter 3 Multiple Linear Regression. 2 3.1 Multiple Regression Models Suppose that the yield in pounds of conversion in a chemical process depends.](https://reader036.fdocuments.in/reader036/viewer/2022062517/56649ec15503460f94bcdbbe/html5/thumbnails/57.jpg)
Linear Regression Analysis 5E Montgomery, Peck & Vining
57
![Page 58: 1 Chapter 3 Multiple Linear Regression. 2 3.1 Multiple Regression Models Suppose that the yield in pounds of conversion in a chemical process depends.](https://reader036.fdocuments.in/reader036/viewer/2022062517/56649ec15503460f94bcdbbe/html5/thumbnails/58.jpg)
Linear Regression Analysis 5E Montgomery, Peck & Vining
58
3.4.2. Confidence Interval Estimation of the Mean Response
• 100(1-) percent CI on the mean response at the point x01, x02, …, x0k is
• See Example 3-9 on page 95 and the discussion that follows
2 10 / 2, 0 0 0
2 10 / 2, 0 0
ˆ ˆ ' ( ) ( | )
ˆ ˆ ' ( )
n p
n p
y t E y
y t
x X'X x x
x X'X x
![Page 59: 1 Chapter 3 Multiple Linear Regression. 2 3.1 Multiple Regression Models Suppose that the yield in pounds of conversion in a chemical process depends.](https://reader036.fdocuments.in/reader036/viewer/2022062517/56649ec15503460f94bcdbbe/html5/thumbnails/59.jpg)
Linear Regression Analysis 5E Montgomery, Peck & Vining
59
![Page 60: 1 Chapter 3 Multiple Linear Regression. 2 3.1 Multiple Regression Models Suppose that the yield in pounds of conversion in a chemical process depends.](https://reader036.fdocuments.in/reader036/viewer/2022062517/56649ec15503460f94bcdbbe/html5/thumbnails/60.jpg)
Linear Regression Analysis 5E Montgomery, Peck & Vining
60
![Page 61: 1 Chapter 3 Multiple Linear Regression. 2 3.1 Multiple Regression Models Suppose that the yield in pounds of conversion in a chemical process depends.](https://reader036.fdocuments.in/reader036/viewer/2022062517/56649ec15503460f94bcdbbe/html5/thumbnails/61.jpg)
Linear Regression Analysis 5E Montgomery, Peck & Vining
61
![Page 62: 1 Chapter 3 Multiple Linear Regression. 2 3.1 Multiple Regression Models Suppose that the yield in pounds of conversion in a chemical process depends.](https://reader036.fdocuments.in/reader036/viewer/2022062517/56649ec15503460f94bcdbbe/html5/thumbnails/62.jpg)
Linear Regression Analysis 5E Montgomery, Peck & Vining
62
3.4.3. Simultaneous Confidence Intervals on Regression Coefficients
It can be shown that
From this result, the joint confidence region for all parameters in is
,Re
ˆ ˆ( ) ' ' ( )~ p n p
s
FpMS
β β X X β β
, ,Re
ˆ ˆ( ) ' ' ( )p n p
s
FpMS
β β X X β β
![Page 63: 1 Chapter 3 Multiple Linear Regression. 2 3.1 Multiple Regression Models Suppose that the yield in pounds of conversion in a chemical process depends.](https://reader036.fdocuments.in/reader036/viewer/2022062517/56649ec15503460f94bcdbbe/html5/thumbnails/63.jpg)
Linear Regression Analysis 5E Montgomery, Peck & Vining
63
3.5 Prediction of New Observations
• A 100(1-) percent prediction interval for a future observation is
2 10 / 2, 0 0 0
2 10 / 2, 0 0
ˆ ˆ (1 ' ( ) )
ˆ ˆ (1 ' ( ) )
n p
n p
y t y
y t
x X'X x
x X'X x
![Page 64: 1 Chapter 3 Multiple Linear Regression. 2 3.1 Multiple Regression Models Suppose that the yield in pounds of conversion in a chemical process depends.](https://reader036.fdocuments.in/reader036/viewer/2022062517/56649ec15503460f94bcdbbe/html5/thumbnails/64.jpg)
Linear Regression Analysis 5E Montgomery, Peck & Vining
64
![Page 65: 1 Chapter 3 Multiple Linear Regression. 2 3.1 Multiple Regression Models Suppose that the yield in pounds of conversion in a chemical process depends.](https://reader036.fdocuments.in/reader036/viewer/2022062517/56649ec15503460f94bcdbbe/html5/thumbnails/65.jpg)
Linear Regression Analysis 5E Montgomery, Peck & Vining
65
3.6 Hidden Extrapolation in Multiple Regression
• In prediction, exercise care about potentially extrapolating beyond the region containing the original observations.
Figure 3.10 An example of extrapolation in multiple regression.
![Page 66: 1 Chapter 3 Multiple Linear Regression. 2 3.1 Multiple Regression Models Suppose that the yield in pounds of conversion in a chemical process depends.](https://reader036.fdocuments.in/reader036/viewer/2022062517/56649ec15503460f94bcdbbe/html5/thumbnails/66.jpg)
Linear Regression Analysis 5E Montgomery, Peck & Vining
66
3.6 Hidden Extrapolation in Multiple Regression
• We will define the smallest convex set containing all of the original n data points (xi1, xi2, … xik), i = 1, 2, …, n, as the regressor variable hull RVH.
• If a point x01, x02, …, x0k lies inside or on the boundary of the RVH, then prediction or estimation involves interpolation, while if this point lies outside the RVH, extrapolation is required.
![Page 67: 1 Chapter 3 Multiple Linear Regression. 2 3.1 Multiple Regression Models Suppose that the yield in pounds of conversion in a chemical process depends.](https://reader036.fdocuments.in/reader036/viewer/2022062517/56649ec15503460f94bcdbbe/html5/thumbnails/67.jpg)
Linear Regression Analysis 5E Montgomery, Peck & Vining
67
3.6 Hidden Extrapolation in Multiple Regression
• Diagonal elements of the matrix H = X(X’X)-1X’ can aid in determining if hidden extrapolation exists:
• The set of points x (not necessarily data points used to fit the model) that satisfy
is an ellipsoid enclosing all points inside the RVH.
![Page 68: 1 Chapter 3 Multiple Linear Regression. 2 3.1 Multiple Regression Models Suppose that the yield in pounds of conversion in a chemical process depends.](https://reader036.fdocuments.in/reader036/viewer/2022062517/56649ec15503460f94bcdbbe/html5/thumbnails/68.jpg)
Linear Regression Analysis 5E Montgomery, Peck & Vining
68
3.6 Hidden Extrapolation in Multiple Regression
• Let x0 be a point at which prediction or estimation is of interest. Then
• If h00 > hmax then the point is a point of extrapolation.
![Page 69: 1 Chapter 3 Multiple Linear Regression. 2 3.1 Multiple Regression Models Suppose that the yield in pounds of conversion in a chemical process depends.](https://reader036.fdocuments.in/reader036/viewer/2022062517/56649ec15503460f94bcdbbe/html5/thumbnails/69.jpg)
Linear Regression Analysis 5E Montgomery, Peck & Vining
69
Example 3.13
Consider prediction or estimation at:
![Page 70: 1 Chapter 3 Multiple Linear Regression. 2 3.1 Multiple Regression Models Suppose that the yield in pounds of conversion in a chemical process depends.](https://reader036.fdocuments.in/reader036/viewer/2022062517/56649ec15503460f94bcdbbe/html5/thumbnails/70.jpg)
Linear Regression Analysis 5E Montgomery, Peck & Vining
70
Figure 3.10 Scatterplot of cases and distance for the delivery time data.
#9
ab
c
d