Linear Regression Analysis 5E Montgomery, Peck & Vining 1 Chapter 13 Generalized Linear Models.
-
Upload
samson-jordan -
Category
Documents
-
view
299 -
download
1
Transcript of Linear Regression Analysis 5E Montgomery, Peck & Vining 1 Chapter 13 Generalized Linear Models.
![Page 1: Linear Regression Analysis 5E Montgomery, Peck & Vining 1 Chapter 13 Generalized Linear Models.](https://reader033.fdocuments.in/reader033/viewer/2022061520/56649ddb5503460f94ad2066/html5/thumbnails/1.jpg)
Linear Regression Analysis 5E Montgomery, Peck & Vining
1
Chapter 13Generalized Linear Models
![Page 2: Linear Regression Analysis 5E Montgomery, Peck & Vining 1 Chapter 13 Generalized Linear Models.](https://reader033.fdocuments.in/reader033/viewer/2022061520/56649ddb5503460f94ad2066/html5/thumbnails/2.jpg)
Linear Regression Analysis 5E Montgomery, Peck & Vining
2
Generalized Linear Models
• Traditional applications of linear models, such as DOX and multiple linear regression, assume that the response variable is – Normally distributed– Constant variance– Independent
• There are many situations where these assumptions are inappropriate– The response is either binary (0,1), or a count– The response is continuous, but nonnormal
![Page 3: Linear Regression Analysis 5E Montgomery, Peck & Vining 1 Chapter 13 Generalized Linear Models.](https://reader033.fdocuments.in/reader033/viewer/2022061520/56649ddb5503460f94ad2066/html5/thumbnails/3.jpg)
Linear Regression Analysis 5E Montgomery, Peck & Vining
3
Some Approaches to These Problems• Data transformation
– Induce approximate normality– Stabilize variance– Simplify model form
• Weighted least squares– Often used to stabilize variance
• Generalized linear models (GLM)– Approach is about 25-30 years old, unifies linear and
nonlinear regression models– Response distribution is a member of the exponential family
(normal, exponential, gamma, binomial, Poisson)
![Page 4: Linear Regression Analysis 5E Montgomery, Peck & Vining 1 Chapter 13 Generalized Linear Models.](https://reader033.fdocuments.in/reader033/viewer/2022061520/56649ddb5503460f94ad2066/html5/thumbnails/4.jpg)
Linear Regression Analysis 5E Montgomery, Peck & Vining
4
Generalized Linear Models• Original applications were in biopharmaceutical sciences • Lots of recent interest in GLMs in industrial statistics• GLMs are simple models; include linear regression and
OLS as a special case• Parameter estimation is by maximum likelihood
(assume that the response distribution is known)• Inference on parameters is based on large-sample or
asymptotic theory• We will consider logistic regression, Poisson regression,
then the GLM
![Page 5: Linear Regression Analysis 5E Montgomery, Peck & Vining 1 Chapter 13 Generalized Linear Models.](https://reader033.fdocuments.in/reader033/viewer/2022061520/56649ddb5503460f94ad2066/html5/thumbnails/5.jpg)
Linear Regression Analysis 5E Montgomery, Peck & Vining
5
References
• Montgomery, D. C., Peck, E. A5, and Vining, G. G. (2012), Introduction to Linear Regression Analysis, 4th Edition, Wiley, New York (see Chapter 14)
• Myers, R. H., Montgomery, D. C., Vining, G. G. and Robinson, T.J. (2010), Generalized Linear Models with Applications in Engineering and the Sciences, 2nd edition, Wiley, New York
• Hosmer, D. W. and Lemeshow, S. (2000), Applied Logistic Regression, 2nd Edition, Wiley, New York
• Lewis, S. L., Montgomery, D. C., and Myers, R. H. (2001), “Confidence Interval Coverage for Designed Experiments Analyzed with GLMs”, Journal of Quality Technology 33, pp. 279-292
• Lewis, S. L., Montgomery, D. C., and Myers, R. H. (2001), “Examples of Designed Experiments with Nonnormal Responses”, Journal of Quality Technology 33, pp. 265-278
• Myers, R. H. and Montgomery, D. C. (1997), “A Tutorial on Generalized Linear Models”, Journal of Quality Technology 29, pp. 274-291
![Page 6: Linear Regression Analysis 5E Montgomery, Peck & Vining 1 Chapter 13 Generalized Linear Models.](https://reader033.fdocuments.in/reader033/viewer/2022061520/56649ddb5503460f94ad2066/html5/thumbnails/6.jpg)
Linear Regression Analysis 5E Montgomery, Peck & Vining
6
Binary Response Variables
• The outcome ( or response, or endpoint) values 0, 1 can represent “success” and “failure”
• Occurs often in the biopharmaceutical field; dose-response studies, bioassays, clinical trials
• Industrial applications include failure analysis, fatigue testing, reliability testing
• For example, functional electrical testing on a semiconductor can yield: – “success” in which case the device works– “failure” due to a short, an open, or some other failure mode
![Page 7: Linear Regression Analysis 5E Montgomery, Peck & Vining 1 Chapter 13 Generalized Linear Models.](https://reader033.fdocuments.in/reader033/viewer/2022061520/56649ddb5503460f94ad2066/html5/thumbnails/7.jpg)
Linear Regression Analysis 5E Montgomery, Peck & Vining
7
Binary Response Variables
• Possible model:
• The response yi is a Bernoulli random variable
01
1,2,...,
0 or 1
k
i j ij i i ij i
i ny x
y
x
2
( 1) with 0 1
( 0) 1
( )
( ) (1 )i
i i i
i i
i i i i
i y i i
P y
P y
E y
Var y
x
![Page 8: Linear Regression Analysis 5E Montgomery, Peck & Vining 1 Chapter 13 Generalized Linear Models.](https://reader033.fdocuments.in/reader033/viewer/2022061520/56649ddb5503460f94ad2066/html5/thumbnails/8.jpg)
Linear Regression Analysis 5E Montgomery, Peck & Vining
8
Problems With This Model
• The error terms take on only two values, so they can’t possibly be normally distributed
• The variance of the observations is a function of the mean (see previous slide)
• A linear response function could result in predicted values that fall outside the 0, 1 range, and this is impossible because
0 ( ) 1i i i iE y x
![Page 9: Linear Regression Analysis 5E Montgomery, Peck & Vining 1 Chapter 13 Generalized Linear Models.](https://reader033.fdocuments.in/reader033/viewer/2022061520/56649ddb5503460f94ad2066/html5/thumbnails/9.jpg)
Linear Regression Analysis 5E Montgomery, Peck & Vining
9
Binary Response Variables – The Challenger Data
Temperature at Launch
At Least One O-ring Failure
Temperature at Launch
At Least One O-ring Failure
53 1 70 1
56 1 70 1
57 1 72 0
63 0 73 0
66 0 75 0
67 0 75 1
67 0 76 0
67 0 76 0
68 0 78 0
69 0 79 0
70 0 80 0
70 1 81 0
Data for space shuttle launches and static tests prior to the launch of Challenger
![Page 10: Linear Regression Analysis 5E Montgomery, Peck & Vining 1 Chapter 13 Generalized Linear Models.](https://reader033.fdocuments.in/reader033/viewer/2022061520/56649ddb5503460f94ad2066/html5/thumbnails/10.jpg)
Linear Regression Analysis 5E Montgomery, Peck & Vining
10
Binary Response Variables
• There is a lot of empirical evidence that the response function should be nonlinear; an “S” shape is quite logical
• See the scatter plot of the Challenger data
• The logistic response function is a common choice
exp( 1( )
1 exp( 1 exp(E y
x
x x
![Page 11: Linear Regression Analysis 5E Montgomery, Peck & Vining 1 Chapter 13 Generalized Linear Models.](https://reader033.fdocuments.in/reader033/viewer/2022061520/56649ddb5503460f94ad2066/html5/thumbnails/11.jpg)
Linear Regression Analysis 5E Montgomery, Peck & Vining
11
![Page 12: Linear Regression Analysis 5E Montgomery, Peck & Vining 1 Chapter 13 Generalized Linear Models.](https://reader033.fdocuments.in/reader033/viewer/2022061520/56649ddb5503460f94ad2066/html5/thumbnails/12.jpg)
Linear Regression Analysis 5E Montgomery, Peck & Vining
12
The Logistic Response Function
• The logistic response function can be easily linearized. Let:
• Define
• This is called the logit transformation
and ( )E y x
ln1
![Page 13: Linear Regression Analysis 5E Montgomery, Peck & Vining 1 Chapter 13 Generalized Linear Models.](https://reader033.fdocuments.in/reader033/viewer/2022061520/56649ddb5503460f94ad2066/html5/thumbnails/13.jpg)
Linear Regression Analysis 5E Montgomery, Peck & Vining
13
Logistic Regression Model
• Model:
• The model parameters are estimated by the method of maximum likelihood (MLE)
( )
where
( )
exp(
1 exp(
i i i
i i
i
i
y E y
E y
x
x
![Page 14: Linear Regression Analysis 5E Montgomery, Peck & Vining 1 Chapter 13 Generalized Linear Models.](https://reader033.fdocuments.in/reader033/viewer/2022061520/56649ddb5503460f94ad2066/html5/thumbnails/14.jpg)
Linear Regression Analysis 5E Montgomery, Peck & Vining
14
A Logistic Regression Model for the Challenger Data (Using Minitab)
Binary Logistic Regression: O-Ring Fail versus Temperature
Link Function: Logit
Response Information
Variable Value Count
O-Ring F 1 7 (Event)
0 17
Total 24
Logistic Regression Table
Odds 95% CI
Predictor Coef SE Coef Z P Ratio Lower Upper
Constant 10.875 5.703 1.91 0.057
Temperat -0.17132 0.08344 -2.05 0.040 0.84 0.72 0.99
Log-Likelihood = -11.515
![Page 15: Linear Regression Analysis 5E Montgomery, Peck & Vining 1 Chapter 13 Generalized Linear Models.](https://reader033.fdocuments.in/reader033/viewer/2022061520/56649ddb5503460f94ad2066/html5/thumbnails/15.jpg)
Linear Regression Analysis 5E Montgomery, Peck & Vining
15
A Logistic Regression Model for the Challenger Data
Test that all slopes are zero: G = 5.944, DF = 1, P-Value = 0.015
Goodness-of-Fit Tests
Method Chi-Square DF P
Pearson 14.049 15 0.522
Deviance 15.759 15 0.398
Hosmer-Lemeshow 11.834 8 0.159
exp(10.875 0.17132 )ˆ
1 exp(10.875 0.17132 )
xy
x
![Page 16: Linear Regression Analysis 5E Montgomery, Peck & Vining 1 Chapter 13 Generalized Linear Models.](https://reader033.fdocuments.in/reader033/viewer/2022061520/56649ddb5503460f94ad2066/html5/thumbnails/16.jpg)
Linear Regression Analysis 5E Montgomery, Peck & Vining
16
Note that the fitted function has been extended down to 31 deg F, the temperature at which Challenger was launched
![Page 17: Linear Regression Analysis 5E Montgomery, Peck & Vining 1 Chapter 13 Generalized Linear Models.](https://reader033.fdocuments.in/reader033/viewer/2022061520/56649ddb5503460f94ad2066/html5/thumbnails/17.jpg)
Linear Regression Analysis 5E Montgomery, Peck & Vining
17
Maximum Likelihood Estimation in Logistic Regression
• The distribution of each observation yi is
• The likelihood function is
• We usually work with the log-likelihood:
1( ) (1 ) , 1,2,...,i iy yi i i if y i n
1
1
( , ( ) (1 )i i
n ny y
i i i ii i
L f y
y
1 11
ln ( , ln ( ) ln ln(1 )1
n n ni
i i i ii ii i
L f y y
y
![Page 18: Linear Regression Analysis 5E Montgomery, Peck & Vining 1 Chapter 13 Generalized Linear Models.](https://reader033.fdocuments.in/reader033/viewer/2022061520/56649ddb5503460f94ad2066/html5/thumbnails/18.jpg)
Linear Regression Analysis 5E Montgomery, Peck & Vining
18
Maximum Likelihood Estimation in Logistic Regression
• The maximum likelihood estimators (MLEs) of the model parameters are those values that maximize the likelihood (or log-likelihood) function
• ML has been around since the first part of the previous century
• Often gives estimators that are intuitively pleasing• MLEs have nice properties; unbiased (for large
samples), minimum variance (or nearly so), and they have an approximate normal distribution when n is large
![Page 19: Linear Regression Analysis 5E Montgomery, Peck & Vining 1 Chapter 13 Generalized Linear Models.](https://reader033.fdocuments.in/reader033/viewer/2022061520/56649ddb5503460f94ad2066/html5/thumbnails/19.jpg)
Linear Regression Analysis 5E Montgomery, Peck & Vining
19
Maximum Likelihood Estimation in Logistic Regression
• If we have ni trials at each observation, we can write the log-likelihood as
• The derivative of the log-likelihood is 1
ln ( , ln[1 exp(n
i ii
L n
y X y x
1
1
ln ( ,exp(
1 exp(
because )
ni
i ii i
n
i i ii
i i i
nL
n
n
yX y x x
x
X y x
X y X
![Page 20: Linear Regression Analysis 5E Montgomery, Peck & Vining 1 Chapter 13 Generalized Linear Models.](https://reader033.fdocuments.in/reader033/viewer/2022061520/56649ddb5503460f94ad2066/html5/thumbnails/20.jpg)
Linear Regression Analysis 5E Montgomery, Peck & Vining
20
Maximum Likelihood Estimation in Logistic Regression
• Setting this last result to zero gives the maximum likelihood score equations
• These equations look easy to solve…we’ve actually seen them before in linear regression:
( X y 0
1
results from OLS or ML with normal errors
Since ,
ˆ ˆ, and ) (OLS or the normal-theory MLE)
y X
X y 0
X X y X y X 0
X X X y X X X y
![Page 21: Linear Regression Analysis 5E Montgomery, Peck & Vining 1 Chapter 13 Generalized Linear Models.](https://reader033.fdocuments.in/reader033/viewer/2022061520/56649ddb5503460f94ad2066/html5/thumbnails/21.jpg)
Linear Regression Analysis 5E Montgomery, Peck & Vining
21
Maximum Likelihood Estimation in Logistic Regression
• Solving the ML score equations in logistic regression isn’t quite as easy, because
• Logistic regression is a nonlinear model• It turns out that the solution is actually fairly easy, and is based on
iteratively reweighted least squares or IRLS (see Appendix for details)
• An iterative procedure is necessary because parameter estimates must be updated from an initial “guess” through several steps
• Weights are necessary because the variance of the observations is not constant
• The weights are functions of the unknown parameters
, 1, 2,...,1 exp(
ii
i
ni n
x
![Page 22: Linear Regression Analysis 5E Montgomery, Peck & Vining 1 Chapter 13 Generalized Linear Models.](https://reader033.fdocuments.in/reader033/viewer/2022061520/56649ddb5503460f94ad2066/html5/thumbnails/22.jpg)
Linear Regression Analysis 5E Montgomery, Peck & Vining
22
Interpretation of the Parameters in Logistic Regression
• The log-odds at x is
• The log-odds at x + 1 is
• The difference in the log-odds is
0 1
ˆ( ) ˆ ˆˆ( ) lnˆ1 ( )
xx x
x
0 1
ˆ( 1) ˆ ˆˆ( 1) ln ( 1)ˆ1 ( 1)
xx x
x
1̂ˆ ˆ( 1) ( )x x
![Page 23: Linear Regression Analysis 5E Montgomery, Peck & Vining 1 Chapter 13 Generalized Linear Models.](https://reader033.fdocuments.in/reader033/viewer/2022061520/56649ddb5503460f94ad2066/html5/thumbnails/23.jpg)
Linear Regression Analysis 5E Montgomery, Peck & Vining
23
Interpretation of the Parameters in Logistic Regression
• The odds ratio is found by taking antilogs:
• The odds ratio is interpreted as the estimated increase in the probability of “success” associated with a one-unit increase in the value of the predictor variable
1̂1ˆ xR
x
OddsO e
Odds
![Page 24: Linear Regression Analysis 5E Montgomery, Peck & Vining 1 Chapter 13 Generalized Linear Models.](https://reader033.fdocuments.in/reader033/viewer/2022061520/56649ddb5503460f94ad2066/html5/thumbnails/24.jpg)
Linear Regression Analysis 5E Montgomery, Peck & Vining
24
Odds Ratio for the Challenger Data
This implies that every decrease of one degree in temperature increases the odds of O-ring failure by about 1/0.84 = 1.19 or 19 percent
The temperature at Challenger launch was 22 degrees below the lowest observed launch temperature, so now
This results in an increase in the odds of failure of 1/0.0231 = 43.34, or about 4200 percent!! There’s a big extrapolation here, but if you knew this prior to launch, what decision would you have made?
0.17132ˆ 0.84RO e
22( 0.17132)ˆ 0.0231RO e
![Page 25: Linear Regression Analysis 5E Montgomery, Peck & Vining 1 Chapter 13 Generalized Linear Models.](https://reader033.fdocuments.in/reader033/viewer/2022061520/56649ddb5503460f94ad2066/html5/thumbnails/25.jpg)
Linear Regression Analysis 5E Montgomery, Peck & Vining
25
Inference on the Model Parameters
![Page 26: Linear Regression Analysis 5E Montgomery, Peck & Vining 1 Chapter 13 Generalized Linear Models.](https://reader033.fdocuments.in/reader033/viewer/2022061520/56649ddb5503460f94ad2066/html5/thumbnails/26.jpg)
Linear Regression Analysis 5E Montgomery, Peck & Vining
26
Inference on the Model Parameters
See slide 15; Minitab calls this “G”.
![Page 27: Linear Regression Analysis 5E Montgomery, Peck & Vining 1 Chapter 13 Generalized Linear Models.](https://reader033.fdocuments.in/reader033/viewer/2022061520/56649ddb5503460f94ad2066/html5/thumbnails/27.jpg)
Linear Regression Analysis 5E Montgomery, Peck & Vining
27
Testing Goodness of Fit
![Page 28: Linear Regression Analysis 5E Montgomery, Peck & Vining 1 Chapter 13 Generalized Linear Models.](https://reader033.fdocuments.in/reader033/viewer/2022061520/56649ddb5503460f94ad2066/html5/thumbnails/28.jpg)
Linear Regression Analysis 5E Montgomery, Peck & Vining
28
Pearson chi-square goodness-of-fit statistic:
![Page 29: Linear Regression Analysis 5E Montgomery, Peck & Vining 1 Chapter 13 Generalized Linear Models.](https://reader033.fdocuments.in/reader033/viewer/2022061520/56649ddb5503460f94ad2066/html5/thumbnails/29.jpg)
Linear Regression Analysis 5E Montgomery, Peck & Vining
29
The Hosmer-Lemeshow goodness-of-fit statistic:
![Page 30: Linear Regression Analysis 5E Montgomery, Peck & Vining 1 Chapter 13 Generalized Linear Models.](https://reader033.fdocuments.in/reader033/viewer/2022061520/56649ddb5503460f94ad2066/html5/thumbnails/30.jpg)
Linear Regression Analysis 5E Montgomery, Peck & Vining
30
Refer to slide 15 for the Minitab output showing all three goodness-of-fit statistics for the Challenger data
![Page 31: Linear Regression Analysis 5E Montgomery, Peck & Vining 1 Chapter 13 Generalized Linear Models.](https://reader033.fdocuments.in/reader033/viewer/2022061520/56649ddb5503460f94ad2066/html5/thumbnails/31.jpg)
Linear Regression Analysis 5E Montgomery, Peck & Vining
31
Likelihood Inference on the Model Parameters
• Deviance can also be used to test hypotheses about subsets of the model parameters (analogous to the extra SS method)
• Procedure:1 2 2 2
0 2
1 2
1 1
, with parameters, has parameters
This full model has deviance (
:
:
The reduced model is , with deviance ( )
The difference in deviance between the full and reduce
p r
H
H
X X
0
0
X
1 1
1 0
1 0
d models is
( | ) ( ) ( with degrees of freedom
( | ) has a chi-square distribution under :
Large values of ( | ) imply that : should be rejected
r
H
H
0
0
![Page 32: Linear Regression Analysis 5E Montgomery, Peck & Vining 1 Chapter 13 Generalized Linear Models.](https://reader033.fdocuments.in/reader033/viewer/2022061520/56649ddb5503460f94ad2066/html5/thumbnails/32.jpg)
Linear Regression Analysis 5E Montgomery, Peck & Vining
32
Inference on the Model Parameters• Tests on individual model coefficients can also be done using
Wald inference• Uses the result that the MLEs have an approximate normal
distribution, so the distribution of
is standard normal if the true value of the parameter is zero. Some computer programs report the square of Z (which is chi-square), and others calculate the P-value using the t distribution See slide 14 for the Wald test on the temperature parameter for the Challenger data
0
ˆ
ˆ( )Z
se
![Page 33: Linear Regression Analysis 5E Montgomery, Peck & Vining 1 Chapter 13 Generalized Linear Models.](https://reader033.fdocuments.in/reader033/viewer/2022061520/56649ddb5503460f94ad2066/html5/thumbnails/33.jpg)
Linear Regression Analysis 5E Montgomery, Peck & Vining
33
Another Logistic Regression Example: The Pneumoconiosis Data
• A 1959 article in Biometrics reported the data:
![Page 34: Linear Regression Analysis 5E Montgomery, Peck & Vining 1 Chapter 13 Generalized Linear Models.](https://reader033.fdocuments.in/reader033/viewer/2022061520/56649ddb5503460f94ad2066/html5/thumbnails/34.jpg)
Linear Regression Analysis 5E Montgomery, Peck & Vining
34
![Page 35: Linear Regression Analysis 5E Montgomery, Peck & Vining 1 Chapter 13 Generalized Linear Models.](https://reader033.fdocuments.in/reader033/viewer/2022061520/56649ddb5503460f94ad2066/html5/thumbnails/35.jpg)
Linear Regression Analysis 5E Montgomery, Peck & Vining
35
![Page 36: Linear Regression Analysis 5E Montgomery, Peck & Vining 1 Chapter 13 Generalized Linear Models.](https://reader033.fdocuments.in/reader033/viewer/2022061520/56649ddb5503460f94ad2066/html5/thumbnails/36.jpg)
Linear Regression Analysis 5E Montgomery, Peck & Vining
36
The fitted model:
![Page 37: Linear Regression Analysis 5E Montgomery, Peck & Vining 1 Chapter 13 Generalized Linear Models.](https://reader033.fdocuments.in/reader033/viewer/2022061520/56649ddb5503460f94ad2066/html5/thumbnails/37.jpg)
Linear Regression Analysis 5E Montgomery, Peck & Vining
37
![Page 38: Linear Regression Analysis 5E Montgomery, Peck & Vining 1 Chapter 13 Generalized Linear Models.](https://reader033.fdocuments.in/reader033/viewer/2022061520/56649ddb5503460f94ad2066/html5/thumbnails/38.jpg)
Linear Regression Analysis 5E Montgomery, Peck & Vining
38
![Page 39: Linear Regression Analysis 5E Montgomery, Peck & Vining 1 Chapter 13 Generalized Linear Models.](https://reader033.fdocuments.in/reader033/viewer/2022061520/56649ddb5503460f94ad2066/html5/thumbnails/39.jpg)
Linear Regression Analysis 5E Montgomery, Peck & Vining
39
![Page 40: Linear Regression Analysis 5E Montgomery, Peck & Vining 1 Chapter 13 Generalized Linear Models.](https://reader033.fdocuments.in/reader033/viewer/2022061520/56649ddb5503460f94ad2066/html5/thumbnails/40.jpg)
Linear Regression Analysis 5E Montgomery, Peck & Vining
40
Diagnostic Checking
![Page 41: Linear Regression Analysis 5E Montgomery, Peck & Vining 1 Chapter 13 Generalized Linear Models.](https://reader033.fdocuments.in/reader033/viewer/2022061520/56649ddb5503460f94ad2066/html5/thumbnails/41.jpg)
Linear Regression Analysis 5E Montgomery, Peck & Vining
41
![Page 42: Linear Regression Analysis 5E Montgomery, Peck & Vining 1 Chapter 13 Generalized Linear Models.](https://reader033.fdocuments.in/reader033/viewer/2022061520/56649ddb5503460f94ad2066/html5/thumbnails/42.jpg)
Linear Regression Analysis 5E Montgomery, Peck & Vining
42
![Page 43: Linear Regression Analysis 5E Montgomery, Peck & Vining 1 Chapter 13 Generalized Linear Models.](https://reader033.fdocuments.in/reader033/viewer/2022061520/56649ddb5503460f94ad2066/html5/thumbnails/43.jpg)
Linear Regression Analysis 5E Montgomery, Peck & Vining
43
![Page 44: Linear Regression Analysis 5E Montgomery, Peck & Vining 1 Chapter 13 Generalized Linear Models.](https://reader033.fdocuments.in/reader033/viewer/2022061520/56649ddb5503460f94ad2066/html5/thumbnails/44.jpg)
Linear Regression Analysis 5E Montgomery, Peck & Vining
44
Consider Fitting a More Complex Model
![Page 45: Linear Regression Analysis 5E Montgomery, Peck & Vining 1 Chapter 13 Generalized Linear Models.](https://reader033.fdocuments.in/reader033/viewer/2022061520/56649ddb5503460f94ad2066/html5/thumbnails/45.jpg)
Linear Regression Analysis 5E Montgomery, Peck & Vining
45
A More Complex Model
Is the expanded model useful? The Wald test on the term (Years)2 indicates that the term is probably unnecessary.
Consider the difference in deviance:
( (
( | ) ( (
with 1 df (chi-square P-value = 0.0961)
Compare the P-values for the Wald and deviance tests
![Page 46: Linear Regression Analysis 5E Montgomery, Peck & Vining 1 Chapter 13 Generalized Linear Models.](https://reader033.fdocuments.in/reader033/viewer/2022061520/56649ddb5503460f94ad2066/html5/thumbnails/46.jpg)
Linear Regression Analysis 5E Montgomery, Peck & Vining
46
![Page 47: Linear Regression Analysis 5E Montgomery, Peck & Vining 1 Chapter 13 Generalized Linear Models.](https://reader033.fdocuments.in/reader033/viewer/2022061520/56649ddb5503460f94ad2066/html5/thumbnails/47.jpg)
Linear Regression Analysis 5E Montgomery, Peck & Vining
47
![Page 48: Linear Regression Analysis 5E Montgomery, Peck & Vining 1 Chapter 13 Generalized Linear Models.](https://reader033.fdocuments.in/reader033/viewer/2022061520/56649ddb5503460f94ad2066/html5/thumbnails/48.jpg)
Linear Regression Analysis 5E Montgomery, Peck & Vining
48
![Page 49: Linear Regression Analysis 5E Montgomery, Peck & Vining 1 Chapter 13 Generalized Linear Models.](https://reader033.fdocuments.in/reader033/viewer/2022061520/56649ddb5503460f94ad2066/html5/thumbnails/49.jpg)
Linear Regression Analysis 5E Montgomery, Peck & Vining
49
Other models for binary response data
Logit model
Probit model
Complimentary log-log model
![Page 50: Linear Regression Analysis 5E Montgomery, Peck & Vining 1 Chapter 13 Generalized Linear Models.](https://reader033.fdocuments.in/reader033/viewer/2022061520/56649ddb5503460f94ad2066/html5/thumbnails/50.jpg)
Linear Regression Analysis 5E Montgomery, Peck & Vining
50
![Page 51: Linear Regression Analysis 5E Montgomery, Peck & Vining 1 Chapter 13 Generalized Linear Models.](https://reader033.fdocuments.in/reader033/viewer/2022061520/56649ddb5503460f94ad2066/html5/thumbnails/51.jpg)
Linear Regression Analysis 5E Montgomery, Peck & Vining
51
More than two categorical outcomes
![Page 52: Linear Regression Analysis 5E Montgomery, Peck & Vining 1 Chapter 13 Generalized Linear Models.](https://reader033.fdocuments.in/reader033/viewer/2022061520/56649ddb5503460f94ad2066/html5/thumbnails/52.jpg)
Linear Regression Analysis 5E Montgomery, Peck & Vining
52
![Page 53: Linear Regression Analysis 5E Montgomery, Peck & Vining 1 Chapter 13 Generalized Linear Models.](https://reader033.fdocuments.in/reader033/viewer/2022061520/56649ddb5503460f94ad2066/html5/thumbnails/53.jpg)
Linear Regression Analysis 5E Montgomery, Peck & Vining
53
![Page 54: Linear Regression Analysis 5E Montgomery, Peck & Vining 1 Chapter 13 Generalized Linear Models.](https://reader033.fdocuments.in/reader033/viewer/2022061520/56649ddb5503460f94ad2066/html5/thumbnails/54.jpg)
Linear Regression Analysis 5E Montgomery, Peck & Vining
54
Poisson Regression• Consider now the case where the response is a count of
some relatively rare event:– Defects in a unit of product
– Software bugs
– Particulate matter or some pollutant in the environment
– Number of Atlantic hurricanes
• We wish to model the relationship between the count response and one or more regressor or predictor variables
• A logical model for the count response is the Poisson distribution
( ) , 0,1,..., and 0!
yef y y
y
![Page 55: Linear Regression Analysis 5E Montgomery, Peck & Vining 1 Chapter 13 Generalized Linear Models.](https://reader033.fdocuments.in/reader033/viewer/2022061520/56649ddb5503460f94ad2066/html5/thumbnails/55.jpg)
Linear Regression Analysis 5E Montgomery, Peck & Vining
55
Poisson Regression• Poisson regression is another case where the response
variance is related to the mean; in fact, in the Poisson distribution
• The Poisson regression model is
• We assume that there is a function g that relates the mean of the response to a linear predictor
( ) and ( )E y Var y
( ) , 1, 2,...,i i i i iy E y i n
0 1
( )
...i i
i i k ik
i
g
x x
x
![Page 56: Linear Regression Analysis 5E Montgomery, Peck & Vining 1 Chapter 13 Generalized Linear Models.](https://reader033.fdocuments.in/reader033/viewer/2022061520/56649ddb5503460f94ad2066/html5/thumbnails/56.jpg)
Linear Regression Analysis 5E Montgomery, Peck & Vining
56
Poisson Regression
• The function g is called a link function• The relationship between the mean of the response
distribution and the linear predictor is
• Choice of the link function:– Identity link
– Log link (very logical for the Poisson-no negative predicted values)
1 1( ) (i i ig g x
1
( ) ln( )
( i
i i i
i i
g
g e
x
x
x
![Page 57: Linear Regression Analysis 5E Montgomery, Peck & Vining 1 Chapter 13 Generalized Linear Models.](https://reader033.fdocuments.in/reader033/viewer/2022061520/56649ddb5503460f94ad2066/html5/thumbnails/57.jpg)
Linear Regression Analysis 5E Montgomery, Peck & Vining
57
Poisson Regression• The usual form of the Poisson regression model is
• This is a special case of the GLM; Poisson response and a log link
• Parameter estimation in Poisson regression is essentially equivalent to logistic regression; maximum likelihood, implemented by IRLS
• Wald (large sample) and Deviance (likelihood-based) based inference is carried out the same way as in the logistic regression model
, 1, 2,...,ii iy e i n x
![Page 58: Linear Regression Analysis 5E Montgomery, Peck & Vining 1 Chapter 13 Generalized Linear Models.](https://reader033.fdocuments.in/reader033/viewer/2022061520/56649ddb5503460f94ad2066/html5/thumbnails/58.jpg)
Linear Regression Analysis 5E Montgomery, Peck & Vining
58
An Example of Poisson Regression
• The aircraft damage data
• Response y = the number of locations where damage was inflicted on the aircraft
• Regressors:
1
2
3
0 = A-4 type of aircraft
1 = A-6
bomb load (tons)
total months of crew experience
x
x
x
![Page 59: Linear Regression Analysis 5E Montgomery, Peck & Vining 1 Chapter 13 Generalized Linear Models.](https://reader033.fdocuments.in/reader033/viewer/2022061520/56649ddb5503460f94ad2066/html5/thumbnails/59.jpg)
Linear Regression Analysis 5E Montgomery, Peck & Vining
59
The table contains data from 30 strike missions
There is a lot of multicollinearity in this data; the A-6 has a two-man crew and is capable of carrying a heavier bomb load
All three regressors tend to increase monotonically
![Page 60: Linear Regression Analysis 5E Montgomery, Peck & Vining 1 Chapter 13 Generalized Linear Models.](https://reader033.fdocuments.in/reader033/viewer/2022061520/56649ddb5503460f94ad2066/html5/thumbnails/60.jpg)
Linear Regression Analysis 5E Montgomery, Peck & Vining
60
Based on the full model, we can remove x3
However, when x3 is removed, x1 (type of aircraft) is no longer significant – this is not shown, but easily verified
This is probably multicollinearity at work
Note the Type 1 and Type 3 analyses for each variable
Note also that the P-values for the Wald tests and the Type 3 analysis (based on deviance) don’t agree
![Page 61: Linear Regression Analysis 5E Montgomery, Peck & Vining 1 Chapter 13 Generalized Linear Models.](https://reader033.fdocuments.in/reader033/viewer/2022061520/56649ddb5503460f94ad2066/html5/thumbnails/61.jpg)
Linear Regression Analysis 5E Montgomery, Peck & Vining
61
Let’s consider all of the subset regression models:
Deleting either x1 or x2 results in a two-variable model that is worse than the full model
Removing x3 gives a model equivalent to the full model, but as noted before, x1 is insignificant
One of the single-variable models (x2) is equivalent to the full model
![Page 62: Linear Regression Analysis 5E Montgomery, Peck & Vining 1 Chapter 13 Generalized Linear Models.](https://reader033.fdocuments.in/reader033/viewer/2022061520/56649ddb5503460f94ad2066/html5/thumbnails/62.jpg)
Linear Regression Analysis 5E Montgomery, Peck & Vining
62
The one-variable model with x2 displays no lack of fit (Deviance/df = 1.1791)
The prediction equation is
21.6491 0.2282ˆ xy e
![Page 63: Linear Regression Analysis 5E Montgomery, Peck & Vining 1 Chapter 13 Generalized Linear Models.](https://reader033.fdocuments.in/reader033/viewer/2022061520/56649ddb5503460f94ad2066/html5/thumbnails/63.jpg)
Linear Regression Analysis 5E Montgomery, Peck & Vining
63
Another Example Involving Poisson Regression
•The mine fracture data•The response is a count of the number of fractures in the mine•The regressors are:
1
2
3
4
inner burden thickness (feet)
Percent extraction of the lower
previously mined seam
Lower seam height (feet)
Time in years that mine has been open
x
x
x
x
![Page 64: Linear Regression Analysis 5E Montgomery, Peck & Vining 1 Chapter 13 Generalized Linear Models.](https://reader033.fdocuments.in/reader033/viewer/2022061520/56649ddb5503460f94ad2066/html5/thumbnails/64.jpg)
Linear Regression Analysis 5E Montgomery, Peck & Vining
64
The * indicates the best model of a specific subset size
Note that the addition of a term cannot increase the deviance (promoting the analog between deviance and the “usual” residual sum of squares)
To compare the model with only x1, x2, and x4 to the full model, evaluate the difference in deviance:
38.03 - 37.86 = 0.17
with 1 df. This is not significant.
![Page 65: Linear Regression Analysis 5E Montgomery, Peck & Vining 1 Chapter 13 Generalized Linear Models.](https://reader033.fdocuments.in/reader033/viewer/2022061520/56649ddb5503460f94ad2066/html5/thumbnails/65.jpg)
Linear Regression Analysis 5E Montgomery, Peck & Vining
65
There is no indication of lack of fit: deviance/df = 0.9508
The final model is:
1 2 43.721 0.0015 0.0627 0.0317ˆ x x xy e
![Page 66: Linear Regression Analysis 5E Montgomery, Peck & Vining 1 Chapter 13 Generalized Linear Models.](https://reader033.fdocuments.in/reader033/viewer/2022061520/56649ddb5503460f94ad2066/html5/thumbnails/66.jpg)
Linear Regression Analysis 5E Montgomery, Peck & Vining
66
The Generalized Linear Model• Poisson and logistic regression are two special cases of the GLM:
– Binomial response with a logistic link
– Poisson response with a log link
• In the GLM, the response distribution must be a member of the exponential family:
• This includes the binomial, Poisson, normal, inverse normal, exponential, and gamma distributions
( , , ) exp{[ ( )] / ( ) ( , )}
scale parameter
natural location parameter(s)
i i i i i i
i
f y y b a h y
![Page 67: Linear Regression Analysis 5E Montgomery, Peck & Vining 1 Chapter 13 Generalized Linear Models.](https://reader033.fdocuments.in/reader033/viewer/2022061520/56649ddb5503460f94ad2066/html5/thumbnails/67.jpg)
Linear Regression Analysis 5E Montgomery, Peck & Vining
67
The Generalized Linear Model• The relationship between the mean of the response
distribution and the linear predictor is determined by the link function
• The canonical link is specified when
• The canonical link depends on the choice of the response distribution
1 1( ) (i i ig g x
i i
![Page 68: Linear Regression Analysis 5E Montgomery, Peck & Vining 1 Chapter 13 Generalized Linear Models.](https://reader033.fdocuments.in/reader033/viewer/2022061520/56649ddb5503460f94ad2066/html5/thumbnails/68.jpg)
Linear Regression Analysis 5E Montgomery, Peck & Vining
68
Canonical Links for the GLM
![Page 69: Linear Regression Analysis 5E Montgomery, Peck & Vining 1 Chapter 13 Generalized Linear Models.](https://reader033.fdocuments.in/reader033/viewer/2022061520/56649ddb5503460f94ad2066/html5/thumbnails/69.jpg)
Linear Regression Analysis 5E Montgomery, Peck & Vining
69
Links for the GLM
• You do not have to use the canonical link, it just simplifies some of the mathematics
• In fact, the log (non-canonical) link is very often used with the exponential and gamma distributions, especially when the response variable is nonnegative
• Other links can be based on the power family (as in power family transformations), or the complimentary log-log function
![Page 70: Linear Regression Analysis 5E Montgomery, Peck & Vining 1 Chapter 13 Generalized Linear Models.](https://reader033.fdocuments.in/reader033/viewer/2022061520/56649ddb5503460f94ad2066/html5/thumbnails/70.jpg)
Linear Regression Analysis 5E Montgomery, Peck & Vining
70
Parameter Estimation and Inference in the GLM
• Estimation is by maximum likelihood (and IRLS); for the canonical link the score function is
• For the case of a non-canonical link,
• Wald inference and deviance-based inference is conducted just as in logistic and Poisson regression
( X y 0
(
( / )i idiag d d X y 0
![Page 71: Linear Regression Analysis 5E Montgomery, Peck & Vining 1 Chapter 13 Generalized Linear Models.](https://reader033.fdocuments.in/reader033/viewer/2022061520/56649ddb5503460f94ad2066/html5/thumbnails/71.jpg)
Linear Regression Analysis 5E Montgomery, Peck & Vining
71
This is “classical data”; analyzed by many.
y = cycles to failure, x1 = cycle length, x2 = amplitude, x3 = load
The experimental design is a 33 factorial
Most analysts begin by fitting a full quadratic model using ordinary least squares
![Page 72: Linear Regression Analysis 5E Montgomery, Peck & Vining 1 Chapter 13 Generalized Linear Models.](https://reader033.fdocuments.in/reader033/viewer/2022061520/56649ddb5503460f94ad2066/html5/thumbnails/72.jpg)
Linear Regression Analysis 5E Montgomery, Peck & Vining
72
DESIGN-EXPERT PlotCycles
LambdaCurrent = 1Best = -0.19Low C.I. = -0.54High C.I. = 0.22
Recommend transform:Log (Lambda = 0)
Lambda
Ln
(Re
sid
ua
lSS
)
Box-Cox Plot for Power Transforms
12.18
14.27
16.37
18.46
20.56
-3 -2 -1 0 1 2 3
Design-Expert V6 was used to analyze the data
A log transform is suggested
![Page 73: Linear Regression Analysis 5E Montgomery, Peck & Vining 1 Chapter 13 Generalized Linear Models.](https://reader033.fdocuments.in/reader033/viewer/2022061520/56649ddb5503460f94ad2066/html5/thumbnails/73.jpg)
Linear Regression Analysis 5E Montgomery, Peck & Vining
73
Response: Cycles Transform: Natural log Constant: 0.000 ANOVA for Response Surface Linear ModelAnalysis of variance table [Partial sum of squares]
Sum of Mean FSource Squares DF Square Value Prob > FModel 22.32 3 7.44 213.50 < 0.0001A 12.47 1 12.47 357.87 < 0.0001B 7.11 1 7.11 204.04 < 0.0001C 2.74 1 2.74 78.57 < 0.0001Residual 0.80 23 0.035Cor Total 23.12 26
Std. Dev. 0.19 R-Squared 0.9653Mean 6.34 Adj R-Squared 0.9608C.V. 2.95 Pred R-Squared 0.9520
PRESS 1.11 Adeq Precision 51.520
Coefficient Standard 95% CI 95% CIFactor Estimate DF Error Low High
Intercept 6.34 1 0.036 6.26 6.41
A-A 0.83 1 0.044 0.74 0.92 B-B -0.63 1 0.044 -0.72 -0.54 C-C -0.39 1 0.044 -0.48 -0.30
The Final Model is First-Order:
1 2 36.34 0.83 0.63 0.39ˆ x x xy e
![Page 74: Linear Regression Analysis 5E Montgomery, Peck & Vining 1 Chapter 13 Generalized Linear Models.](https://reader033.fdocuments.in/reader033/viewer/2022061520/56649ddb5503460f94ad2066/html5/thumbnails/74.jpg)
Linear Regression Analysis 5E Montgomery, Peck & Vining
74
DESIGN-EXPERT Plot
Ln(Cycles)Design Points
X = A: AY = B: B
Actual FactorC: C = -1.00
Ln(Cycles)
A: A
B: B
-1.00 -0.50 0.00 0.50 1.00
-1.00
-0.50
0.00
0.50
1.00
5.84934
6.33631
6.82328
7.31025
7.89149
DESIGN-EXPERT Plot
Ln(Cycles)X = A: AY = B: B
Actual FactorC: C = -1.00
193.529
1043.84
1894.16
2744.48
3594.79
C
ycle
s
-1.00
-0.50
0.00
0.50
1.00
-1.00
-0.50
0.00
0.50
1.00
A: A B: B
Contour plot (log cycles) & response surface (cycles)
![Page 75: Linear Regression Analysis 5E Montgomery, Peck & Vining 1 Chapter 13 Generalized Linear Models.](https://reader033.fdocuments.in/reader033/viewer/2022061520/56649ddb5503460f94ad2066/html5/thumbnails/75.jpg)
Linear Regression Analysis 5E Montgomery, Peck & Vining
75
A GLM for the Worsted Yarn Data
• We selected a gamma response distribution with a log link
• The resulting GLM (from SAS) is
• Model is adequate; little difference between GLM & OLS
• Contour plots (predictions) very similar
1 2 36.3489 0.8425 0.6313 0.3851ˆ x x xy e
![Page 76: Linear Regression Analysis 5E Montgomery, Peck & Vining 1 Chapter 13 Generalized Linear Models.](https://reader033.fdocuments.in/reader033/viewer/2022061520/56649ddb5503460f94ad2066/html5/thumbnails/76.jpg)
Linear Regression Analysis 5E Montgomery, Peck & Vining
76
The SAS PROC GENMOD output for the worsted yarn experiment, assuming a first-order model in the linear predictor
Scaled deviance divided by df is the appropriate lack of fit measure in the gamma response situation
![Page 77: Linear Regression Analysis 5E Montgomery, Peck & Vining 1 Chapter 13 Generalized Linear Models.](https://reader033.fdocuments.in/reader033/viewer/2022061520/56649ddb5503460f94ad2066/html5/thumbnails/77.jpg)
Linear Regression Analysis 5E Montgomery, Peck & Vining
77
Comparison of the OLS and GLM Models
![Page 78: Linear Regression Analysis 5E Montgomery, Peck & Vining 1 Chapter 13 Generalized Linear Models.](https://reader033.fdocuments.in/reader033/viewer/2022061520/56649ddb5503460f94ad2066/html5/thumbnails/78.jpg)
Linear Regression Analysis 5E Montgomery, Peck & Vining
78
A GLM for the Worsted Yarn Data
• Confidence intervals on the mean response are uniformly shorter from the GLM than from least squares
• See Lewis, S. L., Montgomery, D. C., and Myers, R. H. (2001), “Confidence Interval Coverage for Designed Experiments Analyzed with GLMs”, JQT, 33, pp. 279-292
• While point estimates are very similar, the GLM provides better precision of estimation
![Page 79: Linear Regression Analysis 5E Montgomery, Peck & Vining 1 Chapter 13 Generalized Linear Models.](https://reader033.fdocuments.in/reader033/viewer/2022061520/56649ddb5503460f94ad2066/html5/thumbnails/79.jpg)
Linear Regression Analysis 5E Montgomery, Peck & Vining
79
Residual Analysis in the GLM• Analysis of residuals is important in any model-fitting
procedure
• The ordinary or raw residuals are not the best choice for the GLM, because the approximate normality and constant variance assumptions are not satisfied
• Typically, deviance residuals are employed for model adequacy checking in the GLM.
• The deviance residuals are the square roots of the contribution to the deviance from each observation, multiplied by the sign of the corresponding raw residual:
ˆ( )iD i i ir d sign y y
![Page 80: Linear Regression Analysis 5E Montgomery, Peck & Vining 1 Chapter 13 Generalized Linear Models.](https://reader033.fdocuments.in/reader033/viewer/2022061520/56649ddb5503460f94ad2066/html5/thumbnails/80.jpg)
Linear Regression Analysis 5E Montgomery, Peck & Vining
80
Deviance Residuals:• Logistic regression:
• Poisson regression:
ˆ
1 ( / )ln ( )
ˆ ˆ1
1ˆ
1 i
i i ii i i i
i i i
i
y y nd y n y
n
e
x
ˆ
ˆln ( )i
i
ii i i
yd y y e
e
x
x
![Page 81: Linear Regression Analysis 5E Montgomery, Peck & Vining 1 Chapter 13 Generalized Linear Models.](https://reader033.fdocuments.in/reader033/viewer/2022061520/56649ddb5503460f94ad2066/html5/thumbnails/81.jpg)
Linear Regression Analysis 5E Montgomery, Peck & Vining
81
Deviance Residual Plots
• Deviance residuals behave much like ordinary residual in normal-theory linear models
• Normal probability plot is appropriate• Plot versus fitted values, usually transformed to
the constant-information scale:
1
ˆNormal responses,
ˆBinomial responses, 2sin ( )
ˆPoisson responses, 2
ˆGamma responses, 2 ln( )
i
i
i
i
y
y
y
y
![Page 82: Linear Regression Analysis 5E Montgomery, Peck & Vining 1 Chapter 13 Generalized Linear Models.](https://reader033.fdocuments.in/reader033/viewer/2022061520/56649ddb5503460f94ad2066/html5/thumbnails/82.jpg)
Linear Regression Analysis 5E Montgomery, Peck & Vining
82
![Page 83: Linear Regression Analysis 5E Montgomery, Peck & Vining 1 Chapter 13 Generalized Linear Models.](https://reader033.fdocuments.in/reader033/viewer/2022061520/56649ddb5503460f94ad2066/html5/thumbnails/83.jpg)
Linear Regression Analysis 5E Montgomery, Peck & Vining
83
Deviance Residual Plots for the Worsted Yarn Experiment
![Page 84: Linear Regression Analysis 5E Montgomery, Peck & Vining 1 Chapter 13 Generalized Linear Models.](https://reader033.fdocuments.in/reader033/viewer/2022061520/56649ddb5503460f94ad2066/html5/thumbnails/84.jpg)
Linear Regression Analysis 5E Montgomery, Peck & Vining
84
Overdispersion
• Occurs occasionally with Poisson or binomial data• The variance of the response is greater than one would
anticipate based on the choice of response distribution• For example, in the Poisson distribution, we expect the
variance to be approximately equal to the mean – if the observed variance is greater, this indicates overdispersion
• Diagnosis – if deviance/df greatly exceeds unity, overdispersion may be present
• There may be other reasons for deviance/df to be large, such as a poorly specified model, missing regressors, etc (the same things that cause the mean square for error to be inflated in ordinary least squares modeling)
![Page 85: Linear Regression Analysis 5E Montgomery, Peck & Vining 1 Chapter 13 Generalized Linear Models.](https://reader033.fdocuments.in/reader033/viewer/2022061520/56649ddb5503460f94ad2066/html5/thumbnails/85.jpg)
Linear Regression Analysis 5E Montgomery, Peck & Vining
85
Overdispersion• Most direct way to model overdispersion is with a
multiplicative dispersion parameter, say , where
• A logical estimate for is deviance/df• Unless overdispersion is accounted for, the standard errors
will be too small.• The adjustment consists of multiplying the standard errors
by
( ) (1 ), binomial
( ) , Poisson
Var y
Var y
deviance/df
![Page 86: Linear Regression Analysis 5E Montgomery, Peck & Vining 1 Chapter 13 Generalized Linear Models.](https://reader033.fdocuments.in/reader033/viewer/2022061520/56649ddb5503460f94ad2066/html5/thumbnails/86.jpg)
Linear Regression Analysis 5E Montgomery, Peck & Vining
86
The Wave-Soldering Experiment
• Response is the number of defects• Seven design variables:
– A = prebake condition
– B = flux denisty
– C = conveyor speed
– D = preheat condition
– E = cooling time
– F = ultrasonic solder agitator
– G = solder temperature
![Page 87: Linear Regression Analysis 5E Montgomery, Peck & Vining 1 Chapter 13 Generalized Linear Models.](https://reader033.fdocuments.in/reader033/viewer/2022061520/56649ddb5503460f94ad2066/html5/thumbnails/87.jpg)
Linear Regression Analysis 5E Montgomery, Peck & Vining
87
The Wave-Soldering Experiment
One observation has been discarded, as it was suspected to be an outlier
This is a resolution IV design
![Page 88: Linear Regression Analysis 5E Montgomery, Peck & Vining 1 Chapter 13 Generalized Linear Models.](https://reader033.fdocuments.in/reader033/viewer/2022061520/56649ddb5503460f94ad2066/html5/thumbnails/88.jpg)
Linear Regression Analysis 5E Montgomery, Peck & Vining
88
The Wave-Soldering Experiment
5 of 7 main effects significant; AC, AD, BC, and BD also significant
Overdispersion is a possible problem, as deviance/df is large
Overdispersion causes standard errors to be underestimated, and this could lead to identifying too many effects as significant
/
4.234 2.0577
deviance df
![Page 89: Linear Regression Analysis 5E Montgomery, Peck & Vining 1 Chapter 13 Generalized Linear Models.](https://reader033.fdocuments.in/reader033/viewer/2022061520/56649ddb5503460f94ad2066/html5/thumbnails/89.jpg)
Linear Regression Analysis 5E Montgomery, Peck & Vining
89
After adjusting for overdispersion, fewer effects are significant
C, G, AC, and BD the important factors, assuming a 5% significance level
Note that the standard errors are larger than they were before, having been multiplied by
/
4.234 2.0577
deviance df
![Page 90: Linear Regression Analysis 5E Montgomery, Peck & Vining 1 Chapter 13 Generalized Linear Models.](https://reader033.fdocuments.in/reader033/viewer/2022061520/56649ddb5503460f94ad2066/html5/thumbnails/90.jpg)
Linear Regression Analysis 5E Montgomery, Peck & Vining
90
The Edited Model for the
Wave-Soldering Experiment
![Page 91: Linear Regression Analysis 5E Montgomery, Peck & Vining 1 Chapter 13 Generalized Linear Models.](https://reader033.fdocuments.in/reader033/viewer/2022061520/56649ddb5503460f94ad2066/html5/thumbnails/91.jpg)
Linear Regression Analysis 5E Montgomery, Peck & Vining
91
Generalized Linear Models
• The GLM is a unification of linear and nonlinear models that can accommodate a wide variety of response distributions
• Can be used with both regression models and designed experiments
• Computer implementations in Minitab, JMP, SAS (PROC GENMOD), S-Plus
• Logistic regression available in many basic packages• GLMs are a useful alternative to data transformation, and
should always be considered when data transformations are not entirely satisfactory
• Unlike data transformations, GLMs directly attack the unequal variance problem and use the maximum likelihood approach to account for the form of the response distribution