COM2023
Mathematics Methods for
Computing II
Lecture 17& 18
Gianne DerksDepartment of Mathematics (36AA04)
http://www.maths.surrey.ac.uk/Modules/COM2023
Autumn 2010
Use channel 04 on your EVS handset
OverviewLinear regression Example Predictions Matlab Quadratic regression Multiple regression Other nonlinear regression
● Regression modelling
◆ Linear regression
● Recap
● Example
● Predictions
● R and linear regression
◆ Quadratic regression
◆ Other non-linear regression
◆ Multiple regression
Linear regression model (§6.1)
Linear regression Example Predictions Matlab Quadratic regression Multiple regression Other nonlinear regression
Linear regression modelling: recapLinear regression Example Predictions Matlab Quadratic regression Multiple regression Other nonlinear regression
Given a data set {(x1, y1), (x2, y2), . . . , (xn, yn)}, which looks linear. Todetermine the linear regression line:
● Determine the means x = 1n
n∑
i=1
xi and y = 1n
n∑
i=1
yi;
● Determine the quantities
Sxx =
n∑
i=1
(xi − x)2 and Sxy =
n∑
i=1
(xi − x)(yi − y);
● The slope is b =Sxy
Sxx
;
● The intercept is a = y − bx;
● The least squares regression line of y on x is y = a+ bx;
● The residuals are εi = yi − (a+ bxi). Check that they are normallydistributed.
Forbes’ data: linear regression plotLinear regression Example Predictions Matlab Quadratic regression Multiple regression Other nonlinear regression
22 24 26 28 30
195
200
205
210
atmospheric pressure (inch Hg)
boili
ng p
oint
(F
)
AAAAK
boiling point = 155.3 + 1.9 × atmospheric pressure
Forbes’ data: normal residual checkLinear regression Example Predictions Matlab Quadratic regression Multiple regression Other nonlinear regression
To check the assumption that the noise in Forbes’ data is normallydistributed, we plot the quantiles of the residuals against the theoreticalquantiles of a normal distribution (QQ plot).
−2 −1 0 1 2
−1.
0−
0.5
0.0
0.5
Theoretical quantiles
For
bes’
dat
a qu
antil
es
Example
Linear regression Example Predictions Matlab Quadratic regression Multiple regression Other nonlinear regression
Example: Properties of Synthetic RubberLinear regression Example Predictions Matlab Quadratic regression Multiple regression Other nonlinear regression
The following table gives the hardness and the abrasion loss of twelve differentspecimens of synthetic rubber.
Specimen Hardness x Abrasion loss y Product x× y
1 45 372 167402 55 206 113303 61 175 106754 68 113 76845 59 249 146916 74 145 107307 81 55 44558 86 45 44559 83 97 8051
10 71 219 1554911 53 221 1171312 88 64 5632
Properties of Synthetic Rubber: plotLinear regression Example Predictions Matlab Quadratic regression Multiple regression Other nonlinear regression
50 60 70 80
5010
015
020
025
030
035
0
Hardness x
Abr
asio
n lo
ss y
Synthetic Rubber: linear regression lineLinear regression Example Predictions Matlab Quadratic regression Multiple regression Other nonlinear regression
The totals of the data are as follows (n=12):
Tx =
n∑
i=1
xi = 824, Ty =
n∑
i=1
yi = 1961 and Txy =
n∑
i=1
xiyi = 121120.
And,
Txx =n∑
i=1
x2i = 58792 and Tyy =n∑
i=1
y2i = 422597.
Using this, we get
Sxx = Txx −1nT 2x = and Sxy = Txy −
1nTxTy =
andx = and y =
Thus b =Sxy
Sxx= , a = y − bx = and the line is
Synthetic Rubber: linear regression lineLinear regression Example Predictions Matlab Quadratic regression Multiple regression Other nonlinear regression
The totals of the data are as follows (n=12):
Tx =
n∑
i=1
xi = 824, Ty =
n∑
i=1
yi = 1961 and Txy =
n∑
i=1
xiyi = 121120.
And,
Txx =n∑
i=1
x2i = 58792 and Tyy =n∑
i=1
y2i = 422597.
Using this, we get
Sxx = Txx −1nT 2x = 2210.667 and Sxy = Txy −
1nTxTy =
andx = and y =
Thus b =Sxy
Sxx= , a = y − bx = and the line is
Synthetic Rubber: linear regression lineLinear regression Example Predictions Matlab Quadratic regression Multiple regression Other nonlinear regression
The totals of the data are as follows (n=12):
Tx =
n∑
i=1
xi = 824, Ty =
n∑
i=1
yi = 1961 and Txy =
n∑
i=1
xiyi = 121120.
And,
Txx =n∑
i=1
x2i = 58792 and Tyy =n∑
i=1
y2i = 422597.
Using this, we get
Sxx = Txx −1nT 2x = 2210.667 and Sxy = Txy −
1nTxTy = −13535.33.
andx = and y =
Thus b =Sxy
Sxx= , a = y − bx = and the line is
Synthetic Rubber: linear regression lineLinear regression Example Predictions Matlab Quadratic regression Multiple regression Other nonlinear regression
The totals of the data are as follows (n=12):
Tx =
n∑
i=1
xi = 824, Ty =
n∑
i=1
yi = 1961 and Txy =
n∑
i=1
xiyi = 121120.
And,
Txx =n∑
i=1
x2i = 58792 and Tyy =n∑
i=1
y2i = 422597.
Using this, we get
Sxx = Txx −1nT 2x = 2210.667 and Sxy = Txy −
1nTxTy = −13535.33.
andx = 68.66667 and y =
Thus b =Sxy
Sxx= , a = y − bx = and the line is
Synthetic Rubber: linear regression lineLinear regression Example Predictions Matlab Quadratic regression Multiple regression Other nonlinear regression
The totals of the data are as follows (n=12):
Tx =
n∑
i=1
xi = 824, Ty =
n∑
i=1
yi = 1961 and Txy =
n∑
i=1
xiyi = 121120.
And,
Txx =n∑
i=1
x2i = 58792 and Tyy =n∑
i=1
y2i = 422597.
Using this, we get
Sxx = Txx −1nT 2x = 2210.667 and Sxy = Txy −
1nTxTy = −13535.33.
andx = 68.66667 and y = 163.4167.
Thus b =Sxy
Sxx= , a = y − bx = and the line is
Synthetic Rubber: linear regression lineLinear regression Example Predictions Matlab Quadratic regression Multiple regression Other nonlinear regression
The totals of the data are as follows (n=12):
Tx =
n∑
i=1
xi = 824, Ty =
n∑
i=1
yi = 1961 and Txy =
n∑
i=1
xiyi = 121120.
And,
Txx =n∑
i=1
x2i = 58792 and Tyy =n∑
i=1
y2i = 422597.
Using this, we get
Sxx = Txx −1nT 2x = 2210.667 and Sxy = Txy −
1nTxTy = −13535.33.
andx = 68.66667 and y = 163.4167.
Thus b =Sxy
Sxx= −6.123, a = y − bx = and the line is
Synthetic Rubber: linear regression lineLinear regression Example Predictions Matlab Quadratic regression Multiple regression Other nonlinear regression
The totals of the data are as follows (n=12):
Tx =
n∑
i=1
xi = 824, Ty =
n∑
i=1
yi = 1961 and Txy =
n∑
i=1
xiyi = 121120.
And,
Txx =n∑
i=1
x2i = 58792 and Tyy =n∑
i=1
y2i = 422597.
Using this, we get
Sxx = Txx −1nT 2x = 2210.667 and Sxy = Txy −
1nTxTy = −13535.33.
andx = 68.66667 and y = 163.4167.
Thus b =Sxy
Sxx= −6.123, a = y − bx = 583.845 and the line is
Synthetic Rubber: linear regression lineLinear regression Example Predictions Matlab Quadratic regression Multiple regression Other nonlinear regression
The totals of the data are as follows (n=12):
Tx =
n∑
i=1
xi = 824, Ty =
n∑
i=1
yi = 1961 and Txy =
n∑
i=1
xiyi = 121120.
And,
Txx =n∑
i=1
x2i = 58792 and Tyy =n∑
i=1
y2i = 422597.
Using this, we get
Sxx = Txx −1nT 2x = 2210.667 and Sxy = Txy −
1nTxTy = −13535.33.
andx = 68.66667 and y = 163.4167.
Thus b =Sxy
Sxx= −6.123, a = y − bx = 583.845 and the line is
Abrasion loss = −6.123× Hardness+ 583.845.
Synthetic Rubber: linear regression plotLinear regression Example Predictions Matlab Quadratic regression Multiple regression Other nonlinear regression
50 60 70 80
5010
015
020
025
030
035
0
Hardness x
Abr
asio
n lo
ss y
�����
abrasion loss = 583.845− 6.123 × hardness
Synthetic Rubber: normal residual checkLinear regression Example Predictions Matlab Quadratic regression Multiple regression Other nonlinear regression
−1.5 −1.0 −0.5 0.0 0.5 1.0 1.5
−40
−20
020
4060
Normal Q−Q Plot
Theoretical Quantiles
Sam
ple
Qua
ntile
s
Predictions (§6.1, page 44-45)
Linear regression Example Predictions Matlab Quadratic regression Multiple regression Other nonlinear regression
Prediction for regression modellingLinear regression Example Predictions Matlab Quadratic regression Multiple regression Other nonlinear regression
● In order to predict a future y-value, y say, at a given x-value, x∗ say, usethe fitted model and take
y = a+ bx∗.
Prediction for regression modellingLinear regression Example Predictions Matlab Quadratic regression Multiple regression Other nonlinear regression
● In order to predict a future y-value, y say, at a given x-value, x∗ say, usethe fitted model and take
y = a+ bx∗.
● The fitted model can usually safely be used for interpolation, i.e., x∗ isan x-value inside the minimum and maximum sample x-values.
Prediction for regression modellingLinear regression Example Predictions Matlab Quadratic regression Multiple regression Other nonlinear regression
● In order to predict a future y-value, y say, at a given x-value, x∗ say, usethe fitted model and take
y = a+ bx∗.
● The fitted model can usually safely be used for interpolation, i.e., x∗ isan x-value inside the minimum and maximum sample x-values.
● Be very cautious about using the fitted straight line model forextrapolation. There is no guarantee that the model will be valid or evenpossible for such an x-value outside the sample x-values.
Prediction for regression modellingLinear regression Example Predictions Matlab Quadratic regression Multiple regression Other nonlinear regression
● In order to predict a future y-value, y say, at a given x-value, x∗ say, usethe fitted model and take
y = a+ bx∗.
● The fitted model can usually safely be used for interpolation, i.e., x∗ isan x-value inside the minimum and maximum sample x-values.
● Be very cautious about using the fitted straight line model forextrapolation. There is no guarantee that the model will be valid or evenpossible for such an x-value outside the sample x-values.
● The prediction is likely to be imprecise if
◆ the variation of the residuals is large;
◆ If the number of data points n is small.
Synthetic Rubber example, predictionLinear regression Example Predictions Matlab Quadratic regression Multiple regression Other nonlinear regression
Item Hardness x Abrasion loss y
1 45 3722 55 2063 61 1754 68 1135 59 2496 74 1457 81 558 86 459 83 9710 71 21911 53 22112 88 64
Linear regression line:
y = 584− 6× x
Predictions:
● What is the expected abrasion lossat hardness 50?
Synthetic Rubber example, predictionLinear regression Example Predictions Matlab Quadratic regression Multiple regression Other nonlinear regression
Item Hardness x Abrasion loss y
1 45 3722 55 2063 61 1754 68 1135 59 2496 74 1457 81 558 86 459 83 9710 71 21911 53 22112 88 64
Linear regression line:
y = 584− 6× x
Predictions:
● What is the expected abrasion lossat hardness 50?Take x = 50 in regression line,then y = 284, thus the expectedabrasion loss is 284.
Synthetic Rubber example, predictionLinear regression Example Predictions Matlab Quadratic regression Multiple regression Other nonlinear regression
Item Hardness x Abrasion loss y
1 45 3722 55 2063 61 1754 68 1135 59 2496 74 1457 81 558 86 459 83 9710 71 21911 53 22112 88 64
Linear regression line:
y = 584− 6× x
Predictions:
● What is the expected abrasion lossat hardness 50?Take x = 50 in regression line,then y = 284, thus the expectedabrasion loss is 284.
● What is the expected abrabsionloss at hardness 100?
Synthetic Rubber example, predictionLinear regression Example Predictions Matlab Quadratic regression Multiple regression Other nonlinear regression
Item Hardness x Abrasion loss y
1 45 3722 55 2063 61 1754 68 1135 59 2496 74 1457 81 558 86 459 83 9710 71 21911 53 22112 88 64
Linear regression line:
y = 584− 6× x
Predictions:
● What is the expected abrasion lossat hardness 50?Take x = 50 in regression line,then y = 284, thus the expectedabrasion loss is 284.
● What is the expected abrabsionloss at hardness 100?We are not sure if the model isvalid for hardness less than 45 orlarger than 88. The answer fromthe regression line is y = 584 −600 = −16, hence not realistic.
Regression and Matlab (§6.1, p.
41-43)
Linear regression Example Predictions Matlab Quadratic regression Multiple regression Other nonlinear regression
“Women” data set in Matlab
Linear regression Example Predictions Matlab Quadratic regression Multiple regression Other nonlinear regression
Weight and height of a sample group of women:
58 60 62 64 66 68 70 72
120
130
140
150
160
height
wei
ght
Analysis of data sets in Matlab
Linear regression Example Predictions Matlab Quadratic regression Multiple regression Other nonlinear regression
● The weight vector is such that weight[i] lbs is the average weight ofwomen who are height[i] inches tall.
55 60 65 70 75100
120
140
160
180W
eigh
t (lb
)
Height (in)
Analysis of data sets in Matlab
Linear regression Example Predictions Matlab Quadratic regression Multiple regression Other nonlinear regression
● The weight vector is such that weight[i] lbs is the average weight ofwomen who are height[i] inches tall.
● Check the correlation and rank correlation:
◆ corr(height,weight)=0.9955
◆ corr(height,weight,’Type’,’Spearman’)=1.
Analysis of data sets in Matlab
Linear regression Example Predictions Matlab Quadratic regression Multiple regression Other nonlinear regression
● The weight vector is such that weight[i] lbs is the average weight ofwomen who are height[i] inches tall.
● Check the correlation and rank correlation:
◆ corr(height,weight)=0.9955
◆ corr(height,weight,’Type’,’Spearman’)=1.
● Next we fit a linear model y = a+ bx with x the height, y the weight.
◆ myfit = polyfit(height,weight,1)
◆ This gives myfit = 3.4500 -87.5167,hence the coefficients a and b with a = −87.52 and b = 3.45.
◆ Thus the linear regression line is “weight = 3.45 × height - 87.52”
Analysis of data sets in Matlab
Linear regression Example Predictions Matlab Quadratic regression Multiple regression Other nonlinear regression
● The weight vector is such that weight[i] lbs is the average weight ofwomen who are height[i] inches tall.
● Check the correlation and rank correlation:
◆ corr(height,weight)=0.9955
◆ corr(height,weight,’Type’,’Spearman’)=1.
● Next we fit a linear model y = a+ bx with x the height, y the weight.
◆ myfit = polyfit(height,weight,1)
◆ This gives myfit = 3.4500 -87.5167,hence the coefficients a and b with a = −87.52 and b = 3.45.
◆ Thus the linear regression line is “weight = 3.45 × height - 87.52”
● To get analyse the residuals, use
◆ weight approx = polyval(myfit,height);the model y-values.
◆ residuals = weight-weight approx; gives the residuals.
◆ mean(residuals) gives 2.8422e-14 and var(residuals)=2.1595
Plot for the “women” dataLinear regression Example Predictions Matlab Quadratic regression Multiple regression Other nonlinear regression
scatter(height,weight); hold on; plot(height,weight approx)
title(’American women aged 30-39’); xlabel(’Height (in)’);
ylabel(’Weight (lb)’);
58 60 62 64 66 68 70 72
120
130
140
150
160
American women aged 30−39
Height (in)
Wei
ght (
lb)
Plot for the residuals in “women” dataLinear regression Example Predictions Matlab Quadratic regression Multiple regression Other nonlinear regression
figure; scatter(height,residuals); hold on;
plot(height,zeros(length(height))); title(’Residual Plot’);
xlabel(’Height (in)’); ylabel(’Residual (lb)’);
58 60 62 64 66 68 70 72
−1
01
23
Residual Plot
Height (in)
Res
idua
l (lb
)
Normal check of residuals “women” dataLinear regression Example Predictions Matlab Quadratic regression Multiple regression Other nonlinear regression
figure; qqplot(residuals)
−1 0 1
−1
01
23
normal Q−Q plot for women data
Theoretical Quantiles
Qua
ntile
s w
omen
dat
a
Quadratic regression (§6.2)
Linear regression Example Predictions Matlab Quadratic regression Multiple regression Other nonlinear regression
Non-linear regression and women dataLinear regression Example Predictions Matlab Quadratic regression Multiple regression Other nonlinear regression
Looking at the plots obtained from fitting a straight line to the women data,it appears that a non-linear relationship may be more appropriate.
Assume that the appropriate relationship between the variables “weight” and“height” is:
weight = α+ β × height + γ × (height)2 + noise
We will try to fit a quadratic of the form:
weight = a+ b× height + c× (height)2
to the data, where a, b and c have to be found by using the observations.This will be done by using procedures in Matlab.
Using Matlab to fit a quadraticLinear regression Example Predictions Matlab Quadratic regression Multiple regression Other nonlinear regression
Using Matlab, we fit a quadratic of the form
weight = a+ b× height + c× (height)2
to the women data.
Using Matlab to fit a quadraticLinear regression Example Predictions Matlab Quadratic regression Multiple regression Other nonlinear regression
Using Matlab, we fit a quadratic of the form
weight = a+ b× height + c× (height)2
to the women data.
Matlab can fit any order polynomial with polyfit, so now we just use theoption “2” for a second order polynomial:
myfit2 = polyfit(height,weight,2)
This gives c = 0.0831, b = −7.3483 and a = 261.8782.
Using Matlab to fit a quadraticLinear regression Example Predictions Matlab Quadratic regression Multiple regression Other nonlinear regression
Using Matlab, we fit a quadratic of the form
weight = a+ b× height + c× (height)2
to the women data.
Matlab can fit any order polynomial with polyfit, so now we just use theoption “2” for a second order polynomial:
myfit2 = polyfit(height,weight,2)
This gives c = 0.0831, b = −7.3483 and a = 261.8782.
Thus the approximate quadratic is
weight = 261.878184− 7.348319× height+ 0.083064× (height)2
Quadratic & linear fit for women dataLinear regression Example Predictions Matlab Quadratic regression Multiple regression Other nonlinear regression
58 60 62 64 66 68 70 72
120
130
140
150
160
Height (in)
Wei
ght (
lb)
Fitted quadratic modelFitted linear model
Residuals of quadratic fit for women dataLinear regression Example Predictions Matlab Quadratic regression Multiple regression Other nonlinear regression
58 60 62 64 66 68 70 72
−0.
4−
0.2
0.0
0.2
0.4
0.6
Residual plot
height
resi
dual
s
−1 0 1
−0.
4−
0.2
0.0
0.2
0.4
0.6
Normal Q−Q Plot
Theoretical Quantiles
quan
tiles
of r
esid
uals
of q
uadr
atic
fit
Another quadratic fitting exampleLinear regression Example Predictions Matlab Quadratic regression Multiple regression Other nonlinear regression
We “polute” the quadratic y = (x−6)2
4 = 9− 3x+ x2
4 by adding noise and getthe following dataset:
x 1 2 3 4 5 6 7 8 9 10 11 12 13 14
y 7.1 4.2 0.3 3.0 -1.0 -1.5 0.3 -1.0 1.5 1.4 6.3 8.5 13.5 15.4
y 6.3 4.0 2.3 1.0 0.3 0.0 0.3 1.0 2.3 4.0 6.3 9.0 12.3 16.0
Here y is the “poluted” data and y are the “pure” values.
This data is obtained in Matlab by using the following commands:
x = 1:14;
y = 0.25*(x-6).^2 + normrnd(0,1,1,14);
yhat = 0.25*(x-6).^2
Fitting the quadratic dataLinear regression Example Predictions Matlab Quadratic regression Multiple regression Other nonlinear regression
Next we fit a quadratic by using:
myfit = polyfit(x,y,2)
approx y = polyval(myfit,x)
This gives the fitted curve y = 10.3− 3.6x+ 0.29x2 and a set of values y(x),called approx y.
Fitting the quadratic dataLinear regression Example Predictions Matlab Quadratic regression Multiple regression Other nonlinear regression
Next we fit a quadratic by using:
myfit = polyfit(x,y,2)
approx y = polyval(myfit,x)
This gives the fitted curve y = 10.3− 3.6x+ 0.29x2 and a set of values y(x),called approx y.
To plot this, we use
scatter(x,y); hold on; plot(x,approx y);
plot(x,yhat,’--’);
title(’Quadratic regression example’);legend(’Data points’,’Fitted quadratic’,’Original
quadratic’,’Location’, ’West’)
Fitted & original plot of quadraticLinear regression Example Predictions Matlab Quadratic regression Multiple regression Other nonlinear regression
2 4 6 8 10 12 14
05
1015
Quadratic regression example
x
y
Fitted quadraticOriginal quadratic
Residuals in quadratic regressionLinear regression Example Predictions Matlab Quadratic regression Multiple regression Other nonlinear regression
Finally we analyse the residuals:residuals = y - approx y;
subplot(1,2,1); scatter(x,residuals); plot(x,zeros(14));
title(’Residual plot’);
subplot(1,2,2);
qqplot(residuals)
2 4 6 8 10 12 14
−1
01
2
Residual plot
x
Res
idua
ls
−1 0 1
−1
01
2
Normal Q−Q Plot
Theoretical Quantiles
Sam
ple
Qua
ntile
s
Regression and Matlab
Linear regression Example Predictions Matlab Quadratic regression Multiple regression Other nonlinear regression
Given a data set {(x1, y1), (x2, y2), . . . , (xn, yn)}. Using Matlab:
● The coefficients of a polynomial approximation of order n is obtained by
myfit = polyfit(x,y,n)
● The approximate y values are obtained by
approx = polyval(myfit,x)
Multiple regression (§6.2)
Linear regression Example Predictions Matlab Quadratic regression Multiple regression Other nonlinear regression
Data with three componentsLinear regression Example Predictions Matlab Quadratic regression Multiple regression Other nonlinear regression
The trees data set gives the relation between the height, volume and girth(diameter) of 31 cherry trees.
4.1 4.2 4.3 4.4 4.5 4.62
2.5
3
3.5
4
4.5
5
log(height)
log(
volu
me)
2 2.5 3 3.52
2.5
3
3.5
4
4.5
5
log(girth)
log(
volu
me)
2 2.5 3 3.54
4.2
4.4
4.6
4.8
log(girth)
log(
heig
ht)
corr(Volume’,Height’) = 0.5982
corr(Girth’,Volume’) = 0.9671
corr(Girth’,Height’) = 0.5193
corr(log(Volume)’,log(Height)’)
= 0.6486
corr(log(Girth)’,log(Volume)’)
= 0.9767
corr(log(Girth)’,log(Height)’)
= 0.5302
Linear regressionLinear regression Example Predictions Matlab Quadratic regression Multiple regression Other nonlinear regression
First linear regression on the data concerning the girth and volume, usinglog(girth) and log(volume), ignoring any information about the height.
2.2 2.4 2.6 2.8 3.0
2.5
3.0
3.5
4.0
log.g
log.
v
Multiple regressionLinear regression Example Predictions Matlab Quadratic regression Multiple regression Other nonlinear regression
Next multiple regression, writing log(volume) as a linear combination oflog(girth) and log(height). Shown are the observed volume against the fittedvolume
10 20 30 40 50 60 70
1020
3040
5060
7080
observed volume
fitte
d vo
lum
e
Comparing residuals (1)Linear regression Example Predictions Matlab Quadratic regression Multiple regression Other nonlinear regression
A plot of the residuals of the linear (left) and multiple regression (right)models
2.5 3.0 3.5 4.0
−0.
2−
0.1
0.0
0.1
0.2
Residuals Linear Regression
log(volume)
resi
dual
s
2.5 3.0 3.5 4.0
−0.
15−
0.10
−0.
050.
000.
050.
10
Residuals Multiple Regression
log(volume)
resi
dual
s
The residuals for the linear regression are mainly between −0.2 and +0.2 andthe residuals for the multiple regression are mainly between −0.1 and +0.1.
Comparing residuals (2)Linear regression Example Predictions Matlab Quadratic regression Multiple regression Other nonlinear regression
Finally we compare the residuals of the linear regression (left) and multipleregression (right) in normal qqplots:
−2 −1 0 1 2
−0.
2−
0.1
0.0
0.1
0.2
Normal Q−Q Plot
Theoretical Quantiles
Sam
ple
Qua
ntile
s
−2 −1 0 1 2
−0.
15−
0.10
−0.
050.
000.
050.
10
Normal QQ plot multiple regression
Theoretical Quantiles
Sam
ple
Qua
ntile
s
Matlab commands used (1)Linear regression Example Predictions Matlab Quadratic regression Multiple regression Other nonlinear regression
● Overview plots:scatter(log(Girth),log(Height));
scatter(log(Height),log(Volume));
scatter(log(Girth),log(Volume));
4.1 4.2 4.3 4.4 4.5 4.62
2.5
3
3.5
4
4.5
5
log(height)
log(
volu
me)
2 2.5 3 3.52
2.5
3
3.5
4
4.5
5
log(girth)
log(
volu
me)
2 2.5 3 3.54
4.2
4.4
4.6
4.8
log(girth)
log(
heig
ht)
Matlab commands used (1)Linear regression Example Predictions Matlab Quadratic regression Multiple regression Other nonlinear regression
● Overview plots:scatter(log(Girth),log(Height));
scatter(log(Height),log(Volume));
scatter(log(Girth),log(Volume));
● Linear regression:myfit lin = polyfit(log(Girth),log(Volume),1);
approx lin = polyval(myfit lin,Girth)
scatter(log(Girth),log(Volume)); plot(Girth,approx lin);
2.2 2.4 2.6 2.8 3.0
2.5
3.0
3.5
4.0
log.g
log.
v
Matlab commands used (2)Linear regression Example Predictions Matlab Quadratic regression Multiple regression Other nonlinear regression
● Multiple regression:X = [ones(length(Girth),1),Girth,Height];
mul = regress(Volume,X);
approx mul = mul(1) + mul(2)*Girth + mul(3)*Height;
scatter(volume,exp(approx mul))
plot([Volume(1),Volume(1)],[Volume(end),Volume(end)])
10 20 30 40 50 60 70
1020
3040
5060
7080
observed volume
fitte
d vo
lum
e
Matlab commands used (3)Linear regression Example Predictions Matlab Quadratic regression Multiple regression Other nonlinear regression
● Residuals:res lin = log(Volume)-log(approx lin);
res mul = log(Volume)-log(approx mul);
subplot(1,2,1); plot(log(Volume),res lin);
subplot(1,2,2); plot(log(Volume),res mul);
2.5 3.0 3.5 4.0
−0.
2−
0.1
0.0
0.1
0.2
Residuals Linear Regression
log(volume)
resi
dual
s
2.5 3.0 3.5 4.0
−0.
15−
0.10
−0.
050.
000.
050.
10
Residuals Multiple Regression
log(volume)
resi
dual
s
Matlab commands used (3)Linear regression Example Predictions Matlab Quadratic regression Multiple regression Other nonlinear regression
● Residuals:res lin = log(Volume)-log(approx lin);
res mul = log(Volume)-log(approx mul);
subplot(1,2,1); plot(log(Volume),res lin);
subplot(1,2,2); plot(log(Volume),res mul);
● Quantum-quantum plots:subplot(1,2,1); qqplot(res lin);
subplot(1,2,1); qqplot(res mul);
−2 −1 0 1 2
−0.
2−
0.1
0.0
0.1
0.2
Normal Q−Q Plot
Theoretical Quantiles
Sam
ple
Qua
ntile
s
−2 −1 0 1 2
−0.
15−
0.10
−0.
050.
000.
050.
10
Normal QQ plot multiple regression
Theoretical Quantiles
Sam
ple
Qua
ntile
s
Other nonlinear regression
Linear regression Example Predictions Matlab Quadratic regression Multiple regression Other nonlinear regression
Nonlinear regressionLinear regression Example Predictions Matlab Quadratic regression Multiple regression Other nonlinear regression
Some nonlinear regression can be done by introducing new variables, whichmake the relation linear (and hence we can use the simple lm command in Ras before). For example:
● The exponential regression model: Y = exp(α+ βX).Note that log Y = α+ βX, hence with U = log Y , we have a linearrelation between U and X. This is U = α+ βX;
Nonlinear regressionLinear regression Example Predictions Matlab Quadratic regression Multiple regression Other nonlinear regression
Some nonlinear regression can be done by introducing new variables, whichmake the relation linear (and hence we can use the simple lm command in Ras before). For example:
● The exponential regression model: Y = exp(α+ βX).Note that log Y = α+ βX, hence with U = log Y , we have a linearrelation between U and X. This is U = α+ βX;
● The logarithmic regression model: Y = log(α+ βX).Note that expY = α+ βX, hence with V = expY , we have a linearrelation between V and X. This is V = α+ βX;
Nonlinear regressionLinear regression Example Predictions Matlab Quadratic regression Multiple regression Other nonlinear regression
Some nonlinear regression can be done by introducing new variables, whichmake the relation linear (and hence we can use the simple lm command in Ras before). For example:
● The exponential regression model: Y = exp(α+ βX).Note that log Y = α+ βX, hence with U = log Y , we have a linearrelation between U and X. This is U = α+ βX;
● The logarithmic regression model: Y = log(α+ βX).Note that expY = α+ βX, hence with V = expY , we have a linearrelation between V and X. This is V = α+ βX;
● The power regression model: Y = αXβ .Note that log Y = logα+ β logX, hence with W = log Y andZ = logX, we have a linear relation between W and Z. This isW = logα+ βZ.
Nonlinear regressionLinear regression Example Predictions Matlab Quadratic regression Multiple regression Other nonlinear regression
Some nonlinear regression can be done by introducing new variables, whichmake the relation linear (and hence we can use the simple lm command in Ras before). For example:
● The exponential regression model: Y = exp(α+ βX).Note that log Y = α+ βX, hence with U = log Y , we have a linearrelation between U and X. This is U = α+ βX;
● The logarithmic regression model: Y = log(α+ βX).Note that expY = α+ βX, hence with V = expY , we have a linearrelation between V and X. This is V = α+ βX;
● The power regression model: Y = αXβ .Note that log Y = logα+ β logX, hence with W = log Y andZ = logX, we have a linear relation between W and Z. This isW = logα+ βZ.
However, most nonlinear regression models are more complicated.
Top Related