Linear Regression · Web viewlinearize (transform) data to find constants of some nonlinear...
28
Chapter 06.04 Nonlinear Models for Regression After reading this chapter, you should be able to 1. derive constants of nonlinear regression models, 2. use in examples, the derived formula for the constants of the nonlinear regression model, and 3. linearize (transform) data to find constants of some nonlinear regression models. From fundamental theories, we may know the relationship between two variables. An example in chemical engineering is the Clausius-Clapeyron equation that relates vapor pressure of a vapor to its absolute temperature, . (1) where and are the unknown parameters to be determined. The above equation is not linear in the unknown parameters. Any model that is not linear in the unknown parameters is described as a nonlinear regression model. Nonlinear models using least squares The development of the least squares estimation for nonlinear models does not generally yield equations that are 06.04.1
Transcript of Linear Regression · Web viewlinearize (transform) data to find constants of some nonlinear...
Linear Regression
06.04.2
Chapter 06.04
Nonlinear Regression 06.04.3
Chapter 06.04
Nonlinear Models for Regression
After reading this chapter, you should be able to
1. derive constants of nonlinear regression models,
2. use in examples, the derived formula for the constants of the nonlinear regression model, and
3. linearize (transform) data to find constants of some nonlinear regression models.
From fundamental theories, we may know the relationship between two variables. An example in chemical engineering is the Clausius‑Clapeyron equation that relates vapor pressure
)
F
(
o
T
of a vapor to its absolute temperature,
T
.
(
)
T
B
A
P
+
=
log
(1)
where
A
and
B
are the unknown parameters to be determined. The above equation is not linear in the unknown parameters. Any model that is not linear in the unknown parameters is described as a nonlinear regression model.
Nonlinear models using least squares
The development of the least squares estimation for nonlinear models does not generally yield equations that are linear and hence easy to solve. An example of a nonlinear regression model is the exponential model.
Exponential model
Given
(
)
1
1
,y
x
,
(
)
2
2
,y
x
, . . .
(
)
n
n
,y
x
, best fit
bx
ae
y
=
to the data. The variables
a
and
b
are the constants of the exponential model. The residual at each data point
i
x
is
i
bx
i
i
ae
y
E
-
=
(2)
The sum of the square of the residuals is
å
=
=
n
i
i
r
E
S
1
2
(
)
å
=
-
=
n
i
bx
i
i
ae
y
1
2
(3)
To find the constants
a
and
b
of the exponential model, we minimize
r
S
by differentiating with respect to
a
and
b
and equating the resulting equations to zero.
(
)
(
)
0
2
1
=
-
-
=
¶
¶
å
=
i
i
bx
n
i
bx
i
r
e
ae
y
a
S
(
)
(
)
0
2
1
=
-
-
=
¶
¶
å
=
i
i
bx
i
n
i
bx
i
r
e
ax
ae
y
b
S
(4a,b)
or
0
1
2
1
=
+
-
å
å
=
=
n
i
bx
n
i
bx
i
i
i
e
a
e
y
0
1
2
1
=
-
å
å
=
=
n
i
bx
i
n
i
bx
i
i
i
i
e
x
a
e
x
y
(5a,b)
Equations (5a) and (5b) are nonlinear in
a
and
b
and thus not in a closed form to be solved as was the case for linear regression. In general, iterative methods (such as Gauss‑Newton iteration method, method of steepest descent, Marquardt's method, direct search, etc) must be used to find values of
a
and
b
.
However, in this case, from Equation (5a),
a
can be written explicitly in terms of
b
as
å
å
=
=
=
n
i
bx
n
i
bx
i
i
i
e
e
y
a
1
2
1
(6)
Substituting Equation (6) in (5b) gives
0
1
2
1
2
1
1
=
-
å
å
å
å
=
=
=
=
n
i
bx
i
n
i
bx
bx
n
i
i
bx
i
n
i
i
i
i
i
i
e
x
e
e
y
e
x
y
(7)
This equation is still a nonlinear equation in
b
and can be solved best by numerical methods such as the bisection method or the secant method.
Example 1
Many patients get concerned when a test involves injection of a radioactive material. For example for scanning a gallbladder, a few drops of Technetium-99m isotope is used. Half of the techritium-99m would be gone in about 6 hours. It, however, takes about 24 hours for the radiation levels to reach what we are exposed to in day-to-day activities. Below is given the relative intensity of radiation as a function of time.
Table 1 Relative intensity of radiation as a function of time
)
hrs
(
t
0
1
3
5
7
9
g
1.000
0.891
0.708
0.562
0.447
0.355
If the level of the relative intensity of radiation is related to time via an exponential formula
t
Ae
l
g
=
, find
a) the value of the regression constants
A
and
l
,
b) the half-life of Technium-99m, and
c) the radiation intensity after 24 hours.
Solution
a) The value of
l
is given by solving the nonlinear Equation (7),
(
)
0
1
2
1
2
1
1
=
-
=
å
å
å
å
=
=
=
=
n
i
t
i
n
i
t
n
i
t
i
t
i
n
i
i
i
i
i
i
e
t
e
e
e
t
f
l
l
l
l
g
g
l
(8)
and then the value of
A
from Equation (6),
å
å
=
=
=
n
i
t
n
i
t
i
i
i
e
e
A
1
2
1
l
l
g
(9)
Equation (8) can be solved for
l
using bisection method. To estimate the initial guesses, we assume
0.23105
-
=
λ
and
0.077016
-
=
λ
. We need to check whether these values first bracket the root of
(
)
0
=
l
f
. At
0.23105
-
=
λ
, the table below shows the evaluation of
(
)
0.23105
-
f
.
Table 2 Summation value for calculation of constants of model
i
i
t
i
g
i
t
i
i
e
t
l
g
i
t
i
e
l
g
i
t
e
l
2
i
t
i
e
t
l
2
1
2
3
4
5
6
0
1
3
5
7
9
1
0.891
0.708
0.562
0.447
0.355
0.00000
0.70719
1.0620
0.88509
0.62087
0.39937
1.00000
0.70719
0.35400
0.17702
0.08870
0.04437
1.00000
0.62996
0.25000
0.09921
0.03937
0.01562
0.00000
0.62996
0.75000
0.49606
0.27560
0.14062
å
=
6
1
i
3.6745
2.3713
2.0342
2.2922
From Table 2
6
=
n
6745
.
3
23105
.
0
6
1
=
-
=
å
i
t
i
i
i
e
t
g
3713
.
2
23105
.
0
6
1
=
-
=
å
i
t
i
i
e
g
(
)
0342
.
2
23105
.
0
2
6
1
=
-
=
å
i
t
i
e
(
)
2922
.
2
23105
.
0
2
6
1
=
-
=
å
i
t
i
i
e
t
(
)
(
)
(
)
2922
.
2
0342
.
2
3713
.
2
6745
.
3
23105
.
0
-
=
-
f
0024
.
1
=
Similarly
(
)
9201
.
3
0077016
.
0
-
=
-
f
Since
(
)
(
)
0
0077016
.
0
23105
.
0
<
-
´
-
f
f
,
the value of
λ
falls in the bracket of
[
]
0.0077016
0.23105,
-
-
. The next guess of the root then is
(
)
2
0077016
.
0
23105
.
0
-
+
-
=
l
11938
.
0
-
=
Continuing with the bisection method, the root of
(
)
0
=
l
f
is found as
11508
.
0
-
=
l
. This value of the root was obtained after 20 iterations with an absolute relative approximate error of less than 0.0002%.
From Equation (9),
A
can be calculated as
å
å
=
=
=
6
1
2
6
1
i
t
i
t
i
i
i
e
e
A
l
l
g
)
9
)(
11508
.
0
(
2
)
7
)(
11508
.
0
(
2
)
5
)(
11508
.
0
(
2
)
3
)(
11508
.
0
(
2
)
1
)(
11508
.
0
(
2
)
0
)(
11508
.
0
(
2
)
9
(
11508
.
0
)
7
(
11508
.
0
)
5
(
11508
.
0
)
3
(
11508
.
0
)
1
(
11508
.
0
)
0
(
11508
.
0
355
.
0
447
.
0
562
.
0
708
.
0
891
.
0
1
-
-
-
-
-
-
-
-
-
-
-
-
+
+
+
+
+
´
+
´
+
´
+
´
+
´
+
´
=
e
e
e
e
e
e
e
e
e
e
e
e
9378
.
2
9373
.
2
=
99983
.
0
=
The regression formula is hence given by
t
e
11508
.
0
99983
.
0
-
=
g
b) Half life of Technetium-99m is when
0
2
1
=
=
t
g
g
(
)
hours
0232
.
6
)
5
.
0
ln(
11508
.
0
5
.
0
99983
.
0
2
1
99983
.
0
11508
.
0
)
0
(
11508
.
0
11508
.
0
=
=
-
=
=
´
-
-
-
t
t
e
e
e
t
t
c) The relative intensity of the radiation after 24 hrs is
(
)
24
11508
.
0
99983
.
0
-
´
=
e
g
2
10
3160
.
6
-
´
=
This implies that only
%
3171
.
6
100
99983
.
0
10
3160
.
6
2
=
´
´
-
of the initial radioactive intensity is left after 24 hrs.
0
0.5
1
0510
Time, t (hrs)
Relative
Intensity of
Radiation
γ
Figure 1 Relative intensity of radiation as a function of temperature using an exponential regression model.
Growth model
Growth models common in scientific fields have been developed and used successfully for specific situations. The growth models are used to describe how something grows with changes in the regressor variable (often the time). Examples in this category include growth of thin films or population with time. Growth models include
cx
be
a
y
-
+
=
1
(10)
where
b
a
,
and
c
are the constants of the model. At
0
=
x
,
b
1
a
y
+
=
and as
¥
®
x
,
a
y
®
.
The residuals at each data point
i
x
, are
i
cx
i
i
be
a
y
E
-
+
-
=
1
(11)
The sum of the square of the residuals is
å
=
=
n
i
i
r
E
S
1
2
2
1
1
å
=
-
÷
ø
ö
ç
è
æ
+
-
=
n
i
cx
i
i
be
a
y
(12)
To find the constants
a
,
b
and
c
we minimize
r
S
by differentiating with respect to
a
,
b
and
c
, and equating the resulting equations to zero.
(
)
[
]
(
)
0
2
1
2
=
÷
÷
ø
ö
ç
ç
è
æ
+
+
-
=
¶
¶
å
=
´
n
i
cx
cx
i
x
c
cx
r
b
e
b
e
y
ae
e
a
S
i
i
i
i
,
(
)
[
]
(
)
0
2
1
3
=
÷
÷
ø
ö
ç
ç
è
æ
+
-
+
=
¶
¶
å
=
n
i
cx
i
cx
i
cx
r
b
e
a
y
e
by
ae
b
S
i
i
i
,
(
)
[
]
(
)
0
2
1
3
=
÷
÷
ø
ö
ç
ç
è
æ
+
-
+
-
=
¶
¶
å
=
n
i
cx
i
cx
i
cx
i
r
b
e
a
y
e
by
e
abx
c
S
i
i
i
.
(13a,b,c)
One can use the Newton-Raphson method to solve the above set of simultaneous nonlinear equations for
a
,
b
and
c
.
Example 2
The height of a child is measured at different ages as follows.
Table 3 Height of the child at different ages.
)
yrs
(
t
0
5.0
8
12
16
18
)
in
(
H
20
36.2
52
60
69.2
70
Estimate the height of the child as an adult of 30 years of age using the growth model,
ct
be
a
H
-
+
=
1
Solution
The saturation growth model of height,
H
vs. age,
t
is given as
ct
be
a
H
-
+
=
1
where the constants
a
,
b
and
c
are the roots of the simultaneous nonlinear equation system
(
)
[
]
(
)
0
2
6
1
2
=
÷
÷
ø
ö
ç
ç
è
æ
+
+
-
å
=
i
ct
ct
i
ct
ct
b
e
b
e
H
ae
e
i
i
i
i
(
)
[
]
(
)
0
2
6
1
3
=
÷
÷
ø
ö
ç
ç
è
æ
+
-
+
å
=
i
ct
i
ct
i
ct
b
e
a
H
e
bH
ae
i
i
i
(
)
[
]
(
)
0
2
6
1
3
=
÷
÷
ø
ö
ç
ç
è
æ
+
-
+
-
å
=
i
ct
i
ct
i
ct
i
b
e
a
H
e
bH
e
abt
i
i
i
(14a,b,c)
We need initial guesses of the roots to get the iterative process started to find the root of those equations. Suppose we use three of the given data points such as (0, 20), (12, 60) and (18, 70) to find the initial guesses of roots; we have
)
0
(
1
20
c
be
a
-
+
=
)
12
(
1
60
c
be
a
-
+
=
)
18
(
1
70
c
be
a
-
+
=
One can solve three unknowns
a
,
b
and
c
from the three equations as
1
10
5534
.
7
´
=
a
7767
.
2
=
b
1
10
9772
.
1
-
´
=
c
Applying the Newton-Raphson method for simultaneous nonlinear equations, one can get the roots
1
10
4321
.
7
´
=
a
8233
.
2
=
b
1
10
1715
.
2
-
´
=
c
The saturation growth model of the height of the child then is
t
e
H
1
10
1715
.
2
1
8233
.
2
1
10
4321
.
7
-
´
-
+
´
=
The height of the child as an adult of 30 years of age is
"
74
8233
.
2
1
10
4321
.
7
)
30
(
10
1715
.
2
1
1
=
+
´
=
´
´
-
-
e
H
Polynomial Models
Given
n
data points
)
,
(
),......,
,
(
),
,
(
2
2
1
1
n
n
y
x
y
x
y
x
use least squares method to regress the data to an
th
m
order polynomial.
n
m
x
a
x
a
x
a
a
y
m
m
<
+
+
+
+
=
,
2
2
1
0
L
L
(15)
The residual at each data point is given by
m
i
m
i
i
i
x
a
x
a
a
y
E
-
-
-
-
=
.
.
.
1
0
(16)
The sum of the square of the residuals is given by
(
)
å
å
=
=
-
-
-
-
=
=
n
i
m
i
m
i
i
n
i
i
r
x
a
x
a
a
y
E
S
1
2
1
0
1
2
.
.
.
(17)
To find the constants of the polynomial regression model, we put the derivatives with respect to
i
a
to zero, that is,
20
40
60
80
0102030
Time, t (yrs)
Height,
h (in)
Figure 2 Height of child as a function of age saturation growth model.
(
)
(
)
(
)
0
)
(
.
.
.
2
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
0
)
(
.
.
.
2
0
)
1
(
.
.
.
2
1
1
0
1
1
0
1
1
1
0
0
=
-
-
-
-
-
=
¶
¶
=
-
-
-
-
-
=
¶
¶
=
-
-
-
-
-
=
¶
¶
å
å
å
=
=
=
m
i
n
i
m
i
m
i
i
m
r
i
n
i
m
i
m
i
i
r
n
i
m
i
m
i
i
r
x
x
a
x
a
a
y
a
S
x
x
a
x
a
a
y
a
S
x
a
x
a
a
y
a
S
(18)
Setting those equations in matrix form gives
ú
ú
ú
ú
ú
ú
ú
ú
ú
ú
û
ù
ê
ê
ê
ê
ê
ê
ê
ê
ê
ê
ë
é
=
ú
ú
ú
ú
û
ù
ê
ê
ê
ê
ë
é
ú
ú
ú
ú
ú
ú
ú
ú
ú
ú
û
ù
ê
ê
ê
ê
ê
ê
ê
ê
ê
ê
ë
é
÷
ø
ö
ç
è
æ
÷
ø
ö
ç
è
æ
÷
ø
ö
ç
è
æ
÷
ø
ö
ç
è
æ
÷
ø
ö
ç
è
æ
÷
ø
ö
ç
è
æ
÷
ø
ö
ç
è
æ
÷
ø
ö
ç
è
æ
å
å
å
å
å
å
å
å
å
å
å
=
=
=
=
=
+
=
=
+
=
=
=
=
n
i
i
m
i
n
i
i
i
n
i
i
m
n
i
m
i
n
i
m
i
n
i
m
i
n
i
m
i
n
i
i
n
i
i
n
i
m
i
n
i
i
y
x
y
x
y
a
a
a
x
x
x
x
x
x
x
x
n
1
1
1
1
0
1
2
1
1
1
1
1
1
2
1
1
1
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
(19)
The above are solved for
m
a
a
a
,......,
,
1
0
Example 3
To find contraction of a steel cylinder, one needs to regress the thermal expansion coefficient data to temperature
Table 4 The thermal expansion coefficient at given different temperatures
Temperature,
T
)
F
(
o
Coefficient of thermal expansion,
a
EMBED Equation.3
)
F
in/in/
(
o
80
6
10
47
.
6
-
´
40
6
10
24
.
6
-
´
-40
6
10
72
.
5
-
´
-120
6
10
09
.
5
-
´
-200
6
10
30
.
4
-
´
-280
6
10
33
.
3
-
´
-340
6
10
45
.
2
-
´
Fit the above data to
2
2
1
0
T
a
T
a
a
α
+
+
=
Solution
Since
2
2
1
0
T
a
T
a
a
α
+
+
=
is the quadratic relationship between the thermal expansion coefficient and the temperature, the coefficients
2
1
0
,
,
a
a
a
are found as follows
ú
ú
ú
ú
ú
ú
ú
û
ù
ê
ê
ê
ê
ê
ê
ê
ë
é
=
ú
ú
ú
û
ù
ê
ê
ê
ë
é
ú
ú
ú
ú
ú
ú
ú
û
ù
ê
ê
ê
ê
ê
ê
ê
ë
é
÷
ø
ö
ç
è
æ
÷
ø
ö
ç
è
æ
÷
ø
ö
ç
è
æ
÷
ø
ö
ç
è
æ
÷
ø
ö
ç
è
æ
÷
ø
ö
ç
è
æ
÷
ø
ö
ç
è
æ
÷
ø
ö
ç
è
æ
å
å
å
å
å
å
å
å
å
å
å
=
=
=
=
=
=
=
=
=
=
=
n
i
i
i
n
i
i
i
n
i
i
n
i
i
n
i
i
n
i
i
n
i
i
n
i
i
n
i
i
n
i
i
n
i
i
T
T
a
a
a
T
T
T
T
T
T
T
T
n
1
2
1
1
2
1
0
1
4
1
3
1
2
1
3
1
2
1
1
2
1
a
a
a
Table 5 Summations for calculating constants of model
i
)
F
(
o
T
a
EMBED Equation.3
)
F
in/in/
(
o
2
T
3
T
1
80
6
10
4700
.
6
-
´
3
10
4000
.
6
´
5
10
1200
.
5
´
2
40
6
10
2400
.
6
-
´
3
10
6000
.
1
´
4
10
4000
.
6
´
3
-40
6
10
7200
.
5
-
´
3
10
6000
.
1
´
4
10
4000
.
6
´
-
4
-120
6
10
0900
.
5
-
´
4
10
4400
.
1
´
6
10
7280
.
1
´
-
5
-200
6
10
3000
.
4
-
´
4
10
0000
.
4
´
6
10
0000
.
8
´
-
6
-280
6
10
3300
.
3
-
´
4
10
8400
.
7
´
7
10
1952
.
2
´
-
7
-340
6
10
4500
.
2
-
´
5
10
1560
.
1
´
7
10
9304
.
3
´
-
å
=
7
1
i
2
10
6000
.
8
´
-
5
10
3600
.
3
-
´
5
10
5800
.
2
´
7
10
0472
.
7
´
-
Table 5 (cont)
i
4
T
a
´
T
a
´
2
T
1
7
10
0960
.
4
´
4
10
1760
.
5
-
´
2
10
1408
.
4
-
´
2
6
10
5600
.
2
´
4
10
4960
.
2
-
´
3
10
9840
.
9
-
´
3
6
10
5600
.
2
´
4
10
2880
.
2
-
´
-
3
10
1520
.
9
-
´
4
8
10
0736
.
2
´
4
10
1080
.
6
-
´
-
2
10
3296
.
7
-
´
5
9
10
6000
.
1
´
4
10
6000
.
8
-
´
-
1
10
7200
.
1
-
´
6
9
10
1466
.
6
´
4
10
3240
.
9
-
´
-
1
10
6107
.
2
-
´
7
10
10
3363
.
1
´
4
10
3300
.
8
-
´
-
1
10
8322
.
2
-
´
å
=
7
1
i
10
10
1363
.
2
´
3
10
6978
.
2
-
´
-
1
10
5013
.
8
-
´
7
=
n
2
7
1
10
6000
.
8
-
=
´
-
=
å
i
i
T
5
7
1
2
10
5580
.
2
´
=
å
=
i
i
T
7
7
1
3
10
0472
.
7
´
-
=
å
=
i
i
T
10
7
1
4
10
1363
.
2
å
=
´
=
i
i
T
5
7
1
10
3600
.
3
-
=
´
=
å
i
i
a
3
7
1
10
6978
.
2
-
=
´
-
=
å
i
i
i
T
a
1
7
1
2
10
5013
.
8
-
=
´
=
å
i
i
i
T
a
We have
ú
ú
ú
û
ù
ê
ê
ê
ë
é
´
´
-
´
=
ú
ú
ú
û
ù
ê
ê
ê
ë
é
ú
ú
ú
û
ù
ê
ê
ê
ë
é
´
´
-
´
´
-
´
´
-
´
´
-
-
-
-
1
3
5
2
1
0
10
7
5
7
5
2
5
2
10
5013
.
8
10
6978
.
2
10
3600
.
3
10
1363
.
2
10
0472
.
7
10
5800
.
2
10
0472
.
7
10
5800
.
2
10
600
.
8
10
5800
.
2
10
6000
.
8
0000
.
7
a
a
a
Solving the above system of simultaneous linear equations, we get
ú
ú
ú
û
ù
ê
ê
ê
ë
é
´
-
´
´
=
ú
ú
ú
û
ù
ê
ê
ê
ë
é
-
-
-
11
9
6
2
1
0
10
2218
.
1
10
2782
.
6
10
0217
.
6
a
a
a
The polynomial regression model is
2
11
9
6
2
2
1
0
10
1.2218
10
6.2782
10
6.0217
T
T
T
a
T
a
a
-
-
-
´
-
´
+
´
=
+
+
=
a
Linearization of data
Examination of the nonlinear models above shows that in general iterative methods are required to estimate the values of the model parameters. It is sometimes useful to use simple linear regression to estimate the parameters of a nonlinear model. This involves first transforming the given data such as to regress it to a linear model. Following the linearization of the data, the evaluation of model parameters lends itself to direct solution approach using the least squares method. Data for nonlinear models such as exponential, power, and growth can be linearized.
Exponential Model
As given in Example 1, many physical and chemical processes are governed by the exponential function.
bx
ae
=
g
(20)
Taking natural log of both sides of Equation (20) gives
bx
a
+
=
ln
ln
g
(21)
Let
g
ln
=
z
a
a
ln
0
=
implying
o
a
e
a
=
b
a
=
1
then
x
a
a
z
1
0
+
=
(22)
P
)
F
in/in/
(
o
a
0.00E+00
4.00E-06
8.00E-06
-400-300-200-1000100
Temperature,
Coefficient of
Thermal
Expansion,
Figure 3 Second-order polynomial regression model for coefficient of thermal expansion as a function of temperature.
The data
z
versus
x
is now a linear model. The constants
0
a
and
1
a
can be found using the equation for the linear model as
_
1
_
0
1
2
1
2
1
1
1
1
x
a
z
a
x
x
n
z
x
z
x
n
a
n
i
n
i
i
i
n
i
i
n
i
n
i
i
i
i
-
=
÷
ø
ö
ç
è
æ
-
-
=
å
å
å
å
å
=
=
=
=
=
(23a,b)
Now since
0
a
and
1
a
are found, the original constants with the model are found as
0
1
a
e
a
a
b
=
=
(24a,b)
Example 4
Repeat Example 1 using linearization of data.
Solution
t
Ae
l
g
=
(
)
(
)
t
A
l
g
+
=
ln
ln
Assuming
g
ln
=
y
(
)
A
a
ln
0
=
l
=
1
a
We get
t
a
a
y
1
0
+
=
This is a linear relationship between
y
and
t
.
å
å
å
å
å
=
=
=
=
=
÷
ø
ö
ç
è
æ
-
-
=
n
i
n
i
i
i
n
i
i
n
i
n
i
i
i
i
t
t
n
y
t
y
t
n
a
1
2
1
2
1
1
1
1
t
a
y
a
1
0
-
=
(25a,b)
Table 6 Summations of data to calculate constants of model.
i
i
t
i
g
i
i
y
g
ln
=
i
i
y
t
2
i
t
1
2
3
4
5
6
0
1
3
5
7
9
1
0.891
0.708
0.562
0.447
0.355
0.00000
-0.11541
-0.34531
-0.57625
-0.80520
-1.0356
0.0000
-0.11541
-1.0359
-2.8813
-5.6364
-9.3207
0.0000
1.0000
9.0000
25.000
49.000
81.000
å
=
6
1
i
25.000
-2.8778
-18.990
165.00
6
=
n
000
.
25
6
1
å
=
=
i
i
t
å
=
-
=
6
1
8778
.
2
i
i
y
å
=
-
=
6
1
990
.
18
i
i
i
y
t
å
=
=
6
1
2
00
.
165
i
i
t
From Equation (25a,b) we have
(
)
(
)
(
)
(
)
(
)
2
1
25
00
.
165
6
8778
.
2
25
990
.
18
6
-
-
-
-
=
a
11505
.
0
-
=
(
)
6
25
11505
.
0
6
8778
.
2
0
-
-
-
=
a
4
10
6150
.
2
-
´
-
=
Since
(
)
A
a
ln
0
=
0
a
e
A
=
4
10
6150
.
2
-
´
-
=
e
99974
.
0
=
11505
.
0
1
-
=
=
a
l
The regression formula then is
t
e
11505
.
0
99974
.
0
-
´
=
g
Compare the formula to the one obtained without data linearization,
t
e
11508
.
0
99983
.
0
-
´
=
g
b) Half-life is when
0
2
1
=
=
t
g
g
(
)
hours
0248
.
6
)
5
.
0
ln(
11505
.
0
5
.
0
99974
.
0
2
1
99974
.
0
11508
.
0
)
0
(
11505
.
0
11505
.
0
=
=
-
=
=
´
-
-
-
t
t
e
e
e
t
t
c) The relative intensity of radiation, after 24 hours is
(
)
24
11505
.
0
99974
.
0
-
=
e
g
063200
.
0
=
This implies that only
%
3216
.
6
100
99974
.
0
10
3200
.
6
2
=
´
´
-
of the initial radioactivity is left after 24 hours.Logarithmic Functions
The form for the log regression models is
(
)
x
y
ln
1
0
b
b
+
=
(26)
This is a linear function between
y
and
(
)
x
ln
and the usual least squares method applies in which
y
is the response variable and
(
)
x
ln
is the regressor.
0
0.5
1
0510
Time, t (hrs)
Relative
Intensity of
Radiation
γ
Figure 4 Exponential regression model with linearized data for relative intensity of radiation as a function of temperature.
Example 5
Sodium borohydride is a potential fuel for fuel cell. The following overpotential
(
)
h
vs. current
(
)
i
data was obtained in a study conducted to evaluate its electrochemical kinetics.
Table 7 Electrochemical Kinetics of borohydride data.
)
(
V
h
-0.29563
-0.24346
-0.19012
-0.18772
-0.13407
-0.0861
)
(
A
i
0.00226
0.00212
0.00206
0.00202
0.00199
0.00195
At the conditions of the study, it is known that the relationship that exists between the overpotential
(
)
h
and current
(
)
i
can be expressed as
i
b
a
ln
+
=
h
(27)
where
a
is an electrochemical kinetics parameter of borohydride on the electrode. Use the data in Table 7 to evaluate the values of
a
and
b
.
Solution
Following the least squares method, Table 8 is tabulated where
i
x
ln
=
h
=
y
We obtain
bx
a
y
+
=
(28)
This is a linear relationship between
y
and
x
, and the coefficients
b
and
a
are found as follow
å
å
å
å
å
=
=
=
=
=
÷
ø
ö
ç
è
æ
-
-
=
n
i
n
i
i
n
i
i
n
i
n
i
i
i
i
x
x
n
y
x
y
x
n
b
1
2
1
2
1
1
1
1
x
b
y
a
-
=
(29a,b)
Table 8 Summation values for calculating constants of model
#
i
h
=
y
)
ln(
i
x
=
2
x
y
x
´
1
0.00226
-0.29563
-6.0924
37.117
1.8011
2
0.00212
-0.24346
-6.1563
37.901
1.4988
3
0.00206
-0.19012
-6.1850
38.255
1.1759
4
0.00202
-0.18772
-6.2047
38.498
1.1647
5
0.00199
-0.13407
-6.2196
38.684
0.83386
6
0.00195
-0.08610
-6.2399
38.937
0.53726
å
=
6
1
i
0.012400
-1.1371
-37.098
229.39
7.0117
6
=
n
å
=
-
=
6
1
098
.
37
i
i
x
å
=
-
=
6
1
1371
.
1
i
i
y
å
=
=
6
1
0117
.
7
i
i
i
y
x
å
=
=
6
1
2
39
.
229
i
i
x
(
)
(
)
(
)
(
)
(
)
3601
.
1
098
.
37
39
.
229
6
1371
.
1
098
.
37
0117
.
7
6
2
-
=
-
-
-
-
-
=
b
(
)
5990
.
8
6
098
.
37
3601
.
1
6
1371
.
1
-
=
-
-
-
-
=
a
Hence
i
ln
3601
.
1
5990
.
8
´
-
-
=
h
-0.3
-0.2
-0.1
0
0.00180.0020.00220.0024
Current, i(A)
Overpotential
η(V)
Figure 5 Overpotential as a function of current.
)
(
V
h
Power Functions
The power function equation describes many scientific and engineering phenomena. In chemical engineering, the rate of chemical reaction is often written in power function form as
b
ax
y
=
(30)
The method of least squares is applied to the power function by first linearizing the data (the assumption is that
b
is not known). If the only unknown is
a
, then a linear relation exists between
b
x
and
y
. The linearization of the data is as follows.
(
)
(
)
(
)
x
b
a
y
ln
ln
ln
+
=
(31)
The resulting equation shows a linear relation between
(
)
y
ln
and
(
)
x
ln
.
Let
)
ln(
ln
x
w
y
z
=
=
a
a
ln
0
=
implying
0
a
e
a
=
b
a
=
1
we get
w
a
a
z
1
0
+
=
(32)
n
w
a
n
z
a
w
w
n
z
w
z
w
n
a
n
i
i
n
i
i
n
i
n
i
i
i
n
i
i
n
i
n
i
i
i
i
å
å
å
å
å
å
å
=
=
=
=
=
=
=
-
=
÷
ø
ö
ç
è
æ
-
-
=
1
1
1
0
1
2
1
2
1
1
1
1
(33a,b)
Since
0
a
and
1
a
can be found, the original constants of the model are
0
1
a
e
a
a
b
=
=
(34a,b)
Example 6
The progress of a homogeneous chemical reaction is followed and it is desired to evaluate the rate constant and the order of the reaction. The rate law expression for the reaction is known to follow the power function form
n
kC
r
=
-
(35)
Use the data provided in the table to obtain
n
and
k
.
Table 9 Chemical kinetics.
gmol/l)
(
A
C
4
2.25
1.45
1.0
0.65
0.25
0.006
s)
gmol/l
(
×
-
A
r
0.398
0.298
0.238
0.198
0.158
0.098
0.048
Solution
Taking the natural log of both sides of Equation (35), we obtain
(
)
(
)
(
)
C
n
k
r
ln
ln
ln
+
=
-
Let
(
)
r
z
-
=
ln
(
)
C
w
ln
=
)
ln(
0
k
a
=
implying that
0
a
e
k
=
(36)
n
a
=
1
(37)
We get
w
a
a
z
1
0
+
=
This is a linear relation between
z
and
w
, where
å
å
å
å
å
=
=
=
=
=
÷
ø
ö
ç
è
æ
-
-
=
n
i
n
i
i
i
n
i
i
n
i
n
i
i
i
i
w
w
n
z
w
z
w
n
a
1
2
1
2
1
1
1
1
÷
÷
÷
÷
ø
ö
ç
ç
ç
ç
è
æ
-
÷
÷
÷
÷
ø
ö
ç
ç
ç
ç
è
æ
=
å
å
=
=
n
w
a
n
z
a
n
i
i
n
i
i
1
1
1
0
(38a,b)
Table 10 Kinetics rate law using power function
i
C
r
-
w
z
z
w
´
2
w
1
4
0.398
1.3863
-0.92130
-1.2772
1.9218
2
2.25
0.298
0.8109
-1.2107
-0.9818
0.65761
3
1.45
0.238
0.3716
-1.4355
-0.5334
0.13806
4
1
0.198
0.0000
-1.6195
0.0000
0.00000
5
0.65
0.158
-0.4308
-1.8452
0.7949
0.18557
6
0.25
0.098
-1.3863
-2.3228
3.2201
1.9218
7
0.006
0.048
-5.1160
-3.0366
15.535
26.173
å
=
7
1
i
-4.3643
-12.391
16.758
30.998
7
=
n
å
=
-
=
7
1
3643
.
4
i
i
w
å
=
-
=
7
1
391
.
12
i
i
z
å
=
=
7
1
758
.
16
i
i
i
z
w
å
=
=
7
1
2
998
.
30
i
i
w
From Equation (38a,b)
(
)
(
)
(
)
(
)
(
)
31943
.
0
3643
.
4
998
.
30
7
391
.
12
3643
.
4
758
.
16
7
2
1
=
-
-
´
-
´
-
-
´
=
a
(
)
5711
.
1
7
3643
.
4
31943
.
7
391
.
12
0
-
=
-
-
-
=
a
From Equation (36) and (37), we obtain
20782
.
0
5711
.
1
=
=
-
e
k
31941
.
0
1
=
=
a
n
Finally, the model of progress of that chemical reaction is
31941
.
0
20782
.
0
C
r
´
=
-
0
0.2
0.4
024
Concentration, C (gmol/l)
Chemical
Reaction
Rate, r
(gmol/l-s)
Figure 6 Kinetic chemical reaction rate as a function of concentration.
Growth Model
Growth models common in scientific fields have been developed and used successfully for specific situations. The growth models are used to describe how something grows with changes in a regressor variable (often the time). Examples in this category include growth of thin films or population with time. In the logistic growth model, an example of a growth model in which a measurable quantity
y
varies with some quantity
x
is
x
b
ax
y
+
=
(39)
For
0
=
x
,
0
=
y
while as
¥
®
x
,
a
y
®
. To linearize the data for this method,
a
x
a
b
ax
x
b
y
1
1
1
+
=
+
=
(40)
Let
y
z
1
=
x
w
1
=
,
a
a
1
0
=
implying that
0
1
a
a
=
a
b
a
=
1
implying
0
1
1
a
a
a
a
b
=
´
=
Then
w
a
a
z
1
0
+
=
(41)
The relationship between
z
and
w
is linear with the coefficients
0
a
and found as follows.
å
å
å
å
å
=
=
=
=
=
÷
ø
ö
ç
è
æ
-
-
=
n
i
n
i
i
i
n
i
i
n
i
n
i
i
i
i
w
w
n
z
w
z
w
n
a
1
2
1
2
1
1
1
1
÷
÷
÷
÷
ø
ö
ç
ç
ç
ç
è
æ
-
÷
÷
÷
÷
ø
ö
ç
ç
ç
ç
è
æ
=
å
å
=
=
n
w
a
n
z
a
n
i
i
n
i
i
1
1
1
0
(42a,b)
Finding
0
a
and
1
a
, then gives the constants of the original growth model as
0
1
a
a
=
0
1
a
a
b
=
(43a,b)
NONLINEAR REGRESSION
Topic
Nonlinear Regression
Summary
Textbook notes of Nonlinear Regression
Major
General Engineering
Authors
Egwu Kalu, Autar Kaw, Cuong Nguyen
Date
December 7, 2008
Web Site
http://numericalmethods.eng.usf.edu
� EMBED Equation.3 ���
� EMBED Equation.3 ���
06.04.1
)
F
(
o
T
)
F
in/in/
(
o
a
_1285755752.unknown
_1287992494.unknown
_1287994137.unknown
_1287996848.unknown
_1287997194.unknown
_1287997547.unknown
_1287998686.unknown
_1288011228.unknown
_1288011283.unknown
_1288011512.unknown
_1288011604.unknown
_1288011252.unknown
_1287998932.unknown
_1287999023.unknown
_1287998738.unknown
_1287998127.unknown
_1287998197.unknown
_1287998242.unknown
_1287998249.unknown
_1287998435.unknown
_1287998226.unknown
_1287998188.unknown
_1287997971.unknown
_1287998024.unknown
_1287997968.unknown
_1287997360.unknown
_1287997480.unknown
_1287997488.unknown
_1287997413.unknown
_1287997335.unknown
_1287997340.unknown
_1287997298.unknown
_1287996935.unknown
_1287997138.unknown
_1287997187.unknown
_1287997132.unknown
_1287996859.unknown
_1287996867.unknown
_1287996855.unknown
_1287994661.unknown
_1287994859.unknown
_1287994898.unknown
_1287994904.unknown
_1287994882.unknown
_1287994690.unknown
_1287994755.unknown
_1287994764.unknown
_1287994706.unknown
_1287994671.unknown
_1287994590.unknown
_1287994638.unknown
_1287994653.unknown
_1287994601.unknown
_1287994386.unknown
_1287994396.unknown
_1287994161.unknown
_1287993175.unknown
_1287993510.unknown
_1287993999.unknown
_1287994011.unknown
_1287993922.unknown
_1287993235.unknown
_1287993405.unknown
_1287993470.unknown
_1287993183.unknown
_1287993146.unknown
_1287993157.unknown
_1287993164.unknown
_1287993008.unknown
_1287993088.unknown
_1287993111.unknown
_1287992510.unknown
_1286092260.unknown
_1286093540.unknown
_1286098245.unknown
_1286532169.unknown
_1286532171.unknown
_1286532172.unknown
_1286532170.unknown
_1286098342.unknown
_1286098502.unknown
_1286098770.unknown
_1286098803.unknown
_1286098845.unknown
_1286098810.unknown
_1286098771.unknown
_1286098516.unknown
_1286098369.unknown
_1286098382.unknown
_1286098357.unknown
_1286098313.unknown
_1286098328.unknown
_1286098289.unknown
_1286094116.unknown
_1286096611.unknown
_1286096700.unknown
_1286096813.unknown
_1286098211.unknown
_1286098214.unknown
_1286097162.unknown
_1286098200.unknown
_1286097152.unknown
_1286096718.unknown
_1286096732.unknown
_1286096709.unknown
_1286096662.unknown
_1286096690.unknown
_1286096654.unknown
_1286096374.unknown
_1286096462.unknown
_1286096478.unknown
_1286096394.unknown
_1286094159.unknown
_1286096354.unknown
_1286094126.unknown
_1286093581.unknown
_1286094034.unknown
_1286094086.unknown
_1286094026.unknown
_1286093562.unknown
_1286093577.unknown
_1286093549.unknown
_1286092778.unknown
_1286093204.unknown
_1286093291.unknown
_1286093334.unknown
_1286093374.unknown
_1286093527.unknown
_1286093315.unknown
_1286093246.unknown
_1286093269.unknown
_1286093210.unknown
_1286092952.unknown
_1286093035.unknown
_1286093059.unknown
_1286093098.unknown
_1286093195.unknown
_1286093079.unknown
_1286093043.unknown
_1286092970.unknown
_1286092828.unknown
_1286092859.unknown
_1286092902.unknown
_1286092922.unknown
_1286092879.unknown
_1286092838.unknown
_1286092802.unknown
_1286092576.unknown
_1286092668.unknown
_1286092740.unknown
_1286092745.unknown
_1286092684.unknown
_1286092626.unknown
_1286092646.unknown
_1286092611.unknown
_1286092409.unknown
_1286092525.unknown
_1286092545.unknown
_1286092466.unknown
_1286092497.unknown
_1286092448.unknown
_1286092378.unknown
_1286092389.unknown
_1286092322.unknown
_1285758191.unknown
_1285762455.unknown
_1285762961.unknown
_1285763099.unknown
_1285763196.unknown
_1285763228.unknown
_1285763131.unknown
_1285763147.unknown
_1285763035.unknown
_1285763057.unknown
_1285763020.unknown
_1285762654.unknown
_1285762690.unknown
_1285762947.unknown
_1285762701.unknown
_1285762676.unknown
_1285762598.unknown
_1285762117.unknown
_1285762235.unknown
_1285762292.unknown
_1285762194.unknown
_1285761877.unknown
_1285762067.unknown
_1285761866.unknown
_1285755885.unknown
_1285756230.unknown
_1285758031.unknown
_1285758173.unknown
_1285757622.unknown
_1285755927.unknown
_1285756143.unknown
_1285755888.unknown
_1285755855.unknown
_1285755881.unknown
_1285755859.unknown
_1285755866.unknown
_1285755767.unknown
_1285755777.unknown
_1285755782.unknown
_1285755770.unknown
_1285755758.unknown
_1194148613.unknown
_1194945065.unknown
_1202539894.unknown
_1285755695.unknown
_1285755739.unknown
_1285755745.unknown
_1285755717.unknown
_1285755669.unknown
_1285755685.unknown
_1285755657.unknown
_1202540632.unknown
_1202178925.unknown
_1202180269.unknown
_1202182107.unknown
_1202182579.unknown
_1202182602.unknown
_1202262952.unknown
_1202182681.unknown
_1202182589.unknown
_1202182457.unknown
_1202182543.unknown
_1202182146.unknown
_1202182192.unknown
_1202181264.unknown
_1202181580.unknown
_1202181084.unknown
_1202179640.unknown
_1202179786.unknown
_1202179618.unknown
_1202178372.unknown
_1202178842.unknown
_1202178848.unknown
_1202178391.unknown
_1202178041.unknown
_1202178332.unknown
_1194945210.unknown
_1202176832.unknown
_1194942583.unknown
_1194943992.unknown
_1194945020.unknown
_1194945024.unknown
_1194944058.unknown
_1194944821.unknown
_1194944848.unknown
_1194944012.unknown
_1194943972.unknown
_1194943978.unknown
_1194943897.unknown
_1194943958.unknown
_1194943879.unknown
_1194931568.unknown
_1194932085.unknown
_1194940273.unknown
_1194940279.unknown
_1194940354.unknown
_1194939262.unknown
_1194940018.unknown
_1194931588.unknown
_1194931252.unknown
_1194931457.unknown
_1194931476.unknown
_1194931302.unknown
_1194156781.unknown
_1194931212.unknown
_1194148621.unknown
_1191844832.unknown
_1192826895.unknown
_1192877826.unknown
_1193728406.unknown
_1194147597.unknown
_1194148169.unknown
_1194148200.unknown
_1193728682.unknown
_1194116313.unknown
_1193728568.unknown
_1193728380.unknown
_1193728398.unknown
_1193599681.unknown
_1193599732.unknown
_1192946609.unknown
_1192858155.unknown
_1192869566.unknown
_1192869675.unknown
_1192875631.unknown
_1192870301.unknown
_1192870547.unknown
_1192869601.unknown
_1192869614.unknown
_1192869584.unknown
_1192869279.unknown
_1192869286.unknown
_1192858216.unknown
_1192856660.unknown
_1192858116.unknown
_1192856701.unknown
_1192856732.unknown
_1192855622.unknown
_1192856627.unknown
_1192855416.unknown
_1192781845.unknown
_1192823753.unknown
_1192826260.unknown
_1192826874.unknown
_1192823889.unknown
_1192781959.unknown
_1192782158.unknown
_1192781947.unknown
_1192781063.unknown
_1192781680.unknown
_1192781447.unknown
_1192780970.unknown
_1192781007.unknown
_1192780484.unknown
_1192780517.unknown
_1192780885.unknown
_1191845169.unknown
_1191823313.unknown
_1191827717.unknown
_1191828191.unknown
_1191844687.unknown
_1191844753.unknown
_1191844770.unknown
_1191829494.unknown
_1191829838.unknown
_1191844330.unknown
_1191829525.unknown
_1191828686.unknown
_1191828160.unknown
_1191828175.unknown
_1191828152.unknown
_1191824535.unknown
_1191827709.unknown
_1191823361.unknown
_1191744161.unknown
_1191823033.unknown
_1191823093.unknown
_1191751955.unknown
_1191742493.unknown
_1191744078.unknown
_1191744150.unknown
_1191743996.unknown
_1191058614.unknown
_1191065165.unknown
_1191128762.unknown
_1191742485.unknown
_1191128774.unknown
_1191065178.unknown
_1191059087.unknown
_1191064502.unknown
_1191058870.unknown
_1191058847.unknown
_1191057859.unknown
