Post on 15-Apr-2018
Numerical Differentiation and Integration
� Numerical Differentiation
◦ Finite Differences◦ Interpolating Polynomials◦ Taylor Series Expansion◦ Richardson Extrapolation
� Numerical Integration
◦ Basic Numerical Integration◦ Improved Numerical Integration⇒ Trapezoidal, Simpson’s Rules
◦ Rhomberg Integration
ITCS 4133/5133: Numerical Comp. Methods 1 Numerical Differentiation and Integration
Numerical Differentiation and Integration
� Many engineering applications require numerical estimates ofderivatives of functions
� Especially true, when analytical solutions are not possible
� Differentiation: Use finite differences
� Integration (definite integrals): Weighted sum of function valuesat specified points (area under the curve).
ITCS 4133/5133: Numerical Comp. Methods 2 Numerical Differentiation and Integration
Application:Integral of a Normal Distribution
◦ Represented as a Gaussian, a scaled form of f (x) = e−x2
, veryimportant function in statistics
◦ Not easy to determine indefinite integral - use numerical techniques
A =
∫ b
a
e−x2
ITCS 4133/5133: Numerical Comp. Methods 3 Numerical Differentiation and Integration
Application:Integral of a Sinc
f (x) =sin(x)
x
ITCS 4133/5133: Numerical Comp. Methods 4 Numerical Differentiation and Integration
Numerical Differentiation:Approach
⇒ Evaluate the function at two consecutive points, separated by ∆x, ofthe independent variable, and taking their difference:
df (x)
dx≈ f (x + ∆x)− f (x)
∆x
⇒ Fit a function to a set of points that define the relationship betweenthe independent and dependent variables, such as an nth orderpolynomial; then differentiate the polynomial
ITCS 4133/5133: Numerical Comp. Methods 5 Numerical Differentiation and Integration
Finite Differences
Forward Difference
df (x)
dx=f (x + ∆x)− f (x)
∆x
Backward Difference
df (x)
dx=f (x)− f (x−∆x)
∆x
Two Step Method, Central Difference
df (x)
dx=f (x + ∆x)− f (x−∆x)
2∆x
ITCS 4133/5133: Numerical Comp. Methods 6 Numerical Differentiation and Integration
Finite Differences: Example
ITCS 4133/5133: Numerical Comp. Methods 7 Numerical Differentiation and Integration
Forward Differences: Derivation
� Finite difference formulas can be derived from the Taylor series.
f (x + h) = f (x) + hf ′(x) +h2
2f ′′(η),
For h = xi+1 − xi,
f ′(xi) =f (xi+1)− f (xi)
h− h
2f ′′(η),
where xi < η < xi+1.
ITCS 4133/5133: Numerical Comp. Methods 8 Numerical Differentiation and Integration
Backward Differences: Derivation
f (x + h) = f (x) + hf ′(x) +h2
2f ′′f (η),
For h = xi−1 − xi,
f (xi + h) = f (xi + xi−1 − xi) = f (xi−1) = f (xi) + hf ′(xi) +h2
2f ′′f (η)
f ′(xi) =f (xi−1)− f (xi)
h− h
2f ′′(η)
where xi−1 < η < xi.
ITCS 4133/5133: Numerical Comp. Methods 9 Numerical Differentiation and Integration
Central Differences: Derivation
Use the next higher order Taylor polynomial,
f (xi+1) = f (xi) + hf ′(xi) +h2
2f ′′(xi) +
h3
6f ′′′(η1),
f (xi−1) = f (xi)− hf ′(xi) +h2
2f ′′(xi)−
h3
6f ′′′(η2)
with x < η1 < x + h, x− h < η2 < xThus,
f (xi+1)− f (xi−1) = 2hf ′(xi) +h3
6[f ′′′(η1) + f ′′′(η2)]
or,
f ′(xi) =f (xi+1)− f (xi−1)
2h− h2
6f ′′′(η), xi−1 < η < xi + 1
ITCS 4133/5133: Numerical Comp. Methods 10 Numerical Differentiation and Integration
Second Derivatives
f (x + h) = f (x) + hf ′(x) +h2
2f ′′(x) +
h3
3!f ′′′(x) +
h4
4!f (4)(x)(η1)
f (x− h) = f (x)− hf ′(x) +h2
2f ′′(x)− h3
3!f ′′′(x) +
h4
4!f (4)(x)(η2)
Thus
f (x + h) + f (x− h) = 2f (x) + h2f ′′(x) +h4
4![f (4)(x)(η1) + f (4)(x)(η2)]
f ′′(x) ≈ 1
h2[f (x + h)− 2f (x) + f (x− h)]
with truncation error of the O(h4)
ITCS 4133/5133: Numerical Comp. Methods 11 Numerical Differentiation and Integration
Partial Derivatives
� Generally, interested in partial derivatives of functions of 2 variables,(xi, yj), based on a mesh of points.
� Subscripts denote partial derivatives.
ITCS 4133/5133: Numerical Comp. Methods 12 Numerical Differentiation and Integration
Partial Derivatives (contd)
� Laplacian operator: ∆2u = uxx + uyy
� Biharmonic Operator: ∆4u = uxxxx + uxxyy + uyyyy
ITCS 4133/5133: Numerical Comp. Methods 13 Numerical Differentiation and Integration
Using Interpolating Polynomials
� Given a function in discrete form, the data can be interpolated to fitan nth order polynomial
� For those functions that are in analytical form, but difficult to differ-entiate, the function can be discretized and fitted by a polynomial.
f (x) = bnxn + bn−1x
n−1 + · · · + b1x + b0
whose derivative is
df (x)
dx= nbnx
n−1 + (n− 1)bn−2xn−2 + · · · + b1
ITCS 4133/5133: Numerical Comp. Methods 14 Numerical Differentiation and Integration
Power Series Type Interpolating Polynomial
Consider the nth order polynomial passing through (n+1) points; the (n+1) coefficients can be uniquely determined.
Example
f (x) = a0 + a1x + a2x2
Consider f (xi), f (xi + h), f (xi + 2h)
f (xi) = a0 + a1xi + a2x2i
f (xi + h) = a0 + a1(xi + h) + a2(xi + h)2
f (xi + 2h) = a0 + a1(xi + 2h) + a2(xi + 2h)2
ITCS 4133/5133: Numerical Comp. Methods 15 Numerical Differentiation and Integration
Example (contd)
For the 3 points, xi = 0, h, 2h,
fi = a0
fi+1 = a0 + a1h + a2h2
fi+2 = a0 + 2a1h + 4a2h2
which has the solution,
a0 = fi
a1 =−fi+2 + 4fi+1 − 3fi
2h
a2 =fi+2 − 2fi+1 + fi
2h2
ITCS 4133/5133: Numerical Comp. Methods 16 Numerical Differentiation and Integration
Example (contd)
f (x) = a0 + a1x + a2x2
f ′(xi) = f′
i = f ′(xi = 0)
= a1 + 2a2xi
= a1
=−fi+2 + 4fi+1 − 3fi
2h
Similarly, second derivatives can also be obtained.
ITCS 4133/5133: Numerical Comp. Methods 17 Numerical Differentiation and Integration
Numerical Integration
� Integration can be thought of as considering some continuous func-tion f (x) and the area A subtended by it; for instance, within a par-ticular interval
A =
∫ b
a
f (x)dx
� Numerical Integration is needed when f (x) does not have a knownanalytical solution, or, if f(x) is only defined at discrete points.
� There are two approaches to a numerical solution:
⇒ Fitting polynomials to f (x) and integrating using analytical cal-culus,
⇒ Area under f (x) can be approximated using geometric shapesdefined by adjacent points.
ITCS 4133/5133: Numerical Comp. Methods 18 Numerical Differentiation and Integration
Interpolation Approach
We can derive an nth order polynomial (for instance, Gregory-Newtonmethod), which has the general form as
f (x) = b1xn + b2x
n−1 + · · · + bnx + bn+1
After the bis are determined, this can be integrated as∫f (x)dx =
b1xn+1
n + 1+b2x
n
n+ · · · + bnx
2
2+ bn+1x
which can be solved for, within the limits of the integration
ITCS 4133/5133: Numerical Comp. Methods 19 Numerical Differentiation and Integration
Basic Numerical Integration (Newton-CotesClosed Formulas)
� Trapezoid Rule: Approximates function by a straightline to computearea under curve.∫ b
a
f (x)dx ≈ h
2[f (x0) + f (x1)]
� Solution is exact, for polynomials of degree≤ 1, i.e. linear functions.
ITCS 4133/5133: Numerical Comp. Methods 20 Numerical Differentiation and Integration
Basic Numerical Integration (Newton-CotesClosed Formulas)
� Simpson’s Rule: Instead of a linear approximation, use a quadraticpolynomial:
h =b− a
2, x0 = a, x1 = x0 + h =
b + a
2, x2 = b
Approximate integral is given by∫ b
a
f (x)dx ≈ h
3[f (x0) + 4f (x1) + f (x2)
ITCS 4133/5133: Numerical Comp. Methods 21 Numerical Differentiation and Integration
Basic Simpson’s Rule: Examp1e 1
ITCS 4133/5133: Numerical Comp. Methods 22 Numerical Differentiation and Integration
Basic Simpson’s Rule: Examp1e 2
ITCS 4133/5133: Numerical Comp. Methods 23 Numerical Differentiation and Integration
Improved Numerical Integration
� Improve the accuracy by applying lower order methods repeatedlyon several subintervals.
� Known as composite integration
� Simpson, Trapezoid rules use equal subintervals; using unequal(adaptive) intervals leads to Gaussian Quadrature methods.
ITCS 4133/5133: Numerical Comp. Methods 24 Numerical Differentiation and Integration
Composite Trapezoid Rule� Approximates the area under the function f (x) by a set of discrete
trapezoids fitted between each pair of points of the dependent vari-able (and the X axis)
∫ xn
x1
f (x)dx ≈n−1∑i=1
(xi+1 − xi)f (xi+1 + f (xi)
2
f(x)
f(x)
xx1 x2 x3 x4
If the intervals are the same, h = (b− a)/n, we get∫ xn
x1
f (x)dx ≈ b− a2n
[f (a) + 2f (x1) + . . . + f (b)]
ITCS 4133/5133: Numerical Comp. Methods 25 Numerical Differentiation and Integration
Composite Trapezoid Rule:Example
ITCS 4133/5133: Numerical Comp. Methods 26 Numerical Differentiation and Integration
Composite Trapezoid Rule:Algorithm
ITCS 4133/5133: Numerical Comp. Methods 27 Numerical Differentiation and Integration
Composite Trapezoid Rule: Notes
⇒ f(xi+1+f(xi)2 represents the average height of the trapezoid.
⇒ Linear approximation between pairs of successive points used; erroris proportional to the distance between sample points.
Error
⇒ Error in the trapezoidal can be approximated by Taylor’s series
xi+1 − xi
2f ′′(x)
where f ′′(x) is evaluated at the value of x that maximizes the secondderivative between the two sample points.
ITCS 4133/5133: Numerical Comp. Methods 28 Numerical Differentiation and Integration
Composite Simpson’s Rule� Uses the same idea as the composite Trapezoid rule, by using mul-
tiple sub-intervals to improve the Simpson rule approximation.
� For 2 subintervals, Simpson’s rule has the following form:∫ b
a
f (x)dx =
∫ x2
a
f (x)dx +
∫ b
x2
f (x)dx
≈ h
3[f (a) + 4f (x1) + f (x2)] +
h
3[f (x2) + 4f (x3) + f (b)]
≈ h
3[f (a) + 4f (x1) + 2f (x2) + 4f (x3) + f (b)])
Trapezoidal Rule
Simpson’s Rule
f(x)
a b(a+b)/2
f(x)
xITCS 4133/5133: Numerical Comp. Methods 29 Numerical Differentiation and Integration
Composite Simpson’s Rule (contd)
� For n (n must be even) subintervals, h = (b− a)/n we get∫ b
a
f (x)dx =h
3[f (a) + 4f (x1) + 2f (x2) + 4f (x3) +
2f (x4) + . . . + 2f (xn−2 + 4f (xn−1) + f (b)
� n must be even number of subintervals.
� Error Bound: Based on the 4th derivative:
Error =(xi+1 − xi)
5
90f (4)(xi)
where f (4)(xi) is evaluated at the value of x that maximizes the 4thderivative between the two sample points.
ITCS 4133/5133: Numerical Comp. Methods 30 Numerical Differentiation and Integration
Composite Simpson’s Rule:Example
ITCS 4133/5133: Numerical Comp. Methods 31 Numerical Differentiation and Integration
Composite Simpson’s Rule:Algorithm
ITCS 4133/5133: Numerical Comp. Methods 32 Numerical Differentiation and Integration
Romberg Integration
� Simpson’s rule improves on Trapezoid rule with additional evalua-tions of f(x), required to fit a more accurate polynomial
� More evaluations will further improve the integral, leading to theRomberg Integration
I01 =b− a
2(f (a) + f (b))
A second estimate can be obtained with subdividing the interval ab into2 equal intervals is given by
I11 =b− a
2
(1
2f (a) + f (m) +
1
2f (b)
)=
1
2
[I01 + (b− a)f
(a +
b− a2
)]ITCS 4133/5133: Numerical Comp. Methods 33 Numerical Differentiation and Integration
Romberg Integration(contd.)
A third estimate, I21, using 3 intermediate points m1,m2,m3 is given by
I21 =b− a
2
[1
4f (a) +
1
2f (m1) +
1
2f (m2) +
1
2f (m3) +
1
4f (b)
]=
1
2
[I11 +
b− a2
3∑k=1,k 6=2
f
(a +
b− a4
k
)]This leads to the recursive relationship
Ii1 =1
2
[Ii−1,1 +
b− a2i−1
2i−1∑k=1,3,5
f
(a +
b− a2i
k
)], for i = 1, 2, . . .
ITCS 4133/5133: Numerical Comp. Methods 34 Numerical Differentiation and Integration
Romberg Integration(contd.)
Ii1 =1
2
[Ii−1,1 +
b− a2i−1
2i−1∑k=1,3,5
f
(a +
b− a2i
k
)], for i = 1, 2, . . .
We can combine these estimates using Richardson’s formula,
Iij =4j−1Ii+1,j−1 − Ii,j−1
4j−1 − 1
for i = 0, 1, . . . , N − j + 1, j = 1, 2, . . . N . The estimates form an uppertriangular matrix:
I01 I02 I03 I04 · · · I0,N−1 I0,N I0,N+1
I11 I12 I13 . · · · I1,N−1 I1,N
I21 I22 . . · · · I2,N−1
I31 . . . · · ·. . . . · · ·. . IN−2,3
. IN−1,2
IN1
ITCS 4133/5133: Numerical Comp. Methods 35 Numerical Differentiation and Integration
Romberg Integration(contd.)
Algorithm (To solve for I0N� Determine I01 (Trapezoid formula)
� Determine Ii1, for i = 1, 2, . . .
� Determine Ii2, Ii3, etc., using Richardson’s formula.
ITCS 4133/5133: Numerical Comp. Methods 36 Numerical Differentiation and Integration
ITCS 4133/5133: Numerical Comp. Methods 37 Numerical Differentiation and Integration