INTRODUCTION TO THE FINITE DIFFERENCE METHOD

Basic Theory Taylor’s Theorem If a function f and its first n+1 derivatives are continuous in an interval containing the points xi and xi+1, then the value of the function f at the point xi+1 is given by:

2 31 1 1 1

''( ) '''( )( ) ( ) '( ) ( ) ( ) ( )

2! 3!i i

i i i i i i i i i

f x f xf x f x f x x x x x x x

( )

1

( )... ( )

!

nni

i i n

f xx x R

n

where the remainder Rn is defined as:

1 ( 1)1( )( )

!

i

i

nx ni

n x

x tR f t dt

n

If the remainder Rn is omitted in the above series expansion, then the expression obtained is the Taylor polynomial approximation to f(x). After some manipulation, the remainder Rn can be written in the form:

( 1)1

1

( )( )

( 1)!

nn

n i i

fR x x

n

where 1i ix x . A Taylor series provides a means of predicting the value of a function f at a point xi+1 in terms of the value of the function f and its derivatives at a point xi. The more terms are included in the series, the greater the accuracy of the approximation.

Consider the following cases: First Approximation (zero order):

)()( 1 ii xfxf

This approximation is reasonable if the points xi+1 and xi are close to each other, and

is a correct estimate if the function f(x)

is constant.

Second Approximation (first order):

))((')()( 11 iiiii xxxfxfxf

This approximation includes a term which is the slope multiplied by the distance between the points. The expression is now of the form y = c + mx, and

is a correct estimate if the

function f(x) is linear.

Third Approximation (second order):

2111 )(

!2

)(''))((')()( ii

iiiiii xx

xfxxxfxfxf

This approximation now includes the effect of curvature due to the second-order derivative. The expression is a correct estimate if the function f(x) is quadratic. Additional terms can be included as required. Writing

1i ix x h the Taylor series can be written as

( )2 3

1

''( ) '''( ) ( )( ) ( ) '( ) ...

2! 3! !

nni i i

i i i n

f x f x f xf x f x h f x h h h R

n

with the remainder Rn given by

( 1)1 ( )

( 1)!

nn

n

fR h

n

If we truncate the Taylor series after the n+1th term, we will introduce a truncation error of order hn+1, or O(hn+1). The notation O(hn+1) means that the truncation error is of the order hn+1. That is, the error is proportional to the step size h raised to the power n+1. Although this approximation implies nothing regarding the magnitude of the derivatives that multiply hn+1, it is useful in judging the comparative error of numerical methods based on Taylor series expansions. For example, if the error is O(h), halving the step size h will halve the error. On the other hand, if the error is O(h2), halving the step size h will quarter the error. Errors Two types of errors will occur in the numerical approximation of a function by a Taylor series:

Truncation error: due to the truncation of the Taylor series

Round-off error: due to the fact that all types of

computing machines have a finite precision. Taylor series approximations are basically dominated by truncation errors for large values of h, and by round-off errors for small values of h.

Finite Differences Simple difference formulas can be created by truncating the Taylor series after the first derivative term. Examples are: Forward Difference:

21( ) ( ) '( ) ( )i i if x f x h f x O h

1( ) ( )

'( ) ( )i ii

f x f xf x O h

h

Backward Difference:

21( ) ( ) '( ) ( )i i if x f x h f x O h

1( ) ( )

'( ) ( )i ii

f x f xf x O h

h

Higher-order approximations can be created by retaining additional terms of the Taylor series, as follows:

2 31

''( )( ) ( ) '( ) ( )

2!i

i i i

f xf x f x h f x h O h

2 32

''( )( ) ( ) 2 '( ) (2 ) ( )

2!i

i i i

f xf x f x h f x h O h

Multiplying the first equation above by 4 gives:

2 314 ( ) 4 ( ) 4 '( ) 2 ''( ) ( ) (1)i i i if x f x h f x h f x O h

2 3

2( ) ( ) 2 '( ) 2 ''( ) ( ) (2)i i i if x f x h f x h f x O h Subtracting equation (2) from equation (1) gives

31 24 ( ) ( ) 3 ( ) 2 '( ) ( )i i i if x f x f x h f x O h

and rearranging produces the final result:

22 1( ) 4 ( ) 3 ( )'( ) ( )

2i i i

i

f x f x f xf x O h

h

The above expression is a higher-order forward difference approximation. Proceeding in a similar fashion, the following higher-order backward difference approximation can be produced:

21 23 ( ) 4 ( ) ( )'( ) ( )

2i i i

i

f x f x f xf x O h

h

Another possibility is to generate what is known as a central finite difference formula. To this end, we start from the following two Taylor series:

2 31

''( )( ) ( ) '( ) ( )

2!i

i i i

f xf x f x h f x h O h

2 3

1

''( )( ) ( ) '( ) ( )

2!i

i i i

f xf x f x h f x h O h

Subtracting the second expression from the first gives:

31 1( ) ( ) 2 '( ) ( )i i if x f x h f x O h

and rearranging produces the final result:

21 1( ) ( )'( ) ( )

2i i

i

f x f xf x O h

h

Notice that the addition of the above two expressions including the third-order derivative gives:

2 41 1( ) ( ) 2 ( ) ''( ) ( )i i i if x f x f x h f x O h

and rearranging produces a central finite difference approximation for the second-order derivative:

21 12

( ) 2 ( ) ( )''( ) ( )i i i

i

f x f x f xf x O h

h

Forward, backward and central finite difference formulas for higher-order derivatives can also be obtained by using the above procedures (see tables). Note: The appropriate use of forward, backward and central finite difference formulas depends on the available data. Graphical Interpretation Given a function y = u(x), the derivative du/dx at a point x = xi is given by:

0

( ) ( )lim

i

i ih

x x

u x h u xdu

dx h

This represents the slope of the tangent to the curve at the point x = xi.

T

Qy = f (x)

RP

a a + h x

y

f (a + h)

f (a)

0

Approximately, we can write:

( ) ( )

i

i i

x x

u x h u xdu

dx h

where h is a small but finite increment. This forward difference formula represents an approximation of the tangent at x = xi by the chord AB. Similarly, the backward difference formula:

( ) ( )

i

i i

x x

u x u x hdu

dx h

represents an approximation of the tangent at x = xi by the chord AC. The central difference formula:

( ) ( )

2i

i i

x x

u x h u x hdu

dx h

can also be viewed as an approximation of the tangent at x = xi by the chord BC. Example 1 Estimate the first derivative of the function:

4 3 2( ) 0.1 0.15 0.5 0.25 12f x x x x x

at the point x = 0.5, using a step size h = 0.5. Since

3 2'( ) 0.4 0.45 1.0 0.25f x x x x

the correct value of the derivative at the point x = 0.5 is '(0.5) 0.9125f .

We start by using a forward difference formula:

1

1

( ) ( )'( ) i i

ii i

f x f xf x

x x

In this case, xi = 0.5, xi+1 = 1.0 and xi+1 xi = 0.5, thus

(1.0) (0.5)'(0.5) 1.45

0.5

f ff

t

True Approx 0.9125 ( 1.45)% error ε 58.9%

True 0.9125

Now using a backward difference formula:

1

1

( ) ( )'( ) i i

ii i

f x f xf x

x x

with xi = 0.5, xi-1 = 0 and xi xi-1 = 0.5,

(0.5) (0.0)'(0.5) 0.55

0.5

f ff

t

0.9125 ( 0.55)% error ε 39.7%

0.9125

Using a central difference formula:

1 1

1 1

( ) ( )'( ) i i

ii i

f x f xf x

x x

with xi = 0.5, xi+1 = 1.0, xi-1 = 0 and xi+1 xi-1 = 2h = 1.0,

(1.0) (0.0)'(0.5) 1.0

1.0

f ff

t

0.9125 ( 1.0)% error ε 9.6%

0.9125

Let us now decrease the step size from h = 0.5 to h = 0.25: Forward difference:

(0.75) (0.5)'(0.5) 1.155

0.25

f ff

t% error ε 26.5%

Backward difference:

(0.5) (0.25)'(0.5) 0.714

0.25

f ff

t% error ε 21.7%

Central difference:

(0.75) (0.25)'(0.5) 0.934

0.5

f ff

t% error ε 2.4%

Note: The backward and forward difference formulas used in the above example only have an accuracy of O(h). So, halving the step size h halved the error. On the other hand, the central difference formula has an accuracy of O(h2). So, halving the step size h has quartered the error since 0.52 = 0.25.

Example 2 Use high accuracy formulas to estimate the first derivative of the function:

4 3 2( ) 0.1 0.15 0.5 0.25 12f x x x x x

at the point x = 0.5, using a step size h = 0.25. High accuracy forward difference formula:

2 1( ) 4 ( ) 3 ( )'( )

2i i i

i

f x f x f xf x

h

Here xi = 0.5, xi+1 = 0.75, xi+2 = 1.0 and h = 0.25, thus

(1.0) 4 (0.75) 3 (0.5)'(0.5) 0.860

0.5

f f ff

tε 5.82%

High accuracy backward difference formula:

1 23 ( ) 4 ( ) ( )'( )

2i i i

i

f x f x f xf x

h

Here xi = 0.5, xi-1 = 0.25, xi-2 = 0.0 and h = 0.25, thus

3 (0.5) 4 (0.25) (0.0)'(0.5) 0.878

0.5

f f ff

tε 3.77%

High accuracy central difference formula:

2 1 1 2( ) 8 ( ) 8 ( ) ( )'( )

12i i i i

i

f x f x f x f xf x

h

Here xi = 0.5, xi+1 = 0.75, xi+2 = 1.0, xi-1 = 0.25, xi-2 = 0.0 and h = 0.25, thus

(1.0) 8 (0.75) 8 (0.25) (0.0)'(0.5) 0.9125

12 0.25

f f f ff

tε 0 In this case, the high accuracy central difference formula produced the exact solution. This is not unexpected since the exact expression for the derivative is a cubic function and the error of the above formula is O(h4), as can be seen in the table. Example 3 Given a problem described by the following differential equation:

''( ) ( ) 0f x f x x

and the boundary conditions

( ) 0 at 0f x x

( ) 0 at 1f x x

estimate the value of f(x) at the point x = 0.5. Substituting the central difference formula:

1 12

( ) 2 ( ) ( )''( ) i i i

i

f x f x f xf x

h

into the original differential equation applied at the point xi gives:

1 12

( ) 2 ( ) ( )( ) 0i i i

i i

f x f x f xf x x

h

which can be rewritten in the form:

2 21 1( ) 2 ( ) ( ) ( ) 0i i i i if x f x f x h f x h x

2 2

1 12 ( ) ( ) ( )i i i ih f x f x f x h x

In this case, xi = 0.5, xi+1 = 1.0, xi-1 = 0.0 and h = 0.5, thus

2 20.5 2 (0.5) (1.0) (0.0) 0.5 0.5f f f

Since the boundary conditions of the problem state that f(1.0) = 0 and f(0.0) = 0, we have

2

2

0.5 0.5(0.5) 0.0714

0.5 2f

The exact value is f(0.5) = 0.0697, thus the error is t% error ε 2.4%

If we now adopt a step size h = 0.25 instead of h = 0.5, the number of unknowns will increase. This means that the number of equations will also have to be increased, as follows. First, we apply the general equation

2 21 12 ( ) ( ) ( )i i i ih f x f x f x h x

at the point xi = 0.25, giving:

2 20.25 2 (0.25) (0.5) (0.0) 0.25 0.25f f f

But since f(0.0) = 0, the above equation can be rearranged as follows:

2

2

(0.5) 0.25 0.25(0.25) (1)

0.25 2

ff

Now apply the general equation at the point xi = 0.5, giving:

2 20.25 2 (0.5) (0.75) (0.25) 0.25 0.5 (2)f f f

Finally, apply the general equation at the point xi = 0.75, giving:

2 20.25 2 (0.75) (1.0) (0.5) 0.25 0.75f f f

But since f(1.0) = 0, the above equation can be rearranged as follows:

2

2

(0.5) 0.25 0.75(0.75) (3)

0.25 2

ff

Substituting equations (1) and (3) above into equation (2), the value of f(0.5) can be calculated as follows:

2 2

2 22 2

(0.5) 0.25 0.75 (0.5) 0.25 0.250.25 2 (0.5) 0.25 0.5

0.25 2 0.25 2

f ff

(0.5) 0.0701f

with an error

t% error ε 0.6%

We can also calculate the following values:

(0.25) 0.0442 (exact value: (0.25) 0.0440)f f

(0.75) 0.0604 (exact value: (0.75) 0.0601)f f

Exact solution:

sin( )

sin1

xf x x

Example 4 Given the problem of mass balance within a chemical reactor, described by the following advection-dispersion equation with first-order chemical reaction:

2

20

d c dcD U kc

dx dx

where: c: concentration, moles/m3

U: velocity of the water flowing through the reactor, m/h D: dispersion coefficient, m2/h k: first-order decay coefficient, h-1 and the boundary conditions

1 at 0c x 0 atc x L

with L the reactor length, estimate the value of the concentration at the mid-point of the reactor. Central finite difference approximations:

21 1

2 2

( ) 2 ( ) ( )

i

i i i

x x

c x c x c xd c

dx h

1 1( ) ( )

2i

i i

x x

c x c xdc

dx h

Substituting the above approximations into the original differential equation applied at the point xi gives:

1 1 1 12

( ) 2 ( ) ( ) ( ) ( )( ) 0

2i i i i i

i

c x c x c x c x c xD U kc x

h h

which can be rearranged in the form:

1 12 2 2

2( ) ( ) ( ) 0

2 2i i i

D U D D Uc x k c x c x

h h h h h

The above expression is a finite difference approximation of the original differential equation. Given the values: D = 0.2; U = 1; k = 0.5; L = 0.5; and assuming that h = L/2 = 0.25, we have that

2 2

0.2 13.2 2 1.2

2 0.25 2 0.25

D U

h h

2 2

2 2 0.20.5 6.4 0.5 6.9

0.25

Dk

h

2 2

0.2 13.2 2 5.2

2 0.25 2 0.25

D U

h h

Thus:

1.2 (0.5) 6.9 (0.25) 5.2 (0.0) 0c c c

5.2

(0.25) 0.7536.9

c

If we now assume that h = L/4 = 0.125, we have that

2 2

0.2 112.8 4 8.8

2 0.125 2 0.125

D U

h h

2 2

2 2 0.20.5 25.6 0.5 26.1

0.125

Dk

h

2 2

0.2 112.8 4 16.8

2 0.125 2 0.125

D U

h h

General equation:

1 18.8 ( ) 26.1 ( ) 16.8 ( ) 0i i ic x c x c x

Applying the above equation at xi = 0.125 gives:

8.8 (0.25) 26.1 (0.125) 16.8 (0.0) 0c c c

8.8 (0.25) 16.8

(0.125)26.1

cc

Applying the above equation at xi = 0. 25 gives:

8.8 (0.375) 26.1 (0.25) 16.8 (0.125) 0c c c Applying the above equation at xi = 0.375 gives:

8.8 (0.5) 26.1 (0.375) 16.8 (0.25) 0c c c

16.8 (0.25)

(0.375)26.1

cc

Therefore:

16.8 (0.25) 8.8 (0.25) 16.88.8 26.1 (0.25) 16.8 0

26.1 26.1

c cc

The final solution is c(0.25) = 0.732, c(0.125) = 0.890 and c(0.375) = 0.471.

Forward Finite Difference Approximations First Derivative

1( ) ( )'( ) ( )i i

i

f x f xf x O h

h

22 1( ) 4 ( ) 3 ( )'( ) ( )

2i i i

i

f x f x f xf x O h

h

Second Derivative

2 12

( ) 2 ( ) ( )''( ) ( )i i i

i

f x f x f xf x O h

h

23 2 12

( ) 4 ( ) 5 ( ) 2 ( )''( ) ( )i i i i

i

f x f x f x f xf x O h

h

Third Derivative

3 2 13

( ) 3 ( ) 3 ( ) ( )'''( ) ( )i i i i

i

f x f x f x f xf x O h

h

24 3 2 13

3 ( ) 14 ( ) 24 ( ) 18 ( ) 5 ( )'''( ) ( )

2i i i i i

i

f x f x f x f x f xf x O h

h

Fourth Derivative

4 3 2 14

( ) 4 ( ) 6 ( ) 4 ( ) ( )''''( ) ( )i i i i i

i

f x f x f x f x f xf x O h

h

25 4 3 2 14

2 ( ) 11 ( ) 24 ( ) 26 ( ) 14 ( ) 3 ( )'''( ) ( )i i i i i i

i

f x f x f x f x f x f xf x O h

h

Backward Finite Difference Approximations First Derivative

1( ) ( )'( ) ( )i i

i

f x f xf x O h

h

21 23 ( ) 4 ( ) ( )'( ) ( )

2i i i

i

f x f x f xf x O h

h

Second Derivative

1 22

( ) 2 ( ) ( )''( ) ( )i i i

i

f x f x f xf x O h

h

21 2 32

2 ( ) 5 ( ) 4 ( ) ( )''( ) ( )i i i i

i

f x f x f x f xf x O h

h

Third Derivative

1 2 33

( ) 3 ( ) 3 ( ) ( )'''( ) ( )i i i i

i

f x f x f x f xf x O h

h

21 2 3 43

5 ( ) 18 ( ) 24 ( ) 14 ( ) 3 ( )'''( ) ( )

2i i i i i

i

f x f x f x f x f xf x O h

h

Fourth Derivative

1 2 3 44

( ) 4 ( ) 6 ( ) 4 ( ) ( )''''( ) ( )i i i i i

i

f x f x f x f x f xf x O h

h

21 2 3 4 54

3 ( ) 14 ( ) 26 ( ) 24 ( ) 11 ( ) 2 ( )'''( ) ( )i i i i i i

i

f x f x f x f x f x f xf x O h

h

Central Finite Difference Approximations First Derivative

21 1( ) ( )'( ) ( )

2i i

i

f x f xf x O h

h

42 1 1 2( ) 8 ( ) 8 ( ) ( )'( ) ( )

12i i i i

i

f x f x f x f xf x O h

h

Second Derivative

21 12

( ) 2 ( ) ( )''( ) ( )i i i

i

f x f x f xf x O h

h

42 1 1 22

( ) 16 ( ) 30 ( ) 16 ( ) ( )''( ) ( )

12i i i i i

i

f x f x f x f x f xf x O h

h

Third Derivative

22 1 1 23

( ) 2 ( ) 2 ( ) ( )'''( ) ( )

2i i i i

i

f x f x f x f xf x O h

h

43 2 1 1 2 33

( ) 8 ( ) 13 ( ) 13 ( ) 8 ( ) ( )'''( ) ( )

8i i i i i i

i

f x f x f x f x f x f xf x O h

h

Fourth Derivative

22 1 1 24

( ) 4 ( ) 6 ( ) 4 ( ) ( )''''( ) ( )i i i i i

i

f x f x f x f x f xf x O h

h

43 2 1 1 2 3

4

( ) 12 ( ) 39 ( ) 56 ( ) 39 ( ) 12 ( ) ( )'''( ) ( )

6i i i i i i i

i

f x f x f x f x f x f x f xf x O h

h