1

Numerical Differentiation

Jyun-Ming Chen

2

Contents

• Forward, Backward, Central Difference

• Richardson Extrapolation

3

Taylor’s Expansion

)()(2

1)()()(

)()(2

1)()()(

)()(2

1)()()(

2

32

32

hOhxfh

xfhxfxf

hOhxfxfhxfhxf

hOhxfhxfxfhxf

4

Forward Difference Formula for

)(xf

)( )()(

)( hOerrorh

xfhxfxf

)(

)()()( hOerror

h

xfhxfxf

Geometrically

h

)()( xfhxf

)(xf

hx x

)(xf

5

Backward Difference Formula for )(xf

)(

)()()(

hOerrorh

hxfxfxf

)(

)()()(

hOerrorh

hxfxfxf

Similarly

)()(2

1)()()( 32 hOhxfhxfxfhxf

h

)(xf

xxhx

)()( hxfxf

Geometrically

6

Central Difference Formula for

)()(!4

1)(

6

1)(

2

1)()()( 54)4(32 hOhxfhxfhxfhxfxfhxf

)(xf

)()(!4

1)(

6

1)(

2

1)()()( 54)4(32 hOhxfhxfhxfhxfxfhxf –)

)()( hxfhxf )(2 xfh 3)(3

1hxf )( 5hO

)()(3

1)()()(2 53 hOhxfhxfhxfxfh

7

Central Difference Formula for )(xf

h

hxfhxfxf

2

)()()(

)( 2hOerror

)(xf

x

2h

)()( hxfhxf

hx x hx

)(xf

Geometrically

8

Example

3)( xxf 23)(( xxf )3)1( f,

Calculate f’(1) using FD, BD, CD

9

Example (cont)

h=0.1

h=0.1

h=0.1

h=0.05

h=0.05

h=0.05

31.31.0

)1()1.1()1(

ff

f

1525.305.0

)1()05.1()1(

ff

f

71.21.0

)9.0()1()1(

ff

f

8453.205.0

)95.0()1()1(

ff

f

01.32.0

)9.0()1.1()1(

ff

f

00250.31.0

)95.0()05.1()1(

ff

f

31.0error

1525.0error

29.0error

1547.0error

01.0error

00250.0error

herror

2herror

FD:

BD:

CD:

10

Example (cont)

• Remarks:– FD, BD, CD each involves 2 function calls, 1

subtraction, and 1 division: same computation time

– CD is the most accurate (hence, the most recommended method)

– However, sometimes, CD cannot be applied

11

Forward Difference Formula for )(xf

)()2)((6

1)2)((

2

12)()()2( 432 hOhxfhxfhxfxfhxf

)()(6

1 )(

2

1 )( )()( 432 hOhxfhxfhxfxfhxf –2）

)(]3

1

3

4[)(]

2

122[)()()(2)2( 432 hOhxfhxfxfhxfhxf

)()()()(2)2()( 432 hOhxfxfhxfhxfhxf

2

)()(2)2()(

h

xfhxfhxfxf

)(hOerror

12

Backward Difference Formula for )(xf

)()2)((6

1)2)((

2

12)()()2( 432 hOhxfhxfhxfxfhxf

)()(6

1 )(

2

1 )( )()( 432 hOhxfhxfhxfxfhxf –2）

)(]3

1

3

4[)(]

2

122[)()()(2)2( 432 hOhxfhxfxfhxfhxf

)()()()(2)2()( 432 hOhxfxfhxfhxfhxf

2

)2()(2)()(

h

hxfhxfxfxf

)(hOerror

13

Central Difference Formula for

)()(!4

1)(

6

1)(

2

1)()()( 54)4(32 hOhxfhxfhxfhxfxfhxf

)(xf

)()(!4

1)(

6

1)(

2

1)()()( 54)4(32 hOhxfhxfhxfhxfxfhxf +)

)()(12

1 )( )(2)()( 64)4(2 hOhxfhxfxfhxfhxf

)()(2)()(

)( 22

hOh

xfhxfhxfxf

)(

)(2)()()( 2

2hO

h

xfhxfhxfxf

Similar remark on the selection of FD|BD|CD applies for f”(x)

14

More Accurate FD Formula for )(xf

)()(2

1)()()( 32 hOhxfhxfxfhxf

)()]()()(2)2(

[2

1)()()( 3

22 hOhO

h

xfhxfhxfhhxfxfhxf

)()(2

1)()2(

2

1)()( 3hOxfhxfhxfhxfxf

)()(

23

)(2)2(21

)( 2hOh

xfhxfhxfxf

)(2

)(3)(4)2()( 2hO

h

xfhxfhxfxf

)(

2

)(3)(4)2()( 2hO

h

xfhxfhxfxf

15

• Better accuracy can be achieved using this formula

• But, it involves more computations: – 3 function calls, two +/–, one division

• Trade-off: – More computation is the price you paid for better

accuracy

• Similar idea applies to more accurate BD formula

More Accurate FD Formula (cont)

16

Richardson Extrapolation

Idea:• exact

= computed+ error

• The truncation error is of the form: chk

– where c is some constant

• Use different h to estimate the truncation error

• Use extrapolation to get more accurate result

17

• Example: CD for f’(x)

• Using Different h (h1, h2)：

• c1 and c2 could be different

Richardson Extrapolation (cont)

2111 hcRexact

2222 hcR

)(2

)()()( 2hO

h

hxfhxfxf

18

• If

Richardson Extrapolation (cont)

ccc 21

exactchRchR 222

211

1222

21 RRhhc 2

22

1

12

hh

RRc

22

22

212

2

122

222

22

1

122

1

h

hh

h

RRRh

hh

RRRexact

12

122

2

1

hh

RRRexact

19

• f(x) = x3. Use CD with Richardson extrapolation to compute f’(1)

Example

05.0

1.0

2

1

h

h

2

1

0025.3)1(

01.3)1(

Rf

Rf

0000.3

105.01.0

01.30025.30025.3 2

exact

Magic?Coincidence

?

20

Revisit CD Formula for )(xf

5)5(4)4(32 )(!5

1)(

!4

1)(

!3

1)(

2

1)()()( hxfhxfhxfhxfhxfxfhxf

5)5(4)4(32 )(!5

1)(

!4

1)(

!3

1)(

2

1)()()( hxfhxfhxfhxfhxfxfhxf–)

5)5(3 )(!5

2)(

!3

2)(2)()( hxfhxfxfhhxfhxf

4)5(2 )(!5

2)(

!3

2

2

)()()( hxfhxf

h

hxfhxfxf

Change notation:

)()()( 642

21 hOhahahFxf

21

Error Analysis

64

2

2

1 22~

2~)()

2(

hO

ha

haxf

hF

)()()( 642

21 hOhahaxfhF

2211~ and ~ aaaa Assuming

Eliminate a1 to get better accuracy

22

Error Analysis (cont)

)(4

1)()

2(4

3

1)(

)(16

141)(3)()

2(4

642

642

hOhahFh

Fxf

hOhaxfhFh

F

64

2

2

1 222)()

2(

hO

ha

haxf

hF

)()()( 642

21 hOhahaxfhF

4

–)

O(h4)

23

Revisit Previous Example

• f(x) = x3. Use CD with Richardson extrapolation to compute f’(1)

• a2 involves f(5)(x), hence, the exact solution is no surprise.

24

Remark

• How much effort did we use to get this level of accuracy?– F(h): f(x+h), f(x-h); one –, one – F(h/2): f(x+h/2), f(x-h/2); one –, one – R.E.: two , one –

25

Summary (Textbook p.373)

26

27

http://earendil.ath.cx/content/academic/numeralysis1.doc

אקסטרפולציית ריצ'רדסון לביטול איבר השגיאה המוביל

1

1

1

1

0 0 0 01 1

0 00

0 00

2

1

1

i i i

i i

p p pi i

i i

p

p

pp p

ipi

F h a a x F h a a x

F h F hF h

F h F ha a x