1

Ordinary Differential Equations

Jyun-Ming Chen

2

Contents

• Review• Euler’s method• 2nd order methods

– Midpoint

– Heun’s

• Runge-Kutta Method

• Systems of ODE• Stability Issue

3

Review

• DE (Differential Equation)– An equation specifying the relations among the

rate change (derivatives) of variables

• ODE (Ordinary DE) vs. PDE (Partial DE)– The number of independent variables involved

)( 2

2

txxdt

xd ),( 1 yxf

y

f

x

f

4

Review (cont)

• Solution of an equation:

• Solution of DE vs. Solution of Equation

0)1(1 :Solution

01)( 2

fx

xxf

• Geometrically,

f(x)

x

5

Review (cont)

• Solution of an differential equation:

• Geometrically:

solutions validall are

,2,

is,That

:Solution

tt

ttt

exex

ecececx

xdt

dx

t

x

Need additional conditions to

specify a solution

6

Review (cont)

• Order of an ODE– The highest derivative in the equation

• nth order ODE requires n conditions to specify the solution– IVP (initial value problem): All conditions

specified at the same (initial) point– BVP (boundary value problem): otherwise

7

IVP VS. BVPRevisit Shooting Problem

t

g

y

(2) 5

(1) 10

10

)10(let

)(

212

1

2

2

ctcty

ctdt

dydt

yd

ggy

tfy

8

IVP vs. BVP

ttty

c

c

ty

ty

55)(

5

0

5)0(

0)0(

:IVP

2

1

2

tty

ccy

cy

ty

ty

605

6010010500)10(

0)0(

100)10(

0)0(

:BVP

2

11

2

Physical meaning

9

Review (cont)

• Linearity:– No product nor nonlinear functions of y and its

derivatives

• nth order linear ODE

)()()()()( 011

1

1 xbyxadx

dyxa

dx

ydxa

dx

ydxa

n

n

nn

n

n

10

Focus of This Chapter

• Solve IVP of nth order ODE numerically

• e.g.,

0 ),( Solve

0)0(

1)0( conditions initial ,

2

2

xxy

yx

dx

yd

dxdy

11

ODE (IVP)

• First order ODE (canonical form)

• Every nth order ODE can be converted to n first order ODEs in the following method:

0)0(),,( yxyyxfdx

dy

12

)()()()()( 011

1

1 xbyxadx

dyxa

dx

ydxa

dx

ydxa

n

n

nn

n

n

1

11

2

22

3

12

1

)(

)(

)(

)()(

n

nn

n dx

yd

dx

dyxy

dx

yd

dx

dyxy

dx

dy

dx

dyxy

xyxy

1

11

2

22

3

12

1

)(

)(

)(

)()(

n

nn

n dx

yd

dx

dyxy

dx

yd

dx

dyxy

dx

dy

dx

dyxy

xyxy

)()()()()( 10211

1

32

21

xbyxayxayxadx

dyxa

ydx

dy

ydx

dy

ydx

dy

nnn

n

nn

13

Example

xdx

dyx

dx

yd

dx

yd52 2

2

2

3

3

2

22

1

3

2

1

Let

dx

yddxdy

dxdy

dxdy

y

y

yy

21

32

22

33 52

ydx

dy

ydx

dy

xyxydx

dy

xyyx

y

y

yfy

y

y

y

y

52

)( ,Let

322

3

2

3

2

1

14

End of Review

15

xyxftytty

xyxfyxyxxy

xyxfy

yxfdx

dy

x

y

yxyyxfdx

dy

),()()(

),()()(

),(

),(

)(),,( 00

The Canonical Problem

dx

dy

x

yx

,0

This is Euler’s method

16

Example

1)0(, yydx

dy

719.0081.081.01.0)81.0()2.0()3.0(

81.009.09.01.0)9.0()1.0()2.0(

9.01.011.0)1()0()1.0(

1.0 Choose

1)1,0(,),(

xx

xx

xx

t

fyyxf

17

Example (cont)

1)0(

ln

:SolutionExact

x

x

ey

cyecy

cxy

dxy

dy

1.0

1

x

y

y=e–x

18

Error Analysis(Geometric Interpretation)

xyxfyy

xyxfyyy

xyxfy

yxfdx

dy

x

y

yxyyxfdx

dy

),(

),(

),(

),(

)(),,(

0001

0001

00

If the true solution were a straight line, thenEuler is exact

Think in terms of Taylor’s

expansion

19

Error Analysis(From Taylor’s Expansion)

2

62

2

32

),(),()(

)()()()()(x

dxd

xx

yxfxyxfxy

xyxyxxyxyxxy

Euler’sEuler’s truncation errorO(x2) per step

1st order method

20

Cumulative Error

x

y

x = 0 x = T

Number of steps = T/xCumulative Err. = (T/x) O(x2) = O(x)

Remark:x Error But computation time

21

Example (Euler’s)

xexyyydx

dy )(:exact 1, 0

)(9.0)1)(()(

1.0Let

)()()(

)(),(

xyxxyxxy

x

xxyxyxxy

xyyxf

,...729.0)3.0(,81.0)2.0(,9.0)1.0( yyy

22

Methods to Improve Euler

Motivated by Geometric Interpretation

23

Midpoint Method

xxyxfxyxxy

xyxfxyxyxx

xx

))(,()()(

))(,()()(

22

22

xxyxfxyxxy

xyxfxyxyxx

xx

))(,()()(

))(,()()(

22

22

24

Example (Midpoint)

xxyxy

xxyxfxyxxy

xyxyxy

yxfxyxy

yyxf

x

xx

xx

xx

2

22

22

22

1)( )(

))(,()()(

1)()()(

),()()(

),(

)(905.0)1.0)(05.01)(()()1.0(

1.0Let

1)()()( 2

xyxyxyxy

x

xxyxyxxy x

xexyyydx

dy )(:exact 1, 0

,...741.0)3.0(,819.0)2.0(,905.0)1.0( yyy

25

Heun’s Method

xxxyxxfyxfxyxxy

xxyxfxyxxy

))(,(),()()(

)(Predictor ))(,()()(0

21

0

xxxyxxfyxfxyxxy

xxyxfxyxxy

))(,(),()()(

)(Predictor ))(,()()(0

21

0

26

Example (Heun’s)

)(905.01.09.011)()1.0(

1.0Let

1)()()(

)()( )(

))(,(),()()(

1)()()(

),()()(

),(

21

21

021

021

0

xyxyxy

x

xxxyxyxy

xxxyxyxy

xxxyxxfyxfxyxxy

xxyxxyxy

xyxfxyxxy

yyxf

xexyyydx

dy )(:exact 1, 0

,...741.0)3.0(,819.0)2.0(,905.0)1.0( yyy

27

Remark

• Comparison of Euler, Heun, midpoint– 1st order: Euler

– 2nd order: Heun, midpoint

• “order”:• All are special cases of

RK (Runge-Kutta) methods

ExactEuler (error)

Midpoint (error)

y(0.1) 0.9050.9 (0.0

05)0.905

(0)

y(0.2) 0.8190.81 (0.

009)0.819

(0)

y(0.3) 0.7410.729 (0.

012)0.741

(0)

28

RK Methods

)1 to from slope (estimate

function lincrementa :),,(

)( ),(

where

),,(

1

1

ii

hyx

hxyyxyy

hhyxyy

ii

iiii

iiii

29

RK Methods (cont)

expansion sTaylor'match chosen to are ),,( Constants

constants:,

),(

),(

),(

),(

constant:'

11,111,1

22212123

11112

1

2211

ijii

iji

nnnniiin

ii

ii

ii

nn

qpa

qp

nkqnkqyhpxfk

hkqhkqyhpxfk

hkqyhpxfk

yxfk

sa

kakaka

30

Taylor’s Expansion

fy

f

x

f

yy

f

x

f

dx

dy

y

f

dx

dx

x

fyxf

dx

d

'),(

)(][

)(][),()(

)()()()()(

32

32

32

2

2

2

hOfhfy

hOfhyxfxy

hOxyhxyxyhxy

hiiy

f

ixf

ii

hiiy

f

ixf

iii

hidx

diii

31

RK 1st Order

method sEuler' ),(

1

expantion sTaylor' with Compare

),(

),(

1

1

1

11

111

hyxfyy

a

hyxfayy

yxfaka

n

iiii

iiii

ii

32

RK 2nd Order

)(

)(][

)()()(),(

: Expand

22112

21221

211112111

211112

2

hOhfqahpahfahfay

hOhhkqhpfahkayy

hOhkqy

fhp

x

fyxfk

k

iyf

xf

iii

yf

xf

iii

iiii

hkakayy

hkqyhpxfk

yxfk

kakan

ii

ii

i

)(

),(

),(

2

22111

11112

11

2211

33

RK 2nd Order (cont)

unknowns 4

eqns, 3

:

:

1 :

21

1122

21

122

21

qahf

pah

aahf

iyfxfi

21

1121

112 , ,01 qpaa

hyxfyhxfyy

hhkqyhpxfyhky

hkakayy

iiiiii

iiii

ii

)),(,(

),(

)(

21

21

1

11112

22111

11, , 11121

121

2 qpaa

),(),,(

)( )(

21

2121

22111

hfyhxfkyxfk

hkkyhkakayy

iiiii

iii

34

RK 4th Order

• Mostly commonly used one

• Higher order … more evaluation, but less gain on accuracy

)(

)(

)(

)(

where

)22(

34

221

21

3

121

21

2

1

432161

1

hkh,yxfk

hkh,yxfk

hkh,yxfk

,yxfk

hkkkkyy

nn

nn

nn

nn

nn

35

System of ODE

• Convert higher order ODE to 1st order ODEs

• All methods equally apply, in vector form

36

Example (Mass-Spring-Damper System)

• Governing Equation

• After setting the initial conditions x(0) and x’(0), compute the position and velocity of the mass for any t > 0

0 kxxcxm

c

k

Initial Condition

x

m

37

0)0(

1)0(:IC

)(10

then

,

2

1

212

21

122

21

x

x

cxkxm

x

xxkxcxxm

xxxx

Example (cont)

0)0(,),()()(

Euler Dim

yyxyxfxyxxy

n

0)0(,),()()(

Euler Dim

yyxyxfxyxxy

n

38

set t=0.1

27.0

971.01.0

)19.0(99.0

19.0

19.0

99.01.0)2.0),2.0(()2.0()3.0(

19.0

99.01.0

)1.0(1

1.0

1.0

11.0)1.0),1.0(()1.0()2.0(

1.0

11.0

01

0

0

11.0)0.0),0.0(()0.0()1.0(

xfxx

xfxx

xfxx

Example (cont)

)(),(

211

2

2

1

cxkx

xtxf

x

xx

m

Assume m=1,c=1, k=1 (for ease of computation)

21

2

xx

xf

39

Stability: Symptom

xexy

yydx

dy

10)( :solutionexact

1)0(,10

40

Stability (cont)• Example Problem:

xexy

yydx

dy

)( :solutionexact

0,1)0(,

xexy

yydx

dy

)( :solutionexact

0,1)0(,

x

iiii

exy

h

h

hh

h

h

yhhyyy

)( :solutionexact

2limit stability

2

1 1or ,1 1

is,That

11 if unstable

11 if stable

)1(

:Euler

*

1

Conditionally stable

41

Discussion

• Different algorithm different stability limit– Check Midpoint Method

• Different problem different stability limit– use the previous problem as benchmark

42

Implicit Method (Backward Euler)

)(),(

)(),(

),()(

)1,(:Euler Backward

2111

21

1

hOyhyxfy

hOyhyxfy

yxfhOh

yy

ii

iiiii

iiii

iiii

)(),(

),()(

),1( :Euler

),(

21

1

hOhyxfyy

yxfhOh

yy

ii

yxf

iiii

iiii

dxdy

43

Example

• Remark:– Always stable (for this problem)– Truncation error the same as Euler (only

improve the stability)

ihi

ii

iiiiii

yy

hOyyh

hOyhyyhyxfy

yyxf

11

1

21

211111

)()1(

)(),(

),(

44

Linear System of ODE with Constant Coefficients

ii

ii

iii

yhCIy

hOyyhCI

hOhCyyy

Cyy

11

1

21

)(

)(

45

Semi-Implicit Euler

• Not guaranteed to be stable, but usually is

46

c

k

Initial Condition

x

m

0

:Motion ofEquation

kxxcxm

xmkxxcxmf

nn yhh

hyyy

xxx

xx

xx

xx

kcm

1

1

212

21

2

1

1

1

11

10

1Let

xc

force damping

47

c

k

Initial Condition

x

m

0

:Motion ofEquation

2

2

kxxcxm

xmkxxcxmf

221

2

1

21

2

2212

21

2

1

21

1

21

10

)(

1Let

xx

x

hxh

hhyy

xy

f

yfy

xxx

xx

xx

xx

kcm

nn

2

force damping

xc

48

Stiff Set of ODE

Stability limit

Use the change of variable

101)0(

121)0(

1000210002

2

2121

21

100021

ccccv

ccu

eczecy

zvuzvu

yvuyvu

zyvzyu

xx

101)0(

121)0(

1000210002

2

2121

21

100021

ccccv

ccu

eczecy

zvuzvu

yvuyvu

zyvzyu

xxGet the following solution:

49

50

Adaptive Stepsize

• Solving ODE numerically … tracing the integral curve y(x)

• what’s wrong with uniform step size– Uniformly small: waste

effort

– Uniformly large: might miss details

51

Step Doubling

• Idea:– Estimate the truncation error by taking each

step twice: one full step, two half steps– control the step size such that the estimated

error is not too big.

1(2h1)

2(h1)Desired h0

52

Ex: RK 2nd Order

00

11

11

00

3

1

0

1

0

3312

432

431

432

431

error desired theproduce tostepsize the:

stepby error estimated the:

error)n truncatio(the 6

)()(2)()2(

)()(2)2(

)()2()2(

31

h

h

hhh

h

hchyy

hOhcyhOhcy

hOhcyhxy

hOhcyhxy

Overhead: # of f(x,y) evaluations24–2 = 6

Overhead: # of f(x,y) evaluations24–2 = 6

53

Example

xxy

yyy

cos)( :Exact

1)0(,sin

54

Ex (Semi-implicit Euler)

g

ty

h

hh

ty

ty

hty

hty

fh

hyfh

hyy

yy

ff

y

fJ

yygy

yyyygy

nnnnn

)(

0)(

)(

)(

)(

10

1

10

1

00

10

,

,

3)0(,0)0(,3)0(,0)0(,

22

2

1

2

1

1

1

21

21

212

21

hg

ty

ty

ty

hty

hty

hfyy

Euler

nnn

)(

)(

)(

)(

)( 2

2

1

2

1

1

55

nnn hfyhJIyhJI

vu

ff

y

fJ

1

21

1999999

1998998

,

,