Introduction to Simulation - Lecture 14

Multistep Methods II

Jacob White

Thanks to Deepak Ramaswamy, Michal Rewienski, and Karen Veroy

Outline

Small Timestep issues for Multistep MethodsReminder about LTE minimizationA nonconverging exampleStability + Consistency implies convergence

Investigate Large Timestep IssuesAbsolute Stability for two time-scale examples.Oscillators.

Multistep MethodsGeneral Notation

Basic Equations

( ) ( ( ), ( ))d x t f x t u tdt

=Nonlinear Differential Equation:

k-Step Multistep Approach: ( )( )0 0

ˆ ˆ ,k k

l j l jj j l j

j jx t f x u tα β− −

−= =

= ∆∑ ∑

Solution at discrete points

Time discretization

Multistep coefficients

2ˆ lx −

lt1lt −2lt −3lt −l kt −

ˆ lx1ˆ lx −

ˆ l kx −

Multistep MethodsSimplified Problem for

Analysis

( ) ( ) 0( ), 0d v t v t v vdt

λ λ= = ∈

Must Consid ALLer

Scalar ODE:

0 0

ˆ ˆk k

l j l jj j

j jv t vα β λ− −

= =

= ∆∑ ∑Scalar Multistep formula:

λ ∈

Growing Solutions

DecayingSolutions

Oscillations

( )Im λ

( )Re λ

Multistep MethodsConvergence Analysis

Convergence Definition

Definition: A multistep method for solving initial value problems on [0,T] is said to be convergent if given any

initial condition

( )0,

ˆmax 0 as t 0lTlt

v v l t⎡ ⎤∈⎢ ⎥∆⎣ ⎦

− ∆ → ∆ →

exactv

tˆ computed with 2

lv ∆ˆ computed with tlv ∆

Multistep MethodsConvergence Analysis

Two Conditions for Convergence

1) Local Condition: “One step” errors are small (consistency)

Typically verified using Taylor Series

2) Global Condition: The single step errors do not grow too quickly (stability)

Multi-step (k > 1) methods require careful analysis.

Multistep MethodsConvergence Analysis

Global Error Equation

0 0

ˆ ˆ 0k k

l j l jj j

j jv t vα β λ− −

= =

− ∆ =∑ ∑Multistep formula:

Exact solution Almostsatisfies Multistep Formula: ( ) ( )

0 0

k kl

j l j j l jj j

dv t t v t edt

α β− −= =

− ∆ =∑ ∑Local Truncation Error

(LTE)( ) ˆl llE v t v≡ −Global Error:

Difference equation relates LTE to Global error( ) ( ) ( )1

0 0 1 1l l l k l

k kt E t E t E eα λ β α λ β α λ β− −− ∆ + − ∆ + + − ∆ =

Multistep MethodsMaking LTE Small

Exactness Constraints

Local Truncation Error:

LTE

( ) ( ) 1If p pdv t t v t ptdt

−= ⇒ =

( ) ( )Can't be from d v t v tdt

λ=

( ) ( )0 0

k kl

j l j j l jj j

dv t t v t edt

α β− −= =

− ∆ =∑ ∑

( )( )( )

( )( )

( )0 0

1

k jk j

k kk

j jj j

p pk j t t p k

dv t v t

t

d

e

t

jα β

−−

= =

−− ∆ −∆ − ∆ =∑ ∑

Multistep MethodsMaking LTE Small

Exactness Constraint k=2 Example

( ) ( )0 0

1Exactness Cons train 0ts:k k

j jj j

p pk j p k jα β= =

−⎛ ⎞− − − =⎜ ⎟

⎝ ⎠∑ ∑

0

1

2

0

1

2

1 1 1 0 0 0 02 1 0 1 1 1 04 1 0 4 2 0 08 1 0 12 3 0 0

16 1 0 32 4 0 0

αααβββ

⎡ ⎤⎡ ⎤ ⎡ ⎤⎢ ⎥⎢ ⎥ ⎢ ⎥⎢ ⎥− − −⎢ ⎥ ⎢ ⎥⎢ ⎥⎢ ⎥ ⎢ ⎥=− − ⎢ ⎥⎢ ⎥ ⎢ ⎥⎢ ⎥− −⎢ ⎥ ⎢ ⎥⎢ ⎥⎢ ⎥ ⎢ ⎥− − ⎢ ⎥⎣ ⎦ ⎣ ⎦⎢ ⎥⎣ ⎦

For k=2, yields a 5x6 system of equations for Coefficients

p=0

p=1

p=2

p=3

p=4

i 0α =∑Note

Always

Multistep MethodsMaking LTE Small

Exactness Constraint k=2 example, generating methods

1

2

0

1

2

1 1 0 0 0 11 0 1 1 1 21 0 4 2 0 41 0 12 3 0 81 0 32 4 0 16

ααβββ

−⎡ ⎤⎡ ⎤ ⎡ ⎤⎢ ⎥⎢ ⎥ ⎢ ⎥− − − −⎢ ⎥⎢ ⎥ ⎢ ⎥⎢ ⎥⎢ ⎥ ⎢ ⎥=− − −⎢ ⎥⎢ ⎥ ⎢ ⎥− − −⎢ ⎥⎢ ⎥ ⎢ ⎥⎢ ⎥⎢ ⎥ ⎢ ⎥− − −⎣ ⎦ ⎣ ⎦⎣ ⎦

0First introduce a normalization, for example 1α =

0 1 2 0 1 21, 0, 1, 1/ 3, 4 / 3, 1/ 3α α α β β β= = = − = = =

Solve for the 2-step explicit method with lowest LTE

Solve for the 2-step method with lowest LTE

( )5Satisfies all five exactness constraints LTE C t= ∆

( )4Can only satisfy four exactness constraints LTE C t= ∆0 1 2 0 1 21, 4, 5, 0, 4, 2α α α β β β= = = − = = =

10-4 10-3 10-2 10-1 10010-15

10-10

10-5

100

Multistep MethodsMaking LTE Small

( ) ( )d v t v td

=

Trap

Beste

LTE

FE

Timestep

LTE Plots for the FE, Trap, and “Best” Explicit (BESTE).

Best Explicit Method has highest one-step

accurate

10-4 10-3 10-2 10-1 10010-8

10-6

10-4

10-2

100

Multistep MethodsMaking LTE Small

Global Error for the FE, Trap, and “Best” Explicit (BESTE).

[ ]( ) ( ) 0,1d v t v t td

= ∈

FE

Trap

Max Error

Where’s BESTE?

Timestep

10-4

10-3

10-2

10-1

10010

-100

100

10100

10200

Multistep MethodsMaking LTE Small

Global Error for the FE, Trap, and “Best” Explicit (BESTE).

( ) ( )d v t v td

=

FE Trap

Beste

Max Error

Best Explicit Method has lowest one-step error but global errror increases as

timestep decreases

worrysome

Timestep

Multistep MethodsStability of the method

Difference Equation

Why did the “best” 2-step explicit method fail to Converge?

( ) ( ) ( )10 0 1 1

l l l k lk kt E t E t E eα λ β α λ β α λ β− −− ∆ + − ∆ + + − ∆ =

Multistep Method Difference Equation

( ) ˆlv l t v∆ −Global Error

LTE

We made the LTE so small, how come the Global error is so large?

Multistep MethodsStability of the method

Stability Definition

Multistep Method Difference Equation( ) ( ) ( )1

0 0 1 1l l l k l

k kt E t E t E eα λ β α λ β α λ β− −− ∆ + − ∆ + + − ∆ =

Definition: A multistep method is stable if as

( )0, 0,

interval dependent

max max l lT Tl lt t

TE C T et⎡ ⎤ ⎡ ⎤∈ ∈⎢ ⎥ ⎢ ⎥∆ ∆⎣ ⎦ ⎣ ⎦

≤∆

0t∆ →

Global Error is bounded by a constant times the sum of the LTE’sStability means:

Given a kth order difference eqn with zero initial conditions1

0 , 0, , 0l l k l kka x a x u x x− − −+ + = = =

convolution sum

0

ll l j j

jx h u−

=

= ∑

( ) ( )1

,1 0

qMQ

lq m q

q m

lmh l

x can be related to the input u by

γ ς−

= =

=∑∑1

0 1 0k kka z a z a−+ + + =

Roots of

Root multiplicity

Aside on difference Equations

Convolution Sum

Root Relation

Aside on difference Equations

Convolution Sum

Bounding Terms

( ) ( )1

,1 0 0

,

qMQ ll j

q m qq m j

l jm

q mR

x l j uγ ς−

−

= = =

⎛ ⎞= −⎜ ⎟

⎝ ⎠∑∑ ∑

,If then<1, max jq q m jR C uς ≤

Independent of l

( ) 0q ,If t< 1 hen+ , maxl

jq j

eR C uε

ς εε

≤

Bounds distinct Roots

Multistep MethodsStability of the method

Stability Theorem

Theorem: A multistep method is stable if and only if

10 1Roots of 0 either: k k

kz zα α α−+ + + =

1. Have magnitude less than one2. Have magnitude equal to one and are distinct

Multistep MethodsStability of the method

Stability Theorem “Proof”

Given the Multistep Method Difference Equation( ) ( ) ( )1

0 0 1 1l l l k l

k kt E t E t E eα λ β α λ β α λ β− −− ∆ + − ∆ + + − ∆ =

( ) ( )0 0If, as 0, roots o 0f lk kt z tt α λ β α λ β− ∆ +∆ + − ∆→ =

• less than one in magnitude or• are distinct and bounded by 1 , 0tκ κ+ ∆ >

Then from the aside on difference equations

( )0, 0, 0,

max max max l t

l l lT T Tl l lt t t

C T

TCE e e Tt

C et T

eκ κ

⎡ ⎤ ⎡ ⎤ ⎡ ⎤∈ ∈ ∈⎢ ⎥ ⎢ ⎥ ⎢ ⎥∆ ∆ ∆⎣ ⎦ ⎣ ⎦

∆

⎣ ⎦

≤ ≤∆ ∆

Multistep MethodsStability of the method

Stability Theorem Picture

1-1Re

Im

( )0

roots of 0 for a nonzero k

k jj j

jt z tα λ β −

=

− ∆ = ∆∑

As 0, roots t∆ →move inward tomatch polynomial α0

roots of 0k

k jj

jzα −

=

=∑

Multistep MethodsStability of the method

The BESTE Method

Best explicit 2-step method0 1 2 0 1 21, 4, 5, 0, 4, 2α α α β β β= = = − = = =

Re

Im2roots of 4 5 0z z+ − =

Method is Wildly unstable!

-1 1-5

Multistep MethodsStability of the method

Dahlquist’s First Stability Barrier

For a stable, explicit k-step multistep method, the maximum number of exactness constraints that can be satisfied is less than or equal to k (note there are 2k-1

coefficients). For implicit methods, the number of constraints that can be satisfied is either k+2 if k is even

or k+1 if k is odd.

Multistep MethodsConvergence Analysis

Conditions for convergence, stability and consistency

1) Local Condition: One step errors are small (consistency)

Exactness Constraints up to p0 (p0 must be > 0)

( ) 0 11 0

0,max for pl

Tlt

e C t t t+

⎡ ⎤∈⎢ ⎥∆⎣ ⎦

⇒ ≤ ∆ ∆ < ∆

2) Global Condition: One step errors grow slowly (stability)

0roots of 0

kk j

jj

zα −

=

=∑

20, 0,

max maxl lT Tl lt t

TE C et⎡ ⎤ ⎡ ⎤∈ ∈⎢ ⎥ ⎢ ⎥∆ ∆⎣ ⎦ ⎣ ⎦

⇒ ≤∆

( ) 0

0,max pl

Tlt

E CT t⎡ ⎤∈⎢ ⎥∆⎣ ⎦

≤ ∆Convergence Result:

Inside the unit circle or on the unit circle and distinct

Backward-Euler Computed Solution

With Backward-Euler it is easy to use small timesteps for the fast dynamics and then switch to large timesteps for the

slow decay

small t∆

large t∆

( ) ( )d x t Ax tdt

=

( ) 2.1, 0.1eig A = − −

Circuit Example

Multistep MethodsTwo time-constant circuit

Large timestep stability

Forward-Euler Computed Solution

Multistep MethodsFE on two time-constant

circuit?

Large Timestep Stability

The Forward-Euler is accurate for small timesteps, but goes unstable when the timestep is enlarged

Multistep MethodsLarge Timestep Stability

( ) ( ) 0( ), 0d v t v t v vdt

λ λ= = ∈

FE, BE and Trap on the scalar ode problem

Scalar ODE:

( )1ˆ ˆ ˆ ˆ1l l l lv v t v t vλ λ+ = + ∆ = + ∆Forward-Euler:

Backward-Euler: ( )1 1 1 1ˆ ˆ ˆ ˆ ˆ

1l l l l lv v t v v v

tλ

λ+ + += + ∆ ⇒ =

−∆

If 1 1 the solution grows even if <0tλ λ+ ∆ >

1If 1 the solution decays even if 01 t

λλ

< >−∆

( ) ( )( )

1 1 1 1 0.5ˆ ˆ ˆ ˆ ˆ ˆ0.5

1 0.5ll l l l lt

v v t v v v vtλ

λλ

+ + + + ∆= + ∆ + ⇒ =

− ∆Trap Rule:

Multistep MethodsLarge Timestep Stability

FE large timestep region of absolute stability

Difference EqnStability region 1-1

( )1z tλ= + ∆

Im(z)

Re(z)

( )Im λ

( )Re λ

Forward Euler

ODE stability region

2t

−∆

Region ofAbsolute Stability

Multistep MethodsLarge Timestep Stability

FE large timestep stability, circuit example

Difference EqnStability region 1-1

Im(z)

Re(z)

( )Im λ

( )Re λ

ODE stability region

2t

−∆

Circuit example with t = 0.1, 2.1, 0.1λ∆ = − −

Region ofAbsolute Stability

Multistep MethodsLarge Timestep Stability

FE large timestep stability, circuit example

Difference EqnStability region 1-1

Im(z)

Re(z)

( )Im λ

( )Re λ

ODE stability region

2t

−∆

Circuit example with t=1.0, 2.1, 0.1λ∆ = − −

Unstable Difference Equation

Region ofAbsolute Stability

Multistep MethodsLarge Timestep Stability

BE large timestep region of absolute stability

Difference EqnStability region 1-1

Im(z)

Re(z)

( )Im λBackward Euler ( ) 11z tλ −= − ∆

Region ofAbsolute Stability

Multistep MethodsLarge Timestep Stability

BE large timestep stability, circuit example

Difference EqnStability region 1-1

Im(z)

Re(z)

( )Im λCircuit example with t = 0.1, 2.1, 0.1λ∆ = − −

Region ofAbsolute Stability

Multistep MethodsLarge Timestep Stability

BE large timestep stability, circuit example

Difference EqnStability region 1-1

Im(z)

Re(z)

( )Im λCircuit example with t =1.0, 2.1, 0.1λ∆ = − −

Stable Difference Equation

Region ofAbsolute Stability

Multistep MethodsLarge Timestep Stability

Stability Definitions

Region of Absolute Stability for a Multistep method:

A method is A-stable if its region of absolute stability

( )0

Values of where roots of 0k

k jj j

jt t zλ α λ β −

=

∆ − ∆ =∑are inside the unit circle.

includes the entire left-half of the complex plane

A-stable:

Dahlquist’s second Stability barrier:There are no A-stable multistep methods of convergenceorder greater than 2, and the trap rule is the most accurate.

0 5 10 15 20 25 30-6

-4

-2

0

2

4

Multistep methodsNumerical Experiments

Oscillating Strut and Mass

Why does FE result grow, BE result decay and the Trap rule preserve oscillations

Trap rule

Forward-Euler

Backward-Euler

0.1t∆ =

Multistep MethodsLarge Timestep Stability

FE large timestep oscillator example

Difference EqnStability region 1-1

( )1z tλ= + ∆

Im(z)

Re(z)

( )Im λ

( )Re λ

Forward Euler

ODE stability region

2t

−∆

Region ofAbsolute Stability

unstable oscillating

Multistep MethodsLarge Timestep Stability

BE large timestep oscillator example

Difference EqnStability region 1-1

Im(z)

Re(z)

( )Im λBackward Euler ( ) 11z tλ −= − ∆

Region ofAbsolute Stability

decaying oscillating

Difference EqnStability region 1-1

Im(z)

Re(z)

( )Im λTrap Rule

Region ofAbsolute Stability

oscillatingoscillating

( )( )1 0.51 0.5

tz

tλλ

+ ∆=

− ∆

Multistep MethodsLarge Timestep Stability

Trap large timestep oscillator example

Multistep Methods Large Timestep Issues

Two Time-Constant Stable problem (Circuit)FE: stability, not accuracy, limited timestep size.BE was A-stable, any timestep could be used.Trap Rule most accurate A-stable m-step method

Oscillator ProblemForward-Euler generated an unstable difference

equation regardless of timestep size.Backward-Euler generated a stable (decaying)

difference equation regardless of timestep size.Trapezoidal rule mapped the imaginary axis

Summary

Small Timestep issues for Multistep MethodsLocal truncation error and Exactness.Difference equation stability.Stability + Consistency implies convergence.

Investigate Large Timestep IssuesAbsolute Stability for two time-scale examples.Oscillators.

Didn’t talk aboutRunge-Kutta schemes, higher order A-stable methods.