Jacob White Thanks to Deepak Ramaswamy, Michal ......Thanks to Deepak Ramaswamy, Michal Rewienski,...
Transcript of Jacob White Thanks to Deepak Ramaswamy, Michal ......Thanks to Deepak Ramaswamy, Michal Rewienski,...
Introduction to Simulation - Lecture 14
Multistep Methods II
Jacob White
Thanks to Deepak Ramaswamy, Michal Rewienski, and Karen Veroy
Outline
Small Timestep issues for Multistep MethodsReminder about LTE minimizationA nonconverging exampleStability + Consistency implies convergence
Investigate Large Timestep IssuesAbsolute Stability for two time-scale examples.Oscillators.
Multistep MethodsGeneral Notation
Basic Equations
( ) ( ( ), ( ))d x t f x t u tdt
=Nonlinear Differential Equation:
k-Step Multistep Approach: ( )( )0 0
ˆ ˆ ,k k
l j l jj j l j
j jx t f x u tα β− −
−= =
= ∆∑ ∑
Solution at discrete points
Time discretization
Multistep coefficients
2ˆ lx −
lt1lt −2lt −3lt −l kt −
ˆ lx1ˆ lx −
ˆ l kx −
Multistep MethodsSimplified Problem for
Analysis
( ) ( ) 0( ), 0d v t v t v vdt
λ λ= = ∈
Must Consid ALLer
Scalar ODE:
0 0
ˆ ˆk k
l j l jj j
j jv t vα β λ− −
= =
= ∆∑ ∑Scalar Multistep formula:
λ ∈
Growing Solutions
DecayingSolutions
Oscillations
( )Im λ
( )Re λ
Multistep MethodsConvergence Analysis
Convergence Definition
Definition: A multistep method for solving initial value problems on [0,T] is said to be convergent if given any
initial condition
( )0,
ˆmax 0 as t 0lTlt
v v l t⎡ ⎤∈⎢ ⎥∆⎣ ⎦
− ∆ → ∆ →
exactv
tˆ computed with 2
lv ∆ˆ computed with tlv ∆
Multistep MethodsConvergence Analysis
Two Conditions for Convergence
1) Local Condition: “One step” errors are small (consistency)
Typically verified using Taylor Series
2) Global Condition: The single step errors do not grow too quickly (stability)
Multi-step (k > 1) methods require careful analysis.
Multistep MethodsConvergence Analysis
Global Error Equation
0 0
ˆ ˆ 0k k
l j l jj j
j jv t vα β λ− −
= =
− ∆ =∑ ∑Multistep formula:
Exact solution Almostsatisfies Multistep Formula: ( ) ( )
0 0
k kl
j l j j l jj j
dv t t v t edt
α β− −= =
− ∆ =∑ ∑Local Truncation Error
(LTE)( ) ˆl llE v t v≡ −Global Error:
Difference equation relates LTE to Global error( ) ( ) ( )1
0 0 1 1l l l k l
k kt E t E t E eα λ β α λ β α λ β− −− ∆ + − ∆ + + − ∆ =
Multistep MethodsMaking LTE Small
Exactness Constraints
Local Truncation Error:
LTE
( ) ( ) 1If p pdv t t v t ptdt
−= ⇒ =
( ) ( )Can't be from d v t v tdt
λ=
( ) ( )0 0
k kl
j l j j l jj j
dv t t v t edt
α β− −= =
− ∆ =∑ ∑
( )( )( )
( )( )
( )0 0
1
k jk j
k kk
j jj j
p pk j t t p k
dv t v t
t
d
e
t
jα β
−−
= =
−− ∆ −∆ − ∆ =∑ ∑
Multistep MethodsMaking LTE Small
Exactness Constraint k=2 Example
( ) ( )0 0
1Exactness Cons train 0ts:k k
j jj j
p pk j p k jα β= =
−⎛ ⎞− − − =⎜ ⎟
⎝ ⎠∑ ∑
0
1
2
0
1
2
1 1 1 0 0 0 02 1 0 1 1 1 04 1 0 4 2 0 08 1 0 12 3 0 0
16 1 0 32 4 0 0
αααβββ
⎡ ⎤⎡ ⎤ ⎡ ⎤⎢ ⎥⎢ ⎥ ⎢ ⎥⎢ ⎥− − −⎢ ⎥ ⎢ ⎥⎢ ⎥⎢ ⎥ ⎢ ⎥=− − ⎢ ⎥⎢ ⎥ ⎢ ⎥⎢ ⎥− −⎢ ⎥ ⎢ ⎥⎢ ⎥⎢ ⎥ ⎢ ⎥− − ⎢ ⎥⎣ ⎦ ⎣ ⎦⎢ ⎥⎣ ⎦
For k=2, yields a 5x6 system of equations for Coefficients
p=0
p=1
p=2
p=3
p=4
i 0α =∑Note
Always
Multistep MethodsMaking LTE Small
Exactness Constraint k=2 example, generating methods
1
2
0
1
2
1 1 0 0 0 11 0 1 1 1 21 0 4 2 0 41 0 12 3 0 81 0 32 4 0 16
ααβββ
−⎡ ⎤⎡ ⎤ ⎡ ⎤⎢ ⎥⎢ ⎥ ⎢ ⎥− − − −⎢ ⎥⎢ ⎥ ⎢ ⎥⎢ ⎥⎢ ⎥ ⎢ ⎥=− − −⎢ ⎥⎢ ⎥ ⎢ ⎥− − −⎢ ⎥⎢ ⎥ ⎢ ⎥⎢ ⎥⎢ ⎥ ⎢ ⎥− − −⎣ ⎦ ⎣ ⎦⎣ ⎦
0First introduce a normalization, for example 1α =
0 1 2 0 1 21, 0, 1, 1/ 3, 4 / 3, 1/ 3α α α β β β= = = − = = =
Solve for the 2-step explicit method with lowest LTE
Solve for the 2-step method with lowest LTE
( )5Satisfies all five exactness constraints LTE C t= ∆
( )4Can only satisfy four exactness constraints LTE C t= ∆0 1 2 0 1 21, 4, 5, 0, 4, 2α α α β β β= = = − = = =
10-4 10-3 10-2 10-1 10010-15
10-10
10-5
100
Multistep MethodsMaking LTE Small
( ) ( )d v t v td
=
Trap
Beste
LTE
FE
Timestep
LTE Plots for the FE, Trap, and “Best” Explicit (BESTE).
Best Explicit Method has highest one-step
accurate
10-4 10-3 10-2 10-1 10010-8
10-6
10-4
10-2
100
Multistep MethodsMaking LTE Small
Global Error for the FE, Trap, and “Best” Explicit (BESTE).
[ ]( ) ( ) 0,1d v t v t td
= ∈
FE
Trap
Max Error
Where’s BESTE?
Timestep
10-4
10-3
10-2
10-1
10010
-100
100
10100
10200
Multistep MethodsMaking LTE Small
Global Error for the FE, Trap, and “Best” Explicit (BESTE).
( ) ( )d v t v td
=
FE Trap
Beste
Max Error
Best Explicit Method has lowest one-step error but global errror increases as
timestep decreases
worrysome
Timestep
Multistep MethodsStability of the method
Difference Equation
Why did the “best” 2-step explicit method fail to Converge?
( ) ( ) ( )10 0 1 1
l l l k lk kt E t E t E eα λ β α λ β α λ β− −− ∆ + − ∆ + + − ∆ =
Multistep Method Difference Equation
( ) ˆlv l t v∆ −Global Error
LTE
We made the LTE so small, how come the Global error is so large?
Multistep MethodsStability of the method
Stability Definition
Multistep Method Difference Equation( ) ( ) ( )1
0 0 1 1l l l k l
k kt E t E t E eα λ β α λ β α λ β− −− ∆ + − ∆ + + − ∆ =
Definition: A multistep method is stable if as
( )0, 0,
interval dependent
max max l lT Tl lt t
TE C T et⎡ ⎤ ⎡ ⎤∈ ∈⎢ ⎥ ⎢ ⎥∆ ∆⎣ ⎦ ⎣ ⎦
≤∆
0t∆ →
Global Error is bounded by a constant times the sum of the LTE’sStability means:
Given a kth order difference eqn with zero initial conditions1
0 , 0, , 0l l k l kka x a x u x x− − −+ + = = =
convolution sum
0
ll l j j
jx h u−
=
= ∑
( ) ( )1
,1 0
qMQ
lq m q
q m
lmh l
x can be related to the input u by
γ ς−
= =
=∑∑1
0 1 0k kka z a z a−+ + + =
Roots of
Root multiplicity
Aside on difference Equations
Convolution Sum
Root Relation
Aside on difference Equations
Convolution Sum
Bounding Terms
( ) ( )1
,1 0 0
,
qMQ ll j
q m qq m j
l jm
q mR
x l j uγ ς−
−
= = =
⎛ ⎞= −⎜ ⎟
⎝ ⎠∑∑ ∑
,If then<1, max jq q m jR C uς ≤
Independent of l
( ) 0q ,If t< 1 hen+ , maxl
jq j
eR C uε
ς εε
≤
Bounds distinct Roots
Multistep MethodsStability of the method
Stability Theorem
Theorem: A multistep method is stable if and only if
10 1Roots of 0 either: k k
kz zα α α−+ + + =
1. Have magnitude less than one2. Have magnitude equal to one and are distinct
Multistep MethodsStability of the method
Stability Theorem “Proof”
Given the Multistep Method Difference Equation( ) ( ) ( )1
0 0 1 1l l l k l
k kt E t E t E eα λ β α λ β α λ β− −− ∆ + − ∆ + + − ∆ =
( ) ( )0 0If, as 0, roots o 0f lk kt z tt α λ β α λ β− ∆ +∆ + − ∆→ =
• less than one in magnitude or• are distinct and bounded by 1 , 0tκ κ+ ∆ >
Then from the aside on difference equations
( )0, 0, 0,
max max max l t
l l lT T Tl l lt t t
C T
TCE e e Tt
C et T
eκ κ
⎡ ⎤ ⎡ ⎤ ⎡ ⎤∈ ∈ ∈⎢ ⎥ ⎢ ⎥ ⎢ ⎥∆ ∆ ∆⎣ ⎦ ⎣ ⎦
∆
⎣ ⎦
≤ ≤∆ ∆
Multistep MethodsStability of the method
Stability Theorem Picture
1-1Re
Im
( )0
roots of 0 for a nonzero k
k jj j
jt z tα λ β −
=
− ∆ = ∆∑
As 0, roots t∆ →move inward tomatch polynomial α0
roots of 0k
k jj
jzα −
=
=∑
Multistep MethodsStability of the method
The BESTE Method
Best explicit 2-step method0 1 2 0 1 21, 4, 5, 0, 4, 2α α α β β β= = = − = = =
Re
Im2roots of 4 5 0z z+ − =
Method is Wildly unstable!
-1 1-5
Multistep MethodsStability of the method
Dahlquist’s First Stability Barrier
For a stable, explicit k-step multistep method, the maximum number of exactness constraints that can be satisfied is less than or equal to k (note there are 2k-1
coefficients). For implicit methods, the number of constraints that can be satisfied is either k+2 if k is even
or k+1 if k is odd.
Multistep MethodsConvergence Analysis
Conditions for convergence, stability and consistency
1) Local Condition: One step errors are small (consistency)
Exactness Constraints up to p0 (p0 must be > 0)
( ) 0 11 0
0,max for pl
Tlt
e C t t t+
⎡ ⎤∈⎢ ⎥∆⎣ ⎦
⇒ ≤ ∆ ∆ < ∆
2) Global Condition: One step errors grow slowly (stability)
0roots of 0
kk j
jj
zα −
=
=∑
20, 0,
max maxl lT Tl lt t
TE C et⎡ ⎤ ⎡ ⎤∈ ∈⎢ ⎥ ⎢ ⎥∆ ∆⎣ ⎦ ⎣ ⎦
⇒ ≤∆
( ) 0
0,max pl
Tlt
E CT t⎡ ⎤∈⎢ ⎥∆⎣ ⎦
≤ ∆Convergence Result:
Inside the unit circle or on the unit circle and distinct
Backward-Euler Computed Solution
With Backward-Euler it is easy to use small timesteps for the fast dynamics and then switch to large timesteps for the
slow decay
small t∆
large t∆
( ) ( )d x t Ax tdt
=
( ) 2.1, 0.1eig A = − −
Circuit Example
Multistep MethodsTwo time-constant circuit
Large timestep stability
Forward-Euler Computed Solution
Multistep MethodsFE on two time-constant
circuit?
Large Timestep Stability
The Forward-Euler is accurate for small timesteps, but goes unstable when the timestep is enlarged
Multistep MethodsLarge Timestep Stability
( ) ( ) 0( ), 0d v t v t v vdt
λ λ= = ∈
FE, BE and Trap on the scalar ode problem
Scalar ODE:
( )1ˆ ˆ ˆ ˆ1l l l lv v t v t vλ λ+ = + ∆ = + ∆Forward-Euler:
Backward-Euler: ( )1 1 1 1ˆ ˆ ˆ ˆ ˆ
1l l l l lv v t v v v
tλ
λ+ + += + ∆ ⇒ =
−∆
If 1 1 the solution grows even if <0tλ λ+ ∆ >
1If 1 the solution decays even if 01 t
λλ
< >−∆
( ) ( )( )
1 1 1 1 0.5ˆ ˆ ˆ ˆ ˆ ˆ0.5
1 0.5ll l l l lt
v v t v v v vtλ
λλ
+ + + + ∆= + ∆ + ⇒ =
− ∆Trap Rule:
Multistep MethodsLarge Timestep Stability
FE large timestep region of absolute stability
Difference EqnStability region 1-1
( )1z tλ= + ∆
Im(z)
Re(z)
( )Im λ
( )Re λ
Forward Euler
ODE stability region
2t
−∆
Region ofAbsolute Stability
Multistep MethodsLarge Timestep Stability
FE large timestep stability, circuit example
Difference EqnStability region 1-1
Im(z)
Re(z)
( )Im λ
( )Re λ
ODE stability region
2t
−∆
Circuit example with t = 0.1, 2.1, 0.1λ∆ = − −
Region ofAbsolute Stability
Multistep MethodsLarge Timestep Stability
FE large timestep stability, circuit example
Difference EqnStability region 1-1
Im(z)
Re(z)
( )Im λ
( )Re λ
ODE stability region
2t
−∆
Circuit example with t=1.0, 2.1, 0.1λ∆ = − −
Unstable Difference Equation
Region ofAbsolute Stability
Multistep MethodsLarge Timestep Stability
BE large timestep region of absolute stability
Difference EqnStability region 1-1
Im(z)
Re(z)
( )Im λBackward Euler ( ) 11z tλ −= − ∆
Region ofAbsolute Stability
Multistep MethodsLarge Timestep Stability
BE large timestep stability, circuit example
Difference EqnStability region 1-1
Im(z)
Re(z)
( )Im λCircuit example with t = 0.1, 2.1, 0.1λ∆ = − −
Region ofAbsolute Stability
Multistep MethodsLarge Timestep Stability
BE large timestep stability, circuit example
Difference EqnStability region 1-1
Im(z)
Re(z)
( )Im λCircuit example with t =1.0, 2.1, 0.1λ∆ = − −
Stable Difference Equation
Region ofAbsolute Stability
Multistep MethodsLarge Timestep Stability
Stability Definitions
Region of Absolute Stability for a Multistep method:
A method is A-stable if its region of absolute stability
( )0
Values of where roots of 0k
k jj j
jt t zλ α λ β −
=
∆ − ∆ =∑are inside the unit circle.
includes the entire left-half of the complex plane
A-stable:
Dahlquist’s second Stability barrier:There are no A-stable multistep methods of convergenceorder greater than 2, and the trap rule is the most accurate.
0 5 10 15 20 25 30-6
-4
-2
0
2
4
Multistep methodsNumerical Experiments
Oscillating Strut and Mass
Why does FE result grow, BE result decay and the Trap rule preserve oscillations
Trap rule
Forward-Euler
Backward-Euler
0.1t∆ =
Multistep MethodsLarge Timestep Stability
FE large timestep oscillator example
Difference EqnStability region 1-1
( )1z tλ= + ∆
Im(z)
Re(z)
( )Im λ
( )Re λ
Forward Euler
ODE stability region
2t
−∆
Region ofAbsolute Stability
unstable oscillating
Multistep MethodsLarge Timestep Stability
BE large timestep oscillator example
Difference EqnStability region 1-1
Im(z)
Re(z)
( )Im λBackward Euler ( ) 11z tλ −= − ∆
Region ofAbsolute Stability
decaying oscillating
Difference EqnStability region 1-1
Im(z)
Re(z)
( )Im λTrap Rule
Region ofAbsolute Stability
oscillatingoscillating
( )( )1 0.51 0.5
tz
tλλ
+ ∆=
− ∆
Multistep MethodsLarge Timestep Stability
Trap large timestep oscillator example
Multistep Methods Large Timestep Issues
Two Time-Constant Stable problem (Circuit)FE: stability, not accuracy, limited timestep size.BE was A-stable, any timestep could be used.Trap Rule most accurate A-stable m-step method
Oscillator ProblemForward-Euler generated an unstable difference
equation regardless of timestep size.Backward-Euler generated a stable (decaying)
difference equation regardless of timestep size.Trapezoidal rule mapped the imaginary axis
Summary
Small Timestep issues for Multistep MethodsLocal truncation error and Exactness.Difference equation stability.Stability + Consistency implies convergence.
Investigate Large Timestep IssuesAbsolute Stability for two time-scale examples.Oscillators.
Didn’t talk aboutRunge-Kutta schemes, higher order A-stable methods.