Iterative Solvers for Linear Systems of Equations
description
Transcript of Iterative Solvers for Linear Systems of Equations
1
Iterative Solvers for Linear Systems of Equations
Presented by:Kaveh Rahnema
Supervisor:Dr. Stefan [email protected]
2
Outline
1. Short explanation of the system
2. Jacobi method
3. Eigenvalue analysis of the Jacobi method
4. The Multigrid Method
3
Short explanation of the system
nabhfxf
huuu
jj
jjj
,)(22
11
fuu 2
To solve this equation numerically we need to descretize it and bring it into finite difference notation
fAu
f
fff
u
uuu
h
nn
1
3
2
1
1
3
2
1
2
21
121121
12
1
4
Jacobi method
The system can be written as
fDuULDufuULDufuULD
ULDA
11 )()()(
Where D is the Diagonal , L is the lower triangle and U is the upper triangle part of the matrix A
If we define the Jacobi iteration matrix as)(1 ULDPJ
Then the Jacobi method appears in matrix form as
fDuPu J1)0()1(
5
Modified JacobiThe Jacobi method can be modified as
)()1( 1)0()0()1( fDuPwuwu J
Where w is a damping factor to help Jacobi converge faster to the solution
Choosing w = 1 we get the original Jacobi method
Now that we have the formulation of the Jacobi how should we use it to solve the system?
We start with an initial guess (sometimes might be even a bad guess) and solve the system iteratively using the results from the last iteration
to perform the current step until our stopping criteria is satisfied
For stopping criteria we can use different schemes, one might use the difference between 2 iterations , residual etc.
6
Eigenvalue analysis of the Jacobi method
Jww wPIwPfwDuPu )1(,1)0()1(
The Jacobi formulation can also be written as
It follows that the eigenvalues of Pw and A are related by
21
121121
12
2
wIPw
)(2
1)( AwPw
And we have
7
Eigenvalue analysis of the Jacobi method
)2(sin21)(
11,)2(sin4)(
2
2
NkwP
NkNkA
wk
k
NjNjkw jk 0,)sin(,
Now the problem becomes finding the eigenvalues of A
And A and P have the same eigenvectors
We can expand the error of the initial guess using the eigenvalues of A
Rcwce k
N
kkk
,
1
1
)0(
8
The error after n iterations is given by
1
1
1
1
)0()( )(N
k
N
kkw
nkkk
nwk
nw
n wPcwPcePe
As we see after n iterations the error is only reduced by a factor but the fourier modes don‘t mix
and the condition for Modified Jacobi to converge is
101)( wPwk
In the next slide you see the plot of damping factor of Pw for fourier modes for 4 different w
Eigenvalue analysis of the Jacobi method
9
Frequency dependent damping
10
Example solved by Jacobi method
0uHere as an example we solve the equation
In the [0,1] domain with boundary condition 0)1()0( uu
To solve this system we started with a random initial guess, which might not be used often but for some reasons is a good guess for this
presentation
As you will see in the next slide the Jacobi method works good to damp the high frequencies but damps the low frequencies slow,
This makes solving systems of equations with Jacobi hard, and one may look for other better solvers or techniques to solve the equations faster
11
Modified Jacobi Damping Clip
100
90
80
70
60
50
40
30
20
10
00 20 40 60 80 100 120
Grid Points
Val
ue a
t Grid
Poi
nts
SLO
W
12
Example solved by Jacobi method
As said before the high frequencies damp fast but the low frequencies remain and damp slow
This gets even worse going to finer grid points
What shall we do?!
Shall we wait for a long time till we get the result when we have a large number of grid
points?!!
13
The Multigrid MethodAs we saw in the last slides the Jacobi method is slow for the low
frequencies
But let‘s take look at the initial guess at the coarser grid
Let‘s take every second point for the next coarser grid
The next coarse grid has half as much grid points as the fine grid has, so the fourier modes represented in the fine grid can be rewritten as follows
for the coarse grid
hjk
hjk w
Njk
Njkw 2
,2, )2/
sin()2sin(
This means that the kth mode on Ωh becomes the kth mode on Ω2h, which means that in passing from the fine grid to the coarse grid, a mode becomes
more oscillatory
14
15
16
The Multigrid MethodNow that we know the smooth modes in the fine grid are oscilatory in the
next coarse grids, what shall we do?
How about doing some iterations of the Jacobi in the fine grid to get a smoother solution and then pass the residual every second point to the
coarser grid and do more smoothing there?
What shall we do when we are done with the smoothing on the coarse grid?
Shall we return to the fine grid? How? We don‘t have the same dimension!
Shall we go to the next coarser grid? Why?
We continue going down to the next coarser grid till we are in the coarsetst grid where we only have the lowest frequency mode left
And then we need to find a way to go back to the finer grids and check our stopping criteria
17
Going back to the fine gridThe next step is to go from the coarsest grid one level up to the next
fine grid,
But for this step we need to generate the intermediate points which we lost while going to corase grid
The interpolation techniques might help us through this
We can regenerate the intermediate points by interpolating them using the 2 neighboring points
gridcoarsej
gridfinej
gridcoarsej
gridfinej
gridcoarsej
gridcoarsejgridfine
j uuuuuu
u 1222
1
12 ,,2
Now when we are one level finner, we will do some more iterations of the Jacobi method to avoid the high frequency oscillations occured from
going to coarse grid and coming back and continue going up using the same steps till we are in the finest grid
18
Back in the finest grid
Now we will check our resedual and if the resedual is what we expect we stop the solver and we have the solution, else we have to redo the same
operation till we reach our expected residual
Now that we are in the finest (original) grid what shall we do?
Are we done with solving the system?
Is the solution good enough?
One multigrid cycle is also called a V cycle since we go down and come back up
19
Multigrid code1. Pre-smoothing: Perform some iterations of Modified Jacobi to get a rather
smooth solution
2. Construct the residue
3. Restrict your residue to every second point to go to the coarse grid
4. Do step 1
5. Goto step 2 till you are in the coarsest grid
6. Interpolate your results and go back to the finer grid
7. Add the result from the coarse grid to the previous results
8. Post-smoothing: perform some iteration of Modified Jacobi
9. Goto step 5 till you are in the finest grid
10. Check the residual
11. If condition not satisfied start from 1
12. else return the solution
20
The expected residual achieved!!
21
References
• Prof. Bungartz, Numerical Programming course, TUM (CSE), WS 2007
•Dr. Bader, Scientific Computing Course, TUM (CSE), WS 2007
•Dr. Mehl, Scientific Computing Lab Course, TUM (CSE), WS 2007
•William L. Briggs, A Multigirid Tutorial (1987)
22
Thank you for attending this talk