Discretization for PDEsDiscretization for PDEs

Chunfang Chen ,Danny Chunfang Chen ,Danny ThorneThorne

Adam Zornes, Deng LiAdam Zornes, Deng Li

CS 521 CS 521

Feb., 9,2006Feb., 9,2006

Classification of PDEsClassification of PDEsDifferent mathematical and physicalDifferent mathematical and physicalbehaviors:behaviors: Elliptic TypeElliptic Type Parabolic TypeParabolic Type Hyperbolic TypeHyperbolic Type

System of coupled equations for several System of coupled equations for several variables:variables: Time :Time : first-derivative (second-derivative for wave first-derivative (second-derivative for wave

equation)equation) Space:Space: first- and second-derivatives first- and second-derivatives

Classification of PDEs (cont.)Classification of PDEs (cont.)

General form of second-order PDEs ( 2 General form of second-order PDEs ( 2 variables)variables)

PDE Model ProblemsPDE Model Problems

Hyperbolic Hyperbolic (Propagation)(Propagation)

• Advection equation (First-order Advection equation (First-order linear)linear)

• Wave equation (Second-order linearWave equation (Second-order linear

) )

PDE Model Problems (cont.)PDE Model Problems (cont.)

ParabolicParabolic (Time- or space-marching) (Time- or space-marching)

• Burger’s equation(Second-order Burger’s equation(Second-order nonlinear) nonlinear)

• Fourier equation (Second-order Fourier equation (Second-order

linearlinear ) )

(Diffusion / dispersion)

PDE Model Problems (cont.)PDE Model Problems (cont.) Elliptic Elliptic (Diffusion, equilibrium problems)(Diffusion, equilibrium problems)

• Laplace/Poisson (second-order linear)Laplace/Poisson (second-order linear)

• Helmholtz equation (second-order linear)Helmholtz equation (second-order linear)

PDE Model Problems (cont.)PDE Model Problems (cont.)

System of Coupled PDEsSystem of Coupled PDEs • Navier-Stokes EquationsNavier-Stokes Equations

Well-Posed ProblemWell-Posed Problem

Numerically well-posedNumerically well-posed Discretization equations Discretization equations Auxiliary conditions (discretized Auxiliary conditions (discretized

approximated)approximated)

• the computational solution exists (existence)the computational solution exists (existence)• the computational solution is unique (uniqueness)the computational solution is unique (uniqueness)• the computational solution depends continuously on the computational solution depends continuously on

the approximate auxiliary datathe approximate auxiliary data• the algorithm should be well-posed (stable) alsothe algorithm should be well-posed (stable) also

Boundary and InitialBoundary and InitialConditionsConditions

R

s

n

RInitial conditions: starting point for propagation problems

Boundary conditions: specified on domain boundaries to provide the interior solution in computational domain

Numerical MethodsNumerical Methods

Complex geometryComplex geometry Complex equations (nonlinear, Complex equations (nonlinear,

coupled)coupled) Complex initial / boundary Complex initial / boundary

conditionsconditions

No analytic solutionsNo analytic solutions Numerical methods needed !!Numerical methods needed !!

Numerical MethodsNumerical Methods

ObjectiveObjective: Speed, Accuracy at : Speed, Accuracy at minimum costminimum cost

Numerical AccuracyNumerical Accuracy (error analysis)(error analysis) Numerical StabilityNumerical Stability (stability analysis(stability analysis)) Numerical Efficiency Numerical Efficiency (minimize cost)(minimize cost) Validation Validation (model/prototype data, field data, (model/prototype data, field data,

analytic solution, theory, asymptotic solution)analytic solution, theory, asymptotic solution) Reliability and FlexibilityReliability and Flexibility (reduce (reduce

preparation and debugging time)preparation and debugging time) Flow VisualizationFlow Visualization (graphics and animations)(graphics and animations)

computational solution computational solution proceduresprocedures

Governing Equations ICS/BCS

DiscretizationSystem of Algebraic Equations

Equation (Matrix) Solver

ApproximateSolution

Continuous Solutions

Finite-Difference

Finite-Volume

Finite-Element

Spectral

Boundary Element

Hybrid

Discrete Nodal Values

Tridiagonal

ADI

SOR

Gauss-Seidel

Krylov

Multigrid

DAE

Ui (x,y,z,t)

p (x,y,z,t)

T (x,y,z,t)

or

(,,, )

DiscretizationDiscretization Time derivativesTime derivatives almost exclusively by finite-difference methodsalmost exclusively by finite-difference methods Spatial derivativesSpatial derivatives - - Finite-differenceFinite-difference: Taylor-series expansion: Taylor-series expansion - - Finite-elementFinite-element: low-order shape function and : low-order shape function and interpolation function, continuous within eachinterpolation function, continuous within each elementelement - - Finite-volumeFinite-volume: integral form of PDE in each: integral form of PDE in each control volumecontrol volume - There are also other methods, e.g. collocation,- There are also other methods, e.g. collocation, spectral method, spectral element, panelspectral method, spectral element, panel method, boundary element methodmethod, boundary element method

Finite DifferenceFinite Difference Taylor seriesTaylor series

Truncation errorTruncation error

How to reduce truncation errors?How to reduce truncation errors?• Reduce grid spacing, use smaller Reduce grid spacing, use smaller x x = = x-xx-xoo

• Increase order of accuracy, use larger Increase order of accuracy, use larger nn

Finite Difference SchemeFinite Difference Scheme

Forward differenceForward difference

Backward Backward

differencedifference

Central differenceCentral difference

Example : Poisson EquationExample : Poisson Equation

(1,1)

(-1,-1) (1,-1)

(-1,1)

x

y

Example (cont.)Example (cont.)

12

2

2

2 1,,1,,1,,1

y

UUU

x

UUU jijijijijiji

(i,j+1)

y (i-1,j)

x (i,j)

(i+1,j)

(i,j-1)

Rectangular GridRectangular Grid

After we discretize the Poisson equation on aAfter we discretize the Poisson equation on a

rectangular domain, we are left with a finiterectangular domain, we are left with a finite

number of gird points. The boundary valuesnumber of gird points. The boundary values

of the equation are of the equation are

the only known gridthe only known grid

points points

0,4 1,4

2,4 3,4

4,4

0,3 1,3

2,3 3,3

4,3

0,2 1,2

2,2 3,2

4,2

0,1 1,1

2,1 3,1

4,1

0,0 1,0

2,0 3,0

4,0

What to solve?What to solve?

Discretization produces a linear system ofDiscretization produces a linear system ofequations. equations.

The A matrix is aThe A matrix is apentadiagonal bandedpentadiagonal bandedmatrix of a standardmatrix of a standardform:form:

A solution method is to be performed forA solution method is to be performed forsolvingsolving

41010

14101

01410

10141

01014

Matrix StorageMatrix Storage

We could try and take advantage of We could try and take advantage of the banded nature of the system, but the banded nature of the system, but a more general solution is the a more general solution is the adoption of a sparse matrix storage adoption of a sparse matrix storage strategy.strategy.

Limitations of Finite Limitations of Finite DifferencesDifferences

Unfortunately, it is not easy to use Unfortunately, it is not easy to use finite differences in complex finite differences in complex geometries.geometries.

While it is possible to formulate While it is possible to formulate curvilinear finite difference methods, curvilinear finite difference methods, the resulting equations are usually the resulting equations are usually pretty nasty.pretty nasty.

Finite Element MethodFinite Element Method

The finite element method, while more The finite element method, while more complicated than finite difference complicated than finite difference methods, easily extends to complex methods, easily extends to complex geometries.geometries.

A simple (and short) description of the A simple (and short) description of the finite element method is not easy to give.finite element method is not easy to give.

PDEWeak

Form

Matrix

System

Finite Element Method Finite Element Method ((Variational FormulationsVariational Formulations))

Find u in test space H such that a(u,v) = f(v) for all v Find u in test space H such that a(u,v) = f(v) for all v in H, where a is a bilinear form and f is a linear in H, where a is a bilinear form and f is a linear functional.functional.

V(x,y) = V(x,y) = jj V Vjjjj(x,y), j = 1,…,n(x,y), j = 1,…,n I(V) = .5 I(V) = .5 j j jj A AijijVViiVVjj - - jj b biiVVii, i,j = 1,…,n, i,j = 1,…,n AAijij = = a( a( jj, , jj), i = 1,…,n), i = 1,…,n BBii = = f f jj, i = 1,…,n, i = 1,…,n

The coefficients VThe coefficients Vjj are computed and the function are computed and the function V(x,y) is evaluated anyplace that a value is needed.V(x,y) is evaluated anyplace that a value is needed.

The basis functions should have local support (i.e., The basis functions should have local support (i.e., have a limited area where they are nonzero).have a limited area where they are nonzero).

Time Stepping MethodsTime Stepping Methods

Standard methods are common:Standard methods are common:– Forward EulerForward Euler (explicit) (explicit)– Backward EulerBackward Euler (implicit) (implicit)– Crank-NicolsonCrank-Nicolson (implicit) (implicit)

2

11

1

1

2

)1(

x

TTTTL

TLTLt

TT

jjjjxx

njxx

njxx

nj

nj

θ = 0, Fully-Explicit

θ = 1, Fully-Implicit

θ = ½, Crank-Nicolson

Time Stepping Methods (cont.)Time Stepping Methods (cont.)

Variable length time steppingVariable length time stepping– Most common in Method of Lines (Most common in Method of Lines (MOLMOL) codes ) codes

or Differential Algebraic Equation (or Differential Algebraic Equation (DAEDAE) solvers) solvers

Solving the SystemSolving the System

The system may be solved using simple The system may be solved using simple iterative methods - Jacobi, Gauss-Seidel, iterative methods - Jacobi, Gauss-Seidel, SOR, etc.SOR, etc.

• Some advantages:Some advantages: - No explicit storage of the matrix is required- No explicit storage of the matrix is required - The methods are fairly robust and reliable- The methods are fairly robust and reliable

• Some disadvantagesSome disadvantages - Really slow (Gauss-Seidel)- Really slow (Gauss-Seidel) - Really slow (Jacobi)- Really slow (Jacobi)

Solving the SystemSolving the System

Advanced iterative methods (CG, GMRES)Advanced iterative methods (CG, GMRES)CG is a much more powerful way to solve the CG is a much more powerful way to solve the

problem.problem.• Some advantages:Some advantages:

– Easy to program (compared to other advanced Easy to program (compared to other advanced methods)methods)

– Fast (theoretical convergence in N steps for an N Fast (theoretical convergence in N steps for an N by N system)by N system)

• Some disadvantages:Some disadvantages:– Explicit representation of the matrix is probably Explicit representation of the matrix is probably

necessarynecessary– Applies only to SPD matrices Applies only to SPD matrices

Multigrid Algorithm: Multigrid Algorithm: ComponentsComponents

ResidualResidualcompute the error of the compute the error of the

approximationapproximation Iterative method/Smoothing OperatorIterative method/Smoothing Operator

Gauss-Seidel iterationGauss-Seidel iteration RestrictionRestriction

obtain a ‘coarse grid’ obtain a ‘coarse grid’ ProlongationProlongation

from the ‘coarse grid’ back to the from the ‘coarse grid’ back to the original gridoriginal grid

Residual VectorResidual Vector The equation we are to solve is defined as:The equation we are to solve is defined as:

So then the residual is defined to be:So then the residual is defined to be:

Where uWhere uqq is a vector approximation for u is a vector approximation for u As the u approximation becomes better, As the u approximation becomes better,

the components of the residual vector(r) the components of the residual vector(r) , move toward zero, move toward zero

Multigrid Algorithm: Multigrid Algorithm: ComponentsComponents

ResidualResidualcompute the error of your compute the error of your

approximationapproximation Iterative method/Smoothing OperatorIterative method/Smoothing Operator

Gauss-Seidel iterationGauss-Seidel iteration RestrictionRestriction

obtain a ‘coarse grid’ obtain a ‘coarse grid’ ProlongationProlongation

from the ‘coarse grid’ back to the from the ‘coarse grid’ back to the original gridoriginal grid

Multigrid Algorithm: Multigrid Algorithm: ComponentsComponents

ResidualResidualcompute the error of your compute the error of your

approximationapproximation Iterative method/Smoothing OperatorIterative method/Smoothing Operator

Gauss-Seidel iterationGauss-Seidel iteration RestrictionRestriction

obtain a ‘coarse grid’obtain a ‘coarse grid’ ProlongationProlongation

from the ‘coarse grid’ back to the from the ‘coarse grid’ back to the original gridoriginal grid

The Restriction OperatorThe Restriction Operator Defined as ‘half-weighted’ restriction.Defined as ‘half-weighted’ restriction. Each new point in the courser grid, is dependent Each new point in the courser grid, is dependent

upon it’s neighboring points from the fine gridupon it’s neighboring points from the fine grid

Multigrid Algorithm: Multigrid Algorithm: ComponentsComponents

ResidualResidualcompute the error of your compute the error of your

approximationapproximation Iterative method/Smoothing OperatorIterative method/Smoothing Operator

Gauss-Seidel iterationGauss-Seidel iteration RestrictionRestriction

obtain a ‘coarse grid’obtain a ‘coarse grid’ ProlongationProlongation

from the ‘coarse grid’ back to the from the ‘coarse grid’ back to the original gridoriginal grid

The Prolongation OperatorThe Prolongation Operator

The grid change is exactly the The grid change is exactly the opposite of restrictionopposite of restriction

(0,2) (1,2) (2,2)

(0,1) (1,1) (2,1)

(0,0) (1,0) (2,0)

(0,4) (1,4)

(2,4) (3,4)

(4,4)

(0,3) (1,3)

(2,3) (3,3)

(4,3)

(0,2) (1,2)

(2,2) (3,2)

(4,2)

(0,1) (1,1)

(2,1) (3,1)

(4,1)

(0,0) (1,0)

(2,0) (3,0)

(4,0)

Prolongation vs. RestrictionProlongation vs. Restriction

The most efficient multigrid algorithms use The most efficient multigrid algorithms use prolongation and restriction operators that prolongation and restriction operators that are directly related to each other. In the are directly related to each other. In the one dimensional case, the relation one dimensional case, the relation between prolongation and restriction is as between prolongation and restriction is as follows:follows:

Full Multigrid AlgorithmFull Multigrid Algorithm

1.Smooth initial U vector to receive a new approximation Uq

2. Form residual vector: Rq =b -A Uq

3. Restrict Rq to the next courser grid Rq-1

4. Smooth Ae= Rq-1 starting with e=0 to obtain eq-1

5.Form a new residual vector using: Rq-1= Rq-1 -A eq-1

6. Restrict R2 (5x? where ?5) down to R1(3x? where ?3) 7. Solve exactly for Ae= R1 to obtain e1

8. Prolongate e1e2 & add to e2 you got from restriction 9. Smooth Ae= R2 using e2 to obtain a new e2

10. Prolongate eq-1 to eq and add to Uq

ReferenceReference

http://www.mgnet.org/~douglas/http://www.mgnet.org/~douglas/

http://www.mgnet.org/~douglas/Classes/http://www.mgnet.org/~douglas/Classes/cs521/2006-index.htmlcs521/2006-index.html

Practice (Option)Practice (Option)

Introduce the following Introduce the following processingprocessing

by read the book “A Tutorial on Elliptic PDE by read the book “A Tutorial on Elliptic PDE Solvers and Their Parallelization” Solvers and Their Parallelization”

PDEWeak

Form

Discrete

Matrix