Meshfree Approximation with Matlab - Lecture VI: Nonlinear...

Meshfree Approximation with MATLABLecture VI: Nonlinear Problems: Nash Iteration and Implicit

Smoothing

Greg Fasshauer

Department of Applied MathematicsIllinois Institute of Technology

Dolomites Research Week on ApproximationSeptember 8–11, 2008

fasshauer@iit.edu Lecture VI Dolomites 2008

Outline

1 Nonlinear Elliptic PDE

2 Examples of RBFs and MATLAB code

3 Operator Newton Method

4 Smoothing

5 RBF-Collocation

6 Numerical Illustration

7 Conclusions and Future Work

Nonlinear Elliptic PDE

Generic nonlinear elliptic PDE

Lu = f on Ω ⊂ Rs

Approximate Newton Iteration

uk = uk−1 − Thk (uk−1)F (uk−1), k ≥ 1

F (u) = Lu − f (residual),Thk numerical inversion operator, approximates (F ′)−1

−→ RBF collocation

Nash-Moser Iteration [Nash (1956), Moser (1966),Hörmander (1976), Jerome (1985), F. & Jerome (1999)]

uk = uk−1 − Sθk Thk (uk−1)F (uk−1), k ≥ 1

Sθk additional smoothing for accelerated convergence(separated from numerical inversion)−→ implicit RBF smoothing

Lu = f on Ω ⊂ Rs

uk = uk−1 − Thk (uk−1)F (uk−1), k ≥ 1

F (u) = Lu − f (residual),

Thk numerical inversion operator, approximates (F ′)−1

Lu = f on Ω ⊂ Rs

uk = uk−1 − Thk (uk−1)F (uk−1), k ≥ 1

Lu = f on Ω ⊂ Rs

uk = uk−1 − Thk (uk−1)F (uk−1), k ≥ 1

Lu = f on Ω ⊂ Rs

uk = uk−1 − Thk (uk−1)F (uk−1), k ≥ 1

Sθk additional smoothing for accelerated convergence(separated from numerical inversion)

−→ implicit RBF smoothing

Lu = f on Ω ⊂ Rs

uk = uk−1 − Thk (uk−1)F (uk−1), k ≥ 1

Examples of RBFs and MATLAB code Matérn RBFs

Matérn Radial Basic FunctionsDefinition

Φs,β(x) =Kβ− s

2(‖x‖)‖x‖β−

2β−1Γ(β), β >

Kν : modified Bessel function of the second kind of order ν.

Properties:Φs,β strictly positive definite on Rs for all s < 2β since

Φs,β(ω) =(

1 + ‖ω‖2)−β

κ(x ,y) = Φs,β(x − y) are reproducing kernels of Sobolev spacesW β

2 (Ω)

Kν > 0 =⇒ Φs,β > 0

Φs,β(x) =Kβ− s

2(‖x‖)‖x‖β−

2β−1Γ(β), β >

Φs,β(ω) =(

1 + ‖ω‖2)−β

2 (Ω)

Kν > 0 =⇒ Φs,β > 0

Φs,β(x) =Kβ− s

2(‖x‖)‖x‖β−

2β−1Γ(β), β >

Φs,β(ω) =(

1 + ‖ω‖2)−β

2 (Ω)

‖f − Pf‖W k2 (Ω) ≤ Chβ−k‖f‖Wβ

2 (Ω), k ≤ β

[Wu & Schaback (1993)]

Kν > 0 =⇒ Φs,β > 0

Φs,β(x) =Kβ− s

2(‖x‖)‖x‖β−

2β−1Γ(β), β >

Φs,β(ω) =(

1 + ‖ω‖2)−β

2 (Ω)

‖f − Pf‖W kq (Ω) ≤ Chβ−k−s(1/2−1/q)+‖f‖Cβ(Ω), k ≤ β

[Narcowich, Ward & Wendland (2005)]

Kν > 0 =⇒ Φs,β > 0

Φs,β(x) =Kβ− s

2(‖x‖)‖x‖β−

2β−1Γ(β), β >

Φs,β(ω) =(

1 + ‖ω‖2)−β

2 (Ω)

‖f − Pf‖W kq (Ω) ≤ Chβ−k−s(1/2−1/q)+‖f‖Cβ(Ω), k ≤ β

[Narcowich, Ward & Wendland (2005)]Kν > 0 =⇒ Φs,β > 0

Examples

Let r = ε‖x‖, t = ‖ω‖ε

β Φ3,β(r)/√

2π ε3Φ3,β(t)

3 (1 + r) e−r

(1 + t2)−3

4(3 + 3r + r2) e−r

(1 + t2)−4

5(15 + 15r + 6r2 + r3) e−r

(1 + t2)−5

6(105 + 105r + 45r2 + 10r3 + r4) e−r

(1 + t2)−6

Table: Matérn functions and their Fourier transforms for s = 3 and variouschoices of β.

Figure: Matérn functions and Fourier transforms for Φ3,3 (top) and Φ3,6(bottom) centered at the origin in R2 (ε = 10 scaling used).

Implicit Smoothing [F. (1999), Beatson & Bui (2007)]

Crucial property of Matérn RBFs

Φs,β ∗ Φs,α = Φs,α+β , α, β > 0

Therefore with

u(x) =N∑

cjΦs,β(x − x j)

we get

u ∗ Φs,α =

N∑j=1

cjΦs,β(· − x j)

∗ Φs,α

cjΦs,α+β(· − x j)

=: SαuReturn

Implicit Smoothing [F. (1999), Beatson & Bui (2007)]

Crucial property of Matérn RBFs

Φs,β ∗ Φs,α = Φs,α+β , α, β > 0

Therefore with

u(x) =N∑

cjΦs,β(x − x j)

we get

u ∗ Φs,α =

N∑j=1

cjΦs,β(· − x j)

∗ Φs,α

cjΦs,α+β(· − x j)

=: SαuReturn

Noisy and smoothed interpolants

Figure: Solved and evaluated with Φ3,3 (left), evaluated with Φ3,4 (right).

Figure: Solved and evaluated with Φ3,3 (left), evaluated with Φ3,3.2 (right).

Operator Newton Method Practical Newton Iteration for Lu = f

Algorithm (Approximate Newton Iteration)

[F. & Jerome (1999), F., Gartland & Jerome (2000), F. (2002), Bernal & Kindelan (2007)]

Create computational “grids” X1 ⊆ · · · ⊆ XK ⊂ Ω, and chooseinitial guess u0

For k = 1,2, . . . ,K1 Solve the linearized problem

Luk−1v = f − Luk−1 on Xk

2 Perform optional smoothing of Newton correction

v ← Sθk v

3 Perform Newton update of k -th iterate

uk = uk−1 + v

Operator Newton Method Practical Newton Iteration for Lu = f

Algorithm (Nash Iteration)

[F. & Jerome (1999), F., Gartland & Jerome (2000), F. (2002)]

Create computational “grids” X1 ⊆ · · · ⊆ XK ⊂ Ω, and chooseinitial guess u0

For k = 1,2, . . . ,K1 Solve the linearized problem

Luk−1v = f − Luk−1 on Xk

2 Perform optional smoothing of Newton correction

v ← Sθk v

3 Perform Newton update of k -th iterate

uk = uk−1 + v

Smoothing Loss of Derivatives

Why Do We Need Smoothing?

Approximate Newton method based on approximation of (F ′)−1 bynumerical inversion Th, i.e., for u, v in appropriate Banach spaces

‖[F ′(u)Th(u)− I

]v‖ ≤ τ(h)‖v‖

for some continuous monotone increasing function τ(usually τ(h) = O(hp) for some p)

Differentiation reduces the order of approximation, i.e., introducesa loss of derivatives[Jerome (1985)] used Newton-Kantorovich theory to show anappropriate smoothing of the Newton update will yield globalsuperlinear convergence for approximate Newton iteration

‖[F ′(u)Th(u)− I

]v‖ ≤ τ(h)‖v‖

for some continuous monotone increasing function τ(usually τ(h) = O(hp) for some p)Differentiation reduces the order of approximation, i.e., introducesa loss of derivatives

[Jerome (1985)] used Newton-Kantorovich theory to show anappropriate smoothing of the Newton update will yield globalsuperlinear convergence for approximate Newton iteration

‖[F ′(u)Th(u)− I

]v‖ ≤ τ(h)‖v‖

for some continuous monotone increasing function τ(usually τ(h) = O(hp) for some p)Differentiation reduces the order of approximation, i.e., introducesa loss of derivatives[Jerome (1985)] used Newton-Kantorovich theory to show anappropriate smoothing of the Newton update will yield globalsuperlinear convergence for approximate Newton iteration

Smoothing Hörmander’s Smoothing

Hörmander’s SmoothingTheorem ([Hörmander (1976), F. & Jerome (1999)])

Let 0 ≤ ` ≤ k and p be integers. In Sobolev spaces W kp (Ω) there exist

smoothings Sθ satisfying1 Semigroup property: ‖Sθu − u‖Lp → 0 as θ →∞2 Bernstein inequality: ‖Sθu‖W k

p≤ Cθk−`‖u‖W `

3 Jackson inequality: ‖Sθu − u‖W `p≤ Cθ`−k‖u‖W k

Remark: Also true in intermediate Besov spaces Bσp,∞(Ω)

Hörmander defined Sθ by convolution

Sθu = φθ ∗ u, φθ = θsφ(θ·)

New: Use φθ = Φs,α Matérn RBFs

Note: Jackson and Bernstein theorems known for interpolation withMatérn functions, but not for smoothing [Beatson & Bui (2007)]

RBF-Collocation Kansa’s Method

Non-symmetric RBF Collocation

Linear(ized) BVP

Lu(x) = f (x), x ∈ Ω ⊂ Rs

Bu(x) = g(x), x ∈ ∂Ω

Use Ansatz u(x) =N∑

cjϕ(‖x − x j‖) [Kansa (1990)]

Collocation at x1, . . . ,x I︸︷︷︸∈Ω

,x I+1, . . . ,xN︸︷︷︸∈∂Ω

leads to linear system

Ac = y

], y =

]fasshauer@iit.edu Lecture VI Dolomites 2008

RBF-Collocation Kansa’s Method

Computational Grids for N = 289

Figure: Uniform (left), Chebyshev (center), and Halton (right) collocationpoints.

Numerical Illustration Nonlinear 2D-BVP

Numerical Illustration

Nonlinear PDE: Lu = f

−µ2∇2u − u + u3 = f , in Ω = (0,1)× (0,1)

u = 0, on ∂Ω

Linearized equation: Luv = f − Lu

−µ2∇2v + (3u2 − 1)v = f + µ2∇2u + u − u3

Computational grids: uniformly spaced, Chebyshev, or Haltonpoints in [0,1]× [0,1]

Use µ = 0.1 for all examples

Numerical Illustration (cont.)

RBFs used: Matérn functions

Φs,β(x) =Kβ− s

2(‖εx‖)‖εx‖β−

2β−1Γ(β), β >

Φs,β(0) =Γ(β − s

2sΓ(β)

∇2Φs,β(x) =[(‖εx‖2 + 4(β − s

Kβ− s2(‖εx‖)

−2(β − s2

)‖εx‖Kβ− s2 +1(‖εx‖)

] ε2‖εx‖β−s2−2

2β−1Γ(β)

∇2Φs,β(0) =ε2Γ(β − s

2 − 1)√

2sΓ(β)

Fixed shape parameter ε =√

function rbf_definitionMaternglobal rbf Lrbfrbf = @(ep,r,s,b) matern(ep,r,s,b); % Matern functionsLrbf = @(ep,r,s,b) Lmatern(ep,r,s,b); % Laplacian

function rbf = matern(ep,r,s,b)scale = gamma(b-s/2)*2^(-s/2)/gamma(b);rbf = scale*ones(size(r));nz = find(r~=0);rbf(nz) = 1/(2^(b-1)*gamma(b))*besselk(b-s/2,ep*r(nz))...

.*(ep*r(nz)).^(b-s/2);

function Lrbf = Lmatern(ep,r,s,b)scale = -ep^2*gamma(b-s/2-1) / (2^(s/2)*gamma(b));Lrbf = scale*ones(size(r));nz = find(r~=0);Lrbf(nz) = ep^2/(2^(b-1)*gamma(b))*(ep*r(nz)).^(b-s/2-2).*...

(((ep*r(nz)).^2+4*(b-s/2)^2).* besselk(b-s/2,ep*r(nz))...-2*(b-s/2)*(ep*r(nz)).*besselk(b-s/2+1,ep*r(nz)));

Exact solution and initial guess

Figure: Solution u (left), initial guess u(x , y) = 16x(1− x)y(1− y) (right).

Numerical Illustration Newton and Nash Iteration on Single Grid

Newton and Nash Iteration on Single Uniform Grid

Newton NashN RMS-error K RMS-error K ρ

25(41) 1.356070 10−1 7 1.064151 10−1 5 0.32881(113) 2.404571 10−2 9 2.183223 10−2 10 0.527289(353) 4.237178 10−3 9 2.276646 10−3 20 0.953

1089(1217) 8.982388 10−4 9 3.450676 10−4 37 0.9994225(4481) 1.855711 10−4 10 7.780351 10−5 32 0.999

Matérn parameters: s = 3, β = 4, uniform points

Nash smoothing: α = ρθbkwith θ = 1.1435, b = 1.2446

Sample MATLAB calls: Newton_NLPDE(289,’u’,3,4,0),Newton_NLPDE(289,’u’,3,4,0.953)

Newton approximations and updates for N = 289

Figure: Approximate solution (left), and updates (right).

Nash approximations and updates for N = 289

Error drops and smoothing parameters for N = 289

Figure: Drop of RMS error (left), and smoothing parameter α (right).

Newton and Nash Iteration on Single Chebyshev Grid

25(41) 8.809920 10−2 8 7.825548 10−2 8 0.29981(113) 3.546179 10−3 9 3.277817 10−3 8 0.541289(353) 6.198255 10−4 9 8.420461 10−5 35 0.999

1089(1217) 1.495895 10−4 8 5.470357 10−6 37 0.9994225(4481) 3.734340 10−4 7 7.790757 10−6 35 0.999

Matérn parameters: s = 3, β = 4, Chebyshev points

Newton and Nash Iteration on Single Halton Grid

25(41) 3.160062 10−2 7 2.597881 10−2 7 0.38981(113) 9.828342 10−3 9 8.125240 10−3 13 0.791289(353) 2.896087 10−3 9 1.981563 10−3 15 0.953

1089(1217) 9.480208 10−4 9 3.305680 10−4 36 0.9994225(4481) 3.563199 10−4 8 1.330167 10−4 37 0.999

Matérn parameters: s = 3, β = 4, Halton points

Convergence for Different Collocation Point Sets

Figure: Convergence of Newton and Nash iteration for different choices ofcollocation points.

Newton and Nash Iteration on Single Chebyshev Grid

Newton Nashβ RMS-error K RMS-error K ρ

3 4.022065 10−3 7 9.757401 10−4 38 0.9994 6.198255 10−4 9 8.420461 10−5 35 0.9995 1.803903 10−4 9 9.620937 10−5 8 0.4476 2.715679 10−4 8 1.259029 10−4 8 0.3767 2.279834 10−4 8 1.237608 10−4 9 0.320

Matérn parameters: N = 289, s = 3, Chebyshev points

Convergence for Different Matérn Functions

Figure: Convergence of Newton and Nash iteration for different Matérnfunctions (β).

Conclusions and Future Work

ConclusionsImplicit smoothing improves convergence of non-symmetric RBFcollocation for nonlinear test caseImplicit smoothing easy and cheap to implement for RBF collocationSmoothing with Matérn kernels recovers some of the “loss ofderivative” of numerical inversion. Can’t really work since saturated.More accurate results than earlier with MQ-RBFsRequired more than 20002 points with earlier FD experiments[F., Gartland & Jerome (2000)] (without smoothing) for sameaccuracy as 1089 points here

Future WorkTry mesh refinement within Newton algorithm via adaptivecollocationFurther investigate use of different Matérn parametersCouple smoothing parameter to current residualsDo smoothing with an approximate smoothing kernelApply similar ideas in RBF-PS framework

Conclusions and Future Work

ConclusionsImplicit smoothing improves convergence of non-symmetric RBFcollocation for nonlinear test caseImplicit smoothing easy and cheap to implement for RBF collocationSmoothing with Matérn kernels recovers some of the “loss ofderivative” of numerical inversion. Can’t really work since saturated.More accurate results than earlier with MQ-RBFsRequired more than 20002 points with earlier FD experiments[F., Gartland & Jerome (2000)] (without smoothing) for sameaccuracy as 1089 points here

Future WorkTry mesh refinement within Newton algorithm via adaptivecollocationFurther investigate use of different Matérn parametersCouple smoothing parameter to current residualsDo smoothing with an approximate smoothing kernelApply similar ideas in RBF-PS framework

Appendix References

References I

Buhmann, M. D. (2003).Radial Basis Functions: Theory and Implementations.Cambridge University Press.

Fasshauer, G. E. (2007).Meshfree Approximation Methods with MATLAB.World Scientific Publishers.

Higham, D. J. and Higham, N. J. (2005).MATLAB Guide.SIAM (2nd ed.), Philadelphia.

Wendland, H. (2005).Scattered Data Approximation.Cambridge University Press.

Beatson, R. K. and Bui, H.-Q. (2007).Mollification formulas and implicit smoothing.Adv. in Comput. Math., 27, pp. 125–149.

Appendix References

References II

Bernal, F. and Kindelan, M. (2007).Meshless solution of isothermal Hele-Shaw flow.In Meshless Methods 2007, A. Ferreira, E. Kansa, G. Fasshauer, and V. Leitão(eds.), INGENI Edições Porto, pp. 41–49.

Fasshauer, G. E. (1999).On smoothing for multilevel approximation with radial basis functions.In Approximation Theory XI, Vol.II: Computational Aspects, C. K. Chui andL. L. Schumaker (eds.), Vanderbilt University Press, pp. 55–62.

Fasshauer, G. E. (2002).Newton iteration with multiquadrics for the solution of nonlinear PDEs.Comput. Math. Applic. 43, pp. 423–438.

Fasshauer, G. E., Gartland, E. C. and Jerome, J. W. (2000).Newton iteration for partial differential equations and the approximation of theidentity.Numerical Algorithms 25, pp. 181–195.

Appendix References

References III

Fasshauer, G. E. and Jerome, J. W. (1999).Multistep approximation algorithms: Improved convergence rates throughpostconditioning with smoothing kernels.Adv. in Comput. Math. 10, pp. 1–27.

Hörmander, L. (1976).The boundary problems of physical geodesy.Arch. Ration. Mech. Anal. 62, pp. 1–52.

Jerome, J. W. (1985).An adaptive Newton algorithm based on numerical inversion: regularization aspostconditioner.Numer. Math. 47, pp. 123–138.

Kansa, E. J. (1990).Multiquadrics — A scattered data approximation scheme with applications tocomputational fluid-dynamics — II: Solutions to parabolic, hyperbolic and ellipticpartial differential equations.Comput. Math. Applic. 19, pp. 147–161.

Appendix References

References IV

Moser, J. (1966).A rapidly convergent iteration method and nonlinear partial differential equationsI.Ann. Scoula Norm. Pisa XX, pp. 265–315.

Narcowich, F. J., Ward, J. D. and Wendland, H. (2005).Sobolev bounds on functions with scattered zeros, with applications to radialbasis function surface fitting.Math. Comp. 74, pp. 743–763.

Narcowich, F. J., Ward, J. D. and Wendland, H. (2006).Sobolev error estimates and a Bernstein inequality for scattered datainterpolation via radial basis functions.Constr. Approx. 24, pp. 175–186.

Nash, J. (1956).The imbedding problem for Riemannian manifolds.Ann. Math. 63, pp. 20–63.

Appendix References

References V

Schaback, R. and Wendland, H. (2002).Inverse and saturation theorems for radial basis function interpolation.Math. Comp. 71, pp. 669–681.

Wu, Z. and Schaback, R. (1993).Local error estimates for radial basis function interpolation of scattered data.IMA J. Numer. Anal. 13, pp. 13–27.

Meshfree Approximation with Matlab - Lecture VI: Nonlinear...

Documents

Transcript of Meshfree Approximation with Matlab - Lecture VI: Nonlinear...

MATH 100 Introduction to the Profession - Logicmath.iit.edu/~fass/Notes100_Logic.pdf · MATH 100 – Introduction to the Profession Logic Greg Fasshauer Department of Applied Mathematics

Nonlinear systems - Design of nonlinear controllers

Turbulent Fluid Motion VI--- - NASA · NASA Technical Memorandum 107028 Turbulent Fluid Motion VI---Turbulence, Nonlinear Dynamics, and Deterministic Chaos Robert G. Deissler Lewis

Use of nonlinear regression and nonlinear mathematical ...

I I, I...I I I I I I I I I I ( I I I I I I I I Fisrure No. VI-19 VI-20 VI-21 VI-22 VI-23 VI-24 VI-25 VI-26 VI-27 VI-28 VI-29 VI-30 VI-31 VI-32 VI-33 VI-34 VI-35 Title Sulfate Salt

Positive De nite Kernels: Past, Present and Futurefass/PDKernels.pdf · Positive De nite Kernels: Past, Present and Future Gregory E. Fasshauer Department of Applied Mathematics Illinois

CB - POWERBOX – Mastering Power | Powerbox · VI VI VI VI VI VI VI VI VI. Created Date: 8/25/2016 11:53:45 AM ...

MATH 590: Meshfree Methodsfass/590/notes/Notes590_Ch17Print.pdfMATH 590: Meshfree Methods Machine Learning Greg Fasshauer Department of Applied Mathematics Illinois Institute of Technology

Slotine • Li APPLIED NONLINEAR CONTROL - …read.pudn.com/downloads97/ebook/394594/applied nonlinear control... · Slotine • Li APPLIED NONLINEAR CONTROL! i. APPLIED NONLINEAR

On nonlinear Markov chain Monte Carlodoucet/andrieu_jasra_doucet_delmoral...On nonlinear Markov chain Monte Carlo 991 3. Nonlinear MCMC 3.1. Nonlinear Markov kernels Nonlinear MCMC

MATH 100 – Introduction to the Profession - Proofsamadeus.math.iit.edu/~fass/Notes100_ProofPrint.pdf · MATH 100 – Introduction to the Profession Proofs Greg Fasshauer Department

MATH 590: Meshfree Methodsamadeus.math.iit.edu/~fass/590/notes/Notes590_Ch1Part2.pdf · MATH 590: Meshfree Methods Chapter 1 — Part 2: Scattered Data Interpolation in Rd Greg Fasshauer

MATH 532: Linear Algebra › ~fass › Notes532_Ch4.pdf · 2015-02-18 · MATH 532: Linear Algebra Chapter 4: Vector Spaces Greg Fasshauer Department of Applied Mathematics Illinois

Mixed-Integer Nonlinear Optimization: Applications ... · Mixed-Integer Nonlinear Optimization: Applications, Algorithms, and Computation VI Sven Ley er ... run any algorithm for

Variational approaches in the study of nonlinear problems ...cjoita/IMAR/HABILITATION_THESIS_MATEI_ANDALUZIA.pdf · vi steps in a new research direction, more complex models related

Integer Nonlinear Optimization€¦ · Introduction to Integer Nonlinear Optimization Nonlinear Branch-and-Cut Theoretical and Computational Challenges Integer Nonlinear Optimization

MATH 590: Meshfree Methods - math.iit.edufass/Notes590_Ch43.pdf · MATH 590: Meshfree Methods Chapter 43: RBF-PS Methods in MATLAB Greg Fasshauer Department of Applied Mathematics

Nonlinear Modal Substructuring of Geometrically Nonlinear ...€¦ · Nonlinear Modal Substructuring of Geometrically Nonlinear Finite Element Models by Robert John Kuether A dissertation

Meshfree Approximation with Matlab - Lecture IV: RBF ... · Meshfree Approximation with MATLAB Lecture IV: RBF Collocation and Polynomial Pseudospectral Methods Greg Fasshauer Department

Modelling and solution of Nonlinear Programs - … · Nonlinear Programming Mathematical methods for modelling and solving nonlinear problems ⇒NonLinear Programming (NLP) Nonconvex