Algebraic Multigrid in LatticeQCD computations
Karsten Kahl
Bergische Universitat Wuppertal
October 4, 2011
Outline
The Multigrid Principle
Algebraic MultigridTheory of AMG
Algebraic Multigrid for QCDSmoothers for QCDInterpolation for QCD
Bootstrap Algebraic MultigridLeast Squares InterpolationBootstrap Setup
Numerical Experiments
Outlook
Algebraic Multigrid in Lattice QCD computations, Karsten Kahl 2/26
The Multigrid PrincipleAlgebraic Multigrid
Algebraic Multigrid for QCDBootstrap Algebraic Multigrid
Numerical ExperimentsOutlook
The continuum limit scaling problem — Simple Example
Partial differential equation
∆ψ(x) = ϕ, x ∈ (0, 1)2
ψ(x) = 0, x ∈ ∂(0, 1)2
Discretization by finite differences
Au = f, A ∈ RN2×N2
, u, f ∈ RN2
I smallest eigenvalue λmin ∼ h2 = N−2
I largest eigenvalue λmax = const
Continuum limit: condition number κh→0−→∞
Algebraic Multigrid in Lattice QCD computations, Karsten Kahl 3/26
The Multigrid PrincipleAlgebraic Multigrid
Algebraic Multigrid for QCDBootstrap Algebraic Multigrid
Numerical ExperimentsOutlook
Multigrid in a Nutshell
Fewer
First Coarse Grid
Finest Grid
SmoothThe Multigrid V−cycle
Restriction
Prolongation
Dofs
Algebraic Multigrid in Lattice QCD computations, Karsten Kahl 4/26
The Multigrid PrincipleAlgebraic Multigrid
Algebraic Multigrid for QCDBootstrap Algebraic Multigrid
Numerical ExperimentsOutlook
Theory of AMG
Components of Algebraic Multigrid
Given: Linear system of equationsAu = f
Wanted: Hierarchy of systems A0 = AA`u` = f`, ` = 0, . . . , L
Needed: Coarse spaces (coarse variables)Cn0 ⊃ Cn1 ⊃ . . . ⊃ CnL
Smoother
S` : Cn` −→ Cn`
High modes
Interpolation
P ``+1 : Cn`+1 −→ Cn`
Low modes
Complementarity of Smoother and InterpolationAlgebraic Multigrid in Lattice QCD computations, Karsten Kahl 5/26
The Multigrid PrincipleAlgebraic Multigrid
Algebraic Multigrid for QCDBootstrap Algebraic Multigrid
Numerical ExperimentsOutlook
Theory of AMG
Coarse variables in Algebraic Multigrid
Given: Variables u1, . . . , unWanted: Coarse variables v1, . . . , vnc with nc � n
vi =∑j∈Ni
αijuj
Aggregation
Nl ∩Nk = ∅ for all k, l
C − F splitting
αij = δij =⇒ P =
(WI
)Algebraic Multigrid in Lattice QCD computations, Karsten Kahl 6/26
The Multigrid PrincipleAlgebraic Multigrid
Algebraic Multigrid for QCDBootstrap Algebraic Multigrid
Numerical ExperimentsOutlook
Theory of AMG
Generic Multigrid Algorithm
MG`(A`, b`)
if ` < L thenu` = 0for i = 1, . . . , ν1 dou` = S`(A`, u`, b`) “Pre-smooothing”
end foru`+1 = MG(A`+1,
(P ``+1
)H(b` −A`u`))
u` = u` + P ``+1u`+1 “Coarse-grid correction”for i = 1, . . . , ν2 dou` = S`(A`, u`, b`) “Post-smoothing”
end forelseu`+1 = A−1
`+1
(P ``+1
)H(b` −A`u`)
end ifAlgebraic Multigrid in Lattice QCD computations, Karsten Kahl 7/26
The Multigrid PrincipleAlgebraic Multigrid
Algebraic Multigrid for QCDBootstrap Algebraic Multigrid
Numerical ExperimentsOutlook
Theory of AMG
Two-grid convergence strong approximation property
I Strong approximation property
‖(I − PA−1c PHA)e‖2A ≤ K‖Ae‖22
I Smoothing property
‖Se‖2A ≤ ‖e‖2A − α1〈Ae,Ae〉2‖Se‖2A ≤ ‖e‖2A − α2〈ASe,Ae〉2
I Two-grid convergence
‖S(I − PA−1
c PHA)Se‖2A ≤
1− α1K
1 + α2K
‖e‖2A
I Assumptions fulfilled on all levels ⇒ Multigrid convergence
Algebraic Multigrid in Lattice QCD computations, Karsten Kahl 8/26
The Multigrid PrincipleAlgebraic Multigrid
Algebraic Multigrid for QCDBootstrap Algebraic Multigrid
Numerical ExperimentsOutlook
Theory of AMG
Two-grid convergence weak approximation property
I Weak approximation property
‖ (I − PR) e‖2 ≤K
‖A‖2‖e‖A
I Convergent hermitian positive definite smoother S
λmin(I − SA) ≥ 0 and λmin(S) ≥ 1
c2S‖A‖22
I Two-grid convergence
‖S(I − PA−1
c PHA)Se‖2A ≤
(1− 1
c0
)‖e‖A
with c0 = (2 +KcS)2
Algebraic Multigrid in Lattice QCD computations, Karsten Kahl 9/26
The Multigrid PrincipleAlgebraic Multigrid
Algebraic Multigrid for QCDBootstrap Algebraic Multigrid
Numerical ExperimentsOutlook
Smoothers for QCDInterpolation for QCD
The continuum limit scaling problem in QCD
Wilson discretization of the Dirac equation
D =∑µ
γµ ⊗ dcµ + a · dµµ + 1 ·m
Continuum limit scaling problems:
I condition number
κ ∼ (a ·m)−1 m,a→0−→ ∞
I eigenvalues K below threshold (e.g. 100 MeV)
K ∼ V = N3sNt
Scalability in a,m and V needed (possible in optimal complexity?)(Algebraic) Multigrid
Algebraic Multigrid in Lattice QCD computations, Karsten Kahl 10/26
The Multigrid PrincipleAlgebraic Multigrid
Algebraic Multigrid for QCDBootstrap Algebraic Multigrid
Numerical ExperimentsOutlook
Smoothers for QCDInterpolation for QCD
Choices for AMG in QCD Computations
Application of AMG to linear system D of Wilson fermions
Du = f,D ∈ CNdnsnc , with D† = Γ5DΓ5,
where D is non-hermitian pos. real and Γ5D hermitian indefinite
Define R`, P`, S` such that
I the Γ5-’ambiguity’ is hidden to the AMG algorithm
I the Γ5-structure is preserved in the multigrid hierarchy
D†c = Γ5DcΓ5
Algebraic Multigrid in Lattice QCD computations, Karsten Kahl 11/26
The Multigrid PrincipleAlgebraic Multigrid
Algebraic Multigrid for QCDBootstrap Algebraic Multigrid
Numerical ExperimentsOutlook
Smoothers for QCDInterpolation for QCD
Domain Decomposition smoother S`
Decomposition of L
L2 L3
L1 L4
I easily parallelizable
I implementation available
I local Dirichlet boundaries
I no theory available
Canonical restrictions
Ri : L −→ Li
Block inverses
Bi := RTi (RiDR
Ti )−1Ri
RB Schwarz
for color = 1, 2 dor = b−Dzz = z + (
∑i with Li⊂L|color
Bi)rend for
[1] Hermann Schwarz 1870[2] Martin Luscher 2003
Algebraic Multigrid in Lattice QCD computations, Karsten Kahl 12/26
The Multigrid PrincipleAlgebraic Multigrid
Algebraic Multigrid for QCDBootstrap Algebraic Multigrid
Numerical ExperimentsOutlook
Smoothers for QCDInterpolation for QCD
Polynomial smoother S`
Polynomial best-approximationsFor E ⊂ C \ 0 containing high modesfind polynomial q s.t.
minq∈Πn
maxz∈E|z−1 − q(z)|
I minimizer q∗ known in shortrecursion for ellipsoidal E
I easily parallelizable
I theory available
I estimate of ‖A‖ required
0 2 4 6 8−4
−3
−2
−1
0
1
2
3
4
|z−1 − q∗(z)|Algebraic Multigrid in Lattice QCD computations, Karsten Kahl 13/26
The Multigrid PrincipleAlgebraic Multigrid
Algebraic Multigrid for QCDBootstrap Algebraic Multigrid
Numerical ExperimentsOutlook
Smoothers for QCDInterpolation for QCD
Structure-preserving restriction and prolongation
Preserve γ5 symmetry ↔ symmetry of spectrum to real axis
Pγ5 = γ5P
Dc = P †DP=⇒ D†c = γ5Dγ5
For γ5 =
(I−I
)choose prolongation P =
(P1 00 P2
)⇒ γ5-structure transferred to coarse-grid in Dc
Algebraic Multigrid in Lattice QCD computations, Karsten Kahl 14/26
The Multigrid PrincipleAlgebraic Multigrid
Algebraic Multigrid for QCDBootstrap Algebraic Multigrid
Numerical ExperimentsOutlook
Smoothers for QCDInterpolation for QCD
0th and 1st order interpolation...
0th order 1st order
I aggregation
I easy to implement
I piece-wise constant
Aggregation interpolation
I C/F splitting
I next-neighbor interpolation
I linear interpolation
C/F interpolation
Bootstrap Algebraic Multigrid
Algebraic Multigrid in Lattice QCD computations, Karsten Kahl 15/26
The Multigrid PrincipleAlgebraic Multigrid
Algebraic Multigrid for QCDBootstrap Algebraic Multigrid
Numerical ExperimentsOutlook
Smoothers for QCDInterpolation for QCD
... for Wilson fermions
In 4D with next neighbor coupling
kA
I next neighbor coupling oncoarse grid
I operator complexity
1 +k
|A|+
k2
|A|2. . .
I additional diagonal coupling(preserved on coarser grids)
I operator complexity
1 +81
9
1
16
(1 +
1
16· · ·
Algebraic Multigrid in Lattice QCD computations, Karsten Kahl 16/26
The Multigrid PrincipleAlgebraic Multigrid
Algebraic Multigrid for QCDBootstrap Algebraic Multigrid
Numerical ExperimentsOutlook
Least Squares InterpolationBootstrap Setup
Bootstrap AMG
Bootstrap Algebraic Multigrid Framework
“Continous updating components of the MG hierarchy using practical
tools and measures built from the evolving MG solver”
I Compatible Relaxation and Algebraic DistancesI Adaptive computation of nested subspacesI Quality measurement of grid hierarchies
I Least Squares interpolationI Adaptive definition of interpolationI Quality and complexity control of the multigrid method
I Bootstrap setup techniquesI Multigrid discovery of algebraically smooth errorI Multigrid quality control mechanisms
Algebraic Multigrid in Lattice QCD computations, Karsten Kahl 17/26
The Multigrid PrincipleAlgebraic Multigrid
Algebraic Multigrid for QCDBootstrap Algebraic Multigrid
Numerical ExperimentsOutlook
Least Squares InterpolationBootstrap Setup
Least Squares Interpolation
Given: test vectors u(1), . . . , u(k) ∈ Cn representing low modesWanted: interpolation P accurate for test vectors u(s)
Least Squares Interpolation
LCi(pi) =
k∑s=1
ωs(u(s)i −
∑j∈Ci
(pi)ju(s)j )2 → min
pi
I interpolatory points Ci ⊂ C for i ∈ FI in neighborhood of i
I weights ωs ∼ ‖u(s)‖‖Au(s)‖−1 ∈ R+
I test vectors (→ bootstrap)
∈ Fi
pij2
pij4
pij1
pij3
∈ C
Algebraic Multigrid in Lattice QCD computations, Karsten Kahl 18/26
The Multigrid PrincipleAlgebraic Multigrid
Algebraic Multigrid for QCDBootstrap Algebraic Multigrid
Numerical ExperimentsOutlook
Least Squares InterpolationBootstrap Setup
Least Squares Interpolation – Choice of Weights ωs
Idea: Weights ωs bias Least Squares fit u(s) to low modes
⇒ Intuitive choice ωs ∼ ‖u(s)‖‖Au(s)‖−1
Problem: Local errors in test vectors weighted by global weight
Remedies:
I For u(s) with large ωs and |(Au(s)
)i|
u(s)i = u
(s)i −
1
aii
(Au(s)
)i
Modified Least Squares Fit
LCi(pi) =k∑s=1
ωs(u(s)i −
1
aii
(Au(s)
)i−∑j∈Ci
(pi)ju(s)j )2 → min
pi
Algebraic Multigrid in Lattice QCD computations, Karsten Kahl 19/26
The Multigrid PrincipleAlgebraic Multigrid
Algebraic Multigrid for QCDBootstrap Algebraic Multigrid
Numerical ExperimentsOutlook
Least Squares InterpolationBootstrap Setup
Bootstrap Setup - Multigrid Eigensolver
Assumption: No a priori information on low modes available!
I smoother action known, initial test vectors
u(s) = Sηu(s), u(s) random
I observation (P` = P 01 · · ·P
`−1` , A` = PH` A0P`, T` = PH` P`)
〈v`, v`〉A`〈v`, v`〉T`
=〈P`v`, P`v`〉A〈P`v`, P`v`〉2
Bootstrap Idea
Eigenpairs(v`, λ`) of (A`, T`)
−→Eigenpairs
(P`v`, λ`) of A+ interpolation error
Algebraic Multigrid in Lattice QCD computations, Karsten Kahl 20/26
The Multigrid PrincipleAlgebraic Multigrid
Algebraic Multigrid for QCDBootstrap Algebraic Multigrid
Numerical ExperimentsOutlook
Least Squares InterpolationBootstrap Setup
Bootstrap Setup – Cycling Strategies
Relax on Au = 0, u ∈ U and Av = λTv, v ∈ V
Relax on Au = 0, u ∈ U
Compute V , s.t., Av = λTv, v ∈ V
Relax on Av = λTv, v ∈ V
Algebraic Multigrid in Lattice QCD computations, Karsten Kahl 21/26
The Multigrid PrincipleAlgebraic Multigrid
Algebraic Multigrid for QCDBootstrap Algebraic Multigrid
Numerical ExperimentsOutlook
Numerical Experiments — Settings
I GMRESI restart 32I tolerance 10−8
I Bootstrap AMGI V 2-cycle setupI polynomial smoother of degree 4I at most 16 points used in prolongation for each pointI 24(+24) test vectorsI coarsest grid 24 or 34
I PlatformI single processorI development platform: MATLAB
Algebraic Multigrid in Lattice QCD computations, Karsten Kahl 22/26
The Multigrid PrincipleAlgebraic Multigrid
Algebraic Multigrid for QCDBootstrap Algebraic Multigrid
Numerical ExperimentsOutlook
Numerical Experiments
GMRES(32) GMRES(32) CGN+ BootAMG
V 84 124 164 84 124 164 84 124 164
setup (time) 22.3 110 428 — — — — — —solve (time) 5.77 68 222 17.3 532 2359 23.8 407 885#iterations 17 31 33 502 2106 2536 347 944 699
total (time) 28.1 178 650 17.3 532 2536 23.8 363 885
164 κ-study
−2 −1 0 1 2 3 4 5
x 10−3
0
500
1000
1500
2000
2500
3000
3500
4000
#ite
ratio
ns
bare subtracted mass
AMG GMRES(32)
GMRES(32)
κ
κc
84, 124 Lattice
unknown parameters
(λmin) = 10−5
3-level BootAMG
164 Lattice
β = 5.8 (0.05 fm)
κ = .15462 (400 MeV)
κc = .1548
4-level BootAMG
Algebraic Multigrid in Lattice QCD computations, Karsten Kahl 23/26
The Multigrid PrincipleAlgebraic Multigrid
Algebraic Multigrid for QCDBootstrap Algebraic Multigrid
Numerical ExperimentsOutlook
Results: 48× 243
FGMRES+ αaMG(8) CGN
setup timings 45.22s -solve iterations 9 4476
timings 6.97s 121.82stotal timings 52.19s 121.82s
Configuration:
I 48× 243 Clover Wilson-Dirac
I parameters:I pion mass: mπ = 630 MeVI csw = 1.90952
I generated with parameters from[Del Debbio et al. 2007]with public code
Settings:
I tolerance: 10−10
I restart: 25
I block-size: 34
I local solver: 3 iterations GMRES
I coarse solver: 12 iterations GMRES
I levels: 2
I coarsening: 34 × 6→ 20
I 512 cores (Juropa @ JSC)
Algebraic Multigrid in Lattice QCD computations, Karsten Kahl 24/26
The Multigrid PrincipleAlgebraic Multigrid
Algebraic Multigrid for QCDBootstrap Algebraic Multigrid
Numerical ExperimentsOutlook
Results: 64× 323
FGMRES+ αaMG(8) CGN
setup timings 24.85s -solve iterations 17 9181
timings 5.05s 119.04stotal timings 29.90s 119.04s
Configuration:
I 64× 323 Wilson-Dirac
I 2HEX smeared tree level improvedClover
I parameters:I pion mass: mπ = 300 MeVI csw = 1.0
I BMW collaboration[arXiv:1011.2403][arXiv:1011.2711]
Settings:
I tolerance: 10−10
I restart: 25
I block-size: 24
I local solver: 3 iterations GMRES
I coarse solver: 12 iterations GMRES
I levels: 2
I coarsening: 44 × 6→ 20
I 4096 cores (Juropa @ JSC)
Algebraic Multigrid in Lattice QCD computations, Karsten Kahl 25/26
The Multigrid PrincipleAlgebraic Multigrid
Algebraic Multigrid for QCDBootstrap Algebraic Multigrid
Numerical ExperimentsOutlook
Summary and Outlook
Summary
I development stage of bootstrap AMG for QCD
I setup done once per configuration (and mass)
I high potential for parallelism in setup and solver
I implementation of 0th order interpolation (almost) available
Outlook
I optimize setup and its numerous parameters
I develop theoretical foundation
I implementation of 1st order approach
I application to physical configurations (w.r.t. β, κ, V )
Algebraic Multigrid in Lattice QCD computations, Karsten Kahl 26/26
Top Related