Introduction to Wavelets andWavelet Based Numerical Homogenization

Olof Runborg

Numerical Analysis,School of Computer Science and Communication, KTH

Summer School on Multiscale Modeling and Simulation in ScienceBosön, 2007-06-06

Wavelet multiresolution decomposition

Wavelet multiresolution decomposition

Wavelet multiresolution decomposition

Wavelet multiresolution decomposition

Wavelet multiresolution decomposition

Wavelet multiresolution decomposition

Wavelet multiresolution decomposition

Wavelet multiresolution decomposition

Mathematical description

An (orthogonal) Multiresolution Analysis (MRA) is a family of closedfunction spaces such that

1 · · · ⊂ Vj ⊂ Vj+1 ⊂ · · · ⊂ L2(R)

2 f (t) ∈ Vj ⇔ f (2t) ∈ Vj+1

3 ∪Vj = L2(R) and ∩Vj = 04 Exists ϕ for which ϕ(x − k)k∈Z is L2-orthonormal basis for V0.

Vj are called scaling spaces and ϕ is the scaling (or shape)function.1+2) gives that Vj contains functions with finer and finer detailswhen j increases, and eventually all of L2 by 3).2+4) gives that ϕj,k := 2j/2ϕ(2jx − k)k∈Z is an ON basis for Vj .(ϕj,k are dilations (2j ) and translations (k ) of the original ϕ.)

Mathematical description

An (orthogonal) Multiresolution Analysis (MRA) is a family of closedfunction spaces such that

1 · · · ⊂ Vj ⊂ Vj+1 ⊂ · · · ⊂ L2(R)

2 f (t) ∈ Vj ⇔ f (2t) ∈ Vj+1

3 ∪Vj = L2(R) and ∩Vj = 04 Exists ϕ for which ϕ(x − k)k∈Z is L2-orthonormal basis for V0.

Remarks:Vj are called scaling spaces and ϕ is the scaling (or shape)function.1+2) gives that Vj contains functions with finer and finer detailswhen j increases, and eventually all of L2 by 3).2+4) gives that ϕj,k := 2j/2ϕ(2jx − k)k∈Z is an ON basis for Vj .(ϕj,k are dilations (2j ) and translations (k ) of the original ϕ.)

Multiresolution Analysis (MRA)More remarks

For numerical calculations:

Good if ϕ(x) is localized spatiallyNeed to replace L2(R) by finite interval, e.g. L2([0,1]). Then

V0 ⊂ · · · ⊂ Vj ⊂ Vj+1 ⊂ · · · ⊂ L2([0,1])

and the spaces Vj are finite dimensional. In fact

Vj ∼ R2j.

Wavelet spaces

The “difference” between two successive scaling spaces Vj ⊂ Vj+1 isthe wavelet space Wj . It is defined as the orthogonal complement to Vjin Vj+1,

Vj+1 = Vj ⊕Wj , Vj ⊥ Wj .

Wj thus contains the details present in Vj+1 but not in Vj .By induction

Vj+1 = Wj ⊕Wj−1 ⊕ · · · ⊕W0 ⊕ V0.

and also ∪Wj = L2(R).

Moreover, there exists a mother wavelet ψ(x), such that

ψ(x − k)k∈Z

is an ON-basis for W0.

Wavelet multiresolution decomposition

Wavelet multiresolution decomposition

Wavelet spacesBasis

Dilations and translations of ψ,

ψj,k := 2j/2ψ(2jx − k)k∈Z,

is an ON-basis for Wj and ψj,k , ϕj,k

k∈Z

is an ON-basis for Vj+1.ψj,kj,k∈Z

is an ON-basis for L2(R).

Wavelet examples

Haar

Scaling function φ(x) Mother wavelet ψ(x)

Wavelet examples

Daubechies 4

Scaling function φ(x) Mother wavelet ψ(x)

Wavelet examples

Daubechies 6

Scaling function φ(x) Mother wavelet ψ(x)

Wavelet examples

Daubechies 8

Scaling function φ(x) Mother wavelet ψ(x)

Approximation Properties

Basic approximation mechanism:vanishing moments + space localization

|ujk | ≤ c2−(j+1/2)M supx∈Ω

∣∣∣u(M)(x)∣∣∣ .

Approximation Properties

Basic approximation mechanism:vanishing moments + space localization

DefinitionThe mother wavelet ψ has M vanishing moments if∫

ψ(x)xmdx = 0, m = 0, . . . ,M − 1.

Note 1: Implies the same for ψjk (x) = 2j/2ψ(2jx − k).

Note 2: There exists compactly supported wavelet systems with arbi-trary many vanishing moments. (Daubechies 88).

|ujk | ≤ c2−(j+1/2)M supx∈Ω

∣∣∣u(M)(x)∣∣∣ .

Approximation Properties

Basic approximation mechanism:vanishing moments + space localization

Suppose u(x) =∑

jk ujkψjk (x), and Ω = suppψjk . Hence,

ujk =

∫Ω

u(x)ψjk (x)dx ,

|ujk | ≤ c2−(j+1/2)M supx∈Ω

∣∣∣u(M)(x)∣∣∣ .

Approximation Properties

Basic approximation mechanism:vanishing moments + space localization

Suppose u(x) =∑

jk ujkψjk (x), and Ω = suppψjk . Hence,

ujk =

∫Ω

u(x)ψjk (x)dx ,

Take x0 ∈ Ω and Taylor expand:

ujk =

∫Ω

M−1∑m=0

u(n)(x0)(x − x0)

m

m!ψjk (x)dx +

∫Ω

R(x)(x − x0)

M

M!ψjk (x)dx

|ujk | ≤ c2−(j+1/2)M supx∈Ω

∣∣∣u(M)(x)∣∣∣ .

Approximation Properties

Basic approximation mechanism:vanishing moments + space localization

Suppose u(x) =∑

jk ujkψjk (x), and Ω = suppψjk . Hence,

ujk =

∫Ω

u(x)ψjk (x)dx ,

Take x0 ∈ Ω and Taylor expand:

ujk =

∫Ω

M−1∑m=0

u(n)(x0)(x − x0)

m

m!ψjk (x)dx +

∫Ω

R(x)(x − x0)

M

M!ψjk (x)dx

M vanishing moments ⇒ First term = 0.

|ujk | ≤ c2−(j+1/2)M supx∈Ω

∣∣∣u(M)(x)∣∣∣ .

Approximation Properties

Basic approximation mechanism:vanishing moments + space localization

ujk =

∫Ω

R(x)(x − x0)

M

M!ψjk (x)dx

|ujk | ≤ c2−(j+1/2)M supx∈Ω

∣∣∣u(M)(x)∣∣∣ .

Approximation Properties

Basic approximation mechanism:vanishing moments + space localization

ujk =

∫Ω

R(x)(x − x0)

M

M!ψjk (x)dx

Noting that supx∈Ω |R(x)| = supx∈Ω |u(M)(x)|

|ujk | ≤ |Ω|M supx∈Ω

∣∣∣u(M)(x)∣∣∣ ∫

Ω|ψjk (x)|dx ≤ |Ω|M+1/2 sup

x∈Ω

∣∣∣u(M)(x)∣∣∣ .

|ujk | ≤ c2−(j+1/2)M supx∈Ω

∣∣∣u(M)(x)∣∣∣ .

Approximation Properties

Basic approximation mechanism:vanishing moments + space localization

ujk =

∫Ω

R(x)(x − x0)

M

M!ψjk (x)dx

Noting that supx∈Ω |R(x)| = supx∈Ω |u(M)(x)|

|ujk | ≤ |Ω|M supx∈Ω

∣∣∣u(M)(x)∣∣∣ ∫

Ω|ψjk (x)|dx ≤ |Ω|M+1/2 sup

x∈Ω

∣∣∣u(M)(x)∣∣∣ .

Compact support of ψ ⇒ |Ω| ∼ 2−j , (since ψjk = 2j/2ψ(2jx − k)).

|ujk | ≤ c2−(j+1/2)M supx∈Ω

∣∣∣u(M)(x)∣∣∣ .

Approximation Properties

Basic approximation mechanism:vanishing moments + space localization

ujk =

∫Ω

R(x)(x − x0)

M

M!ψjk (x)dx

Noting that supx∈Ω |R(x)| = supx∈Ω |u(M)(x)|

|ujk | ≤ |Ω|M supx∈Ω

∣∣∣u(M)(x)∣∣∣ ∫

Ω|ψjk (x)|dx ≤ |Ω|M+1/2 sup

x∈Ω

∣∣∣u(M)(x)∣∣∣ .

Compact support of ψ ⇒ |Ω| ∼ 2−j , (since ψjk = 2j/2ψ(2jx − k)).

|ujk | ≤ c2−(j+1/2)M supx∈Ω

∣∣∣u(M)(x)∣∣∣ .

Approximation Properties

Basic approximation mechanism:vanishing moments + space localization

|ujk | ≤ 2−(j+1/2)M supx∈Ω

∣∣∣u(M)(x)∣∣∣ .

ujk decay rapidly when u(x) smooth locally and M large (manyvanishing moments).

Most fine-scale coefficients can be neglected. Just need to keepthose where u(x) has abrupt changes/discontinuitiesGood approximation also of piecewise smooth functions.

Approximation Properties

Basic approximation mechanism:vanishing moments + space localization

|ujk | ≤ 2−(j+1/2)M supx∈Ω

∣∣∣u(M)(x)∣∣∣ .

ujk decay rapidly when u(x) smooth locally and M large (manyvanishing moments).Most fine-scale coefficients can be neglected. Just need to keepthose where u(x) has abrupt changes/discontinuities

Good approximation also of piecewise smooth functions.

Approximation Properties

Basic approximation mechanism:vanishing moments + space localization

|ujk | ≤ 2−(j+1/2)M supx∈Ω

∣∣∣u(M)(x)∣∣∣ .

ujk decay rapidly when u(x) smooth locally and M large (manyvanishing moments).Most fine-scale coefficients can be neglected. Just need to keepthose where u(x) has abrupt changes/discontinuitiesGood approximation also of piecewise smooth functions.

Time Frequency Representation of Signals

Given a discrete signal f (n) with n = 0,1, . . ..

Time representation — total localization in time

f (n) =∑

j

fjδ(n − j)

Basis functions

Time Frequency Representation of Signals

Given a discrete signal f (n) with n = 0,1, . . ..

Time representation — total localization in time

f (n) =∑

j

fjδ(n − j)

Time Frequency Representation of Signals

Given a discrete signal f (n) with n = 0,1, . . ..

Fourier representation — total localization in frequency

f (n) =∑

j

fjexp(inj2π/N)

Basis functions

Time Frequency Representation of Signals

Given a discrete signal f (n) with n = 0,1, . . ..

Fourier representation — total localization in frequency

f (n) =∑

j

fjexp(inj2π/N)

Time Frequency Representation of Signals

Given a discrete signal f (n) with n = 0,1, . . ..

Wavelet representation — localization in time and frequency

f (n) =∑

ij

wijψij(n/2π)

Basis functions

Time Frequency Representation of Signals

Given a discrete signal f (n) with n = 0,1, . . ..

Wavelet representation — localization in time and frequency

f (n) =∑

ij

wijψij(n/2π)

Time Frequency Representation of Signals

Given a discrete signal f (n) with n = 0,1, . . ..

Wavelet representation — localization in time and frequency

f (n) =∑

ij

wijψij(n/2π)

Time Frequency Representation of SignalsExample

Time representation — total localization in time

Time Frequency Representation of SignalsExample

Fourier representation — total localization in frequency

Time Frequency Representation of SignalsExample

Wavelet representation — localization in time and frequency

Wavelet transforms

MRA system characterized by the filter coefficients gk and hk,

ϕ(x) =∑

k

hkϕ1,k (x) =√

2∑

k

hkϕ(2x − k),

ψ(x) =∑

k

gkϕ1,k (x) =√

2∑

k

gkϕ(2x − k).

(Note, ϕ,ψ ∈ V0 ⊂ V1 which is spanned by ϕ1,k .)Conversely,

ϕj+1,k (x)︸ ︷︷ ︸∈Vj+1

=∑

`

hk−2`ϕj,`(x)︸ ︷︷ ︸∈Vj

+∑

`

gk−2`ψj,`(x)︸ ︷︷ ︸∈Wj

,

In general: Compactly supported wavelets ⇔ gk and hk are finitelength (FIR) filters.Example: Haar has h0 = h1 = 1√

2and g0 = −g1 = 1√

2.

Wavelet multiresolution decomposition

Wavelet transforms, cont.

Suppose u ∈ Vj+1 is given

u(x) =∑

k

ukϕj+1,k (x).

We want to decompose u into its parts in Wj and Vj .Use identity above for ϕj+1,k

u(x) =∑

k

∑`

uk hk−2`ϕj,`(x) +∑

k

∑`

uk gk−2`ψj,`(x)

andu(x) =

∑`

Uc` ϕj,`(x)︸ ︷︷ ︸

∈Vj (“coarse” part)

+∑

`

U f`ψj,`(x)︸ ︷︷ ︸

∈Wj (“fine” part)

,

whereUc

` =∑

k

ukhk−2`, U f` =

∑k

ukgk−2`.

Wavelet transforms, cont.

Simple to extract the coarse and fine part of u = uk:

Wu =

(UfUc

), u ∈ Vj+1 Uf ∈ Wj , Uc ∈ Vj .

For compactly supported wavelets, W is sparse. It is also orthonormal,WTW = I.In Haar basis on [0,1],

W =1√2

1 −1 0 · · ·0 0 1 −1 0 · · ·...

.

.

.. . .

. . .0 0 · · · 0 1 −11 1 0 · · ·0 0 1 1 0 · · ·...

.

.

.. . .

. . .0 0 · · · 0 1 1

∈ R2j+1×2j+1.

Fast Wavelet Transform

Decomposition can be continued hierarchically

Uj+1 → U fj

Ucj → U f

j−1 Uc

j−1 → U fj−2

Ucj−2 → · · ·

O(2j+1) + O(2j) + O(2j−1) +

Total cost = O(N) where N = 2j+1 is the length of the transformedvector.

Wavelets – some contributors

Strömberg – first continuous waveletMorlet, Grossman – "wavelet"Meyer, Mallat, Coifman – multiresolution analysisDaubechies – compactly supported waveletsBeylkin, Cohen, Dahmen, DeVore – PDE methods using waveletsSweldens – lifting, second generation wavelets

. . . and many, many more.

Wavelet based homogenization[Beylkin,Brewster,Engquist,Dorobantu,Levy,Gilbert,O.R.,. . . ]

Suppose

Lj+1u = f , Lj+1 ∈ L(Vj+1,Vj+1) u, f ∈ Vj+1,

is a discretization (e.g. FD, FEM) of a differential equation onscale-level j + 1 where Lj+1 contains small scales.

Want to find an effective discrete operator Lj ′ , with j ′ j that computesthe coarse part of u.

C.f. classical homogenization.

Example (Elliptic eq, Haar)

∂x r(x/ε)∂xuε = f , ⇒ Lj+1 =1h2 ∆+Rε∆−.

where Rε is diagonal matrix sampling r(x/ε), and 2j ∼ 1/ε. Here onecould use

Lj ′ =1h2 ∆+R∆−.

Wavelet based homogenization[Beylkin,Brewster,Engquist,Dorobantu,Levy,Gilbert,O.R.,. . . ]

Suppose

Lj+1u = f , Lj+1 ∈ L(Vj+1,Vj+1) u, f ∈ Vj+1,

is a discretization (e.g. FD, FEM) of a differential equation onscale-level j + 1 where Lj+1 contains small scales.

Want to find an effective discrete operator Lj ′ , with j ′ j that computesthe coarse part of u.

C.f. classical homogenization.

Example (Elliptic eq, Haar)

∂x r(x/ε)∂xuε = f , ⇒ Lj+1 =1h2 ∆+Rε∆−.

where Rε is diagonal matrix sampling r(x/ε), and 2j ∼ 1/ε. Here onecould use

Lj ′ =1h2 ∆+R∆−.

Wavelet based homogenizationWavelet decomposition of operator

Start from equation

Lj+1u = f , Lj+1 ∈ L(Vj+1,Vj+1) u, f ∈ Vj+1.

Decompose equation in coarse and fine part (use WTW = I)

WLj+1WTWu = Wf ⇒(Aj BjCj Lj

)(U f

Uc

)=

(F f

F c

).

Eliminate U f ,(Lj − CjA−1

j Bj)Uc = F c − CjA−1j F f .

Supposing f smooth so F f = 0 and F c = f .

(Lj − CjA−1j Bj)Uc = f .

Wavelet based homogenizationWavelet decomposition of operator

Start from equation

Lj+1u = f , Lj+1 ∈ L(Vj+1,Vj+1) u, f ∈ Vj+1.

Decompose equation in coarse and fine part (use WTW = I)

WLj+1WTWu = Wf ⇒(Aj BjCj Lj

)(U f

Uc

)=

(F f

F c

).

Eliminate U f ,(Lj − CjA−1

j Bj)Uc = F c − CjA−1j F f .

Supposing f smooth so F f = 0 and F c = f .

(Lj − CjA−1j Bj)Uc = f .

Wavelet based homogenizationWavelet decomposition of operator

Start from equation

Lj+1u = f , Lj+1 ∈ L(Vj+1,Vj+1) u, f ∈ Vj+1.

Decompose equation in coarse and fine part (use WTW = I)

WLj+1WTWu = Wf ⇒(Aj BjCj Lj

)(U f

Uc

)=

(F f

F c

).

Eliminate U f ,(Lj − CjA−1

j Bj)Uc = F c − CjA−1j F f .

Supposing f smooth so F f = 0 and F c = f .

(Lj − CjA−1j Bj)Uc = f .

Wavelet based homogenizationNumerically homogenized operator

We call the matrix

Lj = Lj − CjA−1j Bj , Lj ∈ L(Vj ,Vj),

the (numerically) homogenized operator. SinceHalf the size of original Lj+1.Given Lj , f we can solve for coarse part of solution, Uc .Takes influence of fine scales into account.

Compare with classical homogenization:

L = ∇R(x/ε)∇ ⇒ L = ∇∫

R(x)dx∇−∇∫

R(x)∂χ

∂xdx∇

where χ solves the (elliptic) cell problem.

Wavelet based homogenization

Reduction can be repeated,

Lj → Lj−1 → Lj−2 → . . . , Lj ∈ L(Vj ,Vj),

to discard suitably many small scales / to get a suitably coarse grid.Also, condition number improves

κ(Lj) < κ(Lj+1).

Wavelet based homogenization

Problem: L sparse (banded) 6→ L sparse (banded). (Must invert Aj .)

However: Approximation properties of wavelets imply elements of A−1j

decay rapidly away from diagonal.

Therefore: L diagonally dominant in many important cases and can bewell approximated by a banded matrix. (Cf. a (local) differentialoperator.)

Different Approximation Strategies

1 “Crude” truncation to ν diagonals,2 Band projection to ν diagonals, defined by

Mx = band(M, ν)x , ∀x ∈ spanv1,v2, . . . ,vν.

v j = 1j−1,2j−1, . . . ,N j−1T , j = 1, . . . , ν.

C.f. “probing”, [Chan, Mathew], [Axelsson, Pohlman, Wittum].3 The above methods used on the matrix H instead, where e.g.

Lj+1 =1h2 ∆+R∆− ⇒ Lj =

1(2h)2 ∆+H∆−.

H can be seen as the effective material coefficient.4 The above methods used on the matrix A−1

j instead, whereLj = Lj − CjA−1

j Bj . [Levy, Chertock]

Elliptic 1D case

Consider the elliptic one-dimensional problem

∂xaε(x)∂xu = 1, u(0) = u′(1) = 0,

with standard second order discretization.Try two cases:

aε(x) = ”noise” aε(x) = ”narrow slit”

Elliptic 1D case – noiseDifferent approximation strategies

Exact

ν=13 ν=15 ν=17

0 0.5 1−1

−0.8

−0.6

−0.4

−0.2

0trunc(L, ν)

Exact

ν=3 ν=5 ν=7

0 0.5 1−1

−0.8

−0.6

−0.4

−0.2

0trunc(H, ν)

Exact

ν=1 ν=3 ν=5

0 0.5 1−1

−0.8

−0.6

−0.4

−0.2

0band(H, ν)

Exact

ν=3 ν=5 ν=7

0 0.5 1−1

−0.8

−0.6

−0.4

−0.2

0band(L, ν)

Elliptic 1D case – narrow slitDifferent approximation strategies

Exact

ν=13 ν=15 ν=17

0 0.5 1−1

−0.8

−0.6

−0.4

−0.2

0trunc(L,ν)

Exact

ν=3 ν=5 ν=7

0 0.5 1−1

−0.8

−0.6

−0.4

−0.2

0trunc(H, ν)

Exact

ν=1 ν=3 ν=5

0 0.5 1−1

−0.8

−0.6

−0.4

−0.2

0band(H, ν)

Exact

ν=7 ν=9 ν=11

0 0.5 1−1

−0.8

−0.6

−0.4

−0.2

0band(L,ν)

Elliptic 1D case – narrow slitMatrix element size

0

1000

2000

3000

4000

5000

6000

7000

5 10 15 20 25 30

5

10

15

20

25

30

L

0

0.5

1

1.5

2

2.5

3

5 10 15 20 25 30

5

10

15

20

25

30

H

ExamplesHelmholtz 2D case

Simulate a wave hitting a wall witha small opening modeld byHelmholtz

∇a(x , y)∇u + ω2u = 0, 0

0.2

0.4

0.6

0.8

1 00.2

0.40.6

0.81

0

0.2

0.4

0.6

0.8

1

x y

a

Wall with slit

ExamplesHelmholtz 2D case

0

0.5

1 00.5

1

−2

−1

0

1

2

Original

0

0.5

1 00.5

1

−2

−1

0

1

2

1 homogenization

0

0.5

1 00.5

1

−2

−1

0

1

2

2 homogenizations

0

0.5

1 00.5

1

−2

−1

0

1

2

3 homogenizations

ExamplesHelmholtz 2D case

0

0.5

1 00.5

1

−2

0

2

Untruncated operator

x y

0

0.5

1 00.5

1

−10

0

10

ν=5

x y

0

0.5

1 00.5

1

−2

0

2

ν=7

x y

0

0.5

1 00.5

1

−2

0

2

ν=9

x y

Helmholtz 2D caseMatrix element size

0 100 200 300 400 500

0

100

200

300

400

500

nz = 19026