Post on 06-Mar-2018
ON WAVELETS AND MIT OCW 18.327 WAVELETS, FILTERS, AND APPLICATIONS
ERNEST YEUNG
Abstract. I wanted to expound here, from an introductory and pedagogical starting point, on wavelets. Note,
my goal here is not to reinvent the wheel. This is meant to be the “missing companion manual” to the MIT
OCW 18.327 http://ocw.mit.edu/courses/mathematics/18-327-wavelets-filter-banks-and-applications-spring-2003/
material and to any book or course on or involving Wavelets, with (one of) the value being the Python code
closely linked with each concept.
One day on 18 juin 2015, I find myself needing to learn about wavelets, fast. Here are notes, solutions to
various exercises and “textbook” problems, and associated code and code snippets that I found useful in
learning about and applying wavelets.
The first thing I now do when I want to pick up a new skill is to see if the MIT (Massachusetts Institute of
Technology) OCW (Open-Course Ware) has a course on it. So far, I’ve seen 18.327 Wavelets, Filter Banks
and Applications, taught by Strang and Amaratunga in Spring 2003 [1] (http://ocw.mit.edu/courses/
mathematics/18-327-wavelets-filter-banks-and-applications-spring-2003/). I haven’t seen it up-
dated recently. I’ll try to give some updates from my perspective. (Note, this past week, 20150709, I saw that
coursera has Computational Methods for Data Analysis; I’ll try to look into that).
Here are some features I want to mention about this exposition on Wavelets:
• EY : 20150720 I wanted to raise the issue about the wavelet and scaling function coefficients at each jth
resolution. I wanted to make resolve the distinction between doing the Fourier transform or essentially
z-transform on the time t-domain (keep in mind that this could also be the spatial domain) and what
it means when parts of the filter bank “acts” on the coefficients themselves.
My original thought was this (and please give any and all (negative) feedback on where this might
fail); with filtration being a collection of nested subspaces, i.e. V 0 ⊂ V 1 ⊂ V 2 ⊂ . . . (I was also
curious if we can think of multiresolution analysis in terms of topological spaces):
Multiresolution analysis as a filtration of L2(R)-algebras, with each subspace V j equipped
with a k-algebra, kj, that is an algebra for the coefficients of the j-level of resolution
scaling function and wavelets. I explicitly make clear what spaces we are living on and the
mapping involved and make explicit the mapping itself in the so-called filter bank representation with
commutative diagrams 5.2, 7.1.1. The operations of convolution and downsampling and upsampling
on L2(R) functions are clearly defined as an algebra and the corresponding operations in the so-called
z-domain, which is really going to L2(C), is also clearly defined and the correspondence or “dictionary”
between L2(R) and L2(C) clearly shown.
Date: 4 juillet 2015.
1991 Mathematics Subject Classification. Wavelets.
Key words and phrases. Wavelets.
If you like this pdf and LaTeX file and/or find it useful, please consider making a financial contribution to the crowdfunding
campaign I am running at Tilt/Open. Funds will go into new computer equipment so I could push a number of numerical
simulations I want to try out in computational fluid dynamics, first, and quantum super-A-polynomials, which I had written
my Masters thesis on, pay myself, and in general, go into a general fund for supporting basic research in the future: one of my
ultimate goals is to directly fund theoretical and mathematical physics research with the biggest private and crowdsourced funds
seen.
1
Then, with scaling functions and wavelets in a multiresolution analysis, I define a k-algebra, kj ,
associated with each subspace V j , for the j-level resolution scaling function and wavelet coefficients.
This way, we know exactly which (sub)space (V j or V j+1, or kj or kj+1) we are living on at each
step of a filter bank and again, the operations of convolution and downsampling and upsampling are
reduced to an algebra (which is easy).
• Explicit association of where concept and theory meets application and code. Learning
and using wavelets should be as fun as playing video games. The original MIT OCW 18.327 material
does not tell you which piece of code corresponds to directly with which Handout or Lecture. In this
pdf/LaTeX file, I do tell you, and being able to directly do the Wavelet transforms and plot them
immediately and interactively helps out a lot in learning and using wavelets. Also, all the code is on
github, making it more amenable to editing and playing around with, and won’t get dated like the
MIT OCW 18.327 materials.
• Python. The Matlab Wavelet Toolkit is a black box, as exemplified by the documentation 1. We
can do everything we’d want to and more in Python, using the open-source PyWavelets library. With
Homebrew and pip, installing packages is easy; I’ve found that installing packages in GNU Octave
isn’t as easy.
• Again, I want to emphasize that we are not trying to reinvent the wheel. There are plenty of reading
material (I should say that the original lectures by Ingrid Daubechies [5] are excellent and lucid, and
she founded the field) and plenty of disparate course lecture notes online. This is meant to be the
“missing companion manual” to the MIT OCW 18.327 material and to any book or course on or
involving Wavelets, building on top of the MIT OCW 18.327 material. (One of) The value of these
notes is the closely linked Python code, and I want to encourage and foster its use, interactively.
For more exposition (that’d usually go into an “Introduction” section) and the rationale behind all this notes
and code, and how you can help, go to the end part for the Colophon 3.
Part 1. MIT OCW 18.327 Spring 2003 3
Lecture Notes 3
1. Discrete-time Filters: Convolution; Fourier Transform; Lowpass and Highpass Filters: Handout 1 3
2. Sampling Rate Change Operations: Upsampling and Downsampling; Fractional Sampling;
Interpolation: Handout 2 6
3. Filter Banks: Time Domain (Haar example) and Frequency Domain; Conditions for Alias
Cancellation and no Distortion; Handout 3 9
4. Filter Banks (contd.): Perfect Reconstruction; Halfband Filters and Possible Factorizations 14
5. Modulation and Polyphase Representations: Noble Identities; Block Toeplitz Matrices and Block
z-transforms; Polyphase Examples 14
6. MATLAB Wavelet Toolbox 17
7. Orthogonal Filter Banks: Paraunitary Matrices; Orthogonality Condition (Condition O) in the
Time Domain, Modulation Domain and Polyphase Domain 17
8. Maxflat Filters: Daubechies and Meyer Formulas. Spectral Factorization 18
9. Multiresolution Analysis (MRA): Requirements for MRA; Nested Spaces and Complementary
Spaces; Scaling Functions and Wavelets: Handout 9 22
10. Refinement Equation: Interative and Recursive Solution Techniques; Infinite Product Formula;
Filter Bank Approach for Computing Scaling Functions and Wavelets: Handout 10 26
11. Project Brief 29
12. Orthogonal Wavelet Bases: Connection to Orthogonal Filters; Orthogonality in the Frequency
Domain. Biorthogonal Wavelet Bases 33
1http://www.mathworks.com/help/wavelet/ref/upcoef.html
2
13. Mallat Pyramid Algorithm; Handout 12 33
14. Accuracy of Wavelet Approximations (Condition A); Vanishing Moments; Polynomial Cancellation
in Filter Banks 35
15. Smoothness of Wavelet Bases: Convergence of the Cascade Algorithm (Condition E); Splines.
Bases vs. Frames 35
16. Signal and Image Processing: Finite Length Signals; Boundary Filters and Boundary Wavelets;
Wavelet Compression Algorithms 35
17. Guest Lecture. Physical Wavelets and their Sources: Real Physics in Complex Spacetime 36
18. Lifting: Ladder Structure for Filter Banks; Factorization of Polyphase Matrix into Lifting Steps;
Lifting Form of Refinement Equation; Handout 16 36
19. Wavelets and Subdivision: Nonuniform Grids; Multiresolution for Triangular Meshes;
Representation and Compression of Surfaces; Slides and Handouts 17,18 38
20. Numerical Solution of PDEs: Galerkin Approximation; Wavelet Integrals (Projection Coefficients,
Moments and Connection Coefficients); Convergence. Subdivision Wavelets for Integral
Equations. Compression and Convergence Estimates 39
21. M-band Wavelets: DFT Filter Banks and Cosine Modulated Filter Banks. Multiwavelets 39
Part 2. Extra fun 40
22. Fun with Quandl 40
Part 3. Colophon, Rationale, PitchDeck, and Help Me and Help Others 41
23. Why Wavelets (for me)? 41
24. The Problem (and the Solution) 42
References 46
Contents
Part 1. MIT OCW 18.327 Spring 2003
Lecture Notes
Here are my notes (and my personal perspectives) on the Slides and Handouts for the Lecture Notes in
[1].
READINGS refer to the Text: Strang and Nguyen. Wavelets and Filter Banks. Wellesley-Cambridge
Press, 1997.[2]
1. Discrete-time Filters: Convolution; Fourier Transform; Lowpass and Highpass Filters:
Handout 1
Readings: Sec 1.1-1.4, 2.1 [2]
Let x : R→ V
x(t) ∈ V
, where V vector space, or R-module.
Let y : R→ V
y(t) ∈ V
.
x represents input, as a function of time t, and y represents output, as a function of time.3
Let linear L : V → V , L ∈ L(V ;V ) be a linear transform (automorphism?) s.t. Lx = y.
Note for a “time delay” δ ∈ R,
Lx(t− δ) = y(t− δ)
Let x(t) = δ(t), with δ being the Dirac Delta function (in this context). Then
Lx(t) = Lδ(t) = h(t)
where h, as defined as such immediately above, is the “response.”
Then for x(t) =∑∞τ=−∞ x(τ)δ(t− τ)
Lx(t) =
∞∑τ=−∞
x(τ)h(t− τ)
Recall the L1 Fourier transform, such that for x ∈ L1(Rn), with n = 1 for time, in this case [3],
(1.1) x(ω) =
∫ ∞−∞
x(t)e−iωtdt =
∞∑n=−∞
x(n)e−iωn
and the “inverse”, (the inverse is well defined for L2; EY : 20150619, correct me if I’m wrong)
(1.2) x(t) =1
2π
∫ π
−πx(ω)eiωtdω
Convolution. define convolution of f, g on Rn
Definition 1.
(1.3) f ∗ g(x) =
∫Rnf(x− y)g(y)dy
Note f ∗ g = g ∗ f by change of variables.
Now
y(t) =
∫ ∞−∞
x(τ)h(t− τ)dτ =
∫ ∞−∞
dτ1
2π
∫ π
−πx(ω)eiωτdωh(t− τ)e−iωte−ωt =
1
2π
∫ π
−πx(ω)h(ω)eiωtdω
=⇒ y(ω) = x(ω)h(ω)
Discrete case: if f, g are discrete, i.e. f = f(i)
g = g(j)
, i, j ∈ Z, then
∫ ∞−∞
dyf(x− y)g(y) =
∞∑k=−∞
f(n− k)g(k) = (f ∗ g)(n)
Toeplitz Matrix. The so-called Toeplitz Matrix representation is thus (I believe): now
(1.4) y(t) =
∞∑τ=−∞
x(τ)h(t− τ) =
∞∑τ=−∞
h(t− τ)x(τ)
Let τ become matrix index j.
Let t become matrix index i.
Let htoep = (htoep)ij be the so-called Toeplitz matrix. Now from Eq. 1.4,
(htoep)ijxj = h(t− τ)xj .
So (htoep)ij = h(i− j) are the matrix entries.4
One should consider causality : causality means that t − τ ≥ 0 (otherwise, for t = 0, for say τ = 1, x(τ = 1)
causes a response for y(t = 0).
Examples.
(a) Low-pass Filter:
(1.5) y(t) =1
2x(t) +
1
2x(t− 1) =
∞∑τ=−∞
h(t− τ)x(τ)
with h(0) =1
2
h(1) =1
2
and 0 otherwise.
pp. 8 of Sec. 1.2 Lowpass Filter = Moving Average of Strang and Nguyen [2] defines the acronym
FIR, finite impulse response. Again, consider this low-pass filter example immediately above:
∞∑τ=−∞
h(t− τ)x(τ) =
∞∑n=−∞
h(n)x(t− n) = y(t) =1
2x(t) +
1
2x(t− 1)
for n = t− τ
τ = t− nSo h(0) = h(1) = 1
2 . Else, h(n) = 0.
So note that so-called “H” can be rewritten in many ways.
For H : L2(R)→ L2(R), defined as
∞∑τ=−∞
h(t− τ)x(τ) =
∞∑n=−∞
h(n)x(t− n) = y(t)
if h(n) 6= 0 only for finite number of n’s, then H = h(n) is a finite impulse response (FIR) filter.
Then
h(ω) =
∫ ∞−∞
h(t)e−iωtdt =1
2+
1
2e−iω
(imagine doing Fourier transforms of Diract delta functions; they “pick off” values, -Prof. Alan J.
Weinstein, Caltech Ph 12a)
(1.6) =⇒ h(ω) =1
2(1 + e−iω) = cos
(ω2
)e−iω/2
(b) High pass filter:
y(t) =1
2x(t)− 1
2x(t− 1)
h(t) =1
2δ(t)− 1
2δ(t− 1)
h(ω) =1
2− 1
2e−iω = i sin
(ω2
)e−i
ω2 = sin
(ω2
)e−i
(ω−π)2
Why they’re called low-pass and high-pass filters, I think, is because of this: consider |h(ω)|. Then for the
low-pass filter, |h(ω)| = cosω/2 so that low frequencies near 0 have a large magnitude of h relative to high
frequencies near ω = π. Vice-versa for the high-pass filter with |h| = sinω/2.5
Python examples. At this point, I would run example1.py and type into the Python prompt plt.show()
and look at Figure 1 for the Haar lowpass filter and Figure 2 for the Haar lowpass filter.
Notice the use of numpy’s fft in fft module. All we’re doing is 1.1.
If you’re wondering what the other “square” shaped graphs in the Handout 1 was (as I did), run Handout_examples.py
(doing python -i Handout_examples.py in the directory of tools) and do plt.show() and look at Figure 3
and Figure 5. It’s the phase or “angle” of the complex valued f as a function of ω.
Readings: Sec 1.1-1.4, 2.1 [2]
2. Sampling Rate Change Operations: Upsampling and Downsampling; Fractional Sampling;
Interpolation: Handout 2
READING: Sec 3.1-3.3 [2]
Downsampling by M .
(2.1) y(t) = x(Mt)
Apply the Fourier transform, Eq. 1.1, to both sides:
y(ω) =
∞∑n=−∞
x(Mn)e−iωm =∑
m=nN
x(m)e−iωmM
At this point, we need an identity, namely,
(2.2)1
M
M−1∑k=0
(e−i2πM m)k =
1 if m = nM
0 otherwise
For the first few cases:
M = 1,
(e−i2πm)0 = 1 m = n
M = 2
1
2
1∑k=0
(e−iπm)k =1
2(1 + (−1)m) =
1 if m = 2n
0 otherwise
M = 3
1
3
2∑k=0
(e−i2πm3 )k =
1
3(1 + e−i
2πm3 + e−i2πm
(2)3 ) =
1 if m = 3n
0 otherwise
However, I think the = 0 case is nontrivial. The m = nM case, leading to a sum of 1 is clear (multiples of 2π
in the argument yields 1 for e−i2πM m = e−i2πn = 1). But 0 otherwise?
I suspect that this should have something to do with, in complex analysis, when integrating around a circle,
if there are no poles, then the integral (or summation) is 0. I’d like to hear someone show this to me, (very)
explicitly.
Otherwise, inserting in Eq. 2.2, by the clever trick of multiplying by 1,
(2.3)
y(ω) =∑
m=nM
1
M
M−1∑k=0
(e−i2πM m)kx(m)e−iω
mM =
∑m=nM
1
M
M−1∑k=0
e−i2πkmM
1
2π
∫ π
−πdωx(ω′)eiω
′me−iωmM =
=1
M
M−1∑k=0
∫ π
−πdω′x(ω′)
1
2π
∑m=nM
eim(ω′− ωM−
2πkM ) =
1
M
M−1∑k=0
x(ω + 2πk
M)
6
Upscaling by L. Apply the Fourier transform, Eq. 1.1, to both sides:
y(t) = x
(t
L
)
(2.4)
y(ω) =
∞∑t=−∞
y(t)e−iωt =
∞∑m=−∞
x(m)e−iωmL =1
2π
∫ π
−πdω
∞∑m=−∞
x(ω′)eiω′me−iωmL =
=1
2π
∫ π
−πdωx(ω′)
∞∑m=−∞
exp (im(ω′ − ωL)) =
∫ π
−πdωx(ω′)δ(ω′ − ωL) = x(ωL)
EY : 20150620, in Handout 2, I find the asymmetry in the downsampling and upscaling scaling (M vs. L)
disconcerting (Eq. 2.3, Eq. 2.4). Why y(ω) = 1M x( ωM ) for downsampling, but y(ω) = x(ωL) for upscaling (no
constants in front)?
Interpolation. The interpolation filter is applying a low pass filter, after a downsampling by M (cf. Eq.
(2.1), Eq. (2.3)). I don’t know if there’s a general form of a low pass filter, so I’ll use the basic example in
Eq. 1.5, Eq. 1.6.
(1) Apply downsampling by M (↓M):
y(t) = x(Mt)
y(ω) =1
M
M−1∑k=0
x
(ω + 2πk
M
)
(2) Apply low pass filter
h(t) =1
2(x(t) + x(t− 1))
h(ω) = cos(ω
2
)e−iω/2 =
1
2x(ω)(1 + e−iω)
For the very last equation, think of the superposition property of the Fourier transform.
Then for interpolation filter
h ◦ y(t) =1
2(x(Mt) + x(M(t− 1))
h ◦ y(ω) =1
M
M−1∑k=0
1
2x
(ω + 2πk
M
)(1 + e−iω)
Fractional Sampling. I’ll employ this notation for downsampling by M (↓M) and upsampling by L (↑ L),
so that
(2.5)
(↓M) : (↓M)x(t) = x(Mt)
(↓M)x(ω) =1
M
M−1∑k=0
x
(ω + 2πk
M
)
(↑ L) : (↑ L)x(t) = x
(t
L
)(↑ L)x(ω) = x(ωL)
7
Consider
↓M◦ ↑ Lx(t) = x
(Mt
L
)↓M ◦ ↑ Lx(ω) =
1
M
M−1∑k=0
x
(ωL
M+
2πk
M
)=
1
M
M−1∑k=0
x
(L
M(ω +
2πk
L)
)I think this is “synthesis”, “decompressing”, “fractional synthesis.”
↑ L◦ ↓Mx(t) = x
(Mt
L
)↑ L ◦ ↑Mx(ω) =
1
M
M−1∑k=0
x
(L
M(ω + 2πk)
)I think this is “analysis”, “compressing”, or “fractional sampling.”
Look at how the immediately above two blocks of equations are different in the frequency ω domain. The
question to ask, in light of Theorem 3.6 of Strang and Nguyen (1996) [2] is when does ↓ M and ↑ L com-
mute?
Look at the phases in the frequency domain.
We know that x doesn’t change because of the periodicity in ω by 2π. Then, examine the phase of ↓M ◦ ↑ Land ↑ L ◦ ↓M , respectively:
L
M(ω +
2πk
L) =
L
M(ω +
2πk
L+ 2πl)
L
M(ω + 2πk) =
L
M(ω + 2πk + 2πm)
with k = 0 . . .M − 1 and l,m ∈ Z. We see that if M ,L are relatively prime, i.e. M/L, M divided by L doesn’t
yield an integer, the greatest common factor is 1. k < M , so k/L is an integer, L being a multiple of one of
k = 0 . . .M − 1.
At this point, run example1.py by typing in python -i example1.py (where -i is the “flag” to get to the
interactive prompt). To look at that the plots, type in the prompt plt.show().
Open up example1.py in your favorite text editor (I use emacs; you can use TextEdit on a Mac OSX) and
compare the plots with the comments (with a hashtag # in Python) and axis titles and you’ll see what’s going
on. One should note that in the frequency domain, we’re effectively changing the L or M factor multiplying ω
(cf. Eq. 2.5), and that translates to in Python indexing or “slicing.” (take a look at the code in example1.py
for Downsampling)
2.1. On the associated Reading for Handout 2, Sec.3.1-3.3 [2]. READING: Sec 3.1-3.3 [2]
cf. Sec. 3.3. “Sampling Operations in the z-Domain.
On pp. 97, Theorem 3.5, here’s some clarification with some algebra [2].
This is true
(↓M ◦ x)(ω) =1
M
M−1∑k=0
x
(ω + 2πk
M
)
=⇒ (↑M ◦ ↓M ◦ x)(ω) = (↓M ◦ x)(Mω) =1
M
M−1∑k=0
x(ω +2πk
M)
In the z-transform representation, one obtain ∀ k ∈ Z
x(ω +2πk
M) =
∫ ∞−∞
dt exp (−i(ω +2πk
M)t)x(t) =
∫ ∞−∞
(zei2πk/M )−tx(t) = x(zei2πk/M )
8
with z = eiω and for k ∈ Z.
3. Filter Banks: Time Domain (Haar example) and Frequency Domain; Conditions for Alias
Cancellation and no Distortion; Handout 3
READING: Sec 4.1 [2]
Consider e.g. M = 2
y(t) = x(2t)
htj = δ(2t− j) = δ(t− j2 )
(3.1)
(↓ 2)h0 = δ(t− k
2)
1√2
(δ(k − τ) + δ(k − τ − 1)) =1√2
(δ(2t− τ) + δ(2t− (τ + 1))
(↓ 2)h1 =1√2
(δ(2t− τ)− δ(2t− (τ + 1))
Now
((↓ 2)h0)tτxτ = ((↓ 2)h0)tτ
...
x(−2)
x(−1)
x(0)
x(1)
x(2)...
=
1√2
...
x(−2) + x(−3)
x(0) + x(−1)
x(2) + x(1)...
Then
((↓ 2)h0)tτ =1√2
. . .
1 1
1 1
1 1. . .
So also
((↓ 2)h1)tτ =1√2
. . .
−1 1
−1 1
−1 1. . .
(↑ L)
y(t) = x(tL
)= htjxj if j =
t
L, then htj = 1
else, 0
So
htj = δ(t
L− j) = δ(t− jL)
9
so for example, e.g. L = 2,
(↑ 2)tjxj = (↑ 2)
...
x(−2)
x(−1)
x(0)
x(1)
x(2)...
=
...
x(−1)
0
x(0)
0
x(1)...
Now consider
h0(↑ L) =1√2
(δ(t− k) + δ(t− k − 1))δ(k − jL) =1√2
(δ(t− jL) + δ(t− jL− 1))
=⇒ h0(↑ L) =1√2
(δ(t− jL) + δ(t− jL− 1))
e.g.
(h0(↑ 2))tjxj = (h0(↑ 2))tj
...
x(−1)
x(0)
x(1)...
=
1√2
...
x(−1)
x(−1)
x(0)
x(0)
x(1)...
Back to the reading of the 3rd Handout and Slides. I didn’t understand it the first time around reading it:
let’s gather up what we know so far.
The handout begins with the Haar Filter Bank slide. I’m assuming that the example presented is called the
Haar Filter Bank (does it have anything to do with the Haar wavelet? is my question).
2-channel FIR - FIR is Finite Impulse Response; 2 channel (I think it’s called 2-channel because of the low
pass and high pass filter for low and high frequencies, respectively).
Define a linear mapping H : L2(R)→ L2(R such that ∀x = x(t) ∈ L2(R),
Hx(t) = y(t) =
∞∑τ=−∞
h(t− τ)x(τ)
It takes a number of forms or representations:
Hx(t) =
∞∑τ=−∞
h(t− τ)x(τ) =
∞∑n=−∞
h(n)x(t− n) =
∞∑τ=−∞
htτxτ
where htτ is the so-called Toeplitz matrix,
with t− τ ≥ 0 for causality, or n ≥ 0.
Because of the convolution property, that y(ω) = x(ω)h(ω) for y = h ∗ x = x ∗ h, then
y = hx10
Let’s collect the facts that we know about downsampling (↓M) and upsampling (↑ L).
(↓M)x(t) = x(Mt) =
∞∑τ=−∞
δ(Mt− τ)x(τ) =
∞∑n=−∞
δ(n− (1−M)t)x(t− n)
(↑ L)x(t) = x
(t
L
)=
∞∑τ=−∞
δ
(t
L− τ)x(τ) =
∞∑n=−∞
δ(n− (1− 1
L)t)x(t− n) =
∞∑τ=−∞
δ(t− Lτ)x(τ)
Take the simple example for a low pass filter (EY: 20150630 are there other kinds of low pass filters of note
to consider?)
h0x(t) =1√2x(t) +
1√2x(t− 1) =
1√2
∞∑τ=−∞
(δ(t− τ) + δ(t− 1− τ))x(τ) =1√2
∞∑n=−∞
(δ(n) + δ(n− 1))x(t− n)
Take the simple example for a high pass filter
h1x(t) =1√2x(t)− 1√
2x(t− 1) =
1√2
∞∑τ=−∞
(δ(t− τ)− δ(t− 1− τ))x(τ) =1√2
∞∑n=−∞
(δ(n)− δ(n− 1))x(t− n)
So the so-called Haar Filter Bank is just
f0 ◦ (↑ 2) ◦ (↓ 2) ◦ h0 + f1 ◦ (↑ 2) ◦ (↓ 2) ◦ h1 : L2(R)→ L2(R)
with (↓ 2) ◦ h0(↓ 2) ◦ h1
being “analysis” and
f0 ◦ (↑ 2)
f1 ◦ (↑ 2)
being “synthesis”
Instead of doing operations from right to left, one can write the operations going from left to right by “filter
bank” diagram. I’ll write it as a commutative diagram instead. For M = 2
x(t) r0(t) y0(t) t0(t) v0(t)h0 (↓ 2) (↑ 2) f0
x(t) r1(t) y1(t) t1(t) v1(t)h0 (↓ 2) (↑ 2) f0
and then x′(t) = v0(t) + v1(t).
Now work it out: from Eq. 3.1,
(↓ 2)h0x(t) =1√2
(x(2t) + x(2t− 1))
(↓ 2)h1x(t) =1√2
(x(2t)− x(2t− 1))
(↑ 2)(↓ 2)h0x(t) =1√2
(x(2t))
(↑ 2)(↓ 2)h1x(t) =1√2
(x(2t))
f0(↑ 2)(↓ 2)h0x(t) =1
2(x(t) + x(t− 1)) = v0(t)
f1(↑ 2)(↓ 2)h1x(t) =1
2(x(t)− x(t− 1)) = v1(t)
with f0 = h0, f1 = h1. So then x′(t) = v0(t) + v1(t) = x(t). “Perfect reconstruction.”
For x(t) ∈ L2(R), suppose
x(t) =
∞∑τ=−∞
x(τ)δ(t− τ)
11
The z-transform is simply the Fourier transform of x(t):
(3.2) x(ω) =
∫ ∞−∞
dte−iωt∞∑
τ=−∞x(τ)δ(t− τ) =
∞∑τ=−∞
x(τ)e−iωτ =
∞∑τ=−∞
x(τ)z−τ = x(z)
for z = eiω.
Then the z-transform of downsampling ↓ M and upsampling ↑ L is the following (cf. Sec. 3.3 Sampling
Operations in the z-Domain [1]):
(3.3)
(↓M)x(z) =1
M
M−1∑k=0
x
(ω + 2πk
M
)=
1
M
M−1∑k=0
∞∑τ=−∞
x(τ)e−i(ω+2πkM )τ =
1
M
∞∑τ=−∞
x(τ)
M−1∑k=0
(ei2πkM z1/M )−τ =
=1
M
M−1∑k=0
x(ei2πkM z
1M )
(3.4) (↑ L)x(z) = x(Lω) =
∞∑τ=−∞
x(τ)e−iLωτ =
∞∑τ=−∞
x(τ)(zL)−τ = x(zL)
For the simple example of a low pass filter h0, recall that h0x(t) = 1√2(x(t) + x(t− 1)) and
x(t) =∑∞τ=−∞ x(τ)δ(t− τ)
Now
H0x(ω) :=
∫ ∞−∞
dte−iωth0x(t) =1√2
∫ ∞−∞
dte−iωt
( ∞∑τ=−∞
x(τ)δ(t− τ) + x(τ)δ(t− 1− τ)
)=
=1√2
∞∑τ=−∞
x(τ)e−iωτ + x(τ)e−iω(1+τ) =
=1√2x(z)
(1 +
1
z
):= H0x(z)
Then for for the simple example of a high pass filter, h1,
H1x(z) =1√2x(z)
(1− 1
z
)
DTFT - discrete-time Fourier Transform.
The perfect reconstruction condition is
x′(t) = x(t− δ)
where δ is the time delay.
∫ ∞−∞
dte−iωtx(t− δ) =
∫ ∞−∞
dte−iωt∞∑
τ=−∞x(τ)δ(t− δ − τ) =
∞∑τ=−∞
x(t− δ)e−iω(δ+τ) = z−δx(ω)
So
x′(z) = z−δx(z)
12
3.1. Aliasing, alias. Let’s consider this example. Suppose
constant x1(t) =∑n∈Z
δ(t− n)
x1(ω) =∑n∈Z
e−iωn
alternating x2(t) =∑n∈Z
(−1)nδ(t− n)
x2(ω) =∑n∈Z
(−1)ne−iωn =∑n∈Z
e−i(ω−π)n
Apply downsampling (↓ 2).
(↓ 2)x1(t) = x1(2t) =∑n∈Z
δ(2t− n) =∑j∈Z
δ(2(t− j))
(↓ 2)x2(t) = x2(2t) =∑n∈Z
(−1)nδ(2t− n) =∑j∈Z
(−1)2jδ(2(t− j)) =∑j∈Z
δ(2(t− j)) = (↓ 2)x1(t)
So x2(t) is the “alias” of x1(t) with respect to (↓ 2).
So, I want to try to generalize this definition of “aliasing”:
∀x1(t), x2(t) ∈ L2(R), (↓M) : L2(R)→ L2(R),
if (↓M)x1 = (↓M)x2 then x1, x2 are aliases with respect to ↓M
It was interesting to me to consider the frequency domain:
(↓ 2)x1(ω) =1
2
2−1∑k=0
∑n∈Z
e−i(ω+2πk
2 )n =1
2
1∑k=0
∑n∈Z
exp(−i(ω
2+ πk
)n)
=1
2
∑n∈Z
e−iωn2 (1 + e−iπn)
(↓ 2)x2(ω) =1
2
2−1∑k=0
∑n∈Z
e−i(ω+2πk
2 −π)n =1
2
1∑k=0
∑n∈Z
exp(−i(ω
2+ πk − π
)n)
=1
2
∑n∈Z
e−iωn2 (1 + eiπn)
Since(1 + e−iπn)
2
eiπn/2
eiπn/2=
cos(πn2
)eiπn/2
= (−1)m/(−1)m = 1
1 + eiπn
2
e−iπ2 n
e−iπn/2=
cos(πn2
)e−iπn/2
= (−1)m/(−1)m = 1
where n = 2m and m ∈ Z. So in the frequency domain, they’re equal too in this example.
3.2. wkeep. We’re going to try to mimic the behavior of Matlab’s wkeep 2
Let length of list by N0 ∈ Z
N0 ∈ Z so either N0 =
2K0 even
2K0 − 1 oddK0 ∈ Z
Suppose we want l ordered elements of the list l ∈ Z, so l =
2p even
2p− 1 oddp ∈ Z
Suppose N0 = 2K0 − 1
l = 2p− 1
(odd, odd case)
N0 = 2(K0 − 1) + 1
(K0 − 1 is what modulus arithmetic would give you, i.e. [N0/2] = K0 − 1, N0%2 = 1)
2MathWorks Documentation, wkeep Keep part of vector or matrix http://www.mathworks.com/help/wavelet/ref/wkeep.html
13
l = 2(p− 1) + 1
On the “left” side of the center, there are K0−1 “ordered elements and we want the last p−1 elements
K0 − 1− (p− 1) = K0 − p
So start counting the elements to choose from position K0 − p+ 1, counting from 1.
So if we start counting from 0, we want the K0 − p position. K0 − p = (K0 − 1)− (p− 1)
on the “right” side of center,
center’s position is at K0 − 1 + 1 = K0 (counting from 1). Counting from 0, K0 − 1 is the position.
K0 + (p− 1) is position of last element to pick off (counting from 1). K0 + (p− 1)− 1 is the position of the
last element to pick off (counting from 0).
if l = 2p,
let’s try to mimic what wkeep does in Matlab. Looking at the MathWorks Documentation, it seems that if we
pretend that we eliminate the last element of our list, so that the length is now N0−1 = 2K0−2 = 2(K0−1),
then we’d now want the center piece of length l.
start at (K0 − 1)− p+ 1 and last element’s position at K0 + p− 1. (counting from 1)
If N0 = 2K0
l = 2p
, start at K0 − p+ 1; last element’s position at K0 + p (counting from 1)
if l = 2p− 1 = 2(p− 1) + 1, let’s mimic what wkeep does in Matlab. Pretend we eliminate the last element of
our list, so that the length is now N0 − 1 = 2K0 − 1 = 2(K0 − 1) + 1
start at (K0 − 1)− (p− 1) + 1 = K0 − p + 1; last element’s position at (K0 − p) + (2p− 1) = K0 + p− 1
(counting from 1).
4. Filter Banks (contd.): Perfect Reconstruction; Halfband Filters and Possible
Factorizations
READING: Sec 4.1 [2]
5. Modulation and Polyphase Representations: Noble Identities; Block Toeplitz Matrices
and Block z-transforms; Polyphase Examples
READING: Sec 3.4, 4.1-4.4 [2]
5.1. On the Readings.
5.1.1. Noble Identities. cf. Sec. 3.4 “Filters Interchanged with Samplers” [2]
This is all about the so-called Noble Identities.
For clarification, I will work out many steps explicitly, and repeat many previous derivations.
Given x(t) ∈ L2(R), recall these various representations:
(5.1) x(t)F7→ F [x(t)](ω) ≡ x(ω) =
∫ ∞−∞
dtx(t)e−iωt =
∫ ∞−∞
dtx(t)z−t ≡ x(z)
14
Now note how the so-called z-transform is related to the usual Fourier transform. These objects are formally
equivalent and is only a substitution via eiω = z. However, the functions live on L2(C) (see Subsection
2.1).
Thusx
(ω + 2πk
M
)= x
(ei2πkM z1/M
)since exp
(−i(ω + 2πk
M
)t
)=(ei2πkM z1/M
)−t
and x(ωL) = x(zL)
since exp (−iωLt) = (zL)−t
.
Also, note that the algebra on L2(R) in the so-called time domain is different from the algebra in the frequency
domain. Explicitly, recall the convolution and its Fourier transform:
f ∗ g(t) =
∫ ∞−∞
dτf(t− τ)g(τ)
F [f ∗ g(t)](ω) =
∫ ∞−∞
dte−iωt∫ ∞−∞
dτf(t− τ)g(τ)u=t−τ
=
∫ ∞−∞
dτg(τ)
∫ ∞−∞
due−iω(u+τ)f(u) = f(ω) · g(ω)
For L2(R) (which I’ll refer to being in the time domain as convention), the multiplication operation for its
algebra is the convolution ∗, but in the frequency domain (which I’ll denote as F [L2(R)], multiplication · is
by pointwise multiplication! Thus,
L2(R) F [L2(R)]
f ∗ g(t) (f · g)(ω) = f(ω) · g(ω)
F
F
Thus
(5.2)
x(t)↓M7→ (↓M ◦ x)(t) = x(Mt)
x(ω)↓M7→ (↓M ◦ x)(ω) =
1
M
M−1∑k=0
x
(ω + 2πk
M
)=
1
M
M−1∑k=0
x(ei2πkM z1/M )
and
x(t)↑L7→ (↑ L ◦ x)(t) = x
(t
L
)t∈Z7→
x(m) if t = Lm, m ∈ Z
0 otherwise
x(ω)↑L7→ (↑ L ◦ x)(ω) = x(ωL) = x(zL)
witht∈Z7→ denoting the discretization of t ∈ Z.
Notice that the operation of ↓M◦, ↑ L◦, the ◦ composition, is distinguished from the multiplication operator
in the respective algebras. On L2(R), ◦ is distinguished from convolution ∗. On F [L2(R)], ◦ is distinguished
from pointwise multiplication ·.
It’s unclear to me if ◦ can be made to “fit into” the respective algebras with multiplication operators ∗ or ·.Please tell me otherwise. Otherwise, operations involving ↓M , ↑ L are distinguished as such.
15
Theorem 1 (Noble identities). The Noble identities are the following:
↓M ◦ (G(zM )x(z))(z) = G · ( ↓M ◦ x)(z)(5.3)
↑ L ◦ (G · x)(z) = G(zL) · (↑ L ◦ x)(z)(5.4)
while noting that its form in L2(R) is, respectively,
(G∗ ↓M ◦ x)(t) =
∫ ∞−∞
dτG(t− τ)x(Mτ)τ∈Z7→
∑m∈Z
G(t−m)x(Mm)(5.5)
(↑ L) ◦G ∗ x(t) =
∫ ∞−∞
dτG
(t
L− τ)x(τ)
t∈Z7→
∑τ∈ZG(m− τ)x(τ) if t = Lm, m ∈ Z
0 otherwise(5.6)
noting that (z) notation simply denotes that these are functions of z ∈ C, and thatt∈Z7→ denotes discretization
of t.
Proof. Distinguishing between operations and understanding that algebra over a field is just algebra makes
this easy.
Let G(zM )x(z) := t(z). Then
↓M ◦G(zM )x(z) = ↓M ◦ t(z) =1
M
M−1∑k=0
t(ei2πkM z1/M ) =
1
M
M−1∑k=0
G(ei2πkz)x(ei2πkM z1/M ) = G(z)
1
M
M−1∑k=0
x(ei2πkM z1/M ) =
= G(z) · (↓M ◦ x)(z) = G · ( ↓M ◦ x)(z)
For all of this in L2(R) “time” domain,
(G ∗ (↓M ◦ x))(t) =
∫ ∞−∞
dτg(t− τ)(↓M ◦ x)(τ) =
∫ ∞−∞
dτg(t− τ)x(Mτ)τ∈Z7→
∑τ∈Z
g(t− τ)x(Mτ)
Likewise,
↑ L ◦ (G · x)(z) = ↑ L ◦G(z) · x(z) = G(zL)x(zK) = G(zL) · (↑ L ◦ x)(z)
and
(↑ L) ◦G ∗ x(t) =↑ L ◦∫ ∞−∞
dτG(t− τ)x(τ) =
∫ ∞−∞
dτG
(t
L− τ)x(τ)
t∈Z7→
∑τ∈ZG(m− τ)x(τ) if t = Lm, m ∈ Z
0 otherwise
�
5.2. On Perfect Reconstruction. Consider this filter bank:
r0(z) 1M
∑M−1k=0 r0
(ei
2πkM z1/M
)t0(z) F0 · t0(z)
x(z) (F0 · t0 + F1 · t1)(z) = x′(z)
r1(z) 1M
∑M−1k=0 r1
(ei
2πkM z1/M
)t1(z) F1 · t1(z)
H0
H1
↓M ↑M F0
⊕
↓M ↑M F1
⊕
withr0(z) = H0 · x(z)
t0(z) :=1
M
M−1∑k=0
r0(ei2πkM z)
F0 · t0(z) = F0(z) · t0(z) = F0(z) · 1
M
M−1∑k=0
H0(ei2πkM z) · x(ei
2πkM z)
16
and likewise for r1, t1, F1.
Theorem 2. The so-called perfect reconstruction condition is
1
M
M−1∑m=0
Fm(z)Hm(z) = z−δ(5.7)
1
M
M−1∑m=0
Fm(z)Hm(ei2πkM z) = 0 ∀ k = 1 . . .M − 1(5.8)
Note that for M = 2, this becomes
1
2(F0(z)H0(z) + F1(z)H1(z)) = z−δ
1
2(F0(z)H0(−z) + F1(z)H1(−z)) = 0
Proof.
M−1∑m=0
Fm(z) · 1
M
M−1∑k=0
Hm(ei2πkM z) · x(ei
2πkM z) =
1
M
M−1∑k=0
M−1∑m=0
Fm(z)Hm(ei2πkM z) · x(ei
2πkM z) = z−δx(z)
Then match different x(ei2πk/Mz) on both sides. �
6. MATLAB Wavelet Toolbox
SOFTWARE: PyWavelets 3
7. Orthogonal Filter Banks: Paraunitary Matrices; Orthogonality Condition (Condition
O) in the Time Domain, Modulation Domain and Polyphase Domain
READING: Sec 5.1-5.2 [2]
7.1. Reading.
7.1.1. Notes on Sec. 5.2. Orthonormal Filter Banks. Consider filter coefficients c(k), d(k), with
c(k) representing the low pass filter
d(k) representing the low pass filter
For input x = x(t) ∈ L2(R),
recall
C : x(t) 7→∫ ∞−∞
dτc(t− τ)x(τ) =∑τ
c(t− τ)x(τ)k=t−τ
=∑k
c(k)x(t− k) t, k ∈ Z
D : x(t) 7→∫ ∞−∞
dτd(t− τ)x(τ) =∑τ
d(t− τ)x(τ)k=t−τ
=∑k
d(k)x(t− k) t, k ∈ Z
with t−k ≥ 0 or equivalently t−τ ≥ 0 for a nonzero c or d coefficient being the (time) causality condition.
The commutative diagram (equivalent to filter bank diagram) is
3PyWavelets, Python.org, https://pypi.python.org/pypi/PyWavelets/
17
Cx(t) Cx(2t) = y0(t) t0(t) Ct0(t)
x(t) Ct0(t) + Dt1(t)
Dx(t) Dx(2t) = y1(t) t1(t) Dt1(t)
C
D
↓ 2 ↑ 2 C
⊕
↓ 2 ↑ 2 D
⊕
where
t0(t) =
y0(t/2) if t even
0 otherwise
t1(t) =
y1(t/2) if t even
0 otherwise
Claim: The claim by Strang and Nguyen[2] is that in order to achieve x(t) = Ct0(t) + Ct1(t), then
C, D are synthesis filters that are time reversals of the analysis filters:
(7.1)C = CT and c(n) = c(−n)
D = DT and d(n) = d(−n)
Indeed this is true since
∑τ
c(t− τ)t0(τ) =∑τ
ctτ t0(τ)ctτ=(c)Ttτ=
∑τ
cτtt0(τ) =∑τ
c(τ − t)t0(τ)
k=t−τ∑kc(−k)t0(t− k)
8. Maxflat Filters: Daubechies and Meyer Formulas. Spectral Factorization
SOFTWARE: daub.py
READING: Sec 5.3-5.5 [2]
Notes on the Handout:
Then Halfband condition is
P (ω) + P (ω + π) = 2
for P (ω) ∈ R[ω], the set of polynomials of ω. Look at the first slide of Handout 8. Notice how ∀ω ∈ [0, 1], the
two curves representing P (ω) + P (ω + π) add up to 2.
The slide says we want P (ω) to be lowpass, which I think means that P (ω) should “let low frequencies pass”
i.e. P (ω) is bigger when frequency ω is low and smaller (in magnitude) when ω is “high” or big.
The slide also says we want p(n) to be symmetric: I think that means that P (ω) should be symmetric about
ω = 0 so that P (−ω) = P (ω) is a symmetry (Z/2, mod-2, symmetry?).
EY : 20150701 I noticed that
P (ω) =
∫ ∞−∞
dtp(t)e−iωt
P (ω) + P (ω + π) =
∫ ∞−∞
dtp(t)(e−iωt + e−i(ω+π)t)eiπ2 t
eiπt/2=
∫ ∞−∞
p(t)e−iωtcos(π2 t)
eiπt/2
=⇒ F(P (ω) + P (ω + π)) = p(t) cos
(πt
2
)exp (−i(ω +
π
2)t) = 2δ(t)
18
Daubechies’ Approach. Let
(8.1) P (ω) = 2(1− y)pp−1∑k=0
(p+ k + 1
k
)yk
with
y =1− cosω
2=
(1− eiω
2
)(1− e−iω
2
)where y symmetric about ω = 0
Note that 1− y = 1+cosω2 .
Substituting z = eiω, so that y =(1−z2
) ( 1−1/z2
)= 1
4 (1 − 1z − z + 1) = 1
4 (2 − z − 1z ), and so (1 − y) =
14 (2 + z + 1
z ).
Note some properties of P (ω):
• P (y = 1) = 0 for y = 1. Note that y = 1 when ω = π2 .
• P (ω) of degree 2p− 1 (highest order term is 2yp(
2pp−1)yp−1 = 2y2p−1
(2pp−1))
• Let
(8.2) Bp(y) := 1 + py +p(p+ 1)
2y2 + · · ·+
(2p− 2
p− 1
)yp−1 =
p−1∑k=0
(p+ k − 1
k
)yk
which is Eq. (5.64) Sec. 5.5 in Strang and Nguyen (1996) [2]. There’s a typo in Slide 5 of Handout 8
[1].
Now note the binomial expansion formula 4
1
(1− y)p=
∞∑k=0
(p+ k − 1
k
)yk =
p−1∑k=0
(p+ k − 1
k
)yk +O(yp)
Let P (y) = 2(1− y)pBp(y) (EY: 20150701 P (y) = P (ω), they’re formally equivalent).
Then
P (y) = 2(1− y)p(
1
(1− y)p+O(yp)
)= 2 +O(yp)
Then clearly,
P (y = 0) = 2(1− 0)pBp(0) = 2
P (y = 1) = 0
and the derivatives of order l = 1 . . . p− 1 of P is 0
P (l)(y = 0) ≡ dl
dylP (y = 0) = 0
since, the idea is that
P (y) = 2(1− y)p(
1
(1− y)p+O(yp)
)= 2 + 2(1− y)pO(yp) = 2 +O(yp)
and because of the (1− y)p factor,
P (l)(y = 1) ≡ dl
dylP (y = 1) = 0
By the Z/2-symmetry, i.e. P (ω) = P (−ω), then
=⇒ P (z) = P
(1
z
)So if z0 is a root of P (z), i.e. P (z0) = 0, then P
(1z0
)= 0.
4Binomial theorem, wikipedia https://en.wikipedia.org/wiki/Binomial_theorem
19
Cepstral Method. Factor out zeros which lie on unit circle:
P (z) =
((1 +
1
z
)(1 + z)
)pQ(z)
then factor Q(z) into Q(z) = R(z)R( 1z ) s.t.
R(z) has all its zeros inside the unit circle, i.e. R(z0) = 0 for z0 s.t. |z0| < 1. R(z) causal
Now
lnQ(z) = lnR(z) + lnR(z−1)
Apply inverse z-transform (i.e. inverse Fourier transform F−1)
(8.3) F−1[lnQ(z)](t) = F−1[lnR(z)](t) + F−1[lnR(z−1)](t) ≡ q(t) = r(t) + r(−t)
where r(t) ≡ F−1[lnR(z)](t) is Strang and Amaratunga’s notation; my notation isq(t) = F−1[lnQ(z)](t)
r(t) = F−1[lnR(z)](t).
Note that r(−t) = F−1[lnR(z−1)](t) because remember how inverse Fourier transforms are defined: 12π
∫ π−π dωr(ω)z−t =
12π
∫ π−π dωr(ω)e−iωt = r(−t).
So r(t) = 0 for t < 0 so that r(t) causal. (Strang and Amaratunga’s notation).
With r(t) causal, then clearly
q(t) =
0 if t < 0
2r(0) if t = 0
r(t) if t > 0
(using my notation)
F−1[lnQ(z)](t) ≡ q(t) is the “complex cepstrum” of q(t). (Strang and Amaratunga’s notation)
Explanation of causal. Consider this very simple example (because I also forgot a lot of complex analysis):
Let
x(z) = exp
(1
1− zkz
)=
∞∑j=0
(1
1− zkz
)jj!
x(z = zk) = 0
(so z = zk is a pole of infinite order)
Then for y(z) = lnx(z) = 11− zkz
,
|y(zk)| =∞ is a singularity at z = zk.
Now y(ω) = 11−zke−iω , so that since |zk| < 1,
=⇒ y(ω) =
∞∑n=0
(zk)ne−iωn
Taking the inverse Fourier transform,
F−1[y(ω)](t) =1
2π
∫ π
−πdωeiωt
∞∑n=0
(zk)ne−iωn =
∞∑n=0
δ(t− n)(zk)n = ztk
So y(t) = F−1[y(ω)](t) is causal (I think this causal condition is that |y(t)| = |zk|t < 1, that the singularities
lie inside the unit circle.
So y(t) = 0 for t < 0
So analogously, r(t) = 0 for t < 0, so that r(t) causal. (Strang and Amaratunga’s notation)20
Algorithm. The problem is this: for
P (ω) = 2(1− y)pp−1∑k=0
(p+ k + 1
k
)yk = 2
(2 + z + 1
z
4
)p p−1∑k=0
(p+ k + 1
k
)(2− z − 1
z
4
)k= ((1 + z)(1 + 1/z))pQ(z)
Given P (ω) (and thus Q(z)),
we want to find H0(z) (i.e. the spectral factorization problem, cf. Slide 8 of Handout 8 [1]) of
P (z) = H0(z)H0(z−1)
The product filter for the orthogonal case, P (z), is the polynomial P (ω) (they’re the same thing).
With Q(z) = R(z)R(1z
), then
R(z) is a piece of H0(z)
R
(1
z
)is a piece of H0(z−1)
So we want to get H0(z) from R(z).
Notice that Q(z) is a piece of the binomial expansion of (1/(1 − y)p) and so that can be easily obtained by
formula.
EY : 20150701 At this point you should open up daub.py and see the algorithm implemented in Python
alongside, in parallel, with the following explanation.
We’re given Q(z), which can be calculated from the formula for Bp(z) (Eq. 8.2).
So first do
Q(z) Q(k) lnQ(k) q(t) = F−1[lnQ(k)](t)z = eiω = ei
2πkM ln F−1
where z = eiω = ei2πkM ∀ k = 0, 1, . . .M − 1, and M is chosen to be sufficiently large, and
q(t) =1
M
M−1∑k=0
lnQ(k)ei2πkM t Inverse Discrete Fourier transform
Then, find r(t) using the fact (causality) that
r(t) =
0 if t < 0
12q(0) if t = 0
q(t) if t > 0
Next,
r(t) exp ◦F [F−1 lnR(k)(t)] = R(k) H0(k) = R(k)(1− e−i 2πkM )p h0(t)exp ◦F ×(1 + e−i
2πkM )p F−1
where exp ◦F [F−1 lnR(k)(t)] = exp (lnR(k)) = R(k),
with the discrete Fourier transform of r(t) given by
F [r(t)](k) =
∞∑t=0
r(t)e−i(2πkM )t k = 0, 1, . . .M − 1
(EY: 20150701 I completely disagree with step v of Slide 17 Handout 8 [1])
and the multiplication ×(1 + e−i2πkM )p = ×(1 + z−1)p gets us H0(z).
Finally the inverse discrete Fourier transform F−1 is given by
h0(t) =1
M
M−1∑k=0
H0(k)ei2πkM t
21
At this point, one should open up and run daub.py and play with the daub function. I think Nh, the argument
for the function daub is 2p, so try only even values. Remember, daub outputs h0(t).
9. Multiresolution Analysis (MRA): Requirements for MRA; Nested Spaces and
Complementary Spaces; Scaling Functions and Wavelets: Handout 9
READING: Sec 1.5, 6.1 [2]
Looking at Handout 10, instead, I will following closely Wojtaszczyk (1997)[4]. Begin with the definition of a
wavelet :
Definition 2. A wavelet is a function ψ(t) ∈ L2(R s.t. if, for
ψjk := 2j/2ψ(2jx− k) j, k ∈ Z,
{ψjk}j,k∈Z is an orthonormal basis in L2(R), then ψ(t) is a wavelet.
(cf. Def. 2.1. of Wojtaszczyk (1997) [4])
9.1. Multiresolution Analysis. Consider a multiresolution analysis starting here, with these 2 defini-
tions.
Definition 3. ∀ k ∈ R, define translation operator Tk : L(R,R)→ L(R,R) where L(R,R) is the set of all
functions f defined on R (EY: 20150701 Would this set of functions necessarily be qualified to be all linear
functions (which is implied by the notation L)? That’s my question.) s.t.
Tk(f)(x) = f(x− k)
(cf. Def. 2.3. of Wojtaszczyk (1997) [4])
In frequency or wavenumber space, so that for f(t− k) ∈ L2(R),
F [f(t− k)](ω) =
∫ ∞−∞
dte−iωtf(t− k)u=t−k
=
∫ ∞−∞
due−iω(u+k)f(u) = e−iωkf(ω)
so that
Tkf(ω) = e−iωkf(ω)
Definition 4. ∀ j ∈ Z, define dyadic dilation operator Jj : L(R,R)→ L(R,R) s.t.
Jj(f)(x) = f(2jx)
(cf. Def. 2.4. of Wojtaszczyk (1997) [4])
In frequency or wavenumber space, so that for f(jt) ∈ L2(R),
F [f(jt)](ω) =
∫ ∞−∞
dte−iωtf(jt) =u=jt=
∫ ∞−∞
du
je−iω
uj f(u) = f(
ω
j)
so
Jj f(ω) = f(ω
j)
Definition 5. The multiresolution analysis is a sequence (V j)j∈Z of subspaces of L2(R, V j ⊆ L2(R),
∀ j ∈ Z s.t.
(1)
· · · ⊂ V −1 ⊂ V 0 ⊂ V 1 ⊂ . . .
(2)
span{⋃j∈Z
V j} = L2(R)
22
(3) ⋂j∈Z
V j = {0}
(4)
f(x) ∈ V j iff f(2−jx) ∈ V 0 i.e. equivalently V j = Jj(V 0) ∀ j ∈ Z
(5)
f ∈ V 0 iff f(x−m) ∈ V 0 ∀m ∈ Z i.e., equivalently V 0 = Tn(V 0) ∀n ∈ Z
(6) ∃φ ∈ V 0 called scaling function s.t.
{φ(t−m)}m∈Z is an orthonormal basis in V 0
i.e. equivalently, ∀ j ∈ Z, {2j/2φ(2jx− k)}k∈Z is an orthonormal basis in V j
(cf. Def. 2.2. of Wojtaszczyk (1997) [4])
EY : 20150701 Note on notation of t, ω’s and x, k’s and indices n, k,m.n; I really don’t know what’s the
best notation to remain consistent. I’ll trust that from the context, the meaning is clear. In the spirit of
open-source software (OSS), feel free to edit, copy, delete and make your own versions (and remember to share
it!). I’m interested in wavelet collocation on the physical spatial space x ∈ R3, so that’s why I’ll be writing in
terms of position x and wavenumber k.
In this easy lemma, that translation and dyadic dilation Tk, 2j/2Jj is an isometry is a good property to have,
ensuring we stay on L2(R).
Lemma 1. (1) ∀ k ∈ R, Tk is an isometry on L2(R)
(2) ∀ j ∈ Z, 2j/2Jj is an isometry on L2(R)
Proof. Main idea: change of variable
Recall ‖f‖22 =∫∞−∞ dx|f(x)|2.
‖Tkf‖22 =
∫ ∞−∞
dx|f(x− k)|2 =
∫ ∞−∞
dx|f(x)|2 = ‖f‖22
‖2j/2Jjf‖22 =
∫ ∞−∞
dx|2j/2f(2j(x))|2 =
∫ ∞−∞
dx|f(x)|2
�
9.2. W j. We’re going to follow Sec. 2.4 of Wojtaszczyk (1997) [4].
Let subspace W j ⊆ L2(R) s.t. V j ⊕W j = V j+1.
Now Jj(V 1) = V j+1, so then
V j+1 = Jj(V 1) = Jj(V 0 ⊕W 0) = Jj(V 0)⊕ Jj(W 0) = V j ⊕ Jj(W 0).
Then W j = Jj(W 0).
Since we’re given a multiresolution analysis (V j)j∈Z, by def., (EY: 20150701 I’m not sure about this following
step; if you can do this explicitly, let me know)
L2(R) = span(⋃j∈Z
V j) = span(⋃j∈Z
J−j+1(V 0)⊕W j) = ⊕∑j∈Z
W j
=⇒ L2(R) = ⊕∑j∈Z
W j
i.e. W j ’s give an orthogonal decomposition of L2(R). So if we get the orthonormal basis of each of the W j ’s,
then we get an orthogonal decomposition of L2(R).23
In application of this orthogonal decomposition,
L2(R) = V 0 ⊕W 0︸ ︷︷ ︸V 1
⊕W 1 ⊕W 2 ⊕ . . .
where V 0 is a coarse approximation and W j is the jth level of detail.
From Theorem 2.20 and Eq. (2.47) of Wojtaszczyk (1997) [4], we get this fact:
Theorem 3. Given a multiresolution analysis,
(9.1) ψ(x) =∑n∈Z
an(−1)nφ(2x+ n+ 1)
where ψ ∈W 0 = V 1 V 0 is a wavelet and
an =
∫ ∞−∞
φ(x
2)φ(x− n)dx
9.3. On the Question of the construction of the scaling function φ. Start with
(9.2) V 0 ⊂ V 1
which is the so-called scaling equation or refinement equation aka two-scale difference equation or dilation
equation.
Using the facts from the definition of a multiresolution analysis, Def. 5, and in particular fact 6,
for scaling function φ = φ(x) ∈ V 0,
since V 0 ⊂ V 1 and {21/2φ(2x−n)}n∈Z is an orthonormal basis in V 1, then φ ∈ V 0 ⊂ V 1 is of the form
φ(x) =∑n∈Z
an21/2φ(2x− n)
Take the Fourier transform of φ(x):
φ(k) =
∫ ∞−∞
dxφ(x)e−ikx =
∫ ∞−∞
dx∑n∈Z
an21/2φ(2x− n)e−ikx =∑n∈Z
21/2an
∫−∞
duφ(u)e−ik(u+n
2 ) =
=∑n∈Z
21/2ane−ik n2 φ
(k
2
)= mφ
(k
2
)φ
(k
2
)
using the u-substitution
u = 2x− n
du = 2dx
x =u+ n
2
and where m(k2
)≡ mφ
(k2
):=∑n∈Z 21/2ane
−i k2n.
Notice that φ(k) = mφ(k/2)φ(k/2). We have a recursion relation. By induction, for N ∈ Z (with the feeling
that N is large), then
φ(k) =
N∏j=1
mφ
(k
2j
)φ
(k
2N
)So given a sequence of numbers (ak)k∈Z (which gives us mφ), ak ∈ R, one can obtain
θ(k) :=
∞∏j=1
m
(k
2j
)and construct φ(k).
Indeed, for the construction of an appropriate multiresolution analysis,
it’s sufficient to construct a multiresolution analysis with a compactly supported scaling function
φ (and then we’ll get wavelets), from Theorem 4.1 of Wojtaszczyk (1997)[4], which I’ll recap here:24
Theorem 4. Suppose
m(k) =
S∑n=T
ane−ikn
where m(k), a trigonometric polynomial, satisfies the following properties:
|m(k)|2 + |m(k + π)|2 = 1 ∀ k ∈ R
m(0) = 1
m(k) 6= 0 ∀ k ∈[−π2,π
2
]
Then the infinite product
θ(k) =
∞∏j=1
m
(k
2j
)converges almost uniformly.
Thus, θ(k) continuous.
Moreover, θ(k) ∈ L2(R).
Function φ, given by φ = 1√2πθ, has compact support on [T, S] and is a scaling function of a multiresolution
analysis.
Formula
ψ(x) = 2
S∑n=T
an(−1)nφ(2x+ n+ 1)
gives a compactly supported wavelet with compact support suppψ ⊂[T−S−1
2 , S−T−12
].
cf. Theorem 4.1, Wojtaszczyk (1997)[4] Notice that this theorem gives us desirable properties and a wavelet.
EY : 20150701 the properties for m(k) look very familiar to what was discussed about the Daubechies filter
coefficients: are they related?
EY : 20150701 Note, I’m not trying to cuff the question of the construction of a compactly supported scaling
function here by referencing and copying directly from Wojtaszczyk (1997)[4]. Wojtaszczyk has the answer
given by Theorem 4.1 and it is what it is. We’re not trying to reinvent the wheel here in these notes or i.e.
suffer from “Not Invented Here”-syndrome.
Otherwise, if you think that including the full proof here is valuable, let me know; I’ll type it up though it’s
essentially a copy of the proof in Wojtaszczyk (1997)[4] with my notation, or I’d invite you to copy, paste,
and edit the LaTeX and add it yourself (and remember to share!).
Ingrid Daubachies original 10 lectures on wavelets is also a great resource I’m trying to look into; she neatly
summarizes in a theorem the take away facts from this entire section in Theorem 6.3.6. [5]:
Theorem 5. Suppose m0 is a trigonometric polynomial s.t.
|m0(k)|2 + |m0(k + π)|2 = 1
m0(0) = 1
Define φ, ψ by
φ(k) =1
(2π)1/2
∞∏j=1
m0
(k
2j
)
ψ(k) = e−ik2 m0
(k
2+ π
)φ
(k
2
)25
Then φ, ψ compactly supported L2-functions s.t.
φ(x) =√
2∑n
hnφ(2x− n)
ψ(x) =√
2∑n
(−1)nh−n+1φ(2x− n)
where hn determined by m0, via
m0(k) =1√2
∑n
hne−ink
ψjk(x) =1
2j/2ψ(
x
2j− k), j, k ∈ Z
ψjk(x) tight frame for L2(R) with frame constant 1.
This tight frame is an orthonormal basis iff for m0, ∃ compact set K, congruent to [−π, π] modulo 2π,
containing a neighborhood of 0, so that
infn>0
infk∈K|m0(2−nk)| > 0
I think Daubachies’ convention is for j to actually be −j in our understanding of V j ⊂ V j+1 cf. Chapter
6, Sec. 6.1 of Strang and Nguyen (1996) [2]. So the (important) orthonormal bases for the W j subspaces
is
(9.3) ψjk(x) = 2j/2ψ(2jx− k) j, k ∈ Z
10. Refinement Equation: Interative and Recursive Solution Techniques; Infinite Product
Formula; Filter Bank Approach for Computing Scaling Functions and Wavelets:
Handout 10
SOFTWARE: cython for pyWavelets package, pyWavelets package, 5
READING: Sec 6.2-6.4 [2]
Yes, the previous section was mostly theory, but it will payoff dividends here.
Slides 5-8 of Handout 10 [1] are essentially an application of the multiresolution analysis definition, Def. 5,
but on a discrete grid x ∈ Z.
For φ0 ∈ V 0, with orthogonal basis {φ0(x− n)}n∈Z for V 0, then
φ0(x) =∑n∈Z
a0nφ0(x− n)
It’s unclear to me from Handout 10 [1] which quantities do we know and don’t know, but I think we want to
find a0n given the values of scaling function φ at various points x, and vice versa.
Discretize x ∈ Z: x ∈ Z→ x ∈ Z. So
φ0(x) =∑n∈Z
a0nφ0(x− n)
u=x−n=
∑u∈Z
a0x−uφ0(u)
Solve for this matrix equation now.
If x is treated as time t, then (time) causality condition is clear: for
φ(t) =∑u∈Z a
0t−uφ(u)
then the causality condition is a0t−u = 0 if t− u < 0.
EY : 20150701 My question is what happens when x is treated as position in physical space? What is the
physical analogue to time causality? Locality?
5Discrete Wavelet Transform in Python http://www.pybytes.com/pywavelets/
26
Now for Jj(V 0) = V j , i.e. orthonormal basis for V j is {2j/2φ(2jx− n)}, then forφj(x) ∈ V j
φj(x) =∑n∈Z
ajn2j/2φ0(2jx− n)
Suppose j = 1. Suppose x = 2m+12 , x ∈ Z/2 (half integers).
φ1(x =2m+ 1
2) = 21/2
∑n∈Z
a1nφ0(2m+ 1− n)
u=2m+1−n= 21/2
∑u∈Z
a12m+1−uφ0(u)
Treating x as time t, the causality condition is 2m+12 ≥ u or m+ 1
2 ≥ u.
Scaling Relation and Wavelet Equation in Frequency Domain. cf. Slides 9-17. [1]
I think this is an application of Theorem 6.3.6. from Ingrid Daubachies [5]. It’d be nice to see that proof from
[5] in comparison to the presentation in Slides 9-17 [1]
Software. I installed cython and PyWavelets packages. For the Mac OSX, at the command prompt of
Terminal on an administrator account, I installed both with pip (it’s an improvement on easy_install 6. By
the way, I find that pip makes installation very easy and robust (easier than easy_install, heh), so I highly
recommend that if you don’t have pip, install pip.
The website for PyWavelets http://www.pybytes.com/pywavelets/#install says that PyWavelets is writ-
ten in Python, Cython, and C and features over 70 built-in wavelet filters, and that results are compatible
with Matlab’s Wavelet Toolbox. And PyWavelets is open source (MIT licenese)! So it looks very promising.
We’re going to use it as much as possible.
Documentation for PyWavelets is at the bottom, under Contents, and one should look at the API Refer-
ence.
Example 6: Generation of orthogonal scaling functions and wavelets. Open up example6.py. Notice the
use of daub.py again. So daub function gives us the low pass filter coefficients (in the so-called filter bank
form).
Opening up example6.py, you’ll see that the use of function upcoef for the inverse DWT (Discrete Wavelet
Transform). It would be a good time to learn about the function upcoef from the documentation at pybytes7. In the MIT OCW 18.327 material, upcoef wasn’t discussed or documented on its usage and in this notes, I
want to improve on that by explicitly discussing the use of functions in Python (same goes with Matlab/GNU
Octave) in application in relation to what’s learned in the notes here and Handout.
Notice the use of upcoef in example6.m and the requirement to input in a synthesis lowpass filter and
synthesis highpass filter, f0, f1, respectively, in Matlab. The wavelet can be specified in terms of orthogonal
filter banks. I think the explanation can be found in Sec. 6.2. of Nguyen and Strang [2] or look at my notes
on the reading that goes along this Handout 10, Subsection 10.1.
In PyWavelet, for a custom wavelet, one must specify the entire filter bank with a lowpass decomposition, high-
pass decomposition, lowpass reconstruction and highpass reconstruction (i.e. synthesis). Given the lowpass
reconstruction and highpass reconstruction, you can get the lowpass decomposition and highpass decomposi-
tion because the coefficients are simply time-reversed.
6“Why use pip over easy install?” stackoverflow http://stackoverflow.com/questions/3220404/
why-use-pip-over-easy-install7PyWavelets API Reference http://www.pybytes.com/pywavelets/ref/idwt-inverse-discrete-wavelet-transform.html
27
Given the lowpass reconstruction and highpass reconstruction, Lo_R, Hi_R in Matlab notation 8, rec_lo,
rec_hi in PyWavelet notation 9, to obtain the lowpass decomposition filter and highpass decomposition filter,
then recall Eq. 7.1 and the discussion there.
Withc(n) = c(−n)
d(n) = d(−n), then this boils down to reversing the order of the coefficients to get c, d’s from c, d’s.
However, once the PyWavelet Object is constructed, instead of using upcoef, what you want to use is the
function wavefun. It “calculates approximations of scaling function (phi) and wavelet function (psi) at the
given level of refinement.” 10. It directly gives you the scaling function and wavelet you want.
EY : 20150703 My question is about the scaling. I would like some clarification in example6.m on the scaling
by M of the phi done after the inverse DWT. It’s unclear what choice of scaling Matlab uses. And how does
that compare with the scaling that’s done in PyWavelets?
Example 7: Generation of biorthogonal scaling functions and wavelets. biorwavf is a Biorthogonal spline
wavelet filter in Matlab.
In PyWavelets, the wavelet filters coefficients and the corresponding lowpass and highpass decomposition filters
and lowpass and highpass reconstruction filters can be accessed directly by creating a Wavelet object.
10.1. Readings. cf. Chapter 6 “Multiresolution”, Sec. 6.1. The Idea of Multiresolution [2]. Let
φ(t) = 21/2∑k
c(k)φ(2t− k) ∈ L2(R)
Let φ(t) have compact support on [2t−N, 2t] ∀ t, i.e. only c(0), c(1), . . . c(N) nonzero.
Note that c(k)’s are called approximation coefficients, which is handy to remember when using Matlab or
PyWavelets as that’s what they are referred to by, there.
Since
〈φ(t), φ(t−m)〉 =
∫ ∞−∞
dtφ(t)φ(t−m) = δ(m)
then ∑k
c(k)c(k − 2m) = δ(m)
Also, recall that ↓M : x(t) 7→ x(Mt), which is part of the “analysis” or decomposition filter bank.
Then coefficients c(k) “go into an orthonormal filter bank.” (cf. Sec. 6.1 [2]).
c(k) is involved in decomposition.
The key relations for scaling function φ(t) ∈ V 1 and wavelet φ(t) ∈ W 0 ⊂ V 1 are the following (see previous
theorems, such as Thm. 6.3.6 of Daubachies [5]):
(10.1)
φ(t) =√
2∑n
hnφ(2t− n)
ψ(t) =√
2∑n
(−1)nh−n+1φ(2t− n)
If we denote the coefficients for a wavelet d = d(k) (note that they are also called detail coefficients, from the
high-pass filter, because Matlab and PyWavelets both call these d’s detail coefficients) in
ψ(t) =√
2∑k
d(k)φ(2t− k) ∈W 1 ⊂ V 1
8upcoef, Mathworks Documentation http://www.mathworks.com/help/wavelet/ref/upcoef.html9“Using custom wavelets,” PyWavelets, http://www.pybytes.com/pywavelets/ref/wavelets.html10Approximating wavelet and scaling functions - Wavelet wavefun, PyWavelets http://www.pybytes.com/pywavelets/ref/
wavelets.html
28
Then we can conclude that the relationship between coefficients for φ and coefficients for ψ is the follow-
ing:
c(n) = d(−n+ 1)(−1)n ∀n ∈ Z
cf. Sec. 6.2. “Wavelets from Filters” [2]
The claim is this:
c(0) . . . c(N) constitute a low pass filter that determines the scaling function φ(t) =∑n 21/2c(n)φ(2t − n) ∈
V 0 ⊂ V 1
d(0) . . . d(N) constitute a high pass filter that determines the wavelets ψ(t) =∑n 21/2d(n)φ(2t−n) ∈W 0 ⊂ V 1
Now consider f j(t) =∑k a
jkφ
jk(t) ∈ V j
f(t) =∑j,k b
jkψ
jk(t) ∈W j
and now construct φ(t) using V 0 ⊂ V 1 the so-called “dilation equation.”
For V 0 ⊕W 0 with orthonormal bases, respectively,
{φ(t− k)}k∈Z, {ψ0k(t) = ψ(t− k)}k∈Z, then∑a1kφ
1k(t) =
∑a0kφ
0k(t) +
∑k
b0kψ0k(t) :=
∑k
a0kφ(t− k) +∑k
b0kψ(t− k)
φ(t− k) =∑n
21/2c(n)φ(2(t− k)− n)n=l−2k
=∑n
c(l − 2k)φ1l
ψ(t− k) =∑n
21/2d(n)φ(2(t− k)− n)n=l−2k
=∑n
d(l − 2k)φ1l
a0k = 〈f1, φ(t− k)〉 =∑
c(l − 2k)a1l
b0k = 〈f1, ψ(t− k)〉 =∑
d(l − 2k)a1l
11. Project Brief
EY : 20150701 I’m partial to wavelet collocation methods on an adaptive grid (for the spatial space) for
computational fluid dynamics (CFD), but it would be neat to include links, abstracts, and possible project
proposals here of past and future projects involving wavelets and filter banks.
However, inspired by Kyle Kastner’s post on “Wavelets” 11, there are two projects that was fun for me to
scratch the surface on and I’d be happy to see others pick up on.
11.1. Sunspots. SOFTWARE: in tools,
• sunspots_get.py
• sunspots.py
We can do a discrete wavelet transform on the number of sunspots observed over the years. The Royal
Observatory of Belgium in Brussels maintains updated data on worldwide observations of sunspot number12.
Much credit goes to them for doing this service.
sunspots_get.py gets you the latest data from SILSO on the observed sunspot number directly from SILSO
and puts the data in a Python pickle so you could save it locally on your hard drive (and so you don’t have
to keep requesting and overloading the website).
Now, I’ve used requests, BeautifulSoup, and Python 2’s urlparse to do what’s called “webscraping” of the
SILSO website - that is automating going to the website, reading in the html code, and extracting from the
website the desired links to the data. Of course, you can manually click on the website and download the
11Kyle Kastner, “Wavelets”, Garbage In, Garbage Out, http://kastnerkyle.github.io/blog/2014/04/17/wavelets/12Source: WDC-SILSO, Royal Observatory of Belgium, Brussels, http://sidc.be/silso/datafiles
29
Figure 1. Source: NASA
data, but I wanted to illustrate how web scraping can be done because you can imagine the usefulness of
automating data collection when dealing with a large amount.
sunspots.py does the actual Discrete Wavelet Transform (DWT) on the sunspot data for daily and mean
monthly observations, and plot them. While I used the single level DWT using Daubechies 2 and Daubechies
16 wavelets, there are 3 directions I see that can be fun to explore:
• Try different types of wavelets from the wavelist() wavelet list of PyWavelets, such as ’haar’, ’symlet’,
etc.
• Try different levels of resolution with wavedec
• Get the “peaks” of the wavelet decomposition (I tried scipy.signal.find_peaks_cwt from scipy.signal
but I need some work as it’s not working for me).
EY : But as of 20150704, I was able to obtain these plots of the single-level wavelet decomposition of the
observed sunspot number over 200 to 300 years, Fig. 11.2,11.2, 11.2.
11.2. Image Compression and Reconstruction. SOFTWARE: kompresni.py
11.3. Image compression and reconstruction. SOFTWARE: in tools, kompresni.py
PyWavelet, with 2-dimensional Wavelet packets (methods), offers the opportunity to do some image compres-
sion and reconstruction.13. with single level dwt2, idwt2, and 2D multilevel decomposition and reconstruction
with wavedec2 and waverec2.
Note that I was using scipy’s misc or ndimage for reading in the image and matplotlib.pyplot’s imshow to
plot it out.
EY : 20150704 right now, I’m having trouble with the code 14 in terms of using index slicing to put together
a 3-dimensional numpy array to get back the image after doing the DWT2 on each of the (3) colors, RGB
(red,green,blue). So I need help there. It’s outputting out right now multiple images after doing dwt2.
Nevertheless, here’s what I obtained so far as plots. Note that it’s tradition to use the image of Lena Soderberg,
but you should have fun and use an image you like (and use appropriately).
13PyWavelets, API Reference, http://www.pybytes.com/pywavelets/ref/2d-dwt-and-idwt.html14ernestyalumni, github, https://github.com/ernestyalumni/18-327-wavelets-filter-banks/blob/master/tools/
kompresni.py
30
31
Figure 2. Original image
32
12. Orthogonal Wavelet Bases: Connection to Orthogonal Filters; Orthogonality in the
Frequency Domain. Biorthogonal Wavelet Bases
13. Mallat Pyramid Algorithm; Handout 12
READING: Sec 1.6, 6.2 [2]
Consider
φj+1(t) ∈ V j+1 = V j ⊕W j
φj+1(t) =∑k
cj+1k φj+1
k (t) =∑k
cjkφjk(t) +
∑k
djkψjk(t)
which we can write, using the orthonormal basis {φjk(t) = 2j/2φ(2jt−k)}k∈K for V j . {ψjk}k∈K is the orthonor-
mal basis for W j .
For the scaling function φ(t) ∈ V 0, since V 0 ⊂ V 1, then we can write φ(t) as
(13.1) φ(t) =∑l
h0(l)21/2φ(2t− l)
Now, by theorem (such as Daubechies Thm. 6.3.6)
(13.2) ψ(t) =∑l
(−1)lh0(−l + 1)21/2φ(2t− l) =∑l
h1(l)21/2φ(2t− l)
So
φjk(t) = 2j/2φ(2jt− k) = 2j/2∑l
h0(l)21/2φ(2j+1t− 2k − l) =∑l
h0(l)φj+12k+l(t)
ψjk(t) = 2j/2ψ(2jt− k) = 2j/2∑l
h1(l)21/2φ(2j+1t− 2k − l) =∑l
h1(l)φj+12k+l(t)
33
Then
(13.3)
cjk = 〈φj+1(t), φjk(t)〉 =
∫ ∞−∞
dtφj+1
(t)φjk(t) =
∫ ∞−∞
dtφj+1
(t)∑l
h0(l)φj+12k+l(t) =
∑l
h0(l)cj+12k+l
u=2k+l=
∑u
h0(u− 2k)cj+1u
djk = 〈φj+1(t), ψjk(t)〉 =
∫ ∞−∞
dtφj+1
(t)∑l
h1(l)φj+12k+l(t) =
∑l
h1(l)cj+12k+l =
∑u
h1(u− 2k)cj+1u
Thus,
(13.4)
〈φj+1(t), φj+1k (t)〉 = cj+1
k =∑k′
cjk′〈φjk′(t), φ
j+1k (t)〉+
∑k′
djk′〈ψjk′(t), φ
j+1k (t)〉 =
∑k′
cjk′h0(k − 2k′) +∑k′
djk′h1(k − 2k′)
where
〈φjk′(t), φj+1k (t)〉 =
∑l
h0(l)δ(2k′ + l − k) = h0(k − 2k′)
〈ψjk′(t), φj+1k (t)〉 =
∑l
h1(l)δ(2k′ + l − k) = h1(k − 2k′)
EY : 20150707 The coefficients cjk’s djk’s live in a different space from V js, W js. These coefficients obey a
different algebra than the algebra of the scaling function and wavelets.
∀ j ∈ Z, V j is a k-algebra. V j is a ring with addition and multiplication being convolution. k is a commutative
ring with addition and multiplication (in the usual sense, like scalar addition and scalar multiplication). But
k has also addition and multiplication on its “indices.”
Proposition 1. ∀ j ∈ Z, V j is a L2(R)-module.
Proof. Sketch of a proof: ∀ f j , gj , pj ∈ V j , then under addition, it is (additive) group
so that f j + gj ∈ V j
f j + gj = gj + f j (commutativity)
(f j + gj) + pj = f j + (gj + pj) (associativity)
and it is equipped with multiplication being the convolution and this multplication is commutative
(h ∗ f j)(t) =
∫ ∞−∞
dτh(t− τ)f j(τ) = (f j ∗ h)(t)
�
Proposition 2. V j also is equipped as a kj-module over kj which itself a k-algebra of coefficients cjks, djks.
You can do addition and multiplication in kj with kj as a ring:
cjk + djk
h0(k − 2m)cjm
and you can also do addition and multiplication on the “indicies” as a commutative ring (but with no dis-
tributivity):
cjk+l = cjl+k
cj(MN)k = cj(NMk)
So V j+1 is a kj+1-module over kj+1 algebra.
V j+1 = V j ⊕W j and W j is a Dj-module over Dj algebra (consisting of djk coefficients).
I claim that kj+1 = kj ⊕Dj .34
For the filter bank representation involving coefficients cj+1k , cjk, djk, define the convolution function, that
involves the commutative ring k in the indices:
h0 ∗ cj+1k =
∑l
h0(k − l)cj+1l
Then the filter bank for this Mallot Pyramid Algorithm, or for wavelet coefficients does not involve V js at all,
but on its kj algebra:
kj+1 kj kj+1 kj+1
kj+1 kj+1
kj+1 Dj kj+1 kj+1
h0∗
h1∗
↓ 2 ↑ 2 h0∗
⊕
↓ 2 ↑ 2 h1∗ ⊕
∑l h0(k − l)cj+1
l cjk cjk/2∑l h0(k − l)cjl/2
cj+1k cj+1
k
∑l h1(k − l)cj+1
l djk djk/2∑l h1(k − l)djl/2
h0∗
h1∗
↓ 2 ↑ 2 h0∗
+
↓ 2 ↑ 2 h1∗+
with
h0 ∗ cj+1k =
∑l
h0(k − l)cj+1l
↓27→∑l
h0(2k − l)cj+1l =
∑l
h0(l − 2k)cj+1l = cjk
(confirming Eq. 13.3) and likewise,
djk =↓ 2 ◦ h1 ∗ cj+1k
and for the reconstruction part,
h0∗ ↑ 2 ◦ cjk =∑l
h0(k − l)cjl/2 =∑m
h0(k − 2m)cjm
h1∗ ↑ 2 ◦ djk =∑l
h1(k − l)djl/2 =∑m
h1(k − 2m)djm
(confirming Eq. 13.4) and the sum of the above is cj+1k .
Again, my point is that ↓ 2, ↑ 2 does not act on tR, the domain of V j as L2(R), but it acts on the commutative
ring of the indices.
14. Accuracy of Wavelet Approximations (Condition A); Vanishing Moments; Polynomial
Cancellation in Filter Banks
15. Smoothness of Wavelet Bases: Convergence of the Cascade Algorithm (Condition E);
Splines. Bases vs. Frames
16. Signal and Image Processing: Finite Length Signals; Boundary Filters and Boundary
Wavelets; Wavelet Compression Algorithms
READING: Sec 8.1-8.3, 8.5, 10.1, 11.1-11.5 [2]35
17. Guest Lecture. Physical Wavelets and their Sources: Real Physics in Complex
Spacetime
READING: ???
18. Lifting: Ladder Structure for Filter Banks; Factorization of Polyphase Matrix into
Lifting Steps; Lifting Form of Refinement Equation; Handout 16
READING: Sec 6.5 [2]
See Section 13 for the setup.
18.1. Slide 6. See Section 13 for the setup.
The lifting scheme is the following:
Proposition 3 (Lifting scheme for wavelets). In the filter bank representation on the kj algebras of which
kj-algebras V j is over,
(18.1)h]0 = h0 + u ∗ h1
h]1 = h1 − h0 ∗ u
where ] denotes the new h0 or h1, and ∗, in this algebra, is the convolution operator, but on the commutative
ring of the indices ( not the time or spatial domain of the wavelet themselves).
Proof. ] denotes new values, after the lifting transformation.
↓ 2 ◦ h]0 ∗ cj+1k =↓ 2 ◦ (h0 ∗ cj+1
k +∑l,m
u(k −m)h1(m− l)cj+1l ) = cjk +
∑l,m
u(2k −m)h1(l −m)cj+1l = c]jk
djk stays the same:
djk =↓ 2 ◦ h1 ∗ cj+1k
For the reconstruction part,
h0∗ ↑ 2 ◦ c]jk = h0∗ ↑ 2 ◦ cjk +∑l
h0(k − l)∑m,n
u(l −m)h1(n−m)cj+1n
h]1∗ ↑ 2 ◦ djk = h1∗ ↑ 2 ◦ djk − h0 ∗ u∗ ↑ 2 ◦ djk = h1∗ ↑ 2 ◦ djk −∑l,m
h0(k − l)u(l −m)djm/2 =
= h1∗ ↑ 2 ◦ djk −∑l,m
h0(k − l)u(l −m)∑n
h1(m− n)cj+1n = h1∗ ↑ 2 ◦ djk −
∑l,m,n
h0(k − l)u(l −m)h1(n−m)cj+1n
Clearly, summing the above two equations reconstructs the original cj+1k coefficient, noticing the cross term
cancels exactly. �
Note the change (and no change) in the coefficients, which I’ll recap here:
c]jk = cjk +∑l,m
u(2k −m)h1(l −m)cj+1l
djk =∑l
h1(l − 2k)cj+1l
cj+1k =
∑l
c]jl h0(k − l) +∑l
djlh]1(k − 2l)
36
Going back to Slide 6,
Recall Eqs. 13.1, 13.2
φ(t) =∑l
h0(l)21/2φ(2t− l)
ψ(t) =∑l
h1(l)21/2φ(2t− l)
With Proposition 3,
ψ](t) =∑l
h]1(l)21/2φ(2t− l) = 21/2∑l
h1(l)−∑m
h0(l −m)u(m)φ(2t− l) =
= ψ(t) + 21/2∑l,m
u(m)h0(l −m)φ(2t− l +m−m) = ψ(t) +∑m
u(m)φ(t−m/2) = ψ(t) +∑k
u(2k)φ(t− k)
These two expression are from the “reconstruction” part of the filter bank. It is clear now what the first 2
expressions on Slide 6 is for.
For the last 2 expressions of Slide 6, consider the decomposition part of the filter bank involving h0, h1.
Let’s resolve the very last expression, which is essentially
ψ(t) =∑l
h1(l)21/2φ](2t− l)
However,
φ](t) =∑l
h]0(l)21/2φ](2t− l) =∑l
(h0 + u ∗ h1)(l)21/2φ](2t− l) =
=∑l
h0(l)21/2φ](2t− l) +∑l,m
u(m)h1(l −m)φ](2t− l +m−m) =
=∑l
h0(l)21/2φ](2t− l) +∑m
u(m)ψ(t−m/2) =∑l
h0(l)21/2φ](2t− l) +∑k
u(2k)ψ(t− k)
18.2. On the Reading, Sec. 6.5, Biorthogonal Wavelets.
18.2.1. On Lifting. cf. Subsection “Filter Construction by Lifting” for Sec. 6.5 [2]
The so-called “no alias” condition for a 2-channel (i.e. ↓ 2, and the frequencies “separate into” two different
ranges, by 2π/2) filter bank is
F0(z)H0(−z) + F1(z)H1(−z) = 0
There is a choice of filters (transformations) F0, F1 that can be made. Let
F0(z) = H1(−z)
F1(z) = −H0(−z)
Thus, do this dual lifting:
H]0(z) = H0(z) + F0(−z)S(z2)
F ]1(z) = F1(z) +H1(−z)T (z2)
Given the choice above H]0, F
]1 of F0, F1, then
H]0(z) = H0(z) +H1(z)S(z2)
F ]1(z) = F1(z) + F0(z)T (z2)
37
18.3. Software: Handout16 examples.py. At this point, you should run Handout16_examples.py (I made
it myself, I don’t think the MIT OCW 18.327 material has the code for anything beyond Handout 16) and
reproduce the so-called Symmetric 5/3 Wavelets on Slide 16. They are actually the biorthogonal wavelets for
the Cohen-Daubechies-Feauveau wavelets 15. In Matlab, they are called “bior2.2”. The 2 and 2 refer to the
number of vanishing moments of the high pass filter; same in PyWavelets. Running the plots in the file, you
should reobtain the plots for Slide number 12 of Handout 16, and I also plotted the filter bank coefficients,
Figures 18.3, 18.3:
Figure 3. cf. ernestyalumni
19. Wavelets and Subdivision: Nonuniform Grids; Multiresolution for Triangular Meshes;
Representation and Compression of Surfaces; Slides and Handouts 17,18
READING: It appears that there are no reading for this session, other than Slides and Handouts 17, 18.
15“Cohen-Daubechies-Feauveau wavelet”, Wikipedia.org, https://en.wikipedia.org/wiki/Cohen-Daubechies-Feauveau_
wavelet
38
Figure 4. cf. ernestyalumni
20. Numerical Solution of PDEs: Galerkin Approximation; Wavelet Integrals (Projection
Coefficients, Moments and Connection Coefficients); Convergence. Subdivision
Wavelets for Integral Equations. Compression and Convergence Estimates
READING: Sec 11.6 [2]
21. M-band Wavelets: DFT Filter Banks and Cosine Modulated Filter Banks.
Multiwavelets
READING: Sec 7.5, 9.1-9.4 [2]39
Part 2. Extra fun
I wanted to, in this part of an open-sourced LaTeX file, welcome and elicit projects, ideas, resources beyond
the MIT OCW 18.327. You should, if you’re reading this and wanted to add to this pdf/LaTeX document,
share your projects, ideas, and resources here.
22. Fun with Quandl
Install (you can use pip install) Quandl (it’s open-source). Open up fun_w_Quandl.py that’s in the tools
directory of the github for 18-327-wavelets-filter-banks. Quandl provides open data, public data that
they’re trying to make free, open, and (easily) accessible to all.
With fun_w_Quandl.py, I’ve tried to use the Haar wavelet transform to apply discrete wavelet transforms
(dwt) of a few time-series data that seemed relevant at the time (the Shanghai Composite Index and the
Interest Rate Spread of 2 years vs. 10 years for the LCY (Local Currency Renminbi or Chinese Yuan) Bond).
The Python functions in the file, obtenir and pkl_data, gets the time-series data directly from Quandl and
“persists” the data i.e. saves it locally using Python’s pickle library. You don’t want to be making calls to
the Quandl website more than necessary.
See Figures 22, 22
Figure 5. cf. ernestyalumni
40
Figure 6. cf. ernestyalumni
Then you can read in the data onto your Python interactive interpreter with lire and then apply the dwt on all
the data sets with make_dwt and plot all of the approximation and detail coefficients with plot_dwt.
I would invite you to run the file and play around with it, alongside looking at the Python code and trying
out your own time series of interest from Quandl. I’d also would like to know if you guys have any ideas or
interpretations of what wavelets can tell us about time-series data.
Part 3. Colophon, Rationale, PitchDeck, and Help Me and Help Others
23. Why Wavelets (for me)?
One day on 18 juin 2015, I find myself needing to learn about wavelets, fast. I had just found the video
uploaded to YouTube of the talk given by Adam Lichtl and Stephen Jones (of SpaceX), given at the GPU
Tech Conference, San Jose, California, in March 17 - 20, 2015 16, and Lichtl, at the end, strongly recommended
that one look at the research of Vasilyev, Regele, Lamb, Ramaprabhu, Oefelein, and Massot. Lichtl and Jones
mentioned Wavelets and Wavelet compression in their slides and I also uncovered that one should learn and
use something called “wavelet collocation.”
I should mention that my goal is to get humanity to become a truly spacefaring civilization and to enable
all the kinds of exploration as seen in Christopher Nolan and Kip Thorne’s Interstellar (2014). I share in
SpaceX’s mission of getting humanity to become a truly spacefaring civilization, and so I want to meet the
needs of and directly help out SpaceX in anyway.17
16https://youtu.be/txk-VO1hzBY, originally posted at http://on-demand.gputechconf.com/gtc/2015/video/S5398.html17https://www.linkedin.com/in/ernestyalumni
41
I had no idea what a wavelet is, nor even the context of where wavelets come from, or used, or what not. The
first thing I now do when I want to pick up a new skill is to see if the MIT OCW has a course on it, and if any
of the websites providing MOOCs (Massive, Open, Online Courses) - edX, coursera, udacity - has anything.
There was MIT OCW 18.327, Wavelets, Filter Banks, and Applications, taught in Spring 2003 by Strang and
Amaratunga. But the problem was this.
24. The Problem (and the Solution)
24.1. The Problem with Education.
Through a sequence of things that I’ll tell you about, I got really interested in seeing if we
could make higher education available to everybody. I think it’s the most important that a
human being could do, because human resources - people - are the most important resources
on the planet and everything that you can make and do comes through education.
For you, this might be relatively trivial and obvious because you made it into Brown
[University], but if youre in Africa, in India, and in the developing world; if you have medical
problems, that might not be quite as obvious. –Sebastian Thrun, Udacity 18
If we’re to fulfill the promise of making education, at the highest levels, open, openly and freely available to
anyone, anywhere, at anytime, then we need to address these problems I had with this MIT OCW 18.327,
and in general, with MIT OCW and MOOCs.
• The code included in the tools directory of MIT OCW 18.327 used Matlab. A Matlab Individual
license is $2,150 US dollars 19. The cost of using Python is $0. I can’t afford a single individual
license, let alone, if and when I run a company, want to afford licenses for my staff. The cost of using
wavelets should be $0.
Coursera has Computational Methods for Data Analysis that has a section on wavelets, taught by
Nathan Kutz, in January-March 2015, but Matlab is exclusively used.
From my (unscientific) survey of browsing the web of disparate university course websites, courses on
wavelets uses Matlab exclusively.
I’m cognizant of GNU Octave as a direct open-source alternative to Matlab and Matlab code is
presumed to run on GNU Octave, but there are things like the Matlab Wavelet Toolkit that is exclusive
(and exclusively a proprietary black box 20) to Matlab, and loading packages in GNU Octave doesn’t
always work (nor is well-documented).
We can do better, and we can do everything and more in Python. Installing packages is especially
easy with pip and Homebrew. As great as GNU Octave is, and it is great, I can’t get the wavelet
package in GNU Octave to easily run without errors, nor all the file and website IO that I can with
Python.
Again, I want to emphasize that relying on Matlab for research and engineering deployment, and
education is not cost-effective, not amenable to anything other than a black box, nor meets the
promise of making higher education openly and freely available to everybody, anywhere, at anytime.
A Matlab Individual License is $2150. The cost of using Python is $0. Even at the proposed hourly
minimum wage in the U.S. of $15, this is 143 work-hours. The average monthly salary of a Medical
Doctor in the Czech Republic is $961 21. We cannot expect education, let alone online education, to
live up to its promise of reaching anyone, anywhere, at anytime, who desires to learn and improve
18Online Learning: The Challenge - Sebastian Thrun19http://www.mathworks.com/pricing-licensing/20http://www.mathworks.com/help/wavelet/ref/upcoef.html21http://www.worldsalaries.org/czechrepublic.shtml
42
society, if the required equipment for learning costs more than two months of a living wage - Open
Source Software (OSS), and in particular Python and its libraries that I am championing here, delivers
on that promise.
• Powerpoint(-like) slides The serious defect with using Powerpoint (or Powerpoint-like slides, in-
cluding beamer for LaTeX) is this: http://norvig.com/Gettysburg/.
We should stop using Powerpoint slides to convey technical information and ideas. At the very least,
an accompanying document should “flesh out” the slides.
The serious defect(s) when using the MIT OCW material, or any lecture slides, is the Powerpoint
template and structure itself and the fact that the presenter is not there and is not present to explain
the slides. It causes me, and anyone else, to have to reconstruct what the presenter meant in the slides
and it causes confusion and it’s inefficient in learning.
• Typography The typography of the slides and handouts is poor and doesn’t look good or pleasing
at all. Nor is the typography standardized in any way. It leads to a lot of confusion.
Yes, typography matters, because the analogy is this: before the Apple iMac, the convention was
that the PC is this beige, uninspiring plastic box. Nothing was devoted to its design, and for the
manufacturers that did attempt to, such as Sony with its VAIO line, priced design exorbitantly high.
Anyways, the convention for the Wintel PC was to be utilitarian and starkly and functionally beige.
The Apple iMac in 1998 was stunning in its design (by Jonathan Ives). You wanted to use the iMac
more. You wanted to play with it and learn more about it. You enjoyed using it more. And then you
ended up increasing your productivity because you “naturally” wanted to use it more.
The analogy, I think, applies in this case: the typography of the slides for MIT OCW 18.327 is lacking
and so are many of the MIT OCWs (e.g. MIT OCW 16.512 Rocket Propulsion). But if the typography
was better (and elegant), we can help with readability, making it easier to understand (and decrease
confusion), and make them more fun to use (and reuse).
Also, in a lot of the slides, not just for MIT OCW 18.327, but in many of the MIT OCWs, the graphs,
pictures, and figures are blurry and the plot labels are unreadable.
• Books don’t update themselves. They get outdated with progress and so do the material around
it.
• Books aren’t interactive; if you’re reading about a technical area with computer applications, the com-
puter applications are divorced, in section and in structure, from the concepts and theory presented,
when they should go hand in hand and in parallel.
• Typos in books and in the MIT OCW material do cause confusion and there’s no way for the reader
to correct them directly and share this edit with others.
• MOOCs are expensive. The cost of a MOOC is upwards to and beyond $244,000 22 and it’s hugely
time-consuming 23
• There aren’t many MOOCs that serve the graduate and post-graduate level. “Reservoir Geomechanics”
taught by Zoback 24 pulled no punches and was a really good, high, graduate-level course on Reservoir
Geomechanics, a.k.a. oil drilling and fracking. Otherwise, for graduate-level courses, I find the MIT
OCWs to be more complete, as it’s (assumedly) all the exact same material as used in the offline class.
But again, see the problems I mentioned above.
22“How and Why Institutions are Engaging with MOOCsAnswers in Report MOOCs: Expectations and Reality”, https:
//onlinelearninginsights.wordpress.com/tag/costs-of-developing-a-mooc/23Jonah Miller, “Course Review: Practical Numerical Methods with Python by George Washington University”, https:
//www.class-central.com/report/gwu-numerical-mooc-review/24https://lagunita.stanford.edu/courses/EarthSciences/GP202/Spring2014/about
43
• Prof. Strang hasn’t taught this class in over ten years and Dr. Amaratunga had left MIT and had
not responded to inquiries at his current occupation at Scaled Display Technologies. This seems like
a general problem with any class online or offline because there’s a shelf-life for the material (and the
instructors!). The hope for the online format is that there’ll be someone online that can help out.
That’s why I place a lot of hope and admiration for quora and stackexchange.
24.1.1. Solutions.
• Python. I’ve rewritten about half of the Matlab code used in MIT OCW 18.327 in Python and placed
it all on github. I’ve also written up Python scripts to accompany later Handouts (Handout 16 and
on) that are missing in the original MIT OCW 18.327 material, but it was obvious from the Handouts
that more computer-generated plots were there (but wasn’t in the tools directory).
I’ve outlined explicitly, and in conjunction with the Handouts, how and what to use with the different
Python libraries, including PyWavelets, and encourage its use and interactivity.
• In between a textbook and lecture slides: This is an excellent example of how to provide
material for a talk or lecture, and not the Powerpoint slides: Title:The singularity theorems in general
relativity. II, Speaker:Amir Aazami (Kavli IPMU), Date:Fri, May 17, 2013, 10:00 - 11:30, http:
//db.ipmu.jp/seminar/sysimg/seminar/945.pdf. Take a look at the pdf 945, “Lecture I: Basic
properties of Lorentzian manifolds.” They are something in between a textbook and Powerpoint
slides. This is what I tried to do here with this LaTeX file, that I make publicly available on github.
• Use LaTeX for typography and make it publicly available on github (and the pdf on Google Drive or
Dropbox) so people can copy, edit, and add onto the LaTeX or the pdf. Since it’s on github, students
and (older) learners can share their notes and insights directly there.
Look, I know it’s time-consuming and sometimes a pain-in-the-ass to LaTeX typeset everything.
Sometimes, I see some students make fantastic looking and very comprehensive notes of the course
lectures in LaTeX. Why not make the pdf AND the LaTeX available publicly on github (or wherever)
so that we can crowdsource this process and have more insights and questions and answers from the
bottom-up, as opposed to top-down from lecturer to students?
For plots, figures, and pictures, you can make your own plots, as I’ve meticulously outlined and
instructed here, and improve upon them, so they’d be crystal clear.
• On github, anyone can update this LaTeX file and the Python/GNU Octave code anytime.
• Python’s interactive interpreter is one of its most famous and greatest features.
• On github, anyone can make edits and correct for typos.
• Sharing stuff on github and on Google Drive or DropBox costs $0. I’m not maintaining a discussion
forum or moderating it either so that costs $0, also in time.
• This LaTeX file and related code was born out of trying to understand and get the tools for, quickly,
combustion CFD. I write for a high-level. But having this material open-sourced, if someone wants to
“fork” or make their own version with more elementary and pedagogical explanations and language,
he or she can go right ahead (just remember to share it).
• I could appreciate that university professors can’t or don’t want to (I don’t know) answer the questions
from everyone wanting to learn about wavelets. Look, I understand, we’re all busy people. Even TAs
aren’t getting paid enough, and TAs wouldn’t want to answer every emailed question from everyone,
nor moderate an online discussion forum (it’s time-consuming and can be aggravating). Why not
crowdsource and divide out this labor online?44
The kinds of discussion forums on udacity, Stanford online (was edX, now Lagunita) is great for this
and, in my opinion, very underutilized. The discussion forums are a gem and should be front and center
of the learning process that would move the office hours, TA sessions, in continental Europe what
would be called the Tutorial, Tutorium and Exercise session oder Ubungsblatter online. I recognize
that setting up and moderating a discussion forum is a time-consuming effort. Beyond a Usenet-style
email list, I wanted to try here an open-sourced, publicly available LaTeX file that anyone can copy,
edit, reuse, and add onto to further the discussion.
My rationale with this is everybody can be “on the same page,” technical, mathematical typesetting
can be done quickly (from my experience, and please tell me about anything I’m missing, but LaTeX
math typesetting online, on wordpress, a forum, stackexchange, is very time-consuming and not fun at
all - I’m well aware that there are a number of individuals who blog posts or answer on stackexchange
very clearly and voluminously math expressions with specific, custom LaTeX typesetting, but it ends
up being a one-way street of communication, because we’re not going to type that much if we can’t
copy, paste, and edit).
• One single LaTeX/pdf file for the course or subject. How many times has one had to dig up
different handouts or lectures notes on different files to reinforce a previous concept? Just throw it
all in one LaTeX/pdf file. And unlike a lot of the slides and handouts for the MIT OCWs and other
classes, with LaTeX generated pdf files; they’re completely searchable: we don’t have to waste as much
time on creating an indexing scheme.
• All the algebra steps missing during a class lecture is done explicitly and clearly here.
Plenty of questions that I have at least (and maybe some other students) involve showing the steps
explicitly. I can understand that the class time is limited and we don’t have time for that in class. We
can time-shift that to here, which is usually done in TA office hours or Ubungsblatter. But if the aim
is to make education open and freely available, to anyone, anywhere, and at anytime, then we have to
move that one-on-one learning online and in another format.
I don’t have all the answers to education, online or offline, but clearly, we can do better, and do it at a lower
cost and price point. But we cannot fail in meeting the needs and challenges of education. This is not just for
students in university. This is also for employee retraining, employee development, career retaining, whether
at a corporation, government, or a startup, or a freelance individual. Failing to bring open access to
technical education and modular skill acquisition is failing the future workers of the world. As
an American, of course I’m concerned with the huge, termed skills gap between the U.S. labor workforce,
between the portion of the U.S. workforce that finds themselves down-sized out of work, and need to acquire
new skills fast, between companies that are trying to fill and meet their human resources needs by checklisting
qualifications and with potential employees and workers who don’t have access to or know where to begin
to acquire these skills to help meet the needs of companies large and small, mature and startup. But these
issues apply all around the world, including developed countries, in particular the U.K., Ireland, France,
Germany25.
24.2. Helping Out and Helping Others. I will be restarting my crowdfunding campaign on Open/Tilt
to crowdfund for money to do more computational fluid dynamics and wavelet applications on the Mac and
Linux. I want to develop on Swift using the Accelerate framework for numerical computation, including the
wavelet stuff here. Also, I want to do CFD and wavelet stuff on a Linux box. I would be crowdfunding for
about $43,000, $16,000 in new computer equipment for the Mac and Linux, and including $20,000 to pay
myself, for my efforts here and continuing development. About 15-20% goes to taxes. It’s still a fraction of
the cost of a MOOC at $244,000 or the pay and equipment of TAs and an instructor and administrative staff
at an offline university (single high-end TA salary is $23000). I’ll launch the crowdfunding campaign as soon
25“Skills Mismatches and Labour Mobility”, europa.eu http://ec.europa.eu/europe2020/pdf/themes/27_skills_gaps_and_
labour_mobility_02.pdf
45
as I can; any and all questions and negative feedback (and funds) would be welcomed right now. However,
as I’ve mentioned several times above, I’m committed to free and open education and tools so regardless of
how much I raise (if at all), or even if I fail to get others on board to help out in this effort with Wavelets
(I’ve tried emailing Prof. Strang and Dr. Amaratunga a number of times but I’ve either gotten no substantial
help or critical feedback, or no response at all), I’ll always keep all this stuff out for free and public use and
distribution.
If you can’t help out with my own crowdfunding efforts, or I haven’t convinced you enough yet (please tell
me why so I could improve), then at the very least, please read and contribute to the Memorial fund for the
creator of Python library matplotlib, John Hunter. “On August 28 2012, John D. Hunter, the creator of
matplotlib, died from complications arising from cancer treatment, after a brief but intense battle with this
terrible illness. John is survived by his wife Miriam, his three daughters Rahel, Ava and Clara, his sisters
Layne and Mary, and his mother Sarah.”26 I used matplotlib heavily for plotting with wavelets in Python
and you should too, as it’s open-source, it costs effectively $0, and it’s very flexible and amenable to changing
needs. As of right now, I’m still seeking any kind of position at SpaceX, as I believe in their mission to the
core, and I had just gotten my Masters in Physics, and I don’t have any income right now, but I am PayPal’ing
to the John Hunter Memorial Fund next week (20150709) as soon as my bank transfer clears, with whatever I
can. If you can’t help me out, fine, but at least help out the John Hunter Memorial Fund because matplotlib
has been very useful.
References
[1] Gilbert Strang, and Kevin Amaratunga. 18.327 Wavelets, Filter Banks and Applications, Spring 2003. (Massachusetts Institute
of Technology: MIT OpenCourseWare), http://ocw.mit.edu (Accessed 19 Jun, 2015). License: Creative Commons BY-NC-
SA
[2] Gilbert Strang, Truong Nguyen. Wavelets and Filter Banks, Wellesley-Cambridge Press; 2nd edition, 1996. ISBN-13:
978-0961408879
[3] Elliott H. Lieb and Michael Loss, Analysis (Graduate Studies in Mathematics), Book 14, American Mathematical Society,
2nd edition, 2001. ISBN-13: 978-0821827833
[4] P. Wojtaszczyk, A Mathematical Introduction to Wavelets, (London Mathematical Society Student Texts), Cambridge
University Press, 1997. ISBN-13: 978-0521578943
[5] Ingrid Daubechies, Ten Lectures on Wavelets (CBMS-NSF Regional Conference Series in Applied Mathematics), SIAM:
Society for Industrial and Applied Mathematics, 1992. ISBN-13: 978-0898712742
E-mail address: ernestyalumni@gmail.com
URL: http://ernestyalumni.wordpress.com
26John Hunter Memorial Fund, Numfocus.org, http://numfocus.org/news/2012/08/28/johnhunter/
46