Fft Lecture
Transcript of Fft Lecture
-
8/6/2019 Fft Lecture
1/90
Fast Fourier Transform
Key Papers in C
Dima Batenkov
Weizmann Institute of Science
-
8/6/2019 Fft Lecture
2/90
Fast Fourier Transform -
erview
Fast Fourier Transform - Overv
J. W. Cooley and J. W. Tukey. An algorithmcalculation of complex Fourier series. MathComputation, 19:297301, 1965
-
8/6/2019 Fft Lecture
3/90
Fast Fourier Transform -
erview
Fast Fourier Transform - Overv
J. W. Cooley and J. W. Tukey. An algorithmcalculation of complex Fourier series. MathComputation, 19:297301, 1965 A fast algorithm for computing the Discre
Transform
-
8/6/2019 Fft Lecture
4/90
Fast Fourier Transform -
erview
Fast Fourier Transform - Overv
J. W. Cooley and J. W. Tukey. An algorithmcalculation of complex Fourier series. MathComputation, 19:297301, 1965 A fast algorithm for computing the Discre
Transform
(Re)discovered by Cooley & Tukey in 196adopted thereafter
-
8/6/2019 Fft Lecture
5/90
Fast Fourier Transform -
erview
Fast Fourier Transform - Overv
J. W. Cooley and J. W. Tukey. An algorithmcalculation of complex Fourier series. MathComputation, 19:297301, 1965 A fast algorithm for computing the Discre
Transform
(Re)discovered by Cooley & Tukey in 196adopted thereafter
Has a long and fascinating history
-
8/6/2019 Fft Lecture
6/90
Fast Fourier Transform -
erview
urier Analysis
Fourier Series
Continuous Fourier Transform
Discrete Fourier Transform
Useful properties 6
Applications Fourier Analysis
-
8/6/2019 Fft Lecture
7/90
Fast Fourier Transform -
erview
urier Analysis
Fourier Series
Continuous Fourier Transform
Discrete Fourier Transform
Useful properties 6
Applications
Fourier Series
Expresses a (real) periodic function x(t)trigonometric series (L < t < L)
x(t) =1
2a0 +
n=1
(an cosn
Lt + b
Coefficients can be computed by
an =1
L
LL
x(t) cosn
Lt
bn =1
L
LL
x(t) sinn
Lt
-
8/6/2019 Fft Lecture
8/90
Fast Fourier Transform -
erview
urier Analysis
Fourier Series
Continuous Fourier Transform
Discrete Fourier Transform
Useful properties 6
Applications
Fourier Series
Generalized to complex-valued functions
x(t) =
n=
cnei nL
t
cn =1
2L
L
L
x(t)ein
L
-
8/6/2019 Fft Lecture
9/90
Fast Fourier Transform -
erview
urier Analysis
Fourier Series
Continuous Fourier Transform
Discrete Fourier Transform
Useful properties 6
Applications
Fourier Series
Generalized to complex-valued functions
x(t) =
n=
cnei nL
t
cn =1
2L
L
L
x(t)ein
L
Studied by D.Bernoulli and L.Euler
Used by Fourier to solve the heat equatio
-
8/6/2019 Fft Lecture
10/90
Fast Fourier Transform -
erview
urier Analysis
Fourier Series
Continuous Fourier Transform
Discrete Fourier Transform
Useful properties 6
Applications
Fourier Series
Generalized to complex-valued functions
x(t) =
n=
cnei nL
t
cn =1
2L
L
L
x(t)ein
L
Studied by D.Bernoulli and L.Euler
Used by Fourier to solve the heat equatio
Converges for almost all nice functions
L2 etc.)
-
8/6/2019 Fft Lecture
11/90
Fast Fourier Transform -
erview
urier Analysis
Fourier Series
Continuous Fourier Transform
Discrete Fourier Transform
Useful properties 6
Applications
Continuous Fourier Transform
Generalization of Fourier Series for infinit
x(t) =
F(f)e2i f
F(f) =
x(t)e2i f t dt
Can represent continuous, aperiodic sign
Continuous frequency spectrum
-
8/6/2019 Fft Lecture
12/90
Fast Fourier Transform -
erview
urier Analysis
Fourier Series
Continuous Fourier Transform
Discrete Fourier Transform
Useful properties 6
Applications
Discrete Fourier Transform
If the signal X(k) is periodic, band-limitedNyquist frequency or higher, the DFT rep
exactly14
A(r) =N1
k=0
X(k)WrkN
where WN = e
2i
N and r = 0 , 1 , . . . , N
The inverse transform:
X(j) =1
N
N1
k=0
A(k)W
N
-
8/6/2019 Fft Lecture
13/90
Fast Fourier Transform -
erview
urier Analysis
Fourier Series
Continuous Fourier Transform
Discrete Fourier Transform
Useful properties 6
Applications
Useful properties 6
Orthogonality of basis functions
N1
k=0
Wrk
N WrkN = NN(r
Linearity
(Cyclic) Convolution theorem
(x y)n =N1
k=0
x(k)y(
F {(x y)} = F {X}F {Y
Symmetries: real hermitian symmetrichermitian antisymmetric
Shifting theorems
-
8/6/2019 Fft Lecture
14/90
Fast Fourier Transform -
erview
urier Analysis
Fourier Series
Continuous Fourier Transform
Discrete Fourier Transform
Useful properties 6
Applications
Applications
Approximation by trigonometric polynomi Data compression (MP3...)
Spectral analysis of signals
Frequency response of systems
Partial Differential Equations
Convolution via frequency domain Cross-correlation Multiplication of large integers Symbolic multiplication of polynomials
-
8/6/2019 Fft Lecture
15/90
Fast Fourier Transform -
erview
ethods known by 1965
Available methods
Goertzels algorithm 7
Methods known by
-
8/6/2019 Fft Lecture
16/90
Fast Fourier Transform -
erview
ethods known by 1965
Available methods
Goertzels algorithm 7
Available methods
In many applications, large digitized dataavailable, but cannot be processed due torunning time of DFT
All (publicly known) methods for efficient
the symmetry of trigonometric functions,
Goertzels method7
is fastest known
-
8/6/2019 Fft Lecture
17/90
Fast Fourier Transform -
erview
ethods known by 1965
Available methods
Goertzels algorithm 7
Goertzels algorithm7
Given a particular 0 < j < N 1, we wan
A(j) =N1
k=0
x(k) cos
2j
Nk
B(j) =
N
k=
-
8/6/2019 Fft Lecture
18/90
Fast Fourier Transform -
erview
ethods known by 1965
Available methods
Goertzels algorithm 7
Goertzels algorithm7
Given a particular 0 < j < N 1, we wan
A(j) =N1
k=0
x(k) cos
2j
Nk
B(j) =
N
k=
Recursively compute the sequence {u(s)
u(s) = x(s) + 2cos
2j
N
u(s +
u(N) = u(N+ 1) = 0
-
8/6/2019 Fft Lecture
19/90
Fast Fourier Transform -
erview
ethods known by 1965
Available methods
Goertzels algorithm 7
Goertzels algorithm7
Given a particular 0 < j < N 1, we wan
A(j) =N1
k=0
x(k) cos
2j
Nk
B(j) =
N
k=
Recursively compute the sequence {u(s)
u(s) = x(s) + 2cos
2j
N
u(s +
u(N) = u(N+ 1) = 0
Then
B(j) = u(1) sin
2j
N
A(j) = x(0) + cos
2j
N
u(1
-
8/6/2019 Fft Lecture
20/90
Fast Fourier Transform -
erview
ethods known by 1965
Available methods
Goertzels algorithm 7
Goertzels algorithm7
Requires N multiplications and only one
Roundoff errors grow rapidly5
Excellent for computing a very small num(< log N)
Can be implemented using only 2 interme
locations and processing input signal as Used today for Dual-Tone Multi-Frequenc
dialing)
-
8/6/2019 Fft Lecture
21/90
Fast Fourier Transform -
erview
spiration for the FFT
Factorial experiments
Goods paper 9
Goods results - summary
Inspiration for the
-
8/6/2019 Fft Lecture
22/90
Fast Fourier Transform -
erview
spiration for the FFT
Factorial experiments
Goods paper 9
Goods results - summary
Factorial experiments
Purpose: investigate the effect of n factors quantity Each factor can have t distinct levels. Fo
t = 2, so there are 2 levels: + and .
So we conduct tn experiments (for every combination of the n factors) and calcula Main effect of a factor: what is the ave
output as the factor goes from to Interaction effects between two or mo
the difference in output between the cfactors are present compared to whenis present?
-
8/6/2019 Fft Lecture
23/90
Fast Fourier Transform -
erview
spiration for the FFT
Factorial experiments
Goods paper 9
Goods results - summary
Factorial experiments
Example of t = 2, n = 3 experiment Say we have factors A, B, C which can be
example, they can represent levels of 3 dto patients
Perform 23 measurements of some quantpressure
Investigate how each one of the drugs anaffect the blood pressure
-
8/6/2019 Fft Lecture
24/90
Fast Fourier Transform -
erview
spiration for the FFT
Factorial experiments
Goods paper 9
Goods results - summary
Factorial experiments
Exp. Combination A B C Bl
1 (1)
2 a +
3 b + 4 ab + +
5 c +
6 ac + +
7 bc + +
8 abc + + +
-
8/6/2019 Fft Lecture
25/90
Fast Fourier Transform -
erview
spiration for the FFT
Factorial experiments
Goods paper 9
Goods results - summary
Factorial experiments
The main effect of A is(ab b) + (a (1)) + (abc bc) + (ac
The interaction of A, B is(ab b) (a (1)) ((abc bc) (ac
etc.
-
8/6/2019 Fft Lecture
26/90
Fast Fourier Transform -
erview
spiration for the FFT
Factorial experiments
Goods paper 9
Goods results - summary
Factorial experiments
The main effect of A is(ab b) + (a (1)) + (abc bc) + (ac
The interaction of A, B is(ab b) (a (1)) ((abc bc) (ac
etc.
If the main effects and interactions are arrathen we can write y = Ax where
A =
1 1 1 1 1 1
1 1 1 1 1 1
1 1 1 1 1 1
1 1 1 1 1 11 1 1 1 1 1
1 1 1 1 1 1
1 1 1 1 1 1
1 1 1 1 1 1
-
8/6/2019 Fft Lecture
27/90
Fast Fourier Transform -
erview
spiration for the FFT
Factorial experiments
Goods paper 9
Goods results - summary
Factorial experiments
F.Yates devised15 an iterative n-step algosimplifies the calculation. Equivalent to dA = Bn, where (for our case of n = 3):
B =
1 1 0 0 0 0
0 0 1 1 0 0
0 0 0 0 1 1
0 0 0 0 0 0
1 1 0 0 0 0
0 0 1 1 0 0
0 0 0 0 1
0 0 0 0 0 0
so that much less operations are required
-
8/6/2019 Fft Lecture
28/90
Fast Fourier Transform -
erview
spiration for the FFT
Factorial experiments
Goods paper 9
Goods results - summary
Goods paper9
I. J. Good. The interaction algorithm and pranalysis. Journal Roy. Stat. Soc., 20:3613 Essentially develops a prime-factor FFT
The idea is inspired by Yates efficient algproves general theorems which can be aefficiently multiplying a N-vector by certa
The work remains unnoticed until Cooley
Good meets Tukey in 1956 and tells him method
It is revealed later4 that the FFT had not much earlier by a coincedence
-
8/6/2019 Fft Lecture
29/90
Fast Fourier Transform -
erview
spiration for the FFT
Factorial experiments
Goods paper 9
Goods results - summary
Goods paper9
Definition 1 Let M(i) be t t matrices, i =direct product
M(1) M(2) M(n
is atn tn matrix C whose elements are
Cr,s =n
i=1
M(i)ri,si
wherer = (r1, . . . , rn) ands = (s1, . . . sn) arepresentation of the index
-
8/6/2019 Fft Lecture
30/90
Fast Fourier Transform -
erview
spiration for the FFT
Factorial experiments
Goods paper 9
Goods results - summary
Goods paper9
Definition 1 Let M(i) be t t matrices, i =direct product
M(1) M(2) M(n
is atn tn matrix C whose elements are
Cr,s =n
i=1
M(i)ri,si
wherer = (r1, . . . , rn) ands = (s1, . . . sn) arepresentation of the index
Theorem 1 For every M, M[n] = An where
Ar,s = {Mr1,sns1r2
s2r3 . . .
snrn
Interpretation: A has at most tn+1 nonzero
a given B we can find M such that M[n] = Bcompute Bx = Anx in O(ntn) instead of O(
-
8/6/2019 Fft Lecture
31/90
Fast Fourier Transform -
erview
spiration for the FFT
Factorial experiments
Goods paper 9
Goods results - summary
Goods paper9
Goods observations For 2n factorial experiment:
M =
1 1
1 1
-
8/6/2019 Fft Lecture
32/90
Fast Fourier Transform -
erview
spiration for the FFT
Factorial experiments
Goods paper 9
Goods results - summary
Goods paper9
Goods observations n-dimensional DFT, size N in each dimen
A(s1, . . . , sn) =(N1,...,N1)
(r1,...,rn)
x(r)Wr1sN
can be represented as
A = [n]x, = {WrsN} r, s = 0
By the theorem, A = nx where is spa
-
8/6/2019 Fft Lecture
33/90
Fast Fourier Transform -
erview
spiration for the FFT
Factorial experiments
Goods paper 9
Goods results - summary
Goods paper9
Goods observations n-dimensional DFT, size N in each dimen
A(s1, . . . , sn) =(N1,...,N1)
(r1,...,rn)
x(r)Wr1sN
can be represented as
A = [n]x, = {WrsN} r, s = 0
By the theorem, A = nx where is spa
n-dimensional DFT, Ni in each dimension
A = ((1) (n))x = (1)
(i) = {WrsNi}
(i) = IN1 (i) INn
-
8/6/2019 Fft Lecture
34/90
Fast Fourier Transform -
erview
spiration for the FFT
Factorial experiments
Goods paper 9
Goods results - summary
Goods paper9
If N = N1N2 and (N1, N2) = 1 then N-poinrepresented as 2-D DFT
r = N2r1 + N1r2
(Chinese Remainder Thm.)s = N2s1(N12 ) mod N
s s1 mod N1, s
These are index mappings s (s1, s2) and
A(s) = A(s1, s2) =N
r=0
x(r)WrsN =N11
r1=0
(substitute r, s from above) =N11
r1=0
N21
r2=0
x(r1, r2)W
This is exactly 2-D DFT seen above!
-
8/6/2019 Fft Lecture
35/90
Fast Fourier Transform -
erview
spiration for the FFT
Factorial experiments
Goods paper 9
Goods results - summary
Goods paper9
So, as above, we can write
A = x
= P((1) (2))Q1
= P(1)(2)Q1x
where P and Q are permutation matrices. TN(N1 + N2) multiplications instead of N
2.
-
8/6/2019 Fft Lecture
36/90
Fast Fourier Transform -
erview
spiration for the FFT
Factorial experiments
Goods paper 9
Goods results - summary
Goods results - summary
n-dimensionanal, size Ni in each dimens
n
Ni
n
Ni
instead of
1-D, size N = n
Ni
where Nis are mutu
n
Ni
N instead of N
Still thinks that other tricks such as usinand sines and cosines of known angles museful
-
8/6/2019 Fft Lecture
37/90
Fast Fourier Transform -
erview
e article
ames W.Cooley (1926-)
ohn Wilder Tukey
915-2000)
The story - meeting
The story - publication
What if...
Cooley-Tukey FFT
FFT propertiesDIT signal flow
DIF signal flow
The article
-
8/6/2019 Fft Lecture
38/90
Fast Fourier Transform -
erview
e article
ames W.Cooley (1926-)
ohn Wilder Tukey
915-2000)
The story - meeting
The story - publication
What if...
Cooley-Tukey FFT
FFT propertiesDIT signal flow
DIF signal flow
James W.Cooley (1926-)
1949 - B.A. in Mathematics(Manhattan College, NY)
1951 - M.A. in Mathematics(Columbia University)
1961 - Ph.D in Applied Math-
ematics (Columbia Univer-sity)
1962 - Joins IBM Watson Research Cent
The FFT is considered his most significanmathematics
Member of IEEE Digital Signal Processin
Awarded IEEE fellowship on his works on
Contributed much to establishing the term
-
8/6/2019 Fft Lecture
39/90
Fast Fourier Transform -
erview
e article
ames W.Cooley (1926-)
ohn Wilder Tukey
915-2000)
The story - meeting
The story - publication
What if...
Cooley-Tukey FFT
FFT propertiesDIT signal flow
DIF signal flow
John Wilder Tukey (1915-2000)
1936 - Bachelor in Chem-istry (Brown University)
1937 - Master in Chemistry(Brown University)
1939 - PhD in Mathemat-
ics (Princeton) for Denu-merability in Topology
During WWII joins Fire Control Researchcontributes to war effort (mainly invlolving
From 1945 also works with Shannon andBell Labs
Besides co-authoring the FFT in 1965, hacontributions to statistics
-
8/6/2019 Fft Lecture
40/90
Fast Fourier Transform -
erview
e article
ames W.Cooley (1926-)
ohn Wilder Tukey
915-2000)
The story - meeting
The story - publication
What if...
Cooley-Tukey FFT
FFT propertiesDIT signal flow
DIF signal flow
The story - meeting
Richard Garwin (Columbia University IBMcenter) meets John Tukey (professor of mPrinceton) at Kennedys Scientific Adviso1963
-
8/6/2019 Fft Lecture
41/90
Fast Fourier Transform -
erview
e article
ames W.Cooley (1926-)
ohn Wilder Tukey
915-2000)
The story - meeting
The story - publication
What if...
Cooley-Tukey FFT
FFT propertiesDIT signal flow
DIF signal flow
The story - meeting
Richard Garwin (Columbia University IBMcenter) meets John Tukey (professor of mPrinceton) at Kennedys Scientific Adviso1963 Tukey shows how a Fourier series of l
N = ab is composite can be expresse
of b-point subseries, thereby reducingcomputations from N2 to N(a + b)
-
8/6/2019 Fft Lecture
42/90
Fast Fourier Transform -
erview
e article
ames W.Cooley (1926-)
ohn Wilder Tukey
915-2000)
The story - meeting
The story - publication
What if...
Cooley-Tukey FFT
FFT propertiesDIT signal flow
DIF signal flow
The story - meeting
Richard Garwin (Columbia University IBMcenter) meets John Tukey (professor of mPrinceton) at Kennedys Scientific Adviso1963 Tukey shows how a Fourier series of l
N = ab is composite can be expresse
of b-point subseries, thereby reducingcomputations from N2 to N(a + b)
Notes that if N is a power of two, the arepeated, giving Nlog2 N operations
-
8/6/2019 Fft Lecture
43/90
Fast Fourier Transform -
erview
e article
ames W.Cooley (1926-)
ohn Wilder Tukey
915-2000)
The story - meeting
The story - publication
What if...
Cooley-Tukey FFT
FFT propertiesDIT signal flow
DIF signal flow
The story - meeting
Richard Garwin (Columbia University IBMcenter) meets John Tukey (professor of mPrinceton) at Kennedys Scientific Adviso1963 Tukey shows how a Fourier series of l
N = ab is composite can be expresse
of b-point subseries, thereby reducingcomputations from N2 to N(a + b)
Notes that if N is a power of two, the arepeated, giving Nlog2 N operations
Garwin, being aware of applications inwould greatly benefit from an efficient
realizes the importance of this and fropushes the FFT at every opportunity
-
8/6/2019 Fft Lecture
44/90
Fast Fourier Transform -
erview
e article
ames W.Cooley (1926-)
ohn Wilder Tukey
915-2000)
The story - meeting
The story - publication
What if...
Cooley-Tukey FFT
FFT propertiesDIT signal flow
DIF signal flow
The story - publication
Garwin goes to Jim Cooley (IBM Watsonwith the notes from conversation with Tukimplement the algorithm
-
8/6/2019 Fft Lecture
45/90
Fast Fourier Transform -
erview
e article
ames W.Cooley (1926-)
ohn Wilder Tukey
915-2000)
The story - meeting
The story - publication
What if...
Cooley-Tukey FFT
FFT propertiesDIT signal flow
DIF signal flow
The story - publication
Garwin goes to Jim Cooley (IBM Watsonwith the notes from conversation with Tukimplement the algorithm
Says that he needs the FFT for computin
transform of spin orientations of He3. It iswhat Garwin really wanted was to be ableexposions from seismic data
-
8/6/2019 Fft Lecture
46/90
Fast Fourier Transform -
erview
e article
ames W.Cooley (1926-)
ohn Wilder Tukey
915-2000)
The story - meeting
The story - publication
What if...
Cooley-Tukey FFT
FFT propertiesDIT signal flow
DIF signal flow
The story - publication
Garwin goes to Jim Cooley (IBM Watsonwith the notes from conversation with Tukimplement the algorithm
Says that he needs the FFT for computin
transform of spin orientations of He3. It iswhat Garwin really wanted was to be ableexposions from seismic data
Cooley finally writes a 3-D in-place FFT aGarwin
-
8/6/2019 Fft Lecture
47/90
Fast Fourier Transform -
erview
e article
ames W.Cooley (1926-)
ohn Wilder Tukey
915-2000)
The story - meeting
The story - publication
What if...
Cooley-Tukey FFT
FFT propertiesDIT signal flow
DIF signal flow
The story - publication
Garwin goes to Jim Cooley (IBM Watsonwith the notes from conversation with Tukimplement the algorithm
Says that he needs the FFT for computin
transform of spin orientations of He3. It iswhat Garwin really wanted was to be ableexposions from seismic data
Cooley finally writes a 3-D in-place FFT aGarwin
The FFT is exposed to a number of math
-
8/6/2019 Fft Lecture
48/90
Fast Fourier Transform -
erview
e article
ames W.Cooley (1926-)
ohn Wilder Tukey
915-2000)
The story - meeting
The story - publication
What if...
Cooley-Tukey FFT
FFT propertiesDIT signal flow
DIF signal flow
The story - publication
Garwin goes to Jim Cooley (IBM Watsonwith the notes from conversation with Tukimplement the algorithm
Says that he needs the FFT for computin
transform of spin orientations of He3. It iswhat Garwin really wanted was to be ableexposions from seismic data
Cooley finally writes a 3-D in-place FFT aGarwin
The FFT is exposed to a number of math
It is decided that the algorithm should be
domain to prevent patenting
-
8/6/2019 Fft Lecture
49/90
Fast Fourier Transform -
erview
e article
ames W.Cooley (1926-)
ohn Wilder Tukey
915-2000)
The story - meeting
The story - publication
What if...
Cooley-Tukey FFT
FFT propertiesDIT signal flow
DIF signal flow
The story - publication
Garwin goes to Jim Cooley (IBM Watsonwith the notes from conversation with Tukimplement the algorithm
Says that he needs the FFT for computin
transform of spin orientations of He3. It iswhat Garwin really wanted was to be ableexposions from seismic data
Cooley finally writes a 3-D in-place FFT aGarwin
The FFT is exposed to a number of math
It is decided that the algorithm should be
domain to prevent patenting Cooley writes the draft, Tukey adds some
Goods article and the paper is submittedComputation in August 1964
-
8/6/2019 Fft Lecture
50/90
Fast Fourier Transform -
erview
e article
ames W.Cooley (1926-)
ohn Wilder Tukey
915-2000)
The story - meeting
The story - publication
What if...
Cooley-Tukey FFT
FFT propertiesDIT signal flow
DIF signal flow
The story - publication
Garwin goes to Jim Cooley (IBM Watsonwith the notes from conversation with Tukimplement the algorithm
Says that he needs the FFT for computin
transform of spin orientations of He3. It iswhat Garwin really wanted was to be ableexposions from seismic data
Cooley finally writes a 3-D in-place FFT aGarwin
The FFT is exposed to a number of math
It is decided that the algorithm should be
domain to prevent patenting Cooley writes the draft, Tukey adds some
Goods article and the paper is submittedComputation in August 1964
Published in April 1965
-
8/6/2019 Fft Lecture
51/90
Fast Fourier Transform -
erview
e article
ames W.Cooley (1926-)
ohn Wilder Tukey
915-2000)
The story - meeting
The story - publication
What if...
Cooley-Tukey FFT
FFT propertiesDIT signal flow
DIF signal flow
What if...
Good reveals in 19934 that Garwin visite
Garwin told me with great enthusiasm aexperimental work... Unfortunately, not wthe subject..., I didnt mention my FFT woalthough I had intended to do so. He wro
9, 1976 and said: Had we talked about ihave stolen [publicized] it from you insteaTukey...... That would have completely cof the FFT
-
8/6/2019 Fft Lecture
52/90
Cooley-Tukey FFT
A(j) = N1k=0 X(k)WjkN N = N1N2
-
8/6/2019 Fft Lecture
53/90
Cooley-Tukey FFT
A(j) = N1k=0 X(k)WjkN N = N1N2
Convert the 1-D indices into 2-D
j = j0 + j1N1 j0 = 0 , . . . , N1 1 j1 =
k = k0 + k1N2 k0 = 0 , . . . , N2 1 k1 =
-
8/6/2019 Fft Lecture
54/90
Cooley-Tukey FFT
A(j) = N1k=0 X(k)WjkN N = N1N2
Convert the 1-D indices into 2-D
j = j0 + j1N1 j0 = 0 , . . . , N1 1 j1 =
k = k0 + k1N2 k0 = 0 , . . . , N2 1 k1 =
Decimate the original sequence into N2 N1-subsequ
-
8/6/2019 Fft Lecture
55/90
Cooley-Tukey FFT
A(j) = N1k=0 X(k)WjkN N = N1N2
Convert the 1-D indices into 2-D
j = j0 + j1N1 j0 = 0 , . . . , N1 1 j1 =
k = k0 + k1N2 k0 = 0 , . . . , N2 1 k1 =
Decimate the original sequence into N2 N1-subsequ
A(j1, j0) =N1
k=0
X(k)WjkN
-
8/6/2019 Fft Lecture
56/90
Cooley-Tukey FFT
A(j) = N1k=0 X(k)WjkN N = N1N2
Convert the 1-D indices into 2-D
j = j0 + j1N1 j0 = 0 , . . . , N1 1 j1 =
k = k0 + k1N2 k0 = 0 , . . . , N2 1 k1 =
Decimate the original sequence into N2 N1-subsequ
A(j1, j0) =N1
k=0
X(k)WjkN
=
N21
k0=0
N11
k1=0X(k1, k0)W
(j0+j1N1)(k0+k1N1N2
-
8/6/2019 Fft Lecture
57/90
Cooley-Tukey FFT
A(j) = N1k=0 X(k)WjkN N = N1N2
Convert the 1-D indices into 2-D
j = j0 + j1N1 j0 = 0 , . . . , N1 1 j1 =
k = k0 + k1N2 k0 = 0 , . . . , N2 1 k1 =
Decimate the original sequence into N2 N1-subsequ
A(j1, j0) =N1
k=0
X(k)WjkN
=
N21
k0=0
N11
k1=0X(k1, k0)W
(j0+j1N1)(k0+k1N1N2
=N21
k0=0
Wj0k0N W
j1k0N2
N11
k1=0
X(k1, k0)Wj0kN1
-
8/6/2019 Fft Lecture
58/90
Cooley-Tukey FFT
A(j) = N1k=0 X(k)WjkN N = N1N2
Convert the 1-D indices into 2-D
j = j0 + j1N1 j0 = 0 , . . . , N1 1 j1 =
k = k0 + k1N2 k0 = 0 , . . . , N2 1 k1 =
Decimate the original sequence into N2 N1-subsequ
A(j1, j0) =N1
k=0
X(k)WjkN
=
N21
k0=0
N11
k1=0X(k1, k0)W
(j0+j1N1)(k0+k1N2)N1N2
=N21
k0=0
Wj0k0N W
j1k0N2
N11
k1=0
X(k1, k0)Wj0k1N1
=
-
8/6/2019 Fft Lecture
59/90
Cooley-Tukey FFT
The sequences Xj0 have N elements and require NNcompute
Given these, the array A requires NN2 operations to
Overall N(N1 + N2)
In general, if N = ni=0 Ni then N(ni=0 Ni) operation
-
8/6/2019 Fft Lecture
60/90
Cooley-Tukey FFT
If N1 = N2 = = Nn = 2 (i.e. N= 2n):
j = jn12n1 + +j0, k = kn12
A(jn1, . . . ,j0) =1
k0=0
1
kn1=0
X(kn1, . . . , k0)WjN
= k0
Wjk0N
k1
Wjk12N
kn1
Wjkn1N
Xj0 (kn2, . . . , k0) = kn1
X(kn1, . . . , k0)Wj0kn12
Xj1,j0 (kn3, . . . , k0) = kn2
Wj0kn24
Xj0 (kn2, . . . , k0)Wj1k2
Xjm1,...,j0 (knm1, . . . , k0) = knm
W(j0++jm22m2)knm2m
Xjm2,
Wjm1knm2
-
8/6/2019 Fft Lecture
61/90
Cooley-Tukey FFT
Finally, A(jn1, . . . , j0) = Xjn1,...,j0 Two important properties:
In-place: only two terms are involved in the calculaintermediate pair Xjm1,...,j0(. . . ) jm1 = 0, 1: Xjm2,...,j0(0 , . . . )
Xjm2,...,j0(1 , . . . )So the values can be overwritten and no more addused
Bit-reversal It is convenient to store Xjm1,...,j0(knm1, . . . , k0
j02n1 + + jm12
nm + knm12mn1 +
The result must be reversed Cooley & Tukey didnt realize that the algorithm in fac
DFTs from smaller ones with some multiplications alo
The terms W(j0++jm22
m2)knm2m are later called
6 tw
-
8/6/2019 Fft Lecture
62/90
Fast Fourier Transform -
erview
e article
ames W.Cooley (1926-)
ohn Wilder Tukey
915-2000)
The story - meeting
The story - publication
What if...
Cooley-Tukey FFT
FFT propertiesDIT signal flow
DIF signal flow
FFT properties
Roundoff error significantly reduced com
formula6
A lower bound of 12
n log2 n operations ov
algorithms is proved12, so FFT is in a sen
Two canonical FFTs
Decimation-in-Time: the Cooley & TukEquivalent to taking N1 = N/2, N2are the DFT coefficients of even-numbXj0(1) - those of odd-numbered. The fi
simply linear combination of the two. Decimation-in-Frequency (Tukey-Sand
N1 = 2, N2 = N/2. Then Y() = Xeven-numbered and odd-numbered saZ() = X1() is the difference. The evcoefficients are the DFT of Y, and theweighted DFT of Z.
-
8/6/2019 Fft Lecture
63/90
Fast Fourier Transform -
erview
e article
ames W.Cooley (1926-)
ohn Wilder Tukey
915-2000)
The story - meeting
The story - publication
What if...
Cooley-Tukey FFT
FFT propertiesDIT signal flow
DIF signal flow
DIT signal flow
-
8/6/2019 Fft Lecture
64/90
Fast Fourier Transform -
erview
e article
ames W.Cooley (1926-)
ohn Wilder Tukey
915-2000)
The story - meeting
The story - publication
What if...
Cooley-Tukey FFT
FFT propertiesDIT signal flow
DIF signal flow
DIT signal flow
-
8/6/2019 Fft Lecture
65/90
Fast Fourier Transform -
erview
e article
ames W.Cooley (1926-)
ohn Wilder Tukey
915-2000)
The story - meeting
The story - publication
What if...
Cooley-Tukey FFT
FFT propertiesDIT signal flow
DIF signal flow
DIT signal flow
-
8/6/2019 Fft Lecture
66/90
Fast Fourier Transform -
erview
e article
ames W.Cooley (1926-)
ohn Wilder Tukey
915-2000)
The story - meeting
The story - publication
What if...
Cooley-Tukey FFT
FFT propertiesDIT signal flow
DIF signal flow
DIF signal flow
-
8/6/2019 Fft Lecture
67/90
Fast Fourier Transform -
erview
e article
ames W.Cooley (1926-)
ohn Wilder Tukey
915-2000)
The story - meeting
The story - publication
What if...
Cooley-Tukey FFT
FFT propertiesDIT signal flow
DIF signal flow
DIF signal flow
-
8/6/2019 Fft Lecture
68/90
Fast Fourier Transform -
erview
e article
ames W.Cooley (1926-)
ohn Wilder Tukey
915-2000)
The story - meeting
The story - publication
What if...
Cooley-Tukey FFT
FFT propertiesDIT signal flow
DIF signal flow
DIF signal flow
-
8/6/2019 Fft Lecture
69/90
Fast Fourier Transform -
erview
wly discovered history of FFT
What happened after the
blication
Danielson-Lanczos
Gauss Newly discovered histo
-
8/6/2019 Fft Lecture
70/90
Fast Fourier Transform -
erview
wly discovered history of FFT
What happened after the
blication
Danielson-Lanczos
Gauss
What happened after the publi
Rudnicks note13 mentions Danielson-La
19423. Although ... less elegant and is pterms of real quantities, it yields the samebinary form of the Cooley-Tukey algorithmnumber of arithmetical operations.
Historical Notes on FFT2
Also mentions Danielson-Lanczos Thomas and Prime Factor algorithm (
distinct from Cooley-Tukey FFT
A later investigation10 revealed that Gausdiscovered the Cooley-Tukey FFT in 1805
-
8/6/2019 Fft Lecture
71/90
Danielson-Lanczos
G. C. Danielson and C. Lanczos. Some improvementFourier analysis and their application to X-ray scatterFranklin Institute, 233:365380 and 435452, 1942
-
8/6/2019 Fft Lecture
72/90
Danielson-Lanczos
G. C. Danielson and C. Lanczos. Some improvementFourier analysis and their application to X-ray scatterFranklin Institute, 233:365380 and 435452, 1942
... the available standard forms become impractical fof coefficients. We shall show that, by a certain transfit is possible to double the number of ordinates with othan double the labor
-
8/6/2019 Fft Lecture
73/90
Danielson-Lanczos
G. C. Danielson and C. Lanczos. Some improvementFourier analysis and their application to X-ray scatterFranklin Institute, 233:365380 and 435452, 1942
... the available standard forms become impractical fof coefficients. We shall show that, by a certain transfit is possible to double the number of ordinates with othan double the labor
Concerned with improving efficiency of hand calculat
-
8/6/2019 Fft Lecture
74/90
Danielson-Lanczos
G. C. Danielson and C. Lanczos. Some improvementFourier analysis and their application to X-ray scatterFranklin Institute, 233:365380 and 435452, 1942
... the available standard forms become impractical fof coefficients. We shall show that, by a certain transfit is possible to double the number of ordinates with othan double the labor
Concerned with improving efficiency of hand calculat
We shall now describe a method which eliminates thcomplicated schemes by reducing ... analysis for 4n canalyses for 2n coefficients
-
8/6/2019 Fft Lecture
75/90
Danielson-Lanczos
G. C. Danielson and C. Lanczos. Some improvementFourier analysis and their application to X-ray scatterFranklin Institute, 233:365380 and 435452, 1942
... the available standard forms become impractical fof coefficients. We shall show that, by a certain transfit is possible to double the number of ordinates with othan double the labor
Concerned with improving efficiency of hand calculat
We shall now describe a method which eliminates thcomplicated schemes by reducing ... analysis for 4n canalyses for 2n coefficients
If desired, this reduction process can be applied twic(???!!)
Adoping these improvements the approximate timesfor 8 coefficients, 25 min. for 16, 60 min. for 32 and 1( 0.37Nlog2 N)
-
8/6/2019 Fft Lecture
76/90
Danielson-Lanczos
Take the DIT Cooley-Tukey with N1 = N/2, N2 = 2
A(j0 +N
2j1) =
N/21
k1=0
X(2k1)Wj0k1N/2 + W
j0+N
2j1
N
N/21
k1=0
-
8/6/2019 Fft Lecture
77/90
Danielson-Lanczos
Take the DIT Cooley-Tukey with N1 = N/2, N2 = 2
A(j0 +N
2j1) =
N/21
k1=0
X(2k1)Wj0k1N/2 + W
j0+N
2j1
N
N/21
k1=0
Danielson-Lanczos showed completely analogous re
framework of real trigonometric series: ...the contribution of the even ordinates to a Four
coefficients is exactly the same as the contributionordinates to a Fourier analysis of 2n coefficients
..contribution of the odd ordinates is also reducibscheme... by means of a transformation process i
phase difference ...we see that, apart from the weight factors 2cos
2sin(/4n)k, the calculation is identical to the coshalf the number of ordinates
-
8/6/2019 Fft Lecture
78/90
Danielson-Lanczos
Take the DIT Cooley-Tukey with N1 = N/2, N2 = 2
A(j0 +N
2j1) =
N/21
k1=0
X(2k1)Wj0k1N/2 + W
j0+N
2j1
N
N/21
k1=0
Danielson-Lanczos showed completely analogous re
framework of real trigonometric series: ...the contribution of the even ordinates to a Four
coefficients is exactly the same as the contributionordinates to a Fourier analysis of 2n coefficients
..contribution of the odd ordinates is also reducibscheme... by means of a transformation process i
phase difference ...we see that, apart from the weight factors 2cos
2sin(/4n)k, the calculation is identical to the coshalf the number of ordinates
These weight factors are exactly the twiddle fac
-
8/6/2019 Fft Lecture
79/90
Fast Fourier Transform -
erview
wly discovered history of FFT
What happened after the
blication
Danielson-Lanczos
Gauss
Gauss
Herman H. Goldstine writes in a footnoteNumerical Analysis from the 16th through
(1977)8:This fascinating work of Gauss was nrediscovered by Cooley and Tukey in paper in 1965
This goes largely unnoticed until the rese
Johnson and Burrus10 in 1985 Essentially develops an DIF FFT for tw
sequences. Gives examples for N = Declares that it can be generalized to
factors, although no examples are giv Uses the algorithm for solving the proorbit parameters of asteroids
The treatise was published posthumothen other numerical methods were prnobody found this interesting enough.
-
8/6/2019 Fft Lecture
80/90
Fast Fourier Transform -
erview
pact
mpact
Further developments
Concluding thoughts
Impact
-
8/6/2019 Fft Lecture
81/90
Fast Fourier Transform -
erview
pact
mpact
Further developments
Concluding thoughts
Impact
Certainly a classic
Two special issues of IEEE trans. on Aud
Electroacoustics devoted entirely to FFT
Arden House Workshop on FFT11
Brought together people of very divers
someday radio tuners will operate witunits. I have heard this suggested witbut one can speculate. - J.W.Cooley
Many people discovered the DFT via th
-
8/6/2019 Fft Lecture
82/90
Fast Fourier Transform -
erview
pact
mpact
Further developments
Concluding thoughts
Further developments
Real-FFT (DCT...)
Parallel FFT
FFT for prime N
...
-
8/6/2019 Fft Lecture
83/90
Fast Fourier Transform -
erview
pact
mpact
Further developments
Concluding thoughts
Concluding thoughts
It is obvious that prompt recognition and significant achievments is an important g
-
8/6/2019 Fft Lecture
84/90
Fast Fourier Transform -
erview
pact
mpact
Further developments
Concluding thoughts
Concluding thoughts
It is obvious that prompt recognition and significant achievments is an important g
However, the publication in itself may not
-
8/6/2019 Fft Lecture
85/90
Fast Fourier Transform -
erview
pact
mpact
Further developments
Concluding thoughts
Concluding thoughts
It is obvious that prompt recognition and significant achievments is an important g
However, the publication in itself may not
Communication between mathematiciansanalysts and workers in a very wide rang
can be very fruitful
-
8/6/2019 Fft Lecture
86/90
Fast Fourier Transform -
erview
pact
mpact
Further developments
Concluding thoughts
Concluding thoughts
It is obvious that prompt recognition and significant achievments is an important g
However, the publication in itself may not
Communication between mathematiciansanalysts and workers in a very wide rang
can be very fruitful Always seek new analytical methods and
increase in processing speed
-
8/6/2019 Fft Lecture
87/90
Fast Fourier Transform -
erview
pact
mpact
Further developments
Concluding thoughts
Concluding thoughts
It is obvious that prompt recognition and significant achievments is an important g
However, the publication in itself may not
Communication between mathematiciansanalysts and workers in a very wide rang
can be very fruitful Always seek new analytical methods and
increase in processing speed
Careful attention to a review of old literaturewards
-
8/6/2019 Fft Lecture
88/90
Fast Fourier Transform -
erview
pact
mpact
Further developments
Concluding thoughts
Concluding thoughts
It is obvious that prompt recognition and significant achievments is an important g
However, the publication in itself may not
Communication between mathematiciansanalysts and workers in a very wide rang
can be very fruitful Always seek new analytical methods and
increase in processing speed
Careful attention to a review of old literaturewards
Do not publish papers in Journal of Frank
-
8/6/2019 Fft Lecture
89/90
Fast Fourier Transform -
erview
pact
mpact
Further developments
Concluding thoughts
Concluding thoughts
It is obvious that prompt recognition and significant achievments is an important g
However, the publication in itself may not
Communication between mathematiciansanalysts and workers in a very wide rang
can be very fruitful Always seek new analytical methods and
increase in processing speed
Careful attention to a review of old literaturewards
Do not publish papers in Journal of Frank
Do not publish papers in neo-classic Lati
-
8/6/2019 Fft Lecture
90/90
References
1. J. W. Cooley and J. W. Tukey. An algorithm for the machine calculation of complex Fourier series.Mathematics of Computation, 19:297301, 1965.
2. J. W. Cooley, P. A. Lewis, and P. D. Welch. History of the fast Fourier transform. In Proc. IEEE,volume 55, pages 16751677, October 1967.
3. G. C. Danielson and C. Lanczos. Some improvements in practical Fourier analysis and their appli-cation to X-ray scattering from liquids. J. Franklin Institute, 233:365380 and 435452, 1942.
4. D.L.Banks. A conversation with I. J. Good. Statistical Science, 11(1):119, 1996.
5. W. M. Gentleman. An error analysis of Goertzels (Watts) method for computing Fourier coeffi-cients. Comput. J., 12:160165, 1969.
6. W. M. Gentleman and G. Sande. Fast Fourier transformsfor fun and profit. In Fall Joint ComputerConference, volume 29 ofAFIPS Conference Proceedings, pages 563578. Spartan Books, Washington,D.C., 1966.
7. G. Goertzel. An algorithm for the evaluation of finite trigonometric series. The American Mathe-matical Monthly, 65(1):3435, January 1958.
8. Herman H. Goldstine. A History of Numerical Analysis from the 16th through the 19th Century.Springer-Verlag, New York, 1977. ISBN 0-387-90277-5.
9. I. J. Good. The interaction algorithm and practical Fourier analysis. Journal Roy. Stat. Soc., 20:361372, 1958.
10. M. T. Heideman, D. H. Johnson, and C. S. Burrus. Gauss and the history of the Fast FourierTransform. Archive for History of Exact Sciences, 34:265267, 1985.
11. IEEE. Special issue on fast Fourier transform. IEEE Trans. on Audio and Electroacoustics, AU-17:65186, 1969.
12. Morgenstern. Note on a lower bound of the linear complexity of the fast Fourier transform. JACM:Journal of the ACM, 20, 1973.
13. Philip Rudnick. Note on the calculation of Fourier series (in Technical Notes and Short Papers).j-MATH-COMPUT, 20(95):429430, July 1966. ISSN 0025-5718.