Convolution, Fourier Series, and the Fourier Transform

CS414 – Spring 2007Roger Cheng(some slides courtesy of Brian Bailey)

Convolution

A mathematical operator which computes the “amount of overlap” between two functions. Can be thought of as a general moving average

Discrete domain:

Continuous domain:

Discrete domain

Basic steps1. Flip (reverse) one of the digital functions.2. Shift it along the time axis by one sample.3. Multiply the corresponding values of the two digital functions.4. Summate the products from step 3 to get one point of the

digital convolution.5. Repeat steps 1-4 to obtain the digital convolution at all times

that the functions overlap.

Example

Continuous domain example

Continuous domain example

LTI (Linear Time-Invariant) Systems

Convolution can describe the effect of an LTI system on a signal

Assume we have an LTI system H, and its impulse response h[n]

Then if the input signal is x[n], the output signal is y[n] = x[n] * h[n]

Hx[n] y[n] = x[n]*h[n]

Mathematics of Waves

Periodic phenomena occur everywhere– vision, sound, electronic communication, etc.

In order to manipulate physical world, we need mathematical tools

Fourier Series

Any reasonable function can be expressed as a (infinite) linear combination of sines and cosines

F(t) = a0 + a1cos (ωt) + b1sin(ωt) + a2cos (2ωt) + b2sin(2ωt) + …

=

F(t) is a periodic function with

))sin()cos((0

nnn tnbtna

T 2

Reasonable?

F(t) is a periodic function with

must satisfy certain other conditions– finite number of discontinuities within T– finite average within T– finite number of minima and maxima

T 2

Calculate Coefficients

T

k dttktfT

a0

)cos()(2

T

dttfa

T

0

0

)(

T

k dttktfT

b0

)sin()(2

))sin()cos(()(0

nnn tnbtnatF

Example

F(t) = square wave, with T=1.0s ( )0.1

2

0

1

0 1 2 3 4 5 6 7

Series1

Example

ttF

sin4)(

0

1

0 1 2 3 4 5 6 7

Series1

Series2

Example

ttttF

5sin543sin

34sin4)(

0

1

0 1 2 3 4 5 6 7

Series1

Series3

Example

What if – T = .01s;

– T = .05s;

– T = 50s;

ttttF

5sin543sin

34sin4)(

ttttF

5sin543sin

34sin4)(

ttttF

5sin543sin

34sin4)(

Time

Frequency

Time vs. Frequency Domain

0

1

0 1 2 3 4 5 6 7

Series1

Series3

0

0.2

0.4

0.6

0.8

1

1.2

1.4

1w 3w 5w

ttttF

5sin543sin

34sin4)(

Why Frequency Domain?

Allows efficient representation of a good approximation to the original function

Makes filtering easy

Note that convolution in the time domain is equivalent to multiplication in the frequency domain (and vice versa)

– When you are faced with a difficult convolution (or multiplication), you can switch domains and do the complement operation

Fourier Family

Discrete Fourier Transform

Rarely have closed form equation for F(t)

Must sample at periodic intervals– generates a series of numeric values– apply transform to a time window of values– N = number of sample points– x[ ] contains sample points

DFT Notation

ck[i] and sk [i] are the cosine and sine waves, each N points in length– ck is cosine wave for amplitude in ReX[k]– sk is sine wave for amplitude in ImX[k]

k refers to the frequency of the wave– usually between 0 and N/2

)/2sin(][ Nkiisk

)/2cos(][ Nkiick

Complex exponentials

Alternatively, we can use complex exponentials

Euler’s formula: eiw = cos(w) + i*sin(w)

Calculate DFT

Separate sinusoids

Complex exponentials

)/2cos(][][Re1

0

NkiixkXN

i

)/2sin(][][Im1

0

NkiixkXN

i

Calculate Inverse DFT

Coefficients first need to be normalized

With two special cases

2/][Re][Re

NkXkX

NXX ]0[Re]0[Re

2/][Im][Im

NkXkX

NNXNX ]2/[Re]2/[Re

Calculate Inverse DFT

Corresponding points within the basis function contribute to each input value– Equation uses the normalized coefficients

Result is exactly equal to original data (within rounding error), ie IDFT(DFT(x[n])) = x[n]

)/2(sin][Im)/2(cos][Re][2/

0

2/

0

NkikXNkikXixN

k

N

k

DCT (Discrete Cosine Transform)

DCT is very similar to DFT– Sine wave + phase shift equals cosine wave– N coefficients of cosine basis functions– Advantage: Result is purely real– Most common is “Type II”– Used in JPEG and MPEG

FFT (Fast Fourier Transform)

FFT is an efficient algorithm for computing the DFT– Naïve method for DFT requires O(N2) operations– FFT uses divide and conquer to break up problem into many 2-

point DFT’s (which are easy to compute)– 2-point DFT: X[0] = x[0] + x[1]

X[1] = x[0] – x[1]– log2N stages, O(N) operations per stage => O(N log N) total

operations– Ideally, want N to be a power of 2 or close to a power of 2

4-point FFT “butterfly” diagram

Fourier Transform

Definitions:

Can be difficult to compute => Often rely upon table of transforms

Delta function

Definition:

Often, the result of the Fourier Transform needs to be expressed in terms of the delta function

Fourier Transform pairs

There is a duality in all transform pairs