3f3 Digital Signal Processing Part1 1232329900875209 1

Introduction to Digital Signal Processing (DSP)

Elena Punskaya www-sigproc.eng.cam.ac.uk/~op205

Some material adapted from courses by Prof. Simon Godsill, Dr. Arnaud Doucet,

Dr. Malcolm Macleod and Prof. Peter Rayner

Books:

–  J.G. Proakis and D.G. Manolakis, Digital Signal Processing 3rd edition, Prentice-Hall.

–  R.G Lyons, Understanding Digital Signal processing, 2nd edition, , Prentice-Hall.

Digital: operating by the use of discrete signals to represent data in the form of numbers

Signal: a parameter (electrical quantity or effect) that can be varied in such a way as to convey information

Processing: a series operations performed according to programmed instructions

changing or analysing information which is measured as discrete sequences of numbers

What is Digital Signal Processing?

Applications of DSP - Biomedical

Biomedical: analysis of biomedical signals, diagnosis, patient monitoring, preventive health care, artificial organs

Examples:

1) electrocardiogram (ECG) signal – provides doctor with information about the condition of the patient’s heart

2) electroencephalogram (EEG) signal – provides Information about the activity of the brain

Applications of DSP - Speech

Speech applications:

Examples

1) noise reduction – reducing background noise in the sequence produced by a sensing device (microphone)

2) speech recognition – differentiating between various speech sounds

3) synthesis of artificial speech – text to speech systems for blind

Applications of DSP - Communications

Communications:

Examples

1) telephony – transmission of information in digital form via telephone lines, modem technology, mobile phones

2) encoding and decoding of the information sent over a physical channel (to optimise transmission or to detect or correct errors in transmission)

Applications of DSP - Radar

Radar and Sonar:

Examples

2) tracking

1) target detection – position and velocity estimation

Applications of DSP – Image Processing

Image Processing:

Examples

1) content based image retrieval – browsing, searching and retrieving images from database

2) compression - reducing the redundancy in the image data to optimise transmission / storage

2) image enhancement

Applications of DSP – Music

Music Applications:

Examples:

3) Manipulation (mixing, special effects)

2) Playback

1) Recording

Applications of DSP - Multimedia

Multimedia:

generation storage and transmission of sound, still images, motion pictures

Examples: 1) digital TV

2) video conferencing

DSP Implementation - Operations

To implement DSP we must be able to:

DSP Digital Signal

Digital Signal

1) perform numerical operations including, for example, additions, multiplications, data transfers and logical operations

either using computer or special-purpose hardware

Output

DSP chips

•  Introduction of the microprocessor in the late 1970's and early 1980's meant DSP techniques could be used in a much wider range of applications.

DSP chip – a programmable device, with its own native instruction code

designed specifically to meet numerically-intensive requirements of DSP

capable of carrying out millions of floating point operations per second Bluetooth

headset Household appliances

Home theatre system

DSP Implementation – Digital/Analog Conversion

DSP Digital Signal

Digital Signal

Reconstruction Analog Signal

2) convert the digital information, after being processed back to an analog signal

- involves digital-to-analog conversion & reconstruction (recall from 1B Signal and Data Analysis)

e.g. text-to-speech signal (characters are used to generate artificial sound)

DSP Implementation –Analog/Digital Conversion

DSP Digital Signal

Digital Signal

Sampling

Analog Signal

3) convert analog signals into the digital information - sampling & involves analog-to-digital conversion (recall from 1B Signal and Data Analysis)

e.g. Touch-Tone system of telephone dialling (when button is pushed two sinusoid signals are generated (tones) and transmitted, a digital system determines the frequences and uniquely identifies the button – digital (1 to 12) output

DSP Implementation

DSP Digital Signal

Digital Signal

Reconstruction Analog Signal Sampling

Analog Signal

perform both A/D and D/A conversions

e.g. digital recording and playback of music (signal is sensed by microphones, amplified, converted to digital, processed, and converted back to analog to be played

Limitations of DSP - Aliasing

Most signals are analog in nature, and have to be sampled loss of information •  we only take samples of the signals at intervals and

don’t know what happens in between aliasing cannot distinguish between higher and lower frequencies

Sampling theorem: to avoid aliasing, sampling rate must be at least twice the maximum frequency component (`bandwidth’) of the signal

Gjendemsjø, A. Aliasing Applet, Connexions, http://cnx.org/content/m11448/1.14

(recall from 1B Signal and Data Analysis)

•  Sampling theorem says there is enough information to reconstruct the signal, which does not mean sampled signal looks like original one

Limitations of DSP - Antialias Filter

correct reconstruction is not just connecting samples with straight lines

needs antialias filter (to filter out all high frequency components before sampling) and the same for reconstruction – it does remove information though

Each sample is taken at a slightly earlier part of a cycle

Limitations of DSP – Frequency Resolution

Most signals are analog in nature, and have to be sampled loss of information

•  we only take samples for a limited period of time

limited frequency resolution

does not pick up “relatively” slow changes

Limitations of DSP – Quantisation Error

Most signals are analog in nature, and have to be sampled loss of information •  limited (by the number of bits available) precision in data

storage and arithmetic

quantisation error

smoothly varying signal represented by “stepped” waveform

Advantages of Digital over Analog Signal Processing

Why still do it?

•  Digital system can be simply reprogrammed for other applications / ported to different hardware / duplicated

(Reconfiguring analog system means hadware redesign, testing, verification)

•  DSP provides better control of accuracy requirements (Analog system depends on strict components tolerance, response may drift with

temperature) •  Digital signals can be easily stored without deterioration (Analog signals are not easily transportable and often can’t be processed off-line)

•  More sophisticated signal processing algorithms can be implemented

(Difficult to perform precise mathematical operations in analog form)

Brief Review of Fourier Analysis

Time domain

Example: speech recognition

tiny segment

sound /a/ as in father

sound /i/ as in see

difficult to differentiate between different sounds in time domain

How do we hear?

www.uptodate.com

Inner Ear

Cochlea – spiral of tissue with liquid and thousands of tiny hairs that gradually get smaller

Each hair is connected to the nerve

The longer hair resonate with lower frequencies, the shorter hair resonate with higher frequencies

Thus the time-domain air pressure signal is transformed into frequency spectrum, which is then processed by the brain

Our ear is a Natural Fourier Transform Analyser!

Fourier’s Discovery

Jean Baptiste Fourier showed that any signal could be made up by adding together a series of pure tones (sine wave) of appropriate amplitude and phase

(Recall from 1A Maths)

Fourier Series for periodic square wave

infinitely large number of sine waves is required

Fourier Transform

The Fourier transform is an equation to calculate the frequency, amplitude and phase of each sine wave needed to make up any given signal :

1. The Discrete Time Fourier Transform (DTFT) of a sampled sequence{xn}, n = 0,±1,±2, . . . ,±! is given by

X (!) =!!

xne"jn!T .

X (!) =

x(t)e"j!tdt.

where T is the sampling interval.

(a) Explain how the problems with computing the DTFT on a digitalcomputer can be overcome by choosing a uniformly spaced gripof N frequencies covering the range of ! from 0 to 2"/T andtruncating the summation in the expression above to just N datapoints; thus, determine the expression for the N -point DiscreteFourier Transform (DFT). [10%]

Xp =N"1!n=0

xne"j 2!N np, p = 0, . . . , N " 1

(b) Answer. There are two problems with calculating DTFT on a dig-ital computer: first, the frequency variable ! is continuous whichcan be overcome by evaluating the DTFT at a finite collectionof discrete frequencies without any undesirable consequences; sec-ond, the summation involves infinite number of samples which canbe overcome by performing the summation over a finite numberof data points, which does have does have consequences since sig-nals are generally not of finite duration. If we choose a uniformlyspaced grip of N frequencies covering the range of ! from 0 to2"/T and truncate the summation in the expression above to justN data points we get the DFT:

Xp =N"1!n=0

xne"j 2!N np, p = 0, . . . , N " 1

(c) Assume you are interested in detecting the presence of a singlecontinuous-wave sinusoidal tone in a time-domain data sequence.Show how the 64-point DFT of {xn} can be computed at thediscrete frequency of 30kHz if the sampling rate of {xn} was fs =

X (!) =!!

xne"jn!T .

X (!) =

x(t)e"j!tdt.

(a) Explain how the problems with computing the DTFT on a digitalcomputer can be overcome by choosing a uniformly spaced gripof N frequencies covering the range of ! from 0 to 2"/T andtruncating the summation in the expression above to just N datapoints; thus, determine the expression for the N -point DiscreteFourier Transform (DFT). [10%]

Xp =N"1!n=0

xne"j 2!N np, p = 0, . . . , N " 1

Xp =N"1!n=0

xne"j 2!N np, p = 0, . . . , N " 1

(c) Assume you are interested in detecting the presence of a singlecontinuous-wave sinusoidal tone in a time-domain data sequence.Show how the 64-point DFT of {xn} can be computed at thediscrete frequency of 30kHz if the sampling rate of {xn} was fs =

Prism Analogy

Analogy:

a prism which splits white light into a spectrum of colors

white light consists of all frequencies mixed together

the prism breaks them apart so we can see the separate frequencies

White light

Spectrum of colours

Fourier Transform

Signal Spectrum

Every signal has a frequency spectrum. • the signal defines the spectrum • the spectrum defines the signal

We can move back and forth between the time domain and the frequency domain without losing information

Time domain / Frequency domain

•  Some signals are easier to visualise in the frequency domain

•  Some signals are easier to visualise in the time domain

•  Some signals are easier to define in the time domain (amount of information needed)

•  Some signals are easier to define in the frequency domain (amount of information needed)

Fourier Transform is most useful tool for DSP

Fourier Transforms Examples

peaks correspond to the resonances of the vocal tract shape

they can be used to differentiate between sounds

in logarithmis units of dB

sound /i/ as in see

signal spectrum

cosine

added higher frequency component

sound /a/ as in father

in logarithmis units of dB

Back to our sound recognition problem:

Discrete Time Fourier Transform (DTFT)

What about sampled signal?

The DTFT is defined as the Fourier transform of the sampled signal. Define the sampled signal in the usual way:

Take Fourier transform directly

using the “sifting property of the δ-function to reach the last line

X (!) =!!

xne"jn!T .

X (!) =

x(t)e"j!tdt.

xs(t) = x(t)!!

"(t " nT )

Xs (!) =

xs(t)e"j!tdt

Xs (!) =

x(t)!!

"(t " nT )e"j!tdt

Xs (!) =!!

x(nT )e"j!nT

(a) Explain how the problems with computing the DTFT on a digitalcomputer can be overcome by choosing a uniformly spaced gripof N frequencies covering the range of ! from 0 to 2#/T andtruncating the summation in the expression above to just N datapoints; thus, determine the expression for the N -point DiscreteFourier Transform (DFT). [10%]

Xp =N"1!n=0

xne"j 2!N np, p = 0, . . . , N " 1

X (!) =!!

xne"jn!T .

X (!) =

x(t)e"j!tdt.

xs(t) = x(t)!!

"(t " nT )

Xs (!) =

xs(t)e"j!tdt

Xs (!) =

x(t)!!

"(t " nT )e"j!tdt

Xs (!) =!!

x(nT )e"j!nT

Xp =N"1!n=0

xne"j 2!N np, p = 0, . . . , N " 1

X (!) =!!

xne"jn!T .

X (!) =

x(t)e"j!tdt.

xs(t) = x(t)!!

"(t " nT )

Xs (!) =

xs(t)e"j!tdt

Xs (!) =

x(t)!!

"(t " nT )e"j!tdt

Xs (!) =!!

x(nT )e"j!nT

Xp =N"1!n=0

xne"j 2!N np, p = 0, . . . , N " 1

X (!) =!!

xne"jn!T .

X (!) =

x(t)e"j!tdt.

xs(t) = x(t)!!

"(t " nT )

Xs (!) =

xs(t)e"j!tdt

Xs (!) =

x(t)!!

"(t " nT )e"j!tdt

Xs (!) =!!

x(nT )e"j!nT

Xp =N"1!n=0

xne"j 2!N np, p = 0, . . . , N " 1

X (!) =!!

xne"jn!T .

X (!) =

x(t)e"j!tdt.

xs(t) = x(t)!!

"(t " nT )

Xs (!) =

xs(t)e"j!tdt

Xs (!) =

x(t)!!

"(t " nT )e"j!tdt

Xs (!) =!!

x(nT )e"j!nT

Xp =N"1!n=0

xne"j 2!N np, p = 0, . . . , N " 1

X (!) =!!

xne"jn!T .

X (!) =

x(t)e"j!tdt.

xs(t) = x(t)!!

"(t " nT )

Xs (!) =

xs(t)e"j!tdt

Xs (!) =

x(t)!!

"(t " nT )e"j!tdt

Xs (!) =!!

x(nT )e"j!nT

Xp =N"1!n=0

xne"j 2!N np, p = 0, . . . , N " 1

Discrete Time Fourier Transform – Signal Samples

Note that this expression known as DTFT is a periodic function of the frequency usually written as

The signal sample values may be expressed in terms of DTFT by noting that the equation above has the form of Fourier series (as a function of ω) and hence the sampled signal can be obtained directly as

[You can show this for yourself by first noting that (*) is a complex Fourier series with coefficients however it is also covered in one of Part IB Examples Papers]

X (!) =!!

xne"jn!T .

X (!) =

x(t)e"j!tdt.

xs(t) = x(t)!!

"(t " nT )

Xs (!) =

xs(t)e"j!tdt

Xs (!) =

x(t)!!

"(t " nT )e"j!tdt

Xs (!) =!!

x(nT )e"j!nT

Xp =N"1!n=0

xne"j 2!N np, p = 0, . . . , N " 1

X (!) =!!

xne"jn!T .

X (!) =

x(t)e"j!tdt.

xs(t) = x(t)!!

"(t " nT )

Xs (!) =

xs(t)e"j!tdt

Xs (!) =

x(t)!!

"(t " nT )e"j!tdt

Xs (!) =!!

x(nT )e"j!nT

X(!)e+jn!Td!T

Xp =N"1!n=0

xne"j 2!N np, p = 0, . . . , N " 1

Computing DTFT on Digital Computer

The DTFT

expresses the spectrum of a sampled signal in terms of the signal samples but is not computable on a digital computer for two reasons:

1.  The frequency variable ω is continuous. 2.  The summation involves an infinite number of

samples.

X (!) =!!

xne"jn!T .

X (!) =

x(t)e"j!tdt.

xs(t) = x(t)!!

"(t " nT )

Xs (!) =

xs(t)e"j!tdt

Xs (!) =

x(t)!!

"(t " nT )e"j!tdt

Xs (!) =!!

x(nT )e"j!nT

Xp =N"1!n=0

xne"j 2!N np, p = 0, . . . , N " 1

Overcoming problems with computing DTFT

The problems with computing DTFT on a digital computer can be overcome by:

Step 1. Evaluating the DTFT at a finite collection of discrete frequencies.

no undesirable consequences, any frequency of interest can always be included in the collection

Step 2. Performing the summation over a finite number of data points

does have consequences since signals are generally not of finite duration

The Discrete Fourier Transform (DFT)

The discrete set of frequencies chosen is arbitrary. However, since the DTFT is periodic we generally choose a uniformly spaced grid of N frequencies covering the range ωT from 0 to 2π. If the summation is then truncated to just N data points we get the DFT

The inverse DFT can be used to obtain the sampled signal values from the DFT: multiply each side by and sum over p=0 to N-1

Orthogonality property of complex exponentials

is N if n=q and 0 otherwise

Xp = X(2"

N!1!n=0

xne!j 2!N np, p = 0, . . . , N ! 1

(c) Assume you are interested in detecting the presence of a singlecontinuous-wave sinusoidal tone in a time-domain data sequence.Show how the 64-point DFT of {xn} can be computed at thediscrete frequency of 30kHz if the sampling rate of {xn} was fs =128kHz. Determine the total number of complex multiplicationsrequired. [20%].

Answer. 30 kHz corresponds to p = 15 since 15128kHz64 = 30kHz

(a uniformly spaced grip of N = 64 frequencies covering the rangeof ! from 0 to 2"/T is considered, where fs = 1/T = 128kHz )and one obtains

X15 =63!

xne!j 2!64

There are 63 complex multiplications (for n = 0 we multiply by1).

(d) The DFT is typically implemented using a Fast Fourier Transform(FFT) algorithm. Assuming N is even, split the summation in thebasic DFT equation into two parts: one for even n and one forodd n, and then show that the DFT values Xp and Xp+N/2 maybe expressed as

Xp = Ap + W pBp

Xp+N/2 = Ap ! W pBp

Derive Ap, Bp and W in the above expression, and thus define theFFT “butterfly” structure [40%]

Xp = X(2"

N!1!n=0

xne!j 2!N np, p = 0, . . . , N ! 1

(c) Assume you are interested in detecting the presence of a singlecontinuous-wave sinusoidal tone in a time-domain data sequence.Show how the 64-point DFT of {xn} can be computed at thediscrete frequency of 30kHz if the sampling rate of {xn} was fs =128kHz. Determine the total number of complex multiplicationsrequired. [20%].

X15 =63!

xne!j 2!64

(d) The DFT is typically implemented using a Fast Fourier Transform(FFT) algorithm. Assuming N is even, split the summation in thebasic DFT equation into two parts: one for even n and one forodd n, and then show that the DFT values Xp and Xp+N/2 maybe expressed as

Xp = Ap + W pBp

Xp+N/2 = Ap ! W pBp

Derive Ap, Bp and W in the above expression, and thus define theFFT “butterfly” structure [40%]

Xp = X(2"

N!1!n=0

xne!j 2!N np, p = 0, . . . , N ! 1

N!1!p=0

Xpej 2!

N pq =N!1!p=0

N!1!n=0

xne!j 2!

N npp ej 2!

N pq =N!1!n=0

N!1!p=0

N (q!n)pp

(d) Assume you are interested in detecting the presence of a singlecontinuous-wave sinusoidal tone in a time-domain data sequence.Show how the 64-point DFT of {xn} can be computed at thediscrete frequency of 30kHz if the sampling rate of {xn} was fs =128kHz. Determine the total number of complex multiplicationsrequired. [20%].

X15 =63!

xne!j 2!64

(e) The DFT is typically implemented using a Fast Fourier Transform(FFT) algorithm. Assuming N is even, split the summation in thebasic DFT equation into two parts: one for even n and one forodd n, and then show that the DFT values Xp and Xp+N/2 maybe expressed as

Xp = Ap + W pBp

Xp = X(2"

N!1!n=0

xne!j 2!N np, p = 0, . . . , N ! 1

N!1!p=0

Xpej 2!

N pq =N!1!p=0

N!1!n=0

xne!j 2!

N npp ej 2!

N pq =N!1!n=0

N!1!p=0

N (q!n)pp

X15 =63!

xne!j 2!64

Xp = Ap + W pBp

Xp = X(2"

N!1!n=0

xne!j 2!N np, p = 0, . . . , N ! 1

N!1!p=0

Xpej 2!

N pq =N!1!p=0

N!1!n=0

xne!j 2!

N npp ej 2!

N pq =N!1!n=0

N!1!p=0

N (q!n)pp

X15 =63!

xne!j 2!64

Xp = Ap + W pBp

Xp = X(2"

N!1!n=0

xne!j 2!N np, p = 0, . . . , N ! 1

N!1!p=0

Xpej 2!

N pq =N!1!p=0

N!1!n=0

xne!j 2!

N npp ej 2!

N pq =N!1!n=0

N!1!p=0

N (q!n)pp

X15 =63!

xne!j 2!64

Xp = Ap + W pBp

The Discrete Fourier Transform Pair

•  is periodic, for each p

•  is periodic, for each n

•  for real data

[You should check that you can show these results from first principles]

Properties of the Discrete Fourier Transform (DFT)

Xp = X(2"

N!1!n=0

xne!j 2!N np, p = 0, . . . , N ! 1

N!1!p=0

Xpej 2!

N pq =N!1!p=0

N!1!n=0

xne!j 2!

N npp ej 2!

N pq =N!1!n=0

N!1!p=0

N (q!n)pp

Xp+N = Xp

xn+N = xn

Xp = X"

!p = X"

(f) Assume you are interested in detecting the presence of a singlecontinuous-wave sinusoidal tone in a time-domain data sequence.Show how the 64-point DFT of {xn} can be computed at thediscrete frequency of 30kHz if the sampling rate of {xn} was fs =128kHz. Determine the total number of complex multiplicationsrequired. [20%].

X15 =63!

xne!j 2!64

Xp = X(2"

N!1!n=0

xne!j 2!N np, p = 0, . . . , N ! 1

N!1!p=0

Xpej 2!

N pq =N!1!p=0

N!1!n=0

xne!j 2!

N npp ej 2!

N pq =N!1!n=0

N!1!p=0

N (q!n)pp

Xp+N = Xp

xn+N = xn

Xp = X"

!p = X"

X15 =63!

xne!j 2!64

Xp = X(2"

N!1!n=0

xne!j 2!N np, p = 0, . . . , N ! 1

N!1!p=0

Xpej 2!

N pq =N!1!p=0

N!1!n=0

xne!j 2!

N npp ej 2!

N pq =N!1!n=0

N!1!p=0

N (q!n)pp

Xp+N = Xp

xn+N = xn

Xp = X"

!p = X"

X15 =63!

xne!j 2!64

Xp = X(2"

N!1!n=0

xne!j 2!N np, p = 0, . . . , N ! 1

N!1!p=0

Xpej 2!

N pq =N!1!p=0

N!1!n=0

xne!j 2!

N npp ej 2!

N pq =N!1!n=0

N!1!p=0

N (q!n)pp

Xp+N = Xp

xn+N = xn

Xp = X"

!p = X"

X15 =63!

xne!j 2!64

Xp = X(2"

N!1!n=0

xne!j 2!N np, p = 0, . . . , N ! 1

N!1!p=0

Xpej 2!

N pq =N!1!p=0

N!1!n=0

xne!j 2!

N npp ej 2!

N pq =N!1!n=0

N!1!p=0

N (q!n)pp

Xp+N = Xp

xn+N = xn

Xp = X"

!p = X"

X15 =63!

xne!j 2!64

Xp = X(2"

N!1!n=0

xne!j 2!N np, p = 0, . . . , N ! 1

N!1!p=0

Xpej 2!

N pq =N!1!p=0

N!1!n=0

xne!j 2!

N npp ej 2!

N pq =N!1!n=0

N!1!p=0

N (q!n)pp

Xp+N = Xp

xn+N = xn

Xp = X"

!p = X"

X15 =63!

xne!j 2!64

DFT Interpolation

Zero padding

Padded sequence

Zero-padding

just visualisation, not additional information!

Circular Convolution

circular convolution

Example of Circular Convolution

Circular convolution of x1={1,2,0} and x2={3,5,4} clock-wise anticlock-wise

3 5 0 spins 1 spin 2 spins

y(0)=1×3+2×4+0×5 y(1)=1×5+2×3+0×4 y(2)=1×4+2×5+0×3

folded sequence

x1(n)x2(0-n)|mod3 x1(n)x2(1-n)|mod3 x1(n)x2(2-n)|mod3

clock-wise anticlock-wise

Standard Convolution using Circular Convolution

clock-wise anticlock-wise 1

0 spins

y(0)=1×3+2×0+0×4+0×5

folded sequence

x1(n)x2(0-n)|mod3 x1(n)x2(1-n)|mod3 x1(n)x2(2-n)|mod3

4 3 1 spin

0 5 2 spins

0 y(0)=1×5+2×3+0×0+0×4 y(0)=1×4+2×5+0×3+0×0

Standard Convolution using Circular Convolution

Proof of Validity

Circular convolution of the padded sequence corresponds to the standard convolution

Linear Filtering using the DFT

FIR filter:

DFT and then IDFT can be used to compute standard convolution product and thus to perform linear filtering.

Frequency domain equivalent:

Summary So Far

•  Fourier analysis for periodic functions focuses on the study of Fourier series

•  The Fourier Transform (FT) is a way of transforming a continuous signal into the frequency domain

•  The Discrete Time Fourier Transform (DTFT) is a Fourier Transform of a sampled signal

•  The Discrete Fourier Transform (DFT) is a discrete numerical equivalent using sums instead of integrals that can be computed on a digital computer

•  As one of the applications DFT and then Inverse DFT (IDFT) can be used to compute standard convolution product and thus to perform linear filtering

Fast Fourier Transform (FFT)

Computations for Evaluating the DFT

•  Discrete Fourier Transform and its Inverse can be computed on a digital computer

•  What are computational requirements?

Operation Count

How do we evaluate computational complexity? count the number of real multiplications and

additions by ¨real¨ we mean either fixed-point or floating point

operations depending on the specific hardware

subtraction is considered equivalent to additions divisions are counted separately

Other operations such as loading from memory, storing in memory, loop counting, indexing, etc are not counted (depends on implementation/architecture) – present overhead

Operation Count – Modern DSP

Modern DSP: a real multiplication and addition is a single machine

cycle y=ax+b called MAC (multiply/accumulate)

maximum number of multiplications and additions must be counted, not their sum

traditional claim ¨multiplication is much more time- consuming than addition¨ becomes obsolete

Discrete Fourier Transform as a Vector Operation

Again, we have a sequence of sampled values x and transformed values X , given by

Let us denote:

X 0 0x 0 0 0 0…X 1 1x 0 1 2 N-1 …

…… …… … … …

…X N-1 0 N-1 2(N-1) (N-1) 2 x N-1

This Equation can be rewritten in matrix form:

as follows:

Matrix Interpretation of the DFT

N x N matrix

complex numbers

can be computed off-line and stored

DFT Evaluation – Operation Count

Multiplication of a complex N x N matrix by a complex N-dimensional vector.

2 Total: 4(N-1) real ¨x¨s & 4(N-0.5)(N-1) real ¨+¨s

Complex ¨x¨: 4 real ¨x¨s and 2 real ¨+¨s; complex ¨+¨ : 2 real ¨+¨s

2 First row and first column are 1 however

save 2N-1 ¨x¨s, (N-1) complex ¨x¨s & N(N-1) ¨+¨s

No attempt to save operations N complex multiplications (¨x¨s)

N(N-1) complex additions (¨+¨s)

DFT Computational Complexity

Thus, for the input sequence of length N the number of arithmetic operations in direct computation of DFT is proportional to N . 2

For N=1000, about a million operations are needed!

In 1960s such a number was considered prohibitive in most applications.

Discovery of the Fast Fourier Transform (FFT)

When in 1965 Cooley and Tukey ¨first¨ announced discovery of Fast Fourier Transform (FFT) in 1965 it revolutionised Digital Signal Processing.

They were actually 150 years late – the principle of the FFT was later discovered in obscure section of one of Gauss’ (as in Gaussian) own notebooks in 1806.

Their paper is the most cited mathematical paper ever written.

The FFT is a highly elegant and efficient algorithm, which is still one of the most used algorithms in speech processing, communications, frequency estimation, etc – one of the most highly developed area of DSP.

There are many different types and variations.

They are just computational schemes for computing the DFT – not new transforms!

Here we consider the most basic radix-2 algorithm which requires N to be a power of 2.

The Fast Fourier Transform (FFT)

Lets take the basic DFT equation:

Now, split the summation into two parts: one for even n and one for odd n:

Take outside

FFT Derivation 1

Thus, we have:

FFT Derivation 2

Now, take the same equation again where we split the summation into two parts: one for even n and one for odd n:

And evaluate at frequencies p+N/2

FFT Derivation 3

FFT Derivation 4

simplified

FFT Derivation 5

FFT Derivation - Butterfly

Or, in more compact form:

(‘Butterfly’)

FFT Derivation - Redundancy

FFT Derivation - Computational Load

Flow Diagram for a N=8 DFT

Input: Output:

Flow Diagram for a N=8 DFT - Decomposition

Flow Diagram for a N=8 DFT – Decomposition 2

Bit-reversal

FFT Derivation Summary

•  The FFT derivation relies on redundancy in the calculation of the basic DFT

•  A recursive algorithm is derived that repeatedly rearranges the problem into two simpler problems of half the size

•  Hence the basic algorithm operates on signals of length a power of 2, i.e.

(for some integer M)

•  At the bottom of the tree we have the classic FFT `butterfly’ structure

Computational Load of full FFT algorithm

The type of FFT we have considered, where N = 2M, is called a radix-2 FFT.

It has M = log2 N stages, each using N / 2 butterflies

Complex multiplication requires 4 real multiplications and 2 real additions

Complex addition/subtraction requires 2 real additions

Thus, butterfly requires 10 real operations.

Hence the radix-2 N-point FFT requires 10( N / 2 )log2 N real operations compared to about 8N2 real operations for the DFT.

Computational Load of full FFT algorithm

Direct DFT

The radix-2 N-point FFT requires 10( N / 2 )log2 N real operations compared to about 8N2 real operations for the DFT.

This is a huge speed-up in typical applications, where N is 128 – 4096:

Input Output

Further advantage of the FFT algorithm

The Inverse FFT (IFFT)

The IDFT is different from the DFT:

•  it uses positive power of instead of negative ones •  There is an additional division of each output value by N

Any FFT algorithm can be modified to perform the IDFT by •  using positive powers instead of negatives •  Multiplying each component of the output by 1 / N

Hence the algorithm is the same but computational load increases due to N extra multiplications

The form of FFT we have described is called “decimation in time”; there is a form called “decimation in frequency” (but it has no advantages).

The "radix 2" FFT must have length N a power of 2. Slightly more efficient is the "radix 4" FFT, in which 2-input 2-output butterflies are replaced by 4-input 4-output units. The transform length must then be a power of 4 (more restrictive).

A completely different type of algorithm, the Winograd Fourier Transform Algorithm (WFTA), can be used for FFT lengths equal to the product of a number of mutually prime factors (e.g. 9*7*5 = 315 or 5*16 = 80). The WFTA uses fewer multipliers, but more adders, than a similar-length FFT.

Efficient algorithms exist for FFTing real (not complex) data at about 60% the effort of the same-sized complex-data FFT.

The Discrete Cosine and Sine Transforms (DCT and DST) are similar real-signal algorithms used in image coding.

Many types of FFT

There FFT is surely the most widely used signal processing algorithm of all. It is the basic building block for a large percentage of algorithms in current usage

Specific examples include: •  Spectrum analysis – used for analysing and detecting signals •  Coding – audio and speech signals are often coded in the frequency domain using FFT variants (MP3, …) •  Another recent application is in a modulation scheme called OFDM, which is used for digital TV broadcasting (DVB) and digital radio (audio) broadcasting (DAB). •  Background noise reduction for mobile telephony, speech and audio signals is often implemented in the frequency domain using FFTs

Applications of the FFT

Practical Spectral Analysis

•  Say, we have a symphony recording over 40 minutes long – about 2500 seconds

•  Compact-disk recordings are sampled at 44.1 kHz and are in stereo, but say we combine both channels into a single one by summing them at each time point

•  Are we to compute the DFT of a sequence one hundred million samples long?

a)  unrealistic – speed and storage requirements are too high

b)  useless – we will get a very much noiselike wide-band spectrum covering the range from 20 Hz to 20 kHz at high resolution including all notes of all instruments with their harmonics

•  We actually want a sequence of short DFTs, each showing the spectrum of a signal of relatively short interval

•  If violin plays note E during this interval – spectrum would exhibit energies at not the frequency of note E (329 Hz) and characteristic harmonics of the violin

•  If the interval is short enough – we may be able to track not-to-note changes

•  If the frequency resolution is high enough – we will be able to identify musical instruments playing

Remember: our – ear – excellent spectrum analyser?

Short-Time Spectral Analysis

•  If we take a time interval of about 93 milliseconds (signal of length 4096) the frequency resolution will be 11 Hz (adequate)

•  The number of distinct intervals in the symphony is 40 x 60/0.093 - about 26,000

•  We may choose to be conservative and overlap the intervals (say 50%)

•  In total, need to compute 52,000 FFTs of length 4096

the entire computation will be done in considerably less time than the music itself

Short-time spectral analysis also use windowing (more on this later)

Short-Time Spectral Analysis – Computational Load

Case Study: Spectral analysis of a Musical Signal

Extract a short segment

Looks almost periodic over short time interval

Sample rate is 10.025 kHz (T=1/10,025 s)

Load this into Matlab as a vector x

Take an FFT, N=512

X=fft(x(1:512));

Note Conjugate symmetry as data are real: Symmetric

Symmetric Anti-Symmetric

Case Study: Spectral analysis of a Musical Signal, FFT

Case Study: Spectral analysis of a Musical Signal - Frequencies

The Effect of data length N

N=1024

Low resolution

High resolution

The DFT approximation to the DTFT

DTFT at frequency : DFT:

•  Ideally the DFT should be a `good’ approximation to the DTFT

•  Intuitively the approximation gets better as the number of data points N increases

•  This is illustrated in the previous slide – resolution gets better as N increases (more, narrower, peaks in spectrum).

•  How to evaluate this analytically? View the truncation in the summation as a multiplication by a

rectangle window function

Analysis

Pre-multiplying by rectangular window

Spectrum of the windowed signal

Fourier Transform of the Rectangular Window

N=32 Central `Lobe’

Sidelobes

Rectangular Window Spectrum

Lobe width inversely Proportional to N

Rectangular Window Spectrum – Lobe Width

Now, imagine what happens when the sum of two frequency components is DFT-ed:

The DTFT is given by a train of delta functions:

Hence the windowed spectrum is just the convolution of the window spectrum with the delta functions:

Both components separately

Both components Together

DFT for the data

Brief summary

•  The rectangular window introduces broadening of any frequency components (`smearing’) and sidelobes that may overlap with other frequency components (`leakage’).

•  The effect improves as N increases •  However, the rectangle window has poor properties and better choices

of wn can lead to better spectral properties (less leakage, in particular) – i.e. instead of just truncating the summation, we can pre-multiply by a suitable window function wn that has better frequency domain properties.

•  More on window design in the filter design section of the course – see later

Summary so far … •  The Fourier Transform (FT) is a way of transforming

a continuous signal into the frequency domain

•  The Discrete Time Fourier Transform (DTFT) is a Fourier Transform of a sampled signal

•  The Discrete Fourier Transform (DFT) is a discrete numerical equivalent using sums instead of integrals that can be computed on a digital computer

•  The FFT is a highly elegant and efficient algorithm, which is still one of the most used algorithms in speech processing, communications, frequency estimation, etc – one of the most highly developed area of DSP

Thank you!

3f3 Digital Signal Processing Part1 1232329900875209 1

Documents

Transcript of 3f3 Digital Signal Processing Part1 1232329900875209 1

3F3 Digital Filters 2011

3F3 1 Introduction to DSP

3F3 7 Implementation of Digital Filters

Synthesis part1

Fascialplasticity–anew neurobiologicalexplanation: Part1€¦ · Fascialplasticity–anew neurobiologicalexplanation: Part1..... RobertSchleip Inmyofascialmanipulationanimmediatetissuereleaseisoftenfeltundertheworking

Jscript part1

3F3 – Digital Signal Processing (DSP)

S1002 PRO mz-12PRO part1 EN those which are additionally marked with the words CAU-TION or WARNING. The signal word WARNING indicates the poten - tial for serious injury, the signal

Xhtml Part1

Socialmediamagic Part1

Part1 Yesuf

3F3 Tech Narrative (1)

Webinar Presentation TIR Amendments Part1 April 11 2011 Part1

3F3 4 Basics of Digital FIlters

DSP Digital Signal Processing MODULE III PART1

3F3 - Random Processes, Optimal Filtering and Model … · 3F3 - Random Processes, Optimal Filtering and Model-based ... Random Processes Stationarity A stationary process has the

California part1

3F3 – Digital Signal Processing (DSP) - Part1

Tools for Signal Compression€¦ · Table of Contents Introduction..... xi PART1.TOOLS FORSIGNALCOMPRESSION..... 1 Chapter 1. Scalar Quantization..... 3 1.1. Introduction ...

3F3: Signal and Pattern Processinglearning.eng.cam.ac.uk/zoubin/cam-3f3-06/lect1.pdf · • Machine learning is an interdisciplinary ﬁeld focusing on both the mathematical foundations