Wavelets - University of California, Los...
Transcript of Wavelets - University of California, Los...
Dejan [email protected]
Wavelets
EE219A – Spring 2008Special Topics in Circuits and Signal Processing
Lecture 15
Slide 2EE219A – Spring 2008 Lecture 15
Training
Full-day event starting at 9:30am– Break 2-4pm (class)
Training Agenda– Basic PRO– Advanced PRO techniques– Basic Identify– Synplify DSP
Hands-on experience– Labs for each of the four sections– System example: IEE802.11a system with DSP blocks– All-hands-on DSP with a little competition for prizes
Wavelets
Slide 4EE219A – Spring 2008 Lecture 15
Development of Practical IdeasFrom theory to practice– Step 1: mathematicians develop theories (that only a few
people are able to understand at first)– Step 2: physicists and engineers (those who also get to
understand it) adapt the theory to make it more accessible– Step 3: the theory is applied in practice by researchers in
many different fieldsExample– Step 1: in 1807, Joseph Fourier discovered that all
periodic functions can be expressed as a weighted sum of trigonometric functions. Strong opposition, took him 15 years to publish his ideas!
– Step 2: over the next 150 years the ideas were expanded to non-periodic functions and discrete-time sequences
– Step 3: in 1965, the FFT was derived and finds use in EE and many other disciplines that require function analysis
Slide 5EE219A – Spring 2008 Lecture 15
Another Example: Wavelets
Some good reasons for wavelets– Many interesting applications where no other transform
has been applied (because it didn’t work)– Now that we have finally adopted Fourier’s theory, we
need something else to grind about…
So, what did we learn from Fourier? (so we understand the limitations, or save some time grinding…)– A complex function can be approximated with a weighted
sum of basis functions– Fourier used sinusoids with varying frequencies as the
basis functions– This representation provides the frequency content of the
original function
Slide 6EE219A – Spring 2008 Lecture 15
What’s Wrong with the Fourier Transform?Uses sinusoids as basis functions
These are not compactly supported in time domain– Stretch out in time to infinity, so they cannot be used to
approximate non-stationary signalsFourier transform assumes all spectral components are present at all times!– Time domain representation does not provide information
about spectral content of the signalLimitation: non-stationary signals whose spectral content changes in time cannot be supported by FT
Slide 7EE219A – Spring 2008 Lecture 15
Support for Non-stationary SignalsA work-around: modify Fourier transform to allow analysis of non-stationary signals by slicing in time (short-time Fourier transform)– Segment in time by applying windowing functions and
analyzing each segment separately– Many approaches (between late 1940s and early 1970s)
differing in the choice of windowing functionsSTFT, thus, provides full time-frequency representation?Not really, there is still a problem: all approaches used the same window for the analysis of the entire signal– Not applicable to signals that have
● High frequency components with short time spans● Low frequency components with long time spans
– STFT can do high frequency analysis by shortening time window or low frequency analysis by widening the window, but it cannot do both
Slide 8EE219A – Spring 2008 Lecture 15
Back to Applications
Wavelets (to be defined) are a nice example of how practical problems drive theory development– High frequency short time / low frequency long time type
of signals actually occur in geophysics
In 1970s, J. Morlet, a geophysicist solved the problem– He proposed using a different window functions for
analyzing different frequency bands– These windows were generated in a special way:
by dilation or compression of a prototype Gaussian!● Couldn’t argue with Gauss in the 1970s (well established by then),
so you’d expect Morlet to have much easier time than Fourier● Also, Morlet was a practitioner, so skip step 1 and some of step 2● None of the above actually happened, because mathematicians
didn’t like the idea!
Slide 9EE219A – Spring 2008 Lecture 15
Wavelets
The window functions (dilation or compression of a Gaussian) have compact support both in time and frequency– Fourier transform of a Gaussian is also Gaussian
Due to the “small and oscillatory” nature of the window functions, Morlet named his basis functions as wavelets of constant shape
In early 1980s, inverse function was derived– With help from A. Grossman, a theoretical physicist of
quantum mechanics– A re-discovery of Alberto Calderon’s 1964 work on
harmonic analysis
Slide 10EE219A – Spring 2008 Lecture 15
Now, Back to Mathematicians
In 1984, Yves Meyer, a French mathematician, noticed the similarity between Morlet’s and Calderon’s work– Also noticed lot so redundancy in Morlet’s choice of basis
functions, then known as wavelets– He started developing wavelets with better localization
propertiesIn 1985, Meyer constructed orthogonal wavelet basis functions– It turned out that another harmonic analyst, Stromberg,
had discovered the same wavelets about 5 years before– Neither Meyer nor Stromberg, however, have actually
discovered orthogonal wavelet basis functions– The discovery honor goes to German mathematician
Alfred Haar who did it back in 1909!
Slide 11EE219A – Spring 2008 Lecture 15
Wavelets (Cont.)
First and simplest orthogonal wavelets (wavelets = wavelet basis functions)– Little practical use due to limited frequency localization
Another important wavelet person: Ingrid Daubechies– Developed wavelet frames for discretization of time snd
scale parameters– More freedom in the choice of wavelets at the expense of
some redundancy– Ingdid Daubechies and Stehpane Mallat are credited for
developing the transition from continuous to discrete signal analysis
In 1986, Mallat developed the idea of multi-resolution analysis for discrete wavelet transform (DWT)– Ph.D. dissertation (U-Penn, 1988)
Slide 12EE219A – Spring 2008 Lecture 15
Mallat and More Recent Work
Decomposing a discrete signal into its dyadic frequency bands by a series of low-pass and high-pass frequency filters to compute its DWT from the approximations at these various scales– Not surprisingly, the idea was familiar to EEs, for about 20
years, as quadrature mirror filters (QMF) and sub-band filtering developed in the mid-1970s
– Mallat’s work was extension of time localization to the idea of QMF and sub-band coding
– Together with Daubechies’ orthonormal wavelets (1988), Mallat’s work laid foundation of modern wavelets
In the past 20 years– Search for other wavelets with different properties
● Biorthogonal wavelets (Cohen, Feauveau, Daubechies, 1992)● Wavelet packets (Coifman, Meyer, Wickerhauser)
Slide 13EE219A – Spring 2008 Lecture 15
Example
)100*2cos()50*2cos()25*2cos()10*2cos()(1 tttttx Π+Π+Π+Π=
1000ms t800ms )10*cos(2 800ms t600ms )25*cos(2 600ms t300ms )50*cos(2
300mst0 )100*2cos()(2
≤<Π≤<Π≤<Π
≤<Π=
tttttx
Fourier transform, X1(f) Fourier transform, X2(f)
Not all spectralcomponents exist at all times
Slide 14EE219A – Spring 2008 Lecture 15
Time-Frequency Representation
Short Time Fourier Transform (STFT)
– w(t) is the windowing functionMultiply signal by a window and then take a F.T. of the result (segment into stationary short-enough pieces)– S(τ,f) is STFT of x(t) at frequency f and translation τ
Translate the window to get spectral content of signal at different frequencies– w(t-τ) does time-segmentation of x(t)
Haven’t we solved the problem?
translation parameter
Slide 15EE219A – Spring 2008 Lecture 15
Heisenberg's Uncertainty PrincipleThe problem with previous approach is uniform resolution for all frequencies (same window for x(t))There exists a tradeoff between the resolution in time and frequency domain– If the signal has high frequency components for a short
time span, a narrow window is needed for time resolution, but this results in wider frequency bands (poor freq. res.)
– If the signal has low frequency components of longer time span, a wider window need to be used for good frequency resolution (but, with poor time resolution)
This is yet another demonstration of Heisenberg's uncertainty principle– F.T. is an extreme case where all time domain information
is lost to get precise frequency information– Thus, STFT offers fixed resolution which needs to be
chosen keeping the above tradeoff in mind
Slide 16EE219A – Spring 2008 Lecture 15
Continuous Wavelet TransformVarying time and frequency resolutions by using windows of different lengths– First decompose in freq domain– Analyze the signal in time
– a > 0, b: scale and translation parameters
Morlet’s Choice: Gaussian wavelets
Wave: Oscillatory; let: compact support– “small wave”
Redundant representation of signal
Search for orthonormal wavelet basis– Meyer Wavelet / Haar Wavelet Morlet Wavelet
Mother wavelet
Slide 17EE219A – Spring 2008 Lecture 15
Fourier basis functions Wavelet basis functions
Fourier vs. Wavelet Transform
Slide 18EE219A – Spring 2008 Lecture 15
Discrete Wavelet SeriesWe need discrete domain transforms
Discretize the translation and scale parameters (a, b)– Daubechies (a=2j, b=2jk)
Input signal is still continuous
Number of wavelets in the family is still infinite
Slide 19EE219A – Spring 2008 Lecture 15
Towards Practical Implementations…How to limit the number of scales for analysis?
Each wavelet is a like a constant – Q filter
Scaling function
Slide 20EE219A – Spring 2008 Lecture 15
Mallat’s MRARelation between scaling and wavelet functions at different frequency resolutions
Describes wavelet transform in terms of digital filtering and sampling operations– Iterative use of low-pass and high-pass filters,
subsequent down-sampling by 2x
Leads to an easier implementation where wavelets are abstracted away!
Slide 21EE219A – Spring 2008 Lecture 15
Mallat’s DWT
The popular form of DWT
HP LP
HP
LP
Slide 22EE219A – Spring 2008 Lecture 15
Applications
Spike Sorting
FBI finger prints
Image compression
Generality of the WT lets us take a pick for the wavelet used
Slide 23EE219A – Spring 2008 Lecture 15
Computer and Human VisionDavid Marr tried to find an answer to “Why the first attempts to construct a robot capable of understanding its surroundings were unsuccessful?”
He speculated that intensity changes occur at different scales in an image and sudden intensity changes produce a peak or two in the first derivative of the image
Vision filter has two characteristics– It should be a differential operator– It should be capable of being turned to act at any desired
scale
Marr Wavelet
Slide 24EE219A – Spring 2008 Lecture 15
1924-1995, the US FBI has collected about 30 million sets of fingerprints.In 1993, the FBI’s Criminal Justice Information Services Division developed standards for fingerprint digitization and compression in cooperation with the National Institute of Standards and Technology, Los Alamos Nat. Lab, commercial vendors and criminal justice communitiesThe data compression standard WSQ (Wavelet/Scalar Quantization) is implemented
FBI Fingerprint Compression
Slide 25EE219A – Spring 2008 Lecture 15
David Donoho’s technique: Wavelet Shrinkage and Thresholding
Denoising Noisy Data
Slide 26EE219A – Spring 2008 Lecture 15