Machine Vision and Dig. Image Analysis 1 Prof. Heikki Kälviäinen CT50A6100 Lectures 3: Image...

Machine Vision and Dig. Image Analysis

1 Prof. Heikki Kälviäinen CT50A6100

Lectures 3:Image Transforms

Professor Heikki Kälviäinen

Machine Vision and Pattern Recognition Laboratory Department of Information Technology

Faculty of Technology ManagementLappeenranta University of Technology (LUT)

[email protected]://www.lut.fi/~kalviai

http://www.it.lut.fi/ip/research/mvpr/


Prof. Heikki Kälviäinen CT50A6100

2

Content

• Fourier Transform. – Discrete Fourier Transform. – Properties of 2-D Fourier Transform.

• Separability, periodicity, translation, rotation, scaling, convolution, correlation.

– Fast Fourier Transform. – Fourier Transform for image processing and analysis.

• Gabor filtering. – 2-D Gabor filter.– Applications.

• Other transforms.



3

Image Transforms

Fourier TransformCosineSineHadamardHaarSlantKarhunen-LoeveFast KLSinusoidal SDVetc.



4

Fourier Transform

• Jean Baptiste Joseph Fourier. • In 1807:

Any periodic function can be expressed as the sum of sines and/or cosines of different frequencies, each multiplied by a different coefficient. • The sum is nowadays called a Fourier series.• An image is a 2D signal.

From: R.C. Gonzalez and R.E. Woods: Digital Image Processing, 3rd edition, 2008.



5



6



7

Images and Their Fourier Spectra

i = sqrt(-1)F(k) also denoted F(u)

F(x,y) also denoted F(u,v)



8



9

Discrete Fourier Transform (DFT)



10

Properties of 2-D Fourier Transform

• Separability.– 2-D can be computed as series of 1-D.

• Periodicity.• Translation, rotation, and scaling.

– Invariant features. • Convolution and correlation.

– Spatial filtering in the frequency domain. • Etc.



11



12



13



14



15

Fast Fourier Transform (FFT)

• Brute-force implementation of Fourier Transform requires on the order of (MN)^2 summations and additions.

• 1024 x 1024 pixels => the order of a trillion multiplications and summations for one DFT

=> computationally too heavy. • FFT => the order of MN log_2 MN. • Based on the successive doubling method.



16



17



18



19

Fourier Transform for Image Processing and Analysis

Properties of Fourier Transform offer for example the following use:• Feature extraction.

– Frequency domain features. • Image compression. • Image filtering.

– Image enhancement in the frequency domain. – Preprocessing.



20

Fourier Transform: Feature Extraction

• Fourier transforms of texture.• Regular patterns. • Feature extraction.



21

Fourier Transform: Image Compression

Radius (pixels) % Image power

8 95

16 97

32 98

64 99.4

128 99.8

Not so many coefficients needed => image compression.

Lossy/lossless compression?



22

Fourier Transform: Image Filtering

• Enhancement in the frequency domain. • Convolution: f(x)*g(x) F(u) G(u).• Ideal low pass filter: Original (left) and filtered image (right).



23

Enhancement in the Frequency Domain

• Ideal high pass filter:– Original (left) and filtered image (right).



24

Image Processing Using Gabor Filtering

• For local and global feature extraction. • Filtering in time (spatial) space and frequency space.• For image processing and analysis two important parameters:

– Frequency f. – Orientation theta.

• Application example: – Face detection and recognition.– FACEDETECT project (http://www.it.lut.fi/project/facedetect/):

• Machine Vision and Pattern Recognition Laboratory (MVPR), Department of Information Technology, LUT, Finland.

• Centre for Vision, Speech and Signal Processing (CVSSP), University of Surrey, UK.



25

Feature Detector: 2-D Gabor Filter

cossin'

sincos'

),( '2''22

2

22

2

2

yxy

yxx

eef

yx fxjy

fx

f



26

Gabor Features

• Maximal joint localization in the spatial and frequency domain.• Smooth and noise tolerant.• Parameters for invariance manipulation:Frequency Envelope sharpness Orientation



27

Constructing Response Matrix

Filter response r(x,y; f,) can be calculated for variousfrequencies f and orientations to construct a response matrix.

columns represent orientationsrows represent frequenciesimage rotation appears as acircular shift of the columns

image scaling appears as ashift of the rows (highfrequencies may vanish)

A SCALE AND ROTATION INVARIANTTREATMENT OF THE RESPONSE MATRIXCAN BE ESTABLISHED, AND THUS, WECAN CONCENTRATE ONLY HOW TOCLASSIFY THEM IN THE STANDARD POSE



28

2-D Gabor Features

discrete frequency [u]

dis

cre

te fr

eq

ue

ncy

[v]

-1/2 -1/4 0 1/4 1/2

-1/2

-1/4

0

1/4

1/2

What do they ”see”?



29

Face verification (authentication) Validating a claimed identity based on the image of a face: are you Mr./Ms. X?

Face recognition (identification)Identifying a person based on an image of his/her face: who are you?

Face detection/localizationLocation of human faces in images at different positions, scales, orientations, and lighting conditions.

Face Detection and Recognition



30

Proposed Algorithm

• Avoiding a scanning window.• Using feature detectors.• Shape-free texture model for the final decision.



31

Evidence Extraction

Requirements

• Scale invariant extraction.• Rotation invariant extraction.• Provides sufficiently small amount of correct candidate points. (n best points from each class; needs confidence measure).

Preferred

• Estimation of evidence scale and orientation.• Fast extraction (scalability).



32

Classifier Construction

eye

eye

nostrilnostril

eyeeye

Gaussian mixturemodel densities(EM estimation)

• Stability property guarantees approximately the Gaussian form of classes in the feature space.

• One class may still consist of several sub-clusters (open eye, closed eye, etc.).

Bayesianclassificationof features



33

Affine Learned Correspondences

Aligned images of objects andmanually selected features Variability and correspondences

1 2

3 4

5 6



34

Affine Hypothesis Search

2

2

3 11 12

4

5

2

2

1

1. Evidence extraction.

2. Affine search and match to correspon- dence model.

Instanceapproved

False alarms occur and hypothesisverification is needed



35

Face Space

• Normalization of space where shape variations and capture effects are removed from patterns.

• Based on three points on the face -> affine registration.

• Optimal with regard to the photometric variance over a big set of faces.



36

Features & Feature Detectors

• Features = salient parts of face.

• Small localization variance and frequent occurrence over population.

• Illumination, scale, rotation, and translation invariance.

• Automatic analysis using the face space desirable.



37



38



39



40



41



42

Other transforms

• DFT, Cosine, Sine, Hadamar, Haar, Slant, Karhunen-Loeve, Fast KL, Sinusoidal transform, SVD transform.

• Basis images: Haar (wavelets) (left), Hadamard (right).– Haar values: positive-black, negative-white, zero-gray– Hadamard values: one – black, minus one – white.



43

Summary

• Image transforms from the spatial domain into the frequency domain.• Fourier Transform.

– For overall image processing and analysis. – Feature extraction, image compression, and image filtering

(enhancement). – Discrete Fourier Transform (DFT) and Fast Fourier Transform (FFT). – Properties of 2-D Fourier Transform.

• Separability, periodicity, translation, rotation, scaling, convolution, correlation.

• Gabor filtering. – For local and global feature extraction. – Orientation and frequency.– 2-D Gabor filter.

• Other transforms. – For example, Cosine Transform for image compression. – More detailed information later in the course.

Machine Vision and Dig. Image Analysis 1 Prof. Heikki Kälviäinen CT50A6100 Lectures 3: Image...

Documents

Transcript of Machine Vision and Dig. Image Analysis 1 Prof. Heikki Kälviäinen CT50A6100 Lectures 3: Image...