Spatial Processes and Image Analysis

Yassir Moudden & Sandrine Pires

CEA/DAPNIA/SEDI-SAP

Lectures :

• Basic models and tools in signal and image processing.• Multiscale transforms : wavelets, ridgelets, curvelets, etc.• Multiresolution analysis and wavelet bases.• Noise modeling and image restoration.• Problems and methods in multispectral data analysis.

Basic models and tools in signal and image processing

Outline :• Different types of images• Sampling and quantification• Fourier transform, Power spectrum• Linear Filtering, convolution• Non-linear operators mathematical morphology• Statistical properties of images

Signals, Images, etc.

Quantitative dataOrganized in time or space

Different types of signals and images (1)

• Continuous or Discrete index.

• Continuous or Quantized values.

• Finite energy, finite power, etc.

Computer processing requires finite energy, discrete, quantized data.

Different types of signals and images (2)

• Signals and images grouped in terms of regularity properties : – Continuous and global order of differentiability– Local regularity– Fractal dimension

• Statistical properties :– Marginal distributions, moments, etc.– Coherence, correlations and non linear dependencies– Stationarity

Different formal environment for handling indexed data sets.

Periodic signals and images

• Fourier series expansion :

where :

• Plancherel-Parceval formula :

Signals and images as energy distributions in time or space …

• Energy :

• Localization :

• Spread :

• More detailed characterization : higher order moments of the energy distribuition.

… and in Fourier space :

• Fourier transform :

• Parceval :

• Localization :

• Spread :

• Heisenberg Uncertainty Principle :

A few properties of the Fourier transform

• examples :

(Poisson Sommation Formula)

Sampling : from continuous to discrete time (1)

• Ideal sampling : multiplication by a Dirac comb with rate Fs = 1/T.• Properties :

– Linear oprator.– Not shift invariant..

• Shannon-Nyquist sampling theorem : Given a uniform sampling rate of Fs = 1/T, the highest frequency that can be unambiguously represented is Fs/2.

• Reconstruction (interpolation) formula :

where

Sampling : from continuous to discrete time (2)

• Sampling in time “periodizes” in frequency space

resulting in aliasing.

In higher dimensions, separable sampling schemes are most commonly used. But there are other non trivial possibilities.

Linear operators - Filtering

• Simplest possible operators are linear. • Shift invariant linear operators = convolutive systems :

• Harmonic signals are eigenvectors of linear filters :

with

Example (1) : low pass spatial filter

• Used for smoothing (removal of small details prior to large object extraction, bridging small gaps in lines) and noise reduction.

• Low-pass (smoothing) spatial filtering– Neighborhood averaging– Results in spatial blurring

Example (2) : median filter (non-linear)

• Replace the current pixel value by the median pixel value in a given neighborhood.

• Achieves effective noise supression.

• Preserves the sharpness of real boundaries.

Mathematical morphology

• Two basic non-linear operators:– Dilation

– Erosion

• Several composite operators : – Closing

– Opening

– Conditionnal closing, etc.

• A strucutring element is used in each of these operations:

Dilation• Principle : takes the binary image B, places the origin of

structuring element S over each pixel of value 1, and ORs the structuring element S into the output image at the corresponding position.

• It is typically applied to binary image, but there are versions that work on gray scale image.

• The basic effect of the operator on a binary image is to gradually enlarge the boundaries of regions of foreground pixels (i.e. white pixels, typically).

• Thus areas of foreground pixels grow in size while holes within those regions become smaller.

example : dilation using a 3 by 3 square structuring element for gap bridging.

Erosion• Principle : takes the binary image B, places the origin of

structuring element S over each pixel of value 1, and ANDs the structuring element S into the output image at the corresponding position.

• It is typically applied to binary image, but there are versions that work on gray scale image.

• The basic effect of the operator on a binary image is to gradually eliminate small objects.

0 0 1 1 00 0 1 1 00 0 1 1 01 1 1 1 1

111

0 0 0 0 00 0 1 1 00 0 1 1 00 0 0 0 0

B S

origin

erode

B S

• Closing is a dilation followed by an erosion (with the same structuring element). Closing also produces the smoothing of sections of contours but fuses narrow breaks, fills gaps in the contour and eliminates small holes.

• Opening is an erosion followed by dilation (with the same structuring element). Opening smoothes the contours of objects, breaks narrow isthmuses and eliminates thin protrusions.

Closing and opening

Effect of closing using a 3 by 3 square structuring element

• Closing is a dilation followed by an erosion (with the same structuring element). Closing also produces the smoothing of sections of contours but fuses narrow breaks, fills gaps in the contour and eliminates small holes.

• Opening is an erosion followed by dilation (with the same structuring element). Opening smoothes the contours of objects, breaks narrow isthmuses and eliminates thin protrusions.

Closing and opening

Statistical signal and image processing • Another way to build classes of signal and image data.

• Doesn’t mean the signal or image data are stochastic.

• Means that our incomplete prior knowledge of what is noise and what is information requires a probabilistic framework for bayesian inference or maximum likelihood estimation.

• Many algorithms for image denoising, restoration etc. are in this general

framework : MEM, Wiener, shrinkage, detection.

• Prior probabilities express our knowledge of noise and signal.

NOISE = NOT STRUCTUREDSIGNAL = STRUCTURED

Statistical properties of signals and images

• A stochastic process/field is completely defined by its probability law

• Simplest model considers IID processes, isotropic, stationary

• How to account for coherent behaviour of neighboring (or not) samples or pixels, in a generic way?

• Different priors for differents classes of images.

• Gibbs-Markov fields• New representations of structured image data.

Gibbs-Markov field models for images

• The probability distribution of the value of pixel s does not depend on all the other pixels but only on those pixels in the considered neighborhood : sort range local interactions.

• example : Monte-Carlo simulations of an Ising model for different values of coupling.

Gibbs-Markov random fields in segmentation

Segmentation of satelite images of urban areas using MRF.

Multiscale transforms : wavelets, ridgelets, curvelets, etc.

Outline :• The Fourier transform• Transient world and singularities : Gibbs effect, regularity• Time-frequency analysis and the Heisenberg principle• Optimal spatiospectral localization• Wavelets, the continuous transform :coherence, sparsity, redundancy• Cauchy Schwartz inequality• Approximation theory : vanishing moments• Non-linear operators mathematical morphology• Markov random fields• Problems in Astronomical data analysis• Frames, radon, ridgelets, curvelets• Parceval plancherel• 2D wavelets

Multiresolution analysis and wavelet bases

Outline :• Multiresolution analysis• The scaling function and scaling equation• Examples• Fast algorithms• Orthogonal and biorthogonal wavelets• Building wavelet bases• Vanishing moments• Applications in compression, approximation• Trees• Wavelet packets• A trous algorithm• Pyramidal algorithm

Image restoration, noise models, detection, deconvolution

Outline :• Image fornation model, Inverse problems in image processing• Algorithms for deconvolution : Richardson-Lucy, CLEAN, • Wiener filtering, Gaussian filter, Maximumentropy methode• Spike processes• Application of multiresolution methods• Shrinkage, Sparsity, bayesesian approaches• Inpainting• Again Cauchy-Schwartz• Pierpaoli• Complex models : accounting for coherent behaviour of wavelet

coefficients

Multi-dimensional data analysis

Outline :• What is multidimensional data

• Where does it come from

• Gaussianity

• Representations ans sparsity

• Projection pusuit

• Principal Component Analysis : Karhunen-Loeve Basis

• Standard mainstream ICA

• Diversity and separability

• Non gaussianity, Non stationarity

• Linear mixture model

• Applications