Empirical Mode Decomposition (EMD) on Surfaces

Hui Wang1,2 Zhixun Su1 Junjie Cao1 Ye Wang3 Hao Zhang2

Geometric Modeling and Processing 2012

1Dalian University of Technology2Simon Fraser University3Harbin Institute of Technology

Generalize signal processing methods to surfaces

23/4/19 2

Motivation

Original surface Low-pass filtering Enhancement filtering

Previous Works

• Parameterization-based methods Geometry images (Gu et al. 2002) Filtering by spherical harmonics (Zhou et al. 2004) …

• Surface-based methods Fourier transform (Taubin 1995) Subdivision wavelet (Valette and Prost 2004, Wang and Tang 2009) Detail editing via Laplacian coordinates (Wang et al. 2011) Mexican Hat Wavelet (Hou and Qin 2012) …

23/4/19 3

23/4/19 4

Our work

Generalize multi-scale Empirical Mode Decomposition (EMD) to surfaces

Original scalar function IMF 1 IMF 2 Smoothed residue

IMF: Intrinsic Mode Function

Details at different scales

= + +

Contents

• 1. 1D EMD

• 2. Our generalized EMD on surfaces

• 3. Feature-preserving smoothing by EMD

• 4. Conclusion and future works

23/4/19 5

Contents

• 1. 1D EMD

• 2. Our generalized EMD on surfaces

• 3. Feature-preserving smoothing by EMD

• 4. Conclusion and future works

23/4/19 6

1D EMD

23/4/19 7

Cited by 5675!Empirical Mode Decomposition (EMD) and Hilbert-Huang Transform (HHT)

Comparison

23/4/19 8

Fourier Wavelet EMD

Basis a priori a priori adaptive

Non-linear no no yes

Non-stationary no yes yes

The basis is data-driven and adaptive.

Work well for non-linear and non-stationary signals.

Motivate potential applications in geometry processing.

1D EMD Example

23/4/19 9

Data: x

IMF 1: d1

IMF 2: d2

IMF 3: d3

IMF 4: d4

IMF 5: d5

IMF 6: d6

Residue: r6

=+

++

++

+

What is the 1D IMF?

23/4/19 10

Images taken from [Huang et al. 1998]

Typical example of 1D IMF

Use “Sifting Process” to extract each IMFSimilar to the harmonic function

23/4/19 11

Sifting Process

Original data: x

23/4/19 12

Sifting Process

Local maximum

23/4/19 13

Sifting Process

Local maximum and minimum

23/4/19 14

Sifting Process

Interpolated by the Cubic Spline

Envelopes

23/4/19 15

Sifting Process

Mean of envelopes of x: m0

23/4/19 16

Sifting Process

h1

x

m0

=-

23/4/19 17

Sifting Process

h1

23/4/19 18

Sifting Process

Mean of envelopes of h1: m1

23/4/19 19

Sifting Process

h2

h1

m1

=-

23/4/19 20

Sifting Process

h2

23/4/19 21

h1 = x - m0

h2 = h1 – m1

Sifting Process

…hk = hk-1 – mk-1

How to stop the sifting process?

23/4/19 22

Stopping Criterion of Sifting Process

The number of zero-crossings and extrema of hk are the same or differ at most by one.

AND

The stander deviation of hk and hk-1 is smaller than a pre-set value.

IMF 1: d1 = hk

23/4/19 23

First Scale EMD

IMF 1: d1 = h5

Data: x

IMF 1: d1

Residue 1: r1

=+

23/4/19 24

Second Scale EMD

Data: x

IMF 1: d1

IMF 2: d2

Residue 2: r2

=+

+

1D EMD

23/4/19 25

x r1

d1

r2

d2

…

rk-1 rk

dk

…

How to stop the EMD?

Stop Criterion of EMD

23/4/19 26

The residue or IMF becomes so small.

OR

The residue becomes a monotonic function or constant.

Finial EMD

23/4/19 27

Data: x

IMF 1: d1

IMF 2: d2

IMF 3: d3

IMF 4: d4

IMF 5: d5

IMF 6: d6

Residue: r6

=+

++

++

+

Contents

• 1. 1D EMD

• 2. Our generalized EMD on surfaces

• 3. Feature-preserving smoothing by EMD

• 4. Conclusion and future works

23/4/19 28

23/4/19 29

Generalized EMD on Surfaces

The principle is similar to that of 1D

Local extrema detection and interpolation method

Original scalar function IMF 1 IMF 2 Smoothed residue

= + +

Local Extrema Detection

23/4/19 30

Local maximum: functional value isn’t smaller than that of 1-ring neighbors

( ), ( ) ( )j ij N i f f v v

Local minimum: functional value isn’t larger than that of 1-ring neighbors

( ), ( ) ( )j ij N i f f v v

Interpolated Method on Surfaces

23/4/19 31

We minimize the linearized thin-plate energy:

2( )sSf dV

The Euler-Lagrange equation is:

2 0S f

: Laplace-Beltrami operator on surfacess

23/4/19 32

( )

,

1, ( ) , (cot cot )

20, otherwise

ikk N i

s ij ij ij ij iji

w i j

w j N i wA

L

1 2( ( ), ( ), , ( ))Tnf f f f f v v v

2 0, . . ( ) ,i is t f f i C L f v

A bi-harmonic field with Dirichlet boundary conditions:

Interpolated Method on Surfaces

Result of EMD on surfaces

23/4/19 33

Original scalar function IMF 1 IMF 2 IMF 3

IMF 4 IMF 5 Residue

Application: Filtering

23/4/19 34

Original scalar function:

1

J

k Jk

f d r

Filtering result:'

1

J

k k Jk

f d r

1k

0 1k

Enhancing

Smoothing

IMFs represent details at different scales

Filtering Scalar Function

23/4/19 35

Original function Enhancing result Smoothing result

23/4/19 36

Original surface High enhancement Band enhancement

Filtering the SurfaceFiltering the three coordinates functions respectively.

Band smoothing Smoothing

Surface Denoising

23/4/19 37

Original surface Corrupted by noise The first residue

Contents

• 1. 1D EMD

• 2. Our generalized EMD on surfaces

• 3. Feature-preserving smoothing by EMD

• 4. Conclusion and future works

23/4/19 38

Feature-preserving Smoothing?

23/4/19 39

First, the interpolation method is not feature aware.

Our generalized EMD cannot preserve sharp features.

Second, the three coordinates functions are processed separately.

We still propose a feature-preserving smoothing method based on EMD.

23/4/19 40

K. Subr, C. Soler, F. Durand, Edge-preserving multiscale image decomposition based on local extrema, ACM Transactions on Graphics 28 (5) (2009) 1–9.

Edge-preserving Multiscale Image Decomposition

Images taken from [Subr et al. 2009]

1D Edge-preserving Smoothing

23/4/19 41Feature-preserving interpolation

Images taken from [Subr et al. 2009] No sifting

Enlarge the extrema-location kernel

Our Generalization

• Extrema identification: Local extrema of Gaussian curvature of k-ring neighbors at the k-th

scale

• Feature-preserving interpolation:

23/4/19 42

'

2' 'arg min , . . , { , , },d

d id ids t v v d x y z i C V

MV

1.0,

, ( , )

0, otherwise

ij ij

i j

w i j E

M

2

2exp( ), ( )

2ij

ij ij j i ii

dw d

v v n

Feature-preserving Smoothing Result

23/4/19 43

Original noisy surface The 8th level smoothed result

Feature-preserving Smoothing Result

23/4/19 44

Surface with real world noise The first level smoothed result

Compare with the Bilateral Filtering

23/4/19 45

Bilateral Filtering

Our EMD-based method

Need a more robust feature-aware interpolation

Contents

• 1. 1D EMD

• 2. Our generalized EMD on surfaces

• 3. Feature-preserving smoothing by EMD

• 4. Conclusion and future works

23/4/19 46

Conclusions

• We first introduce the EMD from Euclidean space to the setting of surfaces.

• We also make a first try for feature-preserving surface smoothing based EMD.

23/4/19 47

Limitations and Future Works

• Our work is the first step toward generalizing the EMD to geometric processing, some problems needed be improved or investigated:

The behaviour of IMFs on surface Generalize the Hilbert-Huang Transform (HHT) to surfaces More possible applications of the generalized EMD More robust anisotropic extrema detection and feature-

aware interpolation …

23/4/19 48

23/4/19 49