An Introduction to HHT: Instantaneous Frequency, Trend,

Degree of Nonlinearity and Non-stationarity

Norden E. Huang

Research Center for Adaptive Data AnalysisCenter for Dynamical Biomarkers and Translational Medicine

NCU, Zhongli, Taiwan, China

OutlineRather than the implementation details,

I will talk about the physics of the method.

• What is frequency?

• How to quantify the degree of nonlinearity?

• How to define and determine trend?

What is frequency?

It seems to be trivial.

But frequency is an important parameter for us to understand many physical phenomena.

Definition of Frequency

Given the period of a wave as T ; the frequency is defined as

1.

T

Instantaneous Frequency

distanceVelocity ; mean velocity

time

dxNewton v

dt

1Frequency ; mean frequency

period

dHH

So that both v and

T defines the p

can appear in differential equations.

hase functiondt

Other Definitions of Frequency : For any data from linear Processes

j

Ti t

j 0

1. Fourier Analysis :

F( ) x( t ) e dt .

2. Wavelet Analysis

3. Wigner Ville Analysis

Definition of Power Spectral Density

Since a signal with nonzero average power is not square integrable, the Fourier transforms do not exist in this case.

Fortunately, the Wiener-Khinchin Theroem provides a simple alternative. The PSD is the Fourier transform of the auto-correlation function, R(τ), of the signal if the signal is treated as a wide-sense stationary random process:

2i 2S( ) R( )e d S( )d ( t )

Fourier Spectrum

Problem with Fourier Frequency• Limited to linear stationary cases: same

spectrum for white noise and delta function.

• Fourier is essentially a mean over the whole domain; therefore, information on temporal (or spatial) variations is all lost.

• Phase information lost in Fourier Power spectrum: many surrogate signals having the same spectrum.

Surrogate Signal:Non-uniqueness signal vs. Power Spectrum

I. Hello

The original data : Hello

The surrogate data : Hello

The Fourier Spectra : Hello

The Importance of Phase

To utilize the phase to define

Instantaneous Frequency

Prevailing Views onInstantaneous Frequency

The term, Instantaneous Frequency, should be banished forever from the dictionary of the communication engineer.

J. Shekel, 1953

The uncertainty principle makes the concept of an Instantaneous Frequency impossible.

K. Gröchennig, 2001

The Idea and the need of Instantaneous Frequency

k , ;

t

tk

0 .

According to the classic wave theory, the wave conservation law is based on a gradually changing φ(x,t) such that

Therefore, both wave number and frequency must have instantaneous values and differentiable.

p

2 2 1 / 2 1

i ( t )

For any x( t ) L ,

1 x( )y( t ) d ,

t

then, x( t )and y( t ) form the analytic pairs:

z( t ) x( t ) i y( t ) ,

wherey( t )

a( t ) x y and ( t ) tan .x( t )

a( t ) e

Hilbert Transform : Definition

The Traditional View of the Hilbert Transform for Data Analysis

Traditional Viewa la Hahn (1995) : Data LOD

Traditional Viewa la Hahn (1995) : Hilbert

Traditional Approacha la Hahn (1995) : Phase Angle

Traditional Approacha la Hahn (1995) : Phase Angle Details

Traditional Approacha la Hahn (1995) : Frequency

The Real World

Mathematics are well and good but nature keeps dragging us around by the nose.

Albert Einstein

Why the traditional approach does not work?

Hilbert Transform a cos + b : Data

Hilbert Transform a cos + b : Phase Diagram

Hilbert Transform a cos + b : Phase Angle Details

Hilbert Transform a cos + b : Frequency

The Empirical Mode Decomposition Method and Hilbert Spectral Analysis

Sifting

(Other alternatives, e.g., Nonlinear Matching Pursuit)

Empirical Mode Decomposition: Methodology : Test Data

Empirical Mode Decomposition: Methodology : data and m1

Empirical Mode Decomposition: Methodology : data & h1

Empirical Mode Decomposition: Methodology : h1 & m2

Empirical Mode Decomposition: Methodology : h3 & m4

Empirical Mode Decomposition: Methodology : h4 & m5

Empirical Mode DecompositionSifting : to get one IMF component

1 1

1 2 2

k 1 k k

k 1

x( t ) m h ,h m h ,

.....

.....h m h

.h c

.

The Stoppage Criteria

The Cauchy type criterion: when SD is small than a pre-set value, where

T2

k 1 kt 0

T2

k 1t 0

h ( t ) h ( t )SD

h ( t )

Or, simply pre-determine the number of iterations.

Empirical Mode Decomposition: Methodology : IMF c1

Definition of the Intrinsic Mode Function (IMF): a necessary condition only!

Any function having the same numbers ofzero cros sin gs and extrema,and also havingsymmetric envelopes defined by local max imaand min ima respectively is defined as anIntrinsic Mode Function( IMF ).

All IMF enjoys good Hilbert Transfo

i ( t )

rm :

c( t ) a( t )e

Empirical Mode Decomposition: Methodology : data, r1 and m1

Empirical Mode DecompositionSifting : to get all the IMF components

1 1

1 2 2

n 1 n n

n

j nj 1

x( t ) c r ,r c r ,

x( t ) c r

. . .r c r .

.

Definition of Instantaneous Frequency

i ( t )

t

The Fourier Transform of the Instrinsic ModeFunnction, c( t ), gives

W ( ) a( t ) e dt

By Stationary phase approximation we have

d ( t ) ,dt

This is defined as the Ins tan taneous Frequency .

An Example of Sifting &

Time-Frequency Analysis

Length Of Day Data

LOD : IMF

Orthogonality Check

• Pair-wise % • 0.0003• 0.0001• 0.0215• 0.0117• 0.0022• 0.0031• 0.0026• 0.0083• 0.0042• 0.0369• 0.0400

• Overall %

• 0.0452

LOD : Data & c12

LOD : Data & Sum c11-12

LOD : Data & sum c10-12

LOD : Data & c9 - 12

LOD : Data & c8 - 12

LOD : Detailed Data and Sum c8-c12

LOD : Data & c7 - 12

LOD : Detail Data and Sum IMF c7-c12

LOD : Difference Data – sum all IMFs

Traditional Viewa la Hahn (1995) : Hilbert

Mean Annual Cycle & Envelope: 9 CEI Cases

Mean Hilbert Spectrum : All CEs

Properties of EMD Basis

The Adaptive Basis based on and derived from the data by the empirical method satisfy nearly all the traditional requirements for basis empirically and a posteriori:

CompleteConvergentOrthogonalUnique

The combination of Hilbert Spectral Analysis and Empirical Mode Decomposition has been

designated by NASA as

HHT

(HHT vs. FFT)

Comparison between FFT and HHT

j

jt

i tj

j

i ( )d

jj

1. FFT :

x( t ) a e .

2. HHT :

x( t ) a ( t ) e .

Comparisons: Fourier, Hilbert & Wavelet

Surrogate Signal:Non-uniqueness signal vs. Spectrum

I. Hello

The original data : Hello

The IMF of original data : Hello

The surrogate data : Hello

The IMF of Surrogate data : Hello

The Hilbert spectrum of original data : Hello

The Hilbert spectrum of Surrogate data : Hello

To quantify nonlinearity we also need instantaneous frequency.

Additionally

How to define Nonlinearity?

How to quantify it through data alone (model independent)?

The term, ‘Nonlinearity,’ has been loosely used, most of the time, simply

as a fig leaf to cover our ignorance.

Can we be more precise?

How is nonlinearity defined?Based on Linear Algebra: nonlinearity is defined based on input vs. output.

But in reality, such an approach is not practical: natural system are not clearly defined; inputs and out puts are hard to ascertain and quantify. Furthermore, without the governing equations, the order of nonlinearity is not known.

In the autonomous systems the results could depend on initial conditions rather than the magnitude of the ‘inputs.’

The small parameter criteria could be misleading: sometimes, the smaller the parameter, the more nonlinear.

Linear Systems

Linear systems satisfy the properties of superposition and scaling. Given two valid inputs to a system H,

as well as their respective outputs

then a linear system, H, must satisfy

for any scalar values α and β.

x ( t ) and x ( t )1 2

y ( t ) H{ x ( t )} and y (t) = H{ x ( t )}

1 1

2 2

y ( t ) y ( t ) H{ x ( t ) x ( t )} 1 1 1 2

How is nonlinearity defined?Based on Linear Algebra: nonlinearity is defined based on input vs. output.

But in reality, such an approach is not practical: natural system are not clearly defined; inputs and out puts are hard to ascertain and quantify. Furthermore, without the governing equations, the order of nonlinearity is not known.

In the autonomous systems the results could depend on initial conditions rather than the magnitude of the ‘inputs.’

The small parameter criteria could be misleading: sometimes, the smaller the parameter, the more nonlinear.

How should nonlinearity be defined?

The alternative is to define nonlinearity based on data characteristics: Intra-wave frequency modulation.

Intra-wave frequency modulation is known as the harmonic distortion of the wave forms. But it could be better measured through the deviation of the instantaneous frequency from the mean frequency (based on the zero crossing period).

Characteristics of Data from Nonlinear Processes

32

2

2

22

d x x cos tdt

d x x cos tdt

Spring with positiondependent cons tan t ,int ra wave frequency mod ulation;therefore,we need ins tan

x

1

taneous frequenc

x

y.

Duffing Pendulum

2

22( co .) s1d x

x txdt

x

Duffing Equation : Data

Hilbert’s View on Nonlinear DataIntra-wave Frequency Modulation

A simple mathematical model

x( t ) cos t sin 2 t

Duffing Type WaveData: x = cos(wt+0.3 sin2wt)

Duffing Type WavePerturbation Expansion

For 1 , we can have

cos t cos sin 2 t sin t sin sin 2 t

cos t sin t sin 2 t ....

This is very similar to the solutionof D

x( t ) cos t sin 2 t

1 cos t cos 3

uffing equ

t ....2

atio

2

n .

Duffing Type WaveWavelet Spectrum

Duffing Type WaveHilbert Spectrum

Degree of nonlinearity

1/ 22

z

z

Let us consider a generalized intra-wave frequency modulation model as:

d x( t ) cos( t sin t )

IF IFDN (Degree of Nolinearity ) should be

I

cos t IF=

.2

Depending on

d

F

t

1t

he value of , we can have either a up-down symmetric or a asymmetric wave form.

Degree of Nonlinearity• DN is determined by the combination of δη precisely with

Hilbert Spectral Analysis. Either of them equals zero means linearity.

• We can determine δ and η separately:– η can be determined from the instantaneous

frequency modulations relative to the mean frequency.

– δ can be determined from DN with known η. NB: from any IMF, the value of δη cannot be greater than 1.

• The combination of δ and η gives us not only the Degree of Nonlinearity, but also some indications of the basic properties of the controlling Differential Equation, the Order of Nonlinearity.

Stokes Models

22

2

2 with ; =0.1.

25

: is positive ranging from 0.1 to 0.375; beyond 0.375, there is no solution.

: is negative ranging from 0.1 to 0.391

d xx

;

Stokes I

Stokes II

x cos t dt

beyond 0.391, there is no solution.

Data and IFs : C1

0 20 40 60 80 100-3

-2.5

-2

-1.5

-1

-0.5

0

0.5

1

1.5

2

Time: Second

Freq

uenc

y : H

z

Stokes Model c1: e=0.375; DN=0.2959

IFqIFzDataIFq-IFz

Summary Stokes I

0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.450

0.05

0.1

0.15

0.2

0.25

0.3

0.35

Epsilon

Com

bine

d D

egre

e of

Sta

tiona

rity

Summary Stokes I

C1C2CDN

Lorenz Model• Lorenz is highly nonlinear; it is the model

equation that initiated chaotic studies.

• Again it has three parameters. We decided to fix two and varying only one.

• There is no small perturbation parameter.

• We will present the results for ρ=28, the classic chaotic case.

Phase Diagram for ro=28

010

2030

4050

-20-10

010

20-30

-20

-10

0

10

20

30

x

Lorenz Phase : ro=28, sig=10, b=8/3

y

z

X-Component

DN1=0.5147

CDN=0.5027

Data and IF

0 5 10 15 20 25 30-10

0

10

20

30

40

50

Time : second

Freq

uenc

y : H

z

Lorenz X : ro=28, sig=10, b=8/3

IFqIFzDataIFq-IFz

ComparisonsFourier Wavelet Hilbert

Basis a priori a priori Adaptive

Frequency Integral transform: Global

Integral transform: Regional

Differentiation:Local

Presentation Energy-frequency Energy-time-frequency

Energy-time-frequency

Nonlinear no no yes, quantifying

Non-stationary no yes Yes, quantifying

Uncertainty yes yes no

Harmonics yes yes no

How to define and determine Trend ?

Parametric or Non-parametric?Intrinsic vs. extrinsic approach?

The State-of-the arts: Trend

“One economist’s trend is another economist’s cycle”

Watson : Engle, R. F. and Granger, C. W. J. 1991 Long-run Economic Relationships. Cambridge University Press.

Philosophical Problem Anticipated

名不正則言不順言不順則事不成 

                      ——孔夫子

Definition of the TrendProceeding Royal Society of London, 1998

Proceedings of National Academy of Science, 2007

Within the given data span, the trend is an intrinsically fitted monotonic function, or a function in which there can be at most one extremum.

The trend should be an intrinsic and local property of the data; it is determined by the same mechanisms that generate the data.

Being local, it has to associate with a local length scale, and be valid only within that length span, and be part of a full wave length.

The method determining the trend should be intrinsic. Being intrinsic, the method for defining the trend has to be adaptive.

All traditional trend determination methods are extrinsic.

Algorithm for Trend

• Trend should be defined neither parametrically nor non-parametrically.

• It should be the residual obtained by removing cycles of all time scales from the data intrinsically.

• Through EMD.

Global Temperature Anomaly

Annual Data from 1856 to 2003

GSTA

IMF Mean of 10 Sifts : CC(1000, I)

Statistical Significance Test

IPCC Global Mean Temperature Trend

Comparison between non-linear rate with multi-rate of IPCC

1840 1860 1880 1900 1920 1940 1960 1980 2000 2020-0.02

-0.015

-0.01

-0.005

0

0.005

0.01

0.015

0.02

0.025Various Rates of Global Warming

Time : Year

War

min

g R

ate:

o C/Y

r

Blue shadow and blue line are the warming rate of non-linear trend.Magenta shadow and magenta line are the rate of combination of non-linear trend and AMO-like components.Dashed lines are IPCC rates.

Conclusion• With HHT, we can define the true instantaneous

frequency and extract trend from any data.

• We can quantify the degree of nonlinearity. Among the applications, the degree of nonlinearity could be used to set an objective criterion in biomedical and structural health monitoring, and to quantify the degree of nonlinearity in natural phenomena.

• We can also determine the trend, which could be used in financial as well as natural sciences.

• These are all possible because of adaptive data analysis method.

History of HHT1998: The Empirical Mode Decomposition Method and the

Hilbert Spectrum for Non-stationary Time Series Analysis, Proc. Roy. Soc. London, A454, 903-995. The invention of the basic method of EMD, and Hilbert transform for determining the Instantaneous Frequency and energy.

1999: A New View of Nonlinear Water Waves – The Hilbert Spectrum, Ann. Rev. Fluid Mech. 31, 417-457.

Introduction of the intermittence in decomposition. 2003: A confidence Limit for the Empirical mode

decomposition and the Hilbert spectral analysis, Proc. of Roy. Soc. London, A459, 2317-2345.Establishment of a confidence limit without the ergodic assumption.

2004: A Study of the Characteristics of White Noise Using the Empirical Mode Decomposition Method, Proc. Roy. Soc. London, A460, 1179-1611.Defined statistical significance and predictability.

Recent Developments in HHT

2007: On the trend, detrending, and variability of nonlinear and nonstationary time series. Proc. Natl. Acad. Sci., 104, 14,889-14,894.

2009: On Ensemble Empirical Mode Decomposition. Adv. Adaptive data Anal., 1, 1-41

2009: On instantaneous Frequency. Adv. Adaptive Data Anal., 2, 177-229.

2010: The Time-Dependent Intrinsic Correlation Based on the Empirical Mode Decomposition. Adv. Adaptive Data Anal., 2. 233-266.

2010: Multi-Dimensional Ensemble Empirical Mode Decomposition. Adv. Adaptive Data Anal., 3, 339-372.

2011: Degree of Nonlinearity. (Patent and Paper)2012: Data-Driven Time-Frequency Analysis (Applied and

Computational Harmonic Analysis, T. Hou and Z. Shi)

Current Efforts and Applications

• Non-destructive Evaluation for Structural Health Monitoring – (DOT, NSWC, DFRC/NASA, KSC/NASA Shuttle, THSR)

• Vibration, speech, and acoustic signal analyses– (FBI, and DARPA)

• Earthquake Engineering– (DOT)

• Bio-medical applications– (Harvard, Johns Hopkins, UCSD, NIH, NTU, VHT, AS)

• Climate changes – (NASA Goddard, NOAA, CCSP)

• Cosmological Gravity Wave– (NASA Goddard)

• Financial market data analysis– (NCU)

• Theoretical foundations– (Princeton University and Caltech)

Outstanding Mathematical Problems

1. Mathematic rigor on everything we do. (tighten the definitions of IMF,….)

2. Adaptive data analysis (no a priori basis methodology in general)

3. Prediction problem for nonstationary processes (end effects)

4. Optimization problem (the best stoppage criterion and the unique decomposition ….)

5. Convergence problem (finite step of sifting, best spline implement, …)

Do mathematical results make physical sense?

Do mathematical results make physical sense?

Good math, yes; Bad math ,no.

The job of a scientist is to listen carefully to nature, not to tell nature how to behave.

Richard Feynman

To listen is to use adaptive method and let the data sing, and not to force the data to fit preconceived modes.

The Job of a Scientist

All these results depends on adaptive approach.

Thanks