Towards inference for skewed alpha stable Levy processes

Simon Godsill and Tatjana Lemke

Signal Processing and Communications Lab.

University of Cambridge

www-sigproc.eng.cam.ac.uk/~sjg

Overview

Motivation

Continuous-time models

Inference for models with jumps

Inference for alpha-stable Levy processes

Conclusions

Motivation

Traditional tracking applications run in discrete time – discrete time state space models:

– Dynamic models of behaviour:

– Sensor (observation) models:

Hidden state (position/velocity…)

Measurements (range, bearing, …)

State Space Model:

Optimal Filtering (sequential inference):

It is often more natural to model in continuous time:

– Observations can arrive at arbitrary timings, and out of sequence

– Object manoeuvres are carried out in continuous time without reference to observation times

Real objects undergo rapid and random changes of regime, or `jumps’, occurring at unknown times on the continuous-time scale:

To appear

0 50 100 150 200 25099.5

99.6

99.7

99.8

99.9

100

100.1

100.2

100.3

Variable rate - C=60,

C=2,

T=100, N Particles=400

Can approximate this type of behaviour with jump Markov (or semi-Markov) discrete time systems, allowing a finite set of discrete time switching models (HMMs, etc….)

However, seems more natural to model directly in continuous time, without limiting regimes to a finite set (although can extend models to allow in addition regime switching between discrete states)

Bayesian Monte Carlo computational tools are well suited to inference in these continuous time models

Continuous time models

Example: stochastic trend model with Gaussian jumps

0 50 100 150 200 25099.5

99.6

99.7

99.8

99.9

100

100.1

100.2

100.3

Variable rate - C=60,

C=2,

T=100, N Particles=400

Now assume that W(t) is comprised of (scaled) Brownian motion B(t) plus

a pure Gaussian jump processZ(t):

0 20 40 60 80 100 120 140 160 180 200-2.5

-2

-1.5

-1

-0.5

0

0.5

1

1.5

2

0 20 40 60 80 100 120 140 160 180 20085

90

95

100

105

110

115

See Christensen, Murphy and Godsill, 2012

0 20 40 60 80 100 120 140 160 180 200-2.5

-2

-1.5

-1

-0.5

0

0.5

1

1.5

2

...

Observation of the jump process

Now assume discrete-time observations of the continuous-time process, e.g.

Filtering/ smoothing and parameter estimation can now be carried out using Kalman-filter conditional likelihoods:

Fixed Interval Filtering and Smoothing

Particle filter-smoother, See Godsill, Doucet,

West (2004) and Bunch and Godsill (2012)

Tracking examples

®-stable Levy processes

Previously modelled jumps as a finite activity process

Perhaps more realistic to model the jumps as an infinite collection of large/ small/tiny jumps occurring in each finite time interval

®-stable Levy processes provide a natural

way to achieve this – justified in many Communications, Signal Processing and Finance applications

Joint work with Tatjana Lemke,

see Lemke and Godsill Proc.

ICASSP 2011, ICASSP 2012

Stable distributions

Approaches to inference can be made by scale mixtures of normals for symmetric case, and Euler discretisation – Godsill and Kuruoglu 1998,

Godsill and Yang 2006, Tsionas 1998. See also Buckle (1995)

Our current work focuses on a powerful series expansion for the general asymmetric case:

[Here stated in its simplest form for a random variable with

®<1. A similar form applies for the stochastic integral.]

[Samorodnitsky and Taqqu 1994]

iid random variables

satisfying some simple

moment conditions, e.g.

A Gaussian would do.

Arrival times of a unit rate Poisson process

now has a `physical’ interpretation as the scale of the

ith jump.

[Samorodnitsky and Taqqu 1994]

[See also Barndorff-Nielsen and Shephard (2001,2001)]

Now, taking Wi to be iid N(¹W,¾W2), we have

a Gaussian structure, conditional upon the terms ¡i

This can be readily handled using Bayesian Monte Carlo inference.

As a significant bonus, (¹W,¾W2) can be

analytically marginalised, leaving only ® to be

sampled

The trade-off is that we must also simulate an infinite series of ¡i terms – in practice we

do this by truncating the series and approximating the residual.

Truncation of the Series

In practice cannot compute the entire infinite series:

Instead, truncate at a limit, ¡i<c, and

approximate the residual with a Gaussian matched to first two moments of the residual (analytically computed).

Inference Schemes

Consider e.g. A MCMC inference algorithm for the parameters and the latent variables.

Parameters are ®, ¾W, ¹W, m,V

The challenging part is the latent variables (m,V), one for each data point.

Latent Variables (m,V)

Can sample directly from the prior p(m,V) and use rejection sampling (unbounded envelope, but can fix (condition on) the residual terms and make it bounded).

Not good enough for large values of X (slow in the tails!); instead, look at p(m,V):

ca

Observe that for large m, V is approximately m2

This means that we can force prior samples of m into the tails of the

distribution (®-stable and hence pareto) and perform rejection sampling

with likelihood constrained to the line V=m2

This is a rare-event tail approximation for large X values, but the approximation

appears very good and speeds up sampling dramatically.

Can tune accuracy vs. speed by selecting the switchover point between optimal (slow)

and tail approximation (fast).

State-space models and continuous time

Incorporation into discrete-time state-space models is fairly straightforward – likelihood computation can then be done by a mean- and scale-shifted Kalman filter

Continuous time is also a fairly straightforward extension – also need to sample the Vi

Conclusion

A general framework for inference of ®-stable

distribution parameters, linear state-space models and continuous-time Levy processes

Straightforward computations using conditionally Gaussian models (Shephard (1993 Biometrika), Carter and Kohn (1994 Biometrika)), and particle filters

Currently exploring Particle MCMC methods for parameter estimation

References

S. J. Godsill, J. Vermaak, W. Ng and J.F. Li. Variable rate particle filters for tracking applications

in Proceedings of the IEEE, Special Issue on Large Scale Dynamical Systems, 2007.

S. J. Godsill and J. Vermaak. Variable rate particle filters for tracking applications. In Proc. IEEE Stat. Sig. Proc., Bordeaux, 2005.

S. J. Godsill and J. Vermaak, Models and algorithms for tracking using trans-dimensional sequential Monte Carlo. In Proc. IEEE ICASSP 2004

Barndorff-Nielsen, Ole E. and Neil Shephard (2001) "Normal modified stable processes", Theory of Probability and Mathematical Statistics, 2001, 1-19.

Barndorff-Nielsen, Ole E. and Neil Shephard (2001) "Non-Gaussian Ornstein-Uhlenbeck-based models and some of their uses in financial economics", (with discussion) Journal of the Royal Statistical Society, Series B, 63, 167-241.

Buckle, D. J. (1995), Bayesian inference for stable distributions, JASA, 90:605-613.

S. J. Godsill. Inference in symmetric alpha-stable noise using MCMC and the slice sampler. In Proc. IEEE International Conference on Acoustics, Speech and Signal Processing, volume VI, pages 3806-3809, 2000. ISBN 0-7803-6296-9.

S. J. Godsill. MCMC and EM-based methods for inference in heavy-tailed processes with alpha-stable innovations. In Proc. IEEE Signal processing workshop on higher-order statistics, June 1999. Caesarea, Israel.[ bib | .ps ]

S. J. Godsill and E. E. Kuruoglu. Bayesian inference for time series with heavy-tailed symmetric α-stable noise processes. In Proc. Applications of heavy tailed distributions in economics, engineering and statistics, June 1999. Washington DC, USA.[ bib | .ps ]

S.J. Godsill and L. Yang. Bayesian inference for continuous-time AR models driven by non-Gaussian lÉvy processes. In Proc. IEEE International Conference on Acoustics, Speech and Signal Processing, Toulouse, France, May 2006.

Tatjana Lemke, Simon J. Godsill: Enhanced Poisson sum representation for alpha-stable processes. ICASSP 2011: 4100-4103

Tatjana Lemke, Simon J. Godsill: LINEAR GAUSSIAN COMPUTATIONS FOR NEAR-EXACT BAYESIAN MONTE CARLO INFERENCE IN SKEWED ALPHA-STABLE TIME SERIES MODELS, ICASSP 20112