Lecture series: Data analysis

Multi

Lecture series: Data analysisLectures: Each Tuesday at 16:00

(First lecture: May 21, last lecture: June 25)Thomas Kreuz, ISC, CNR

[email protected]

http://www.fi.isc.cnr.it/users/thomas.kreuz/

Lecture 1: Example (Epilepsy & spike train synchrony), Data acquisition, Dynamical systems

Lecture 2: Linear measures, Introduction to non-linear dynamics

Lecture 3: Non-linear measures

Lecture 4: Measures of continuous synchronization

Lecture 5: Measures of discrete synchronization(spike trains)

Lecture 6: Measure comparison & Application to epileptic seizure prediction

ScheduleExample: Epileptic seizure prediction

Data acquisition

Introduction to dynamical systems

First lectureNon-linear model systems

Linear measures

Introduction to non-linear dynamics

Non-linear measures

- Introduction to phase space reconstruction

- Lyapunov exponentSecond lectureNon-linear measures

- Dimension

[ Excursion: Fractals ]

- Entropies

- Relationships among non-linear measuresThird lectureCharacterizition of a dynamic in phase space

Predictability(Information / Entropy)DensitySelf-similarityLinearity / Non-linearityDeterminism /Stochasticity(Dimension)Stability (sensitivityto initial conditions)6Dimension (classical)Number of degrees of freedom necessary to characterize a geometric object

Euclidean geometry: Integer dimensions

Object Dimension Point0 Line1 Square (Area)2 Cube (Volume)3 N-cuben

Time series analysis:Number of equations necessary to model a physical system

7Hausdorff-dimension

8Box-counting

9Box-countingRichardson: Counter-intuitive notion that a coastline's measured length changes with the length of the measuring stick used. Fractal dimension of a coastline: How does the number of measuring sticks required to measure the coastline change with the scale of the stick?

10

Example: Koch-curveSome properties:- Infinite length- Continuous everywhere- Differentiable nowhere- Fractal dimension D=log4/log3 1.2611Strange attractors are fractals

Logistic mapHnon map2,0112Self-similarity of the logistic attractor

13Generalized dimensions14Generalized entropies

15Lyapunov-exponent16Summary17Motivation

Measures of synchronization for continuous data

Linear measures: Cross correlation, coherence

Mutual information

Phase synchronization (Hilbert transform)

Non-linear interdependences

Measure comparison on model systems

Measures of directionality

Granger causality

Transfer entropyTodays lectureMotivationMotivation: Bivariate time series analysisThree different scenarios:

Repeated measurement from one system (different times)

Stationarity, Reliability

Simultaneous measurement from one system (same time)

Coupling, Correlation, Synchronization, Directionality

Simultaneous measurement from two systems (same time)

Coupling, Correlation, Synchronization, Directionality20Synchronization

[Huygens: Horologium Oscillatorium. 1673]21Synchronization

[Pecora & Carroll. Synchronization in chaotic systems. Phys Rev Lett 1990]22Synchronization

[Pikovsky & Rosenblum: Synchronization. Scholarpedia (2007)]In-phase synchronization23Synchronization

[Pikovsky & Rosenblum: Synchronization. Scholarpedia (2007)]Anti-phase synchronization24Synchronization

[Pikovsky & Rosenblum: Synchronization. Scholarpedia (2007)]

Synchronization with phase shift25Synchronization


No synchronization26Synchronization


In-phase synchronizationAnti-phase synchronizationNo synchronizationSynchronization with phase shift27Measures of synchronizationSynchronizationDirectionality

Cross correlation / Coherence

Mutual Information Index of phase synchronization - based on Hilbert transform - based on Wavelet transform

Non-linear interdependence Non-linear interdependence

Event synchronization Delay asymmetry

Transfer entropy

Granger causality28LinearcorrelationStatic linear correlation: Pearsons r -1 - completely anti-correlatedr = 0 - uncorrelated (linearly!) 1 - completely correlated

Two sets of data points:

30Examples: Pearsons r

Undefined[An example of the correlation of x and y for various distributions of (x,y) pairs; Denis Boigelot 2011]31Cross correlation

Maximum cross correlation:32CoherenceLinear correlation in the frequency domain

Cross spectrum:Coherence = Normalized power in the cross spectrumWelchs method: average over estimated periodograms of subintervals of equal lengthComplex number Phase33MutualinformationShannon entropyShannon entropy~ Uncertainty

Binary probabilities:In general:

35Mutual Information

Marginal Shannon entropy:

Joint Shannon entropy:Mutual Information:Estimation based on k-nearest neighbor distances:

[Kraskov, Stgbauer, Grassberger: Estimating Mutual Information. Phys Rev E 2004]Kullback-Leibler entropy compares to probability distributions

Mutual Information = KL-Entropy with respect to independence36Mutual Information Properties:

Non-negativity:

Symmetry:

Minimum: Independent time series

Maximum: for identical systems

Venn diagram (Set theory)37

Cross correlation & Mutual Information1.00.50.0CmaxI1.00.50.0CmaxI1.00.50.0CmaxI38Phasesynchronization Definition of a phase

- Rice phase - Hilbert phase - Wavelet phase

Index of phase synchronization

- Index based on circular variance - [Index based on Shannon entropy] - [Index based on conditional entropy]Phase synchronization[Tass et al. PRL 1998]40Linear interpolation between marker events - threshold crossings (mostly zero, sometimes after demeaning) - discrete events (begin of a new cycle)

Problem: Can be very sensitive to noiseRice phase

41Hilbert phase

[Rosenblum et al., Phys. Rev. Lett. 1996]Analytic signal:

Artificial imaginary part:Instantaneous Hilbert phase:

- Cauchy principal value

42Wavelet phase

Basis functions with finite support

Example: complex Morlet wavelet

Wavelet = Hilbert + filter[Quian Quiroga, Kraskov, Kreuz, Grassberger. Phys. Rev. E 2002]Wavelet phase:43Index of phase synchronization:Circular variance (CV)

44Non-linearinterdependenceTakens embedding theorem[F. Takens. Detecting strange attractors in turbulence. Springer, Berlin, 1980]46Non-linear interdependencesNonlinear interdependence S

Nonlinear interdependence H

SynchronizationDirectionality[Arnhold, Lehnertz, Grassberger, Elger. Physica D 1999]47Non-linear interdependence

Non-linear interdependence









EventsynchronizationEvent synchronization

Event times:Synchronicity:Event synchronization:Delay asymmetry:[Quian Quiroga, Kreuz, Grassberger. Phys Rev E 2002]

Window:

withAvoids double-countingEvent synchronization[Quian Quiroga, Kreuz, Grassberger. Phys Rev E 2002]

QqMeasure comparison on model systemsMeasure comparison on model systems[Kreuz, Mormann, Andrzejak, Kraskov, Lehnertz, Grassberger. Phys D 2007]Model systems &Coupling schemesHnon mapIntroduced by Michel Hnon as a simplified model of the Poincar section of the Lorenz modelOne of the most studied examples of dynamical systems that exhibit chaotic behavior

[M. Hnon. A two-dimensional mapping with a strange attractor. Commun. Math. Phys., 50:69, 1976]

64Hnon map

Coupled Hnon maps

Driver:Responder:

Identical systems:Coupling strength:66Coupled Hnon maps

67Coupled Hnon systems

68Rssler system

designed in 1976, for purely theoretical reasonslater found to be useful in modeling equilibrium in chemical reactions[O. E. Rssler. An equation for continuous chaos. Phys. Lett. A, 57:397, 1976]

69Rssler system

Coupled Rssler systemsDriver:Responder:Parameter mismatch:Coupling strength:

71Coupled Rssler systems

72Coupled Rssler systems

73Lorenz system

Developed in 1963 as a simplified mathematical model for atmospheric convectionArise in simplified models for lasers, dynamos, electric circuits, and chemical reactions[E. N. Lorenz. Deterministic non-periodic flow. J. Atmos. Sci., 20:130, 1963]

74Lorenz system

Coupled Lorenz systemsDriver:Responder:Small parameter mismatch in second componentCoupling strength:

76Coupled Lorenz systems

77Coupled Lorenz systems

78Noise-free caseCriterion I: Degree of monotonicity

= 1 - strictly monotonic increaseM(s) = 0 - flat line (or equal decrease and increase) = -1 - strictly monotonic decrease80Degree of monotonicity: Examples

Sequences:

100 values5050 pairs

Left:Monotonicity

Right:# positive# negative81Comparison: No Noise

82Summary: No-Noise-Comparison Results for Rssler are more consistent than for the other systems

Mutual Information slightly better than cross correlation (Non-linearity matters)

Wavelet phase synchronization not appropiate for broadband systems (inherent filtering looses information)

83Robustnessagainst noiseCriterion II: Robustness against noiseExample: White noise

Hnon system: White noise

87Rssler system : White noise

88Lorenz system: White noise

89Hnon system

90Comparison: White noise

91Summary: White noiseFor systems opposite order as in the noise-free case (Lorenz more robust then Hnon and than Rssler)

the more monotonous a system has been without noise, the less noise is necessary to destroy this monotonicity

Highest robustness is obtained for cross correlation followed by mutual information.92Iso-spectral noise: Example

93Iso-spectral noise: Fourier spectrum complex Autocorrelation Fourier spectrumTime domain Frequency domainx (t)

AmplitudePhysical phenomenonTime series94Generation of iso-spectral noisePhase-randomized surrogates:

Take Fourier transform of original signal

Randomize phases

Take inverse Fourier transform

Iso-spectral surrogate(By construction identical Power spectrum, just different phases)

Add to original signal with given NSR 95Lorenz system: Iso-spectral noise

96Comparison: Iso-spectral noise

97Summary: Iso-spectral noise Again results for Rssler are more consistent than for the other systems

Sometimes M never crosses critical threshold (monotonicity of the noise-free case is not destroyed by iso-spectral noise).

Sometimes synchronization increases for more noise: (spurious) synchronization between contaminating noise-signals, only for narrow-band systems

98Correlation among measuresCorrelation among measures

100Correlation among measures



103Summary: Correlation All correlation values rather high (Minimum: ~0.65)

Highest correlations for cross correlation and Hilbert phase synchronization

Event synchronization and Hilbert phase synchronization appear least correlated

Overall correlation between two phase synchronization methods low (but only due to different frequency sensitivity in the Hnon system)104Overall summary: Comparison of measures Capability to distinguish different coupling strengths Obvious and objective criterion exists only in some special cases (e.g., wavelet phase is not very suitable for a system with a broadband spectrum).

Robustness against noise varies (Important criterion for noisy data)

Pragmatic solution:

Choose measure which most reliably yields valuable information (e.g., information useful for diagnostic purposes) in test applications

105Measures ofdirectionalityMeasures of directionality107Granger causality[Granger: Investigating causal relations by econometric models and cross-spectral methods. Econometrica 37, 424-438 (1969)]108Granger causality

Univariate model:Bivariate model:

Model parameters;

Prediction errors; [Granger: Investigating causal relations by econometric models and cross-spectral methods. Econometrica 37, 424-438 (1969)]Fit via linear regression109Transfer Entropy: Conditional entropy

Venn diagram (Set theory)

Conditional entropy:

Mutual Information:

110

Transfer entropy : Conditional entropy

[T. Schreiber. Measuring information transfer. Phys. Rev. Lett., 85:461, 2000]111

Transfer entropy

[T. Schreiber. Measuring information transfer. Phys. Rev. Lett., 85:461, 2000]112

Transfer entropy

[T. Schreiber. Measuring information transfer. Phys. Rev. Lett., 85:461, 2000]113Motivation

Measures of synchronization for continuous data

Linear measures: Cross correlation, coherence

Mutual information

Phase synchronization (Hilbert transform)

Non-linear interdependences

Measure comparison on model systems

Measures of directionality

Granger causality

Transfer entropyTodays lectureMeasures of synchronization for discrete data (e.g. spike trains)

Victor-Purpura distance

Van Rossum distance

Schreiber correlation measure

ISI-distance

SPIKE-distance

Measure comparison

Next lecture

Lecture series: Data analysis

Documents

Transcript of Lecture series: Data analysis