Dynamical systems, information and time serieshomepage.sns.it/marmi//lezioni/DSITS_1.pdf ·...

Dynamical systems,

information and time series

Stefano Marmi

Scuola Normale Superiore

http://homepage.sns.it/marmi/

Lecture 1- European University Institute

September 18, 2009

• Lecture 1: An introduction to dynamical systems and to time series. Periodic and

quasiperiodic motions. (Tue Jan 13, 2 pm - 4 pm Aula Bianchi)

• Lecture 2: A priori probability vs. statistics: ergodicity, uniform distribution of

orbits. The analysis of return times. Kac inequality. Mixing (Sep 25)

• Lecture 3: Shannon and Kolmogorov-Sinai entropy. Randomness and deterministic

chaos. Relative entropy and Kelly's betting. (Oct 9)

• Lecture 4: Time series analysis and embedology: can we distinguish deterministic

chaos in a noisy environment? (Oct 30)

• Lecture 5: Fractals and multifractals. (Nov 6)

Sept 18, 2009Dynamical systems, information and time

series - S. Marmi2

References:

• Fasano Marmi "Analytical Mechanics" Oxford University Press Chapter 13

• Benjamin Weiss: “Single Orbit Dynamics”, AMS 2000.

• Daniel Kaplan and Leon Glass: "UnderstandingNonlinear Dynamics" Springer (1995)

• Sauer, Yorke, Casdagli: Embedology. J. Stat. Phys. 65 (1991) 579-616

• Michael Small "Applied Nonlinear Time SeriesAnalysis" World Scientific

• K. Falconer: "Fractal Geometry - MathematicalFoundations and Applications" John Wiley,

The slides of all lectures will be available at my personal webpage: http://homepage.sns.it/marmi/


series - S. Marmi3

An overview of today’s lecture

• Dynamical systems

• What is ergodic theory? From statistics to probability

• Determinism and randomness

• Statistical induction, backtesting and black swans

• Information, uncertainty and entropy

• Risk management and information theory


series - S. Marmi4

Dynamical systems

• A dynamical system is a couple (phase space, time evolution law)

• The time variable can be discrete (evolution law = iteration of a map) or continuous (evolution law = flow solving a differential equation)

• The phase space is the set of all possible states (i.e. initial conditions) of our system

• Each initial condition uniquely determines the time evolution (determinism)

• The system evolves in time according to a fixed law (iteration of a map, differential equation, etc.)

• Often (but not necessarily) the evolution law is not linear


series - S. Marmi5

Powers of 2


series - S. Marmi6

Period 4


series - S. Marmi7

Period 20

02 →04 →08 →16 →32 →64 →28 →56 →24 →48 →96 →92 →84 →68 →36 →72 →44 →88 →76 →52 →04 →08 →16 →…


series - S. Marmi8

Not quite periodic…

2 →4 →8 →1 →3 →6 →1 →2 →5 →1 → 2 →4 →8 →1 →3 →6 →1 →2 →5 →1 → 2 →4 →8 →1 →3 →6 →1 →2 →5 →1 → 2 →4 →8 →1 →3 →6 →1 →2 →5 →1

→ 2 →4 →8 →1 →3 →6 →1 →2 →5 →1 → 2 →4 →8 →1 →3 →7→1 →2 →5 →1 →


series - S. Marmi9

Quasiperiodicity

The sequence

2 →4 →8 →1 →3 →6 →1 →2 →5 →1 → 2 →4 →8 →1 →3 →6 →1 →2 →5 →1 → 2 →4 →8 →1 →3 →6 →1 →2 →5 →1 → 2 →4 →8 →1 →3 →6 →1 →2 →5 →1 → 2 →4 →8 →1 →3 →6 →1 →2 →5 →1 → 2 →4 →8 →1 →3 →7→1 →2 →5 →1 →

is the result of a symbolic analysis of the iteration of the map

log xn = log xn−1 + log 2 (mod 1) ,

Since log10 2 = 0.3010…. ≈ 3/10 for a long time the sequenceseems to be periodicSept 18, 2009

Dynamical systems, information and time series - S. Marmi

10

Quasiperiodic dynamics

id f knFor some sequence kn

Sept 18, 2009 11Dynamical systems, information and time

series - S. Marmi

• Quasiperiodic = periodic if precision is finite, but the period→∞ if the precision of measurements is improved

• More formally a dynamics f is quasiperiodic if

return times

ff1nk

Renormalization

approach

1nk

The simplest dynamical systems

• The phase space is the circle: S=R/Z

• Case 1: quasiperiodic dynamics

θ(n+1)=θ(n)+ω (mod 1)

(ω irrational, for example

θ=(√5-1)/2=0.618033989…)

• Case 2: chaotic dynamicsθ(n+1)=2θ(n)(mod 1)


series - S. Marmi12

ω

ω

T(θ) = θ +ω (mod 1).

phase space X = [0, 1)

T : θ → 2 θ (mod 1).

13Dynamical systems, information and time

series - S. MarmiSept 18, 2009


series - S. Marmi14

Sensitivity to initial conditions

• « ... For, in respect to the latter branch of the

supposition, it should be considered that the most

trifling variation in the facts of the two cases might

give rise to the most important miscalculations, by

diverting thoroughly the two courses of events, very

much as, in arithmetic, an error which, in its own

individuality, may be inappreciable, produces, at

length, by dint of multiplication at all points of the

process, a result enormously at variance with truth.

... »


series - S. Marmi

The mystery of Marie Roget, 1843

Edgar Allan Poe (1809-1849)Sept 18, 2009 16


Sensitivity to initial conditions

For the doubling map on the circle (case 2) one has

θ(N)- θ’(N)=2N (θ(0)- θ’(0)) even if the initialdatum is known with a 10 digit accuracy, after 40 iterations one cannot even say if the iterates are largerthan ½ or not

In quasiperiodic dynamics this does not happen: for the rotations on the circle one has θ(N)- θ’(N)= θ(0)- θ’(0)

and long term prediction is possible

The dynamics of the doubling maps is heterogeneous and unpredictable, quasiperiodic dynamics is homogeneousand predictable


series - S. Marmi17

Heterogeneous dynamics of the doubling map

If θ(0) = 1/3 = 0.333.., θ(n) = θ(0) for all n: stationarity

If θ(0) = 3/11 = 0.272727.. θ(2n) = θ(0)

θ(2n+1) = θ(1) = 0.72727..

Period 2

If θ(0) = 1/7 = 0.142857142857.. Period 6

If θ(0) = p - 3 = 0.1415926.. , aperiodic, random…. (?)


series - S. Marmi

Chaotic dynamics

Sensitive dependence on initial conditions

Density of periodic orbits

Some form of irreducibility (topological transitivity,

ergodicity, etc.)

Information is produced at a positive rate: positive entropy (Lyapunov exponents)


series - S. Marmi

Topological transitivity

A dynamical system is topologically transitive if it hasa dense orbit.

In the case of the doubling map: x(n+1)=2x(n) (mod 1)

One observes that it corresponds to a shift on the binarydigits of : if x(n)=Σa(n,k)2-k then

x(n+1)= Σa(n,k)2-(k-1)

The orbit ofx=.01000110110000010100111001011101110000000100100011010001010110011110001001101010111100110111101111…is dense


series - S. Marmi

Ergodic theory

The focus of the analysis is mainly on the asymptotic ditribution of the orbits, and not on transient phenomena.

Ergodic theory is an attempt to study the statistical behaviour of orbits of dynamical systems restricting the attention to their asymptotic distribution.

One waits until all transients have been wiped off and looks for an invariant probability measure describing the distribution of typical orbits.


series - S. Marmi21

Ergodic theory: the setup (measure

preserving transformations, stationary

stochastic process)

X phase space, μ probability measure on X

Φ:X → R observable, μ(Φ) = ∫X Φ dμ expectation valueof Φ

A measurable subset of X (event). A dynamics T:X→X induces a time evolution:

on observables Φ → Φ T

on events A →T-1(A)

T is measure-preserving if μ(Φ)= μ(Φ T) for all Φ, equivalently μ(A)=μ(T-1(A)) for all A


series - S. Marmi

Measure theory vs. probability

theory


series - S. Marmi23

Birkhoff theorem and ergodicity

Birkhoff theorem: if T preserves the measure μ then

almost surely the time averages of the observables

exist (statistical expectations). The system is ergodic

if these time averages do not depend on the orbit

(statistics and a-priori probability agree)


series - S. Marmi24

Law of large numbers:

Statistics of orbits =

a-priori probability

“Historia magistra vitae” or the

mathematical foundation of backtesting

Even without assuming ergodicity, Birkhoff

theorem shows that:

• Time averages exist and they give rise to an

experimental statistics to compare with theory

• Past and future time averages agree almost

everywhere


series - S. Marmi25

Ergodicity and time averages appliedto the powers of 2

Ergodicity of irrational rotations implies that in the sequenceof the most significant digit of the powers of 2


series - S. Marmi26

7 appears more frequently than 8 even if looking at the first iterates one would guess otherwise

The doubling map and coin tosses

Two measure preserving dynamical systems

which are conjugate by a measurable

isomorphism have the same orbits statistics

…The doubling map x→2x (mod 1) and the

Bernoulli scheme B(1/2,1/2) have the same

probabilistic properties

…But the first is clearly deterministic


series - S. Marmi27

Determinism, randomness and the

physical laws (T. Tao, http://terrytao.wordpress.com/2007/04/05

/simons-lecture-i-structure-and-randomness-in-fourier-analysis-and-number-theory/)

It may well be that the universe itself is completely deterministic (though this

depends on what the “true” laws of physics are, and also to some extent on

certain ontological assumptions about reality), in which case randomness is

simply a mathematical concept, modeled using such abstract mathematical

objects as probability spaces. Nevertheless, the concept of pseudorandomness

- objects which “behave” randomly in various statistical senses - still makes

sense in a purely deterministic setting. A typical example are the digits

of π=3.1415926535897932385…this is a deterministic sequence of digits, but

is widely believed to behave pseudorandomly in various precise senses (e.g.

each digit should asymptotically appear 10% of the time). If a deterministic

system exhibits a sufficient amount of pseudorandomness, then random

mathematical models (e.g. statistical mechanics) can yield accurate predictions

of reality, even if the underlying physics of that reality has no randomness in it.28


Sept 18, 2009

Truely random?

1 0 1 1 0 0 1 1 1 0 1 0 1 0 0 0 0 0 1 1 1 1 0 0 1 1 1 0 1 1 0 0 0 0 1 0 1 1 0 0 1 0 0 1 1 1 1 0 1 1 0 1 0 0 1 0 0 0 1 1 1 0 1 0 0 0 0 1 0 1 0 0 1 0 1 0 0 00 0 0 0 1 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 1 0 0 0 1 0 0 1 1 1 0 0 0 0 1 1 0 0 1 0 0 1 1 1 1 1 0 0 0 1 0 0 0 1 0 1 1 1 1 1 1 1 1 0 0 0 0 0 1 1 0 1 1 0 0 0 1 1 1 1 0 0 1 0 1 0 1 1 1 0 1 0 1 0 1 1 0 1 0 0 1 0 0 0 0 1 0 0 0 1 0 0 1 0 1 0 0 1 0 0 0 0 0 0 1 1 1 1 1 1 0 1 0 0 0 0 0 0 1 1 0 1 0 1 0 0 0 0 0 1 0 0 0 1 1 0 0 0 0 0 1 0 0 1 1 0 1 1 1 1 1 0 0 0 1 0 1 0 1 0 0 1 1 0 0 1 0 1 1 0 1 0 1 0 1 0 1 1 0 1 1 1 1 0 0 1 1 0 0 0 1 0 0 0 0 0 1 0 1 0 1 0 0 0 0 0 0 01 0 0 1 0 1 1 1 0 1 1 1 0 1 1 0 0 0 1 1 1 0 0 1 1 1 1 0 1 1 0 0 1 1 0 1 0 1 1 1 1 1 0 1 1 1 1 1 1 1 0 0 1 0 0 1 0 1 1 0 0 0 0 1 0 1 0 1 1 0 1 1 1 1 1 1 1 10 1 1 0 0 1 0 1 1 1 1 0 1 1 0 1 1 1 1 0 1 0 0 0 1 0 0 1 1 1 0 0 1 1 1 0 1 0 1 0 1 1 1 0 1 0 1 0 0 1 1 1 1 0 0 1 0 0 1 0 0 0 0 1 1 1 1 0 1 0 0 0 0 0 1 1 1 11 0 1 0 0 0 0 0 0 0 1 1 0 1 0 1 0 1 1 0 1 1 1 0 1 0 0 1 0 1 0 0 1 1 1 0 1 1 1 1 1 1 1 0 0 0 1 0 0 0 0 1 0 0 1 1 1 0 1 1 1 0 0 0 1 0 1 0 0 1 1 0 1 1 0 0 0 01 1 0 1 1 0 1 0 0 0 1 1 0 1 0 1 1 0 1 1 1 1 0 0 0 1 1 1 0 1 1 1 0 0 0 0 0 0 0 0 1 1 1 1 0 1 1 1 1 1 1 0 1 0 1 1 1 0 1 0 1 0 1 0 1 1 1 0 0 0 0 1 0 1 0 1 1 11 0 1 1 0 1 0 0 1 0 0 0 1 1 0 0 1 1 0 0 0 1 1 0 1 0 1 0 1 1 1 1 1 0 0 1 0 0 0 0 1 1 1 0 0 1 0 1 1 1 0 1 1 1 0 1 0 1 0 0 0 1 1 0 1 0 1 0 1 1 0 0 0 1 0 0 1 0 1 1 0 1 0 0 0 0 1 1 0 0 1 1 0 0 0 1 1 1 1 1 1 1 1 0 0 1 0 0 0 1 0 1 0 0 1 1 1 1 0 1 1 1 0 1 1 1 0 1 1 1 1 0 1 0 0 0 0 0 1 1 0 1 0 0 0 0 1 1 0 0 1 0 1 1 1 11 0 0 1 1 1 0 0 0 0 0 0 1 0 0 1 1 0 0 0 0 0 1 0 0 0 1 1 0 1 0 0 0 0 1 1 1 0 0 1 0 1 1 1 1 1 0 0 1 0 1 0 0 1 0 0 1 1 0 1 1 0 0 1 0 0 1 0 1 0 0 0 1 0 0 1 1 11 0 0 1 0 0 0 1 1 0 0 0 1 1 0 1 0 0 0 1 1 1 0 1 0 0 1 1 0 1 0 1 1 0 1 1 1 0 1 0 1 1 1 0 0 1 0 0 0 1 0 0 1 0 1 0 0 0 0 1 0 0 1 0 0 0 0 0 0 0 0 1 1 1 1 0 0 01 0 1 1 1 0 0 0 1 0 1 0 1 1 0 0 1 0 1 0 0 0 0 0 0 0 1 1 1 1 1 1 0 0 0 1 0 0 0 1 0 0 1 0 0 0 1 1 1 0 0 0 0 0 0 0 1 0 0 1 1 0 0 0 0 0 0 0 0 1 0 0 1 0 1 0 0 00 1 0 1 0 1 0 0 0 1 0 1 0 1 1 1 0 0 1 0 0 1 0 1 1 1 1 1 0 1 1 1 1 0 0 1 0 0 0 0 1 0 0 0 1 1 0 0 0 0 0 0 0 1 0 0 1 1 1 0 0 0 0 0 1 0 1 1 0 0 1 0 1 1 1 1 0 00 0 0 1 0 1 0 1 0 0 1 0 0 1 1 0 0 1 0 0 0 1 1 1 0 0 0 0 1 0 1 1 1 1 1 1 1 0 0 1 1 1 1 1 1 1 1 1 1 1 0 1 1 1 1 1 0 1 0 0 1 1 0 0 0 0 1 0 1 1 0 0 0 0 0 0 0 1 0 0 0 0 1 1 0 0 0 1 0 0 1 1 1 0 0 0 0 1 1 0 0 0 0 0 0 1 0 0 0 0 1 1 1 1 1 1 0 0 0 0 1 1 0 1 1 0 1 1 1 0 0 0 1 1 0 1 1 1 0 0 0 1 0 1 1 0 0 1 1 1 1 1 1 0 1 10 0 1 0 0 0 1 0 0 1 0 0 0 0 1 1 0 0 1 0 0 1 1 1 0 0 0 1 1 1 0 1 0 0 1 1 0 1 0 0 0 0 1 0 0 0 0 0 0 1 0 0 1 1 0 1 1 0 1 0 1 0 1 0 1 1 0 0 1 1 1 0 0 1 0 0 0 1 0 1 0 1 0 1 1 1 0 1 0 0 1 0 1 1 0 1 0 0 0 1 0 0 0 1 1 1 1 0 0 0 0 0 1 1 1 0 0 1 1 0 0 1 0 1 0 1 1 1 0 1 1 0 1 1 0 1 0 0 0 1 0 0 0 1 0 0 0 0 0 1 0 0 1 0 1 0 0 0 1 1 1 0 1 0 0 0 0 0 1 1 0 0 0 1 0 0 0 0 1 1 1 1 1 1 0 1 1 0 0 1 0 0 1 1 0 0 0 0 1 1 0 1 0 0 1 1 1 1 1 1 1 0 1 1 1 0 1 0 1 0 0 1 1 0 1 1 1 1 1 0 1 1 1 10 0 0 0 1 1 0 0 0 0 1 1 1 0 1 0 1 1 0 1 0 1 0 1 0 0 1 1 0 1 1 0 0 0 0 1 0 1 0 1 0 0 1 1 0 0 1 0 0 1 0 1 1 0 1 0 0 0 1 0 0 0 0 1 0 0 0 0 0 1 1 0 0 1 0 0 1 11 0 0 0 1 1 0 1 1 0 0 0 0 0 0 1 0 1 0 0 0 0 1 0 1 1 1 1 1 0 0 1 0 1 0 1 1 1 1 1 0 1 1 0 0 1 1 0 1 0 1 0 1 0 0 1 1 1 0 0 1 0 1 1 1 1 1 1 1 0 1 1 0 0 1 1 1 10 0 1 0 1 0 1 1 1 1 0 1 0 0 1 0 1 1 1 0 0 1 0 0 1 1 0 0 1 1 0 0 0 1 1 0 0 0 1 0 0 0 0 1 0 0 0 1 1 0 1 0 1 0 0 0 0 1 1 0 1 1 0 0 0 0 0 0 0 1 1 0 1 0 0 0 0 1 0 0 0 0 0 1 0 1 0 0 1 0 1 0 0 1 0 1 1 0 0 0 0 1 0 1 0 0 1 1 0 0 0 0 1 0 1 1 1 0 0 0 1 0 1 0 0 1 1 1 0 0 0 0 0 1 1 0 0 1 1 0 0 1 0 1 0 0 0 1 1 0 1 0 1 0 1 0 0 0 0 1 1 0 1 0 0 0 1 0 1 1 1 1 0 0 1 1 0 0 1 0 0 0 0 0 0 1 0 0 0 1 1 0 0 0 0 1 0 0 0 1 0 0 1 1 1 0 0 0 0 0 0 1 1 0 1 0 0 1 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 00 1 1 0 1 1 1 0 1 0 0 0 1 0 0 0 0 0 1 0 1 0 0 1 0 0 0 1 0 1 1 1 0 0 1 1 0 0 0 0 0 0 0 0 0 0 0 0 1 0 1 1 1 0 1 1 0 1 1 1 0 1 1 1 0 0 0 1 0 1 0 0 0 1 1 0 0 01 1 1 1 1 0 1 0 0 0 0 1 1 1 1 0 1 1 0 0 1 0 0 0 0 0 0 0 0 0 0 0 1 1 0 1 0 0 1 0 0 1 0 1 0 0 0 0 1 0



www.random.org

•

True Random Number ServiceIntroduction to Randomness and Random NumbersRANDOM.ORG is a true random number service that generates randomness via atmospheric noise. This page explains why it's hard (and interesting) to get a computer to generate proper random numbers.

Random numbers are useful for a variety of purposes, such as generating data encryption keys, simulating and modeling complex phenomena and for selecting random samples from larger data sets. They have also been used aesthetically, for example in literature and music, and are of course ever popular for games and gambling. When discussing single numbers, a random number is one that is drawn from a set of possible values, each of which is equally probable, i.e., a uniform distribution. When discussing a sequence of random numbers, each number drawn must be statistically independent of the others.



http://en.wikipedia.org/wiki/Uniform_distribution_(discrete)

http://en.wikipedia.org/wiki/Uniform_distribution_(discrete)

http://en.wikipedia.org/wiki/Statistical_independence

Examples of time-series in natural and

social sciences

• Weather measurements (temperature, pressure, rain, wind speed, …) .

If the series is very long …climate

• Earthquakes

• Lightcurves of variable stars

• Sunspots

• Macroeconomic historical time series (inflation, GDP,

employment,…)

• Financial time series (stocks, futures, commodities, bonds, …)

• Populations census (humans or animals)

• Physiological signals (ECG, EEG, …)


series - S. Marmi31

Stochastic or chaotic?

• An important goal of time-series analysis is to

determine, given a times series (e.g. HRV) if the

underlying dynamics (the heart) is:

– Intrinsically random

– Generated by a deterministic nonlinear chaotic

system which generates a random output

– A mix of the two (stochastic perturbations of

deterministic dynamics)


series - S. Marmi32

Deterministic or truly random?


series - S. Marmi

Time delay map


series - S. Marmi

Logit and logistic

The logistic map x→L(x)=4x(1-x) preserves the probability

measure dμ(x)=dx/(π√x(1-x))

The transformation h:[0,1] →R, h(x)=lnx-ln(1-x) conjugates L

with a new map G

h L=G h

definined on R. The new invariant probability measure is

dμ(x)=dx/[π(ex/2 + e-x/2 )]

Clearly G and L have the same dynamics (the differ only by a

coordinates change)


series - S. Marmi35

Embedding method


series - S. Marmi36

• Plot x(t) vs. x(t- ), x(t-2 ), x(t-3 ), …

• x(t) can be any observable

• The embedding dimension is the # of delays

• The choice of and of the dimension are critical

• For a typical deterministic system, the orbit will be

diffeomorphic to the attractor of the system (Takens

theorem)


series - S. Marmi37

Dynamics, probability, statistics and

the problem of induction

• The probability of an event (when it exists) is almost always

impossible to be known a-priori

• The only possibility is to replace it with the frequencies measured by

observing how often the event occurs

• The problem of backtesting

• The problem of ergodicity and of typical points: from a single series

of observations I would like to be able to deduce the invariant

probability

• Bertrand Russell’s chicken (turkey in the USA)

http://www.edge.org/3rd_culture/taleb08/taleb08_index.html

Bertrand Russel(The Problems of Philosophy, Home University Library, 1912. Chapter VI On Induction) Available at the page http://www.ditext.com/russell/rus6.html

Domestic animals expect food when they see the person who feeds them. We know that all these rather crude expectations of uniformity are liable to be misleading. The man who has fed the chicken every day throughout its life at last wrings its neck instead, showing that more refined views as to the uniformity of nature would have been useful to the chicken.


series - S. Marmi38


http://www.ditext.com/russell/rus6.html

http://www.ditext.com/russell/rus6.html



series - S. Marmi39


Dynamical systems, information and time serieshomepage.sns.it/marmi//lezioni/DSITS_1.pdf ·...

Documents

Transcript of Dynamical systems, information and time serieshomepage.sns.it/marmi//lezioni/DSITS_1.pdf ·...