Cellular Automata

Simulation of discrete spatio-temporal systems

Systems with many variables

� Iterative function systems describe systems with a single variable

� A iterative system with two variables was given by the Julia set

� Systems in economy, meteorology, ecology, sociology, etc., consists often of a large number of variables, interacting with each other

� Besides chaos many new phenomena occur in such many variable systems

� These are for example evolution, self-organization, emergence, self-reproduction, phase-transitions

Complexity, Chaos, and Anti-Chaos

� The study of spatio-temporal systems will reveal that complexity and chaos are not the same

� Chaotic processes can produce simple patterns called anti-chaos

� The emergence of order out of chaos can be observed (Cohen, Stewart 1994)

S.A. Kauffman (1991) Antichaos and adaptation, Scientific American,265 (2), 64-70J. Cohen and I. Stewart 1994: The collapse of chaos, Penguin, NY

Simple models of complex systems

� A realistic treatment of complex systems is computationally expensive and often intractable

� Scientists are looking for simple models of complex systems

� Often the predictions made with these simple models are surprisingly realistic or provide deep insights in the dynamics of real world systems (e.g. explaining phase-transitions).

� However, the science of complexity is in its very beginning and a new frontier in nonlinear systems dynamics; definitions are evolving and scientists have not yet discovered unifying theories;

T. Bohr et al. (1998): Dynamical systems approach to turbulence, CambridgeUniversity Press, New York

Locally interacting cell arrays

� One of the simplest models involve a spatial array of cells

� The cells interact with nearby cells by simple rules

� These systems often exhibit spatio-temporal chaos– Spatial patterns in time are aperiodic and difficult

to predict

– Complex, often self-similar,patterns evolve in time

Cellular automata

� Von Neumann introduced cellular automata 1966, Wolfram studied them extensively and classified them (“A new kind of science”)

� CA are perhaps the most simple models of spatio-temporal systems, but their behavioral spectrum is wide and interesting to study

Wolfram, S. (1986) Theory and application of cellular automata, WorldScientific, Singapore

Von Neumann (1966) Theory of self-reproducing cellular automata, Univ.Of Illinois Press,Urbana, Il.

Wolfram S. (2002) A new kind of science, Wolfram Press

A motivating example – the XOR 1D Automaton

� Consider a ring of people

� Each one is wearing a cap with the bill forwards, except one who is wearing the bill to the back

� Now, each one is looking at his/her left and right neighbor, and adapts using these rules:

– Left and right neighbor have bill forwards � wear bill forwards

– Left and right neighbor have bill backwards � wear bill forwards

– If only one neighbor has bill forwards � wear bill backwards

� Evolve system over a number of generations for a large ring of people

Simulation for 15 people, 7 time-steps

BBBB

BBBB

BB

BBBB

BB

BB

B

time

space

Periodic boundary condition

BBBB

BBBB

BBBB

BBBB

BB

BB

BBBBBBBB

Sierpinski cones and maximal-speed of information

� Long term behavior shows self-similar cone-like structures

� They resemble Sierpinskitriangles

� The maximal speed-of-information gives rise to the boundary of the cones, similar to the speed-of-light giving rise to Minkowski’s space-time cones

� Indeed, in CA literature the set of possible states that influenced the system in some past are called ‘light cones’

� Complex organization, but no chaos is evident

Sierpinski Triangle in Nature

Formal expression

� The given game is an example of a 1-D cellular automaton

� Several ways to express a cellular automata rule

– X(t+1,i)=(X(t,i-1)+X(t,i+1)) modulo 2

– X(t+1,i)=X(t,i-1) XOR X(t,i+1)

� The XOR statement is a logical function

� 24 = 16 logical functions could be tried instead of XOR

x411

x301

x210

x100

t+1,it,i+1t,i-1

Rule of a cellular automaton

Initial state

� The evolution of a cellular automaton is defined also by its initial state

� The left figure shows the evolution of a cellular automaton with random initial condition using the XOR function

� The behavior is not chaotic, but propagate the initial conditions in an ordered way forward in time

� Other rules may give rise to chaotic behavior from ordered starting conditions

The size of the rule space

� The discussed XOR automaton is an example of an 1-D Cellular automaton

� The size of the neighborhood and the number of possible states determine the number of possible rules for a cellular automaton

� If we consider N nearest neighbors to each side, the number of possible rules would grow to:

Why?S(N) = 222N

Four dynamical classes of Cellular automata

� Cellular automata were classified by Wolfram (2002), into four classes based on their dynamics

1. Class 1 reach a homogeneous state with all cells the same for all initial conditions

2. Class 2 reach a non-uniform state that is either constant or periodic in time, with a pattern depending on initial conditions

3. Class 3 have somewhat random patterns, are sensitive to initial conditions, and small scale local structure

4. Class 4 have relatively simple localized structures that propagate and interact in very complicated ways

� The four classes correspond roughly to fixed points, periodicity, and chaos in dynamical systems � examples will follow

Langton’s λ λ λ λ quantity

� Langton’s quantity λ is the number of state configurations that map to 1 divided by the total number of state configurations

� For instance in the left figure λ=3/8

� As the numbers of 0 equals 1- λ, only the range from 0 to 0.5 is of particular interest

0111

0011

0101

1001

0110

0010

1100

1000

t+1,it,i+1t,it,i-1

Langton, C. (1986) Studying artificial life with cellular automata, Physica D 22, 120-49

Langton’s λ λ λ λ quantity and dynamic behaviour

Solid at zero temperature

Melting FluidSolid at finite temperature

Turbulent Fluid

Melting fluid

Solid at finite temperature

λ

By increasing λ from 0 to 0.5 (1 downto 0.5) roughly the system goes through the same states than the logistic map for different values of the constant a� Assignment

Higher dimensional Cellular automata

� Cellular automata can be defined not only for 1-D arrays but also on higher dimensional arrays

� Some mathematical notation:

– 1-D arrays are called chains

– 2-D arrays are called grids

– Arrays of any dimension d are called d-dimensional lattices

Cellular automata in 2-D

� A classical cellular automaton was defined by Conway –Conway’s game of life

� Consider a ‘game’ played on a rectangular grid, each grid cell can have two states – dead or alive

� The neighbors of a center cell are the nearest neighbors to the north, south, east, west, north-west, south-west, south-east, north-east

� This is termed the Moore Neighborhood

SESSW

ECW

NENNW

Cellular automata in 2-D

� Rule– A cell that is alive, stays

alive, if it has two or three living neighbors

– A dead cell becomes alive, when it has exactly three living neighbors

– For all other cases a cell dies or remains dead

� Example of outer totalistic rule, i.e. a rule that involves only the sum of neighbor states

SESSW

ECW

NENNW

Evolution of the game of life

Starting froman initially randomConfiguration Colonies of cellsemerge, someof them periodicsome of them fixedor moving throughspace, shooting pixels (glider guns*)etc.

*Berlekamp et al. (1982) Winning ways for your mathematical plays, AcademicPress, New York

The glider gun

� Conway offered 50$ for everyone, who could find an endlessly growing configuration or prove that none exists

� William Gosper and 5 other MIT students discovered the glider gun and won the price

– The glider gun shoots a copy of itself

– On an infinite grid it would grow and evolve without limit

Other possible configuration spaces

� Regular tilings of the 2-D plane (there are three possibilities)

� More than 3-dimensional configuration spaces

� Most generally:– Configuration spaces

represented by Caleygraphs of some group

All possible regular tilings ofthe 2-D plane,i.e. tilings consistingonly of the same objects

hexagonalgrid

General CA definition via Caley graph

� Groups describe symmetric structures

� (M,+) is a group, iff ∀a,b,c∈M:– a+b∈ M and a+(b+c)=(a+b)+c

– There exists e∈M with e+a =a

– each a∈M has an inverse called (-a), such that a+(-a)=e.

� We define a group M via a set of generators X ⊆ M, such that for every element a∈M and generator x, both a+x∈M and a+(-x)∈M; Moreover, all elements belong to the group that can be obtained by concatenated application of generators.

� Given a generator {x1,… ,xm} we can define a Caley graph C=(V,E) of the group:

– vertex set: V=M

– edges E are given by (v1,v2)∈E, iff v1 = v2+a, or v1=v2+(-a) for some a in X.

� The fundamental neighborhood N(C)of the Caley graph is defined by the union of the set of generators and the set of its inverse elements.

� For each element in the graph we can get its neighbors by using the generator elements in N(C).

� A cellular automaton (C, N(C),A, T) is defined as a tuple of a Caley graph with labeled vertices, its fundamental neighborhood, an finite alphabet, and a transition function T

� Node labels are chosen from a finite local state space A

� Transition rule T: A|N(C)+1|�A assigns each element of a cell a new value based on the neighbors in the Caleygraph, obtained by applying the generator.

Fd = 〈x1, . . . , xm|〉 Free group

Zd = 〈x1, . . . , xd|∀m,n : xm + xn = xn + xm〉

Zm × Zn = 〈a, . . . , b|ab = ba,ma = nb〉 Torus

Neighborhood types and sizes

� Von Neumann neighborhood and Moore neighborhood are most commonly used in 2-D grids

� The radius of these neighborhoods can be increased, e.g. by applying group generators twice

Examples of groups and their Caleygraphs

Fd = 〈x1, . . . , xm|〉 Free group

Zd = 〈x1, . . . , xd|∀m,n : xm + xn = xn + xm〉

ab

x1

x2

The group Z2 and its fundamental neighborhood

Freegroup

Zm × Zn = 〈a, . . . , b|ab = ba,ma = nb = e〉 Torus

Questions

� How many transition rules can we define on a cellular automaton (C, N(C), A, T)?

� What could be the set of generators for the -2D integer lattice with Moore neigborhood?

Extensions of Cellular automata

� A multidimensional state space

– In Lattice gas models each cell is assigned a vector (velocity of the fluid flow)

� Memory of states in t-1, t-2, etc.

� Dynamic rule sets, dynamic neighborhoods, etc.

Finite cellular automata and chaos

� Finite CA cannot be truly chaotic because the number of states is finite, and thus the system will eventually return to some previous state and be trapped in a circle from then on

� To obtain maximal periods, prime numbers are chosen as cell array sizes

Self-organization

� Simple rules such as the game of life can cause an initially chaotic state to evolve into a highly ordered one � Self-organization

� This somehow contradicts the third law of thermodynamics (3LT), that the entropy is always increasing

� Haken attributed the self-organizing behavior to cooperative effects of the systems components (synergetics)

� The 3LT is motivated by deterministic systems, but in fact also stochastic systems can self-organize

Forest simulation model by Sprott

� Consider a forest with trees placed on grid cells, 0=fur, 1=oak

� We choose a random tree that dies

� We replace this tree with a new tree

� Five trees in the neighborhood are chosen randomly

� If the vast majority (s=4,5) is oak, the new tree gets an oak

� If the vast majority is fur (s=0,1), the new tree gets a fur

� Otherwise (s=2,3), the same tree than before will grow

� Connected patterns emerge from a random starting set

Broken symmetry

� It is surprising, that despite the highly symmetrical starting conditon the emerging system does not converge to a symmetrical object

� This phenomenon is called spontaneous symmetry breaking and can be observed in highly ordered systems, deterministic systems (Wolfram 2002)

Self-organized critically

� So-called dissipative structures will emerge

� Connected regions with a strange but not necessarily fractal boundary (fat fractals)

� The size distribution of the clusters follows a power laws

� Dissipative patterns are observed in many spatio-temporal processes

– Animal migration– Spread of diseases– Vegetation patterns– Clouds and mud

Prigogine, I. (1997) The end of certainty: time, chaos, and the new laws ofnature, the free press, new york

Self-organized critically

� Systems like the forest converge to a pattern for which there is no characteristic scale size

� Size distributions of objects often obey power laws (this they share with the fractals), i.e. the distribution can be fitted to a function

� Recently, power laws are applied in all kind of applications

– Gene regulatory networks– DNA pattern– Stock prices– City distributions– Letter frequency in human/ape

generated random strings (Zipf)

� Not always SOC is the explanation for the Power law

� In case of city size distribution it related to a least effort principle (Zipf).

d ∼ 1/fα

Diffusion

� Diffusion can be modelled via:

� Note, that there is a conservation rule fulfilled

� Task: Implement diffusion system in 2-D in MATLAB and visualize its behaviour over time

aj(t+1)−aj(t)=aj+1(t)−2aj(t)+aj−1(t)

Sand Pile – the prototype of a SOC system

� Consider a pile of sand to which we add sand continuously

� The sand-pile steepens until it reaches an angle of repose, whereupon avalanches keep the sandpile close to this angle

� The avalanches obey a power law scaling in their size distribution and in their duration

A pile of cheese

Bak’s CA simulation of a pile of sand

� Bak simulated a pile of sand using the following CA model

� The pile is represented by a N ×N matrix of integers

� Initially all cells are chose between 1 and 3

� At each time step choose a random cell i,j and set Z(t+1,i,j)=Z(t,i,j)

� Cells outside the boundary are kept as 0� All other cells which exceed Z=3, and

their von-Neumann neighbors are updated with:Z(t+1,i,j)=Z(t,i,j)-4Z(t+1,i±1,j)=Z(t,i±1,j)+1Z(t+1,I,j±1)=Z(t,I,j±1)+1

� Strictly speaking, this is not a cellular automaton, as it evolves not autonomously

d=2

Power spectrum of sum(Z(i,j)) over t

Dropping sand on the central point

Emergence vs. Reductionism

� Reductionism assumes simple laws that govern natural processes and that these simple laws help to understand/explain global behaviour

� Emergence holds that high level structure is generally unpredictable from low level processes, and does not even depend very much on its properties

� Due to Sprott (2006), systems are complex, if they exhibit emergent behaviour

How to measure degree of (self-)organization

� The term of self-organization is used since 1947, but up to know there is no standard definition except “I know it when I see it”

� Thermodynamic entropy measures the degree of a system’s “mixedupedness”(to use Gibbs’s word), or how far it departs from being in a pure state

� Organisms are essentially never in pure states, and are highly mixed up at the molecular level, but are the paradigmatic examples of organization.

� Furthermore, there are many different kinds of organization, and entropy ignores all the distinctions and gradations between them

W. R. Ashby, “Principles of the self-organizing dynamic system,” Journal of General Psychology 37, pp. 125–128, 1947.

How to measure a degree of self-organization and complexity?

� Another school of thought has been put forward by Kolmogorov and Solomonoff

“A complex phenomena is one which does not admit of descriptions which are both short and accurate”

� Problem exactness: Coin tossing, produces sequences of maximal Kolmogorovcomplexity, though dynamics are simple to describe.

� Grassberger gave a more general definition: ‘The complexity of a process as the minimal amount of information about its state needed for maximally accurate prediction’

� Crutchfield and Young gave operational definitions of “maximally accurate prediction”and “state”

� The Crutchfield-Young “statistical complexity”, C

µ, of a

dynamical process is the Shannon entropy (information content) of the minimal sufficient statistic for predicting the process’s future.

� Shalizi and Shalizi used this measure recently to quantify self-organization in CA practically

� They used cyclic CA to assess their method

Cyclic CA

� Cyclic cellular automata (CCA) are simple models of chemical oscillators.

� Started from random initial conditions, they produce several kinds of spatial structure, depending on their control parameters.

� They were introduced by David Griffeath, and extensively studied by Fisch

� Transition rule

– Each site in a square two-dimensional lattice is in one of κ colors.

– A cell of color k will change its color to k + 1 mod κ if there are already at least T cells of that color in its neighborhood

– Otherwise, the cell retains its current color

Fisch, R. (1990a). "The one-dimensional cyclic cellular automaton: A system with deterministic dynamics that emulates an interacting particle system with stochastic dynamics". Journal of Theoretical Probability 3 (2): 311–338.

Cyclic CA

� The CCA has three generic forms of long-term behavior, depending on the size of the threshold relative to the range.

� At high thresholds, the CCA forms homogeneous blocks of solid colors, which are completely static — so-called fixation behavior.

� At very low thresholds, the entire lattice eventually oscillates periodically;

– sometimes the oscillation takes the form of large spiral waves which grow to engulf the entire lattice.

� There is an intermediate range of thresholds where incoherent traveling waves form, propagate for a while, and then disperse;

– this is called “turbulence”, but whether it has any connection to actual fluid turbulence is unknown.

Spirallingwaves

Turbulent behavior

Spirals engulfing the space for Moore neighborhood

� Cyclic CA for a Moore neighborhood and T=2

� For Moore neighborhood the following transitions can be found:

– T=1: local oscillations

– T=2: spiraling waves

– T=3: turbulence, often metastable in very long run (then spirals can take over)

Cellular Automata and beyond

� Statistical complexity of cyclic CA over time

Cosma Rohilla Shalizi and Kristina Lisa Shalizi: Quantifying Self-Organization in Cyclic Cellular Automata, http://arxiv.org/abs/nlin/0507067v1

Cellular automata and beyond

Partial Differential equations

ContinousContinuousContinuous

DiscreteContinuousContinuous

ContinuousDiscreteContinuous

DiscreteDiscreteContinuous

Coupled Flow Lattices

ContinuousContinuousDiscrete

DiscreteContinuousDiscrete

Coupled Map Lattices

ContinuousDiscreteDiscrete

Cellular AutomatonDiscreteDiscreteDiscrete

ModelStateTimeSpace

Summary (1)

� Cellular automata are defined on a Caley graph (with state labels) with a neighbourhood and transition rule mapping the state of a center cell to a new state based on its neighbor states.

� The number of possible transition rules grows exponentially with the size of the local state space and neighborhood

� Common neighborhood types are von Neumann and Moore neighborhood, and the k-neighbors in 1-D arrays (with periodic boundary conditions)

Summary (2)

� CA are simple models of natural systems� Despite their simplicity the behavior of CA can be

extremely complex and difficult to predict� CA serve as models for studying emergent

phenomena and self-organization� Self organized systems are often at the boundary of

chaotic and ordered states; many open questions remain, and definitions are not yet clarified

� An interesting question if the type of global behavior can be predicted from properties of the rules (e.g. Langtons lambda)

Summary (3)

� As simulators CA models are easy to implemented (also in parallel) and can be used to model phenomena such as diffusion, cell systems, flow, pattern formation, etc.

� CA can be seen as discrete counterparts of partial differential equations

� As such they belong to the class of spatio-temporal models