Patterns, universality and computational algorithmsnigel/articles/RG/Patterns... · • Example:...

91
Patterns, universality and computational algorithms Nigel Goldenfeld Department of Physics University of Illinois at Urbana-Champaign Collaborators: Yoshi Oono, M. Mondello (Cell dynamical systems) Navot Israeli (Cellular automata) Badrinaryan Athri and Jon Dantzig (Multiscale modeling) Work supported by the US National Science Foundation and NASA

Transcript of Patterns, universality and computational algorithmsnigel/articles/RG/Patterns... · • Example:...

Page 1: Patterns, universality and computational algorithmsnigel/articles/RG/Patterns... · • Example: BCS theory of superconductivity – Most successful many-body theory – Spectacular

Patterns, universality and computational algorithms

Nigel GoldenfeldDepartment of Physics

University of Illinois at Urbana-Champaign

Collaborators: Yoshi Oono, M. Mondello (Cell dynamical systems)Navot Israeli (Cellular automata)

Badrinaryan Athri and Jon Dantzig (Multiscale modeling)

Work supported by the US National Science Foundation and NASA

Page 2: Patterns, universality and computational algorithmsnigel/articles/RG/Patterns... · • Example: BCS theory of superconductivity – Most successful many-body theory – Spectacular

Overview• Minimal models and universality

– What aspects of physical phenomena should we try to capture?– What aspects of physical phenomena should we try to predict?

• Modelling without calculus– Cell dynamical systems

• Space-time patterns during phase transitions– Universal features of pattern formation– Comparison with experiment

• Can we make simple theories of complex phenomena?– Computational irreducibility

• Multiscale modeling of patterns– Renormalization group approach to large-scale simulation of materials

processes

Page 3: Patterns, universality and computational algorithmsnigel/articles/RG/Patterns... · • Example: BCS theory of superconductivity – Most successful many-body theory – Spectacular

Perceptions of reality: polymers

Chemist

From Chemistry by McMurry & Fay

Page 4: Patterns, universality and computational algorithmsnigel/articles/RG/Patterns... · • Example: BCS theory of superconductivity – Most successful many-body theory – Spectacular

Perceptions of reality: polymers

Computational biologist

Schultengroup UIUC

Page 5: Patterns, universality and computational algorithmsnigel/articles/RG/Patterns... · • Example: BCS theory of superconductivity – Most successful many-body theory – Spectacular

Perceptions of reality: polymers

Theoretical physicist

The Gutsy Gourmet

Page 6: Patterns, universality and computational algorithmsnigel/articles/RG/Patterns... · • Example: BCS theory of superconductivity – Most successful many-body theory – Spectacular

Perceptions of reality: polymers

What questions can each of these representations answer?

Chemical bonding, reactions, …

Elasticity, large scale motions, folding

Thermodynamics, light scattering, rheology

Page 7: Patterns, universality and computational algorithmsnigel/articles/RG/Patterns... · • Example: BCS theory of superconductivity – Most successful many-body theory – Spectacular

Perceptions of reality: polymers• How big is a polymer?

– Chemistry• model bond angles and energy barriers to

rotational states, solvent molecules• Sample over long times, or many

independent conformations• Predict R in Angstroms

– Physics• Random walk of N steps each of length l• Einstein argument: R2=Nl2 = l L• Excluded volume: polymer cannot cross

itself, because two atoms cannot occupy same position. Expect chain to swell:

R = A(l) L0.588 as L ∞

• The scaling with L is:– Universal, i.e. independent of the chemical structure– Asymptotic for large chains

• Cannot predict R in Angstroms

R

Total length of polymer is L = N l

Page 8: Patterns, universality and computational algorithmsnigel/articles/RG/Patterns... · • Example: BCS theory of superconductivity – Most successful many-body theory – Spectacular

Perceptions of reality: polymers

Osmotic pressure of polystyrene in solution as a function of concentration. Data from Wiltzius et al(1983) for 104 < Mw < 107 in two different solvents. Theory by Ohta and Oono (1982).

Data collapse: Pressure data for different concentrations and molecular weight all fall onto one universal curve.

No adjustable parameters.

No materials specific parameters.

Page 9: Patterns, universality and computational algorithmsnigel/articles/RG/Patterns... · • Example: BCS theory of superconductivity – Most successful many-body theory – Spectacular

Perceptions of reality: polymers

• The spaghetti model of a polymer is an example of a minimal model– Flexible, random walk– Self-avoidance: two atoms cannot occupy same point– Statistical thermodynamic equilibrium

• Good for answering certain questions– Universal quantities: pressure, scaling of size with

number of monomers, …

• Lousy at answering other questions– E.g. what is the size of the polymer coil (in Angstroms)?

Page 10: Patterns, universality and computational algorithmsnigel/articles/RG/Patterns... · • Example: BCS theory of superconductivity – Most successful many-body theory – Spectacular

Calculating universal quantities• Renormalization group theory

– First used in quantum field theory and second order phase transitions

– Explained why thermodynamics near critical points showed universality

• Liquid-gas critical point = ferromagnet-paramagnet critical point: exponents

– Later applied to differential equations, dynamical phenomena

• Explains how to identify universal phenomena, see what is important in model building

Page 11: Patterns, universality and computational algorithmsnigel/articles/RG/Patterns... · • Example: BCS theory of superconductivity – Most successful many-body theory – Spectacular

Universality in physics

• Successful scientific predictions are alwaysthe result of minimal models!– Cannot include every detail in a model.

• Physics: usually only a few details important: some separation of energy scales. “Easy”.

– Standard model of high energy physics is a minimal model (23 adjustable parameters!) Do you think this is a “fundamental” theory?

• Chemistry: usually need a lot of details, but at the atomic level. Separation of energy scales. Don’t worry about quarks. “Hard”.

• Economics: every detail matters, it seems. No obvious separation of scales. “Impossible”.

Page 12: Patterns, universality and computational algorithmsnigel/articles/RG/Patterns... · • Example: BCS theory of superconductivity – Most successful many-body theory – Spectacular

Universality in physics• Example: BCS theory of

superconductivity– Most successful many-body theory– Spectacular agreement with experiment:

• Universal ratios: ∆/kBTc = 3.53• Universal functions: ∆(T)

• Poor ability to predict Tc

• BCS model leaves out much physics– Don’t even need to know what is the mechanism

allowing electrons to form Cooper pairs!

Page 13: Patterns, universality and computational algorithmsnigel/articles/RG/Patterns... · • Example: BCS theory of superconductivity – Most successful many-body theory – Spectacular

Universality in complex patterns• Complex patterns can arise from simple

models, algorithms, or equations– E.g. diffusion-limited interface dynamics

• Can we model microstructure realistically using minimalmodels?

• Shape, scaling: yesDimensions: no– Useful for predicting

morphology phasediagram

Page 14: Patterns, universality and computational algorithmsnigel/articles/RG/Patterns... · • Example: BCS theory of superconductivity – Most successful many-body theory – Spectacular

Overview• Minimal models and universality

– What aspects of physical phenomena should we try to capture?– What aspects of physical phenomena should we try to predict?

• Modelling without calculus– Cell dynamical systems

• Space-time patterns during phase transitions– Universal features of pattern formation– Comparison with experiment

• Can we make simple theories of complex phenomena?– Computational irreducibility

• Multiscale modeling of patterns– Renormalization group approach to large-scale simulation of materials

processes

Page 15: Patterns, universality and computational algorithmsnigel/articles/RG/Patterns... · • Example: BCS theory of superconductivity – Most successful many-body theory – Spectacular

Cell dynamical systems (CDS)

Modelling the universal aspects of phase transition dynamics

Page 16: Patterns, universality and computational algorithmsnigel/articles/RG/Patterns... · • Example: BCS theory of superconductivity – Most successful many-body theory – Spectacular
Page 17: Patterns, universality and computational algorithmsnigel/articles/RG/Patterns... · • Example: BCS theory of superconductivity – Most successful many-body theory – Spectacular

Scaling

• Dynamics of spinodaldecomposition– Scale invariance– Pattern

• Extensions to vector and tensor order parameters– Topological defects– Correlation functions

Page 18: Patterns, universality and computational algorithmsnigel/articles/RG/Patterns... · • Example: BCS theory of superconductivity – Most successful many-body theory – Spectacular
Page 19: Patterns, universality and computational algorithmsnigel/articles/RG/Patterns... · • Example: BCS theory of superconductivity – Most successful many-body theory – Spectacular

Dynamical scaling during the quench of a binary alloy.

Measure the concentration correlation function by neutron scattering

Observe data collapse!

Phenomenon is universal, independent of chemical composition of alloy.

In practice, lattice strain effects influence the scaling function, so direct comparison with theory not possible in this case.

Page 20: Patterns, universality and computational algorithmsnigel/articles/RG/Patterns... · • Example: BCS theory of superconductivity – Most successful many-body theory – Spectacular
Page 21: Patterns, universality and computational algorithmsnigel/articles/RG/Patterns... · • Example: BCS theory of superconductivity – Most successful many-body theory – Spectacular
Page 22: Patterns, universality and computational algorithmsnigel/articles/RG/Patterns... · • Example: BCS theory of superconductivity – Most successful many-body theory – Spectacular
Page 23: Patterns, universality and computational algorithmsnigel/articles/RG/Patterns... · • Example: BCS theory of superconductivity – Most successful many-body theory – Spectacular
Page 24: Patterns, universality and computational algorithmsnigel/articles/RG/Patterns... · • Example: BCS theory of superconductivity – Most successful many-body theory – Spectacular
Page 25: Patterns, universality and computational algorithmsnigel/articles/RG/Patterns... · • Example: BCS theory of superconductivity – Most successful many-body theory – Spectacular
Page 26: Patterns, universality and computational algorithmsnigel/articles/RG/Patterns... · • Example: BCS theory of superconductivity – Most successful many-body theory – Spectacular
Page 27: Patterns, universality and computational algorithmsnigel/articles/RG/Patterns... · • Example: BCS theory of superconductivity – Most successful many-body theory – Spectacular
Page 28: Patterns, universality and computational algorithmsnigel/articles/RG/Patterns... · • Example: BCS theory of superconductivity – Most successful many-body theory – Spectacular
Page 29: Patterns, universality and computational algorithmsnigel/articles/RG/Patterns... · • Example: BCS theory of superconductivity – Most successful many-body theory – Spectacular
Page 30: Patterns, universality and computational algorithmsnigel/articles/RG/Patterns... · • Example: BCS theory of superconductivity – Most successful many-body theory – Spectacular

Overview• Minimal models and universality

– What aspects of physical phenomena should we try to capture?– What aspects of physical phenomena should we try to predict?

• Modelling without calculus– Cell dynamical systems

• Space-time patterns during phase transitions– Universal features of pattern formation– Comparison with experiment

• Can we make simple theories of complex phenomena?– Computational irreducibility

• Multiscale modeling of patterns– Renormalization group approach to large-scale simulation of materials

processes

Page 31: Patterns, universality and computational algorithmsnigel/articles/RG/Patterns... · • Example: BCS theory of superconductivity – Most successful many-body theory – Spectacular
Page 32: Patterns, universality and computational algorithmsnigel/articles/RG/Patterns... · • Example: BCS theory of superconductivity – Most successful many-body theory – Spectacular

Zapotowcky et al. (1995).

Page 33: Patterns, universality and computational algorithmsnigel/articles/RG/Patterns... · • Example: BCS theory of superconductivity – Most successful many-body theory – Spectacular

Nagaya et al. J. Phys. Soc. Jpn. 64, 78 (1995).

Experimental data: dynamics of disclinationcoarsening in liquid crystals film.

Cross-polarisersshows the dynamics of disclination lines and other topological defects

Coarsening reflected in growth of mean separation of defects

Page 34: Patterns, universality and computational algorithmsnigel/articles/RG/Patterns... · • Example: BCS theory of superconductivity – Most successful many-body theory – Spectacular

Scaled defect correlation function as a function of scaled distance.

Raw data are correlation function at different positions at different time.

Data collapse predicted by theory.

Experiment agrees with simulation with no adjustable parameters.

Mean field theory by Liu and Mazenko gives good description except near defect core.

Mondello and NG, Phys. Rev. A 45, 657 (1992).

Page 35: Patterns, universality and computational algorithmsnigel/articles/RG/Patterns... · • Example: BCS theory of superconductivity – Most successful many-body theory – Spectacular

Summary

• Predictions rely on existence of minimal models– Parameter independent, universal quantities

• Computational algorithms can model very simple processes that generate complicated space-time structures

• Quantitative prediction of universal quantities in excellent agreement with experiment

Page 36: Patterns, universality and computational algorithmsnigel/articles/RG/Patterns... · • Example: BCS theory of superconductivity – Most successful many-body theory – Spectacular

Overview• Minimal models and universality

– What aspects of physical phenomena should we try to capture?– What aspects of physical phenomena should we try to predict?

• Modelling without calculus– Cell dynamical systems

• Space-time patterns during phase transitions– Universal features of pattern formation– Comparison with experiment

• Can we make simple theories of complex phenomena?– Computational irreducibility

• Multiscale modeling of patterns– Renormalization group approach to large-scale simulation of materials

processes

Page 37: Patterns, universality and computational algorithmsnigel/articles/RG/Patterns... · • Example: BCS theory of superconductivity – Most successful many-body theory – Spectacular

Computational irreducibility and the predictability of physical systems

Navot Israeli & Nigel GoldenfeldDepartment of Physics

University of Illinois at Urbana-Champaign

Phys. Rev. Lett. 92, 074105 (2004)

Page 38: Patterns, universality and computational algorithmsnigel/articles/RG/Patterns... · • Example: BCS theory of superconductivity – Most successful many-body theory – Spectacular

Overview: Part II• The story so far …

– Universal features of pattern formation– Comparison with experiment

• The question: Can we make simple theories of complex phenomena?– Does computational irreducibility present a fundamental limitation

to modelling complex phenomena?

• The idea: Focus on universality and long-wavelength properties– Coarse-graining of spatially-extended dynamical systems

• What we did: Numerical experiments using one-dimensional nearest neighbour cellular automata (CA)– Construct RG flow picture: CA’s emulating other CA’s in a coarse-

grained way

• What our results suggest: Computational irreducibility not necessarily an obstacle: approximation can be computationally reducible– Good enough for physicists!

Page 39: Patterns, universality and computational algorithmsnigel/articles/RG/Patterns... · • Example: BCS theory of superconductivity – Most successful many-body theory – Spectacular

Modelling Complex Systems

Can we go beyond simulation?

Page 40: Patterns, universality and computational algorithmsnigel/articles/RG/Patterns... · • Example: BCS theory of superconductivity – Most successful many-body theory – Spectacular

Are complex systems unpredictable?

n

t

Cellular automata as a proxy for a complex system

Page 41: Patterns, universality and computational algorithmsnigel/articles/RG/Patterns... · • Example: BCS theory of superconductivity – Most successful many-body theory – Spectacular

n

t

n

t

n

t

t

n

n

t

Wolfram’s classification of CAClassification according to apparent complexity:

Class 3Chaotic behavior

Class 4Interacting structures

Class 2Periodic or nested

Class 1Decaying structures

• Classification is only qualitative, there is no classification algorithm.• CA seem to cover the full range of complexity found in nature.• Some CA are computationally irreducible (Wolfram hypothesized that most of class 3 and 4).• Some CA are universal Turing Machines (Wolfram hypothesized that class 4 is the threshold).

Page 42: Patterns, universality and computational algorithmsnigel/articles/RG/Patterns... · • Example: BCS theory of superconductivity – Most successful many-body theory – Spectacular

Usually in physics, we do not seek exact predictions.

What happens if we coarse-grain computationally-irreducible dynamical systems?

Not obvious how to coarse-grain a spatially-extended dynamical system such as a cellular automata.

Construct using exhaustive search ...

)0(a

)( tTa ⋅

)0(b

)(tb

Time evolution BTime evolution A

Coarse-graining

Coarse-graining

Page 43: Patterns, universality and computational algorithmsnigel/articles/RG/Patterns... · • Example: BCS theory of superconductivity – Most successful many-body theory – Spectacular

Elementary Cellular Automata

)(tan )(1 tan+)(1 tan−

)1( +tan

( )AA fStaA ,),(=

lattice of cells alphabet update function

[ ])(),(),()1(

11 tatatafta

nnnA

n

+−

=+{ }1,0=ASfor elementary CA

(256 possible functions)

A dynamical system capable of rich behavior.

n

tWolfram’s rule notation:readas a binary number.

Page 44: Patterns, universality and computational algorithmsnigel/articles/RG/Patterns... · • Example: BCS theory of superconductivity – Most successful many-body theory – Spectacular

Computational Irreducibility and Predictability

• Some complex CA are computationally irreducible. Wolfram hypothesized that most of class 3 and 4 CA are irreducible.

• Computational irreducibility = the system behavior can be predicted only by running it. There is no computationally more efficient way.

• In analogy to the real world:• Many physical processes that seem complex are probably computationally irreducible.• Most physical systems have too many degrees of freedom for direct simulation.• Are Complex Systems inherently unpredictable?

Page 45: Patterns, universality and computational algorithmsnigel/articles/RG/Patterns... · • Example: BCS theory of superconductivity – Most successful many-body theory – Spectacular

Predictable Coarse-Grained Behavior

• Usually infinite precision is not required in Physics and coarse-grained or even statistical information is sufficient.

• Can CA be coarse-grained?

• Coarse-grain-able CA are compactable. They have a coarse description with a smaller phase space.

• Coarse-grain-able CA are predictable if they or their coarse-grained version are computationally reducible.

• We found that at least some computationally irreducible CA has predictable coarse-grained behavior.

Page 46: Patterns, universality and computational algorithmsnigel/articles/RG/Patterns... · • Example: BCS theory of superconductivity – Most successful many-body theory – Spectacular

Coarse-graining procedure

22 −na

1~

−na

na2 12 +na 22 +na 32 +na

na~ 1~

+na

12 −na

Step 1: Blocking with no information loss

Generate the N’th block version of A.Group every N cells of A into a super-cell .

The super-cells accept values from the alphabet .

Calculate the transition function for the super-cells by running A N time steps for all possible inputs of length 3N.

Af ~

A~

a~

{ }121,0 −NK

Page 47: Patterns, universality and computational algorithmsnigel/articles/RG/Patterns... · • Example: BCS theory of superconductivity – Most successful many-body theory – Spectacular

Coarse-graining procedure

Step 2: Projection back on to the original alphabet with loss of information

Construct the coarse-grained CA B by projecting the alphabet of on a subset of

The cell values of B are given by The transition function of B is given by projecting the

arguments and outcome of

A~ { }.121,0 −NK

( )nn aPb ~=

Af ~

[ ] [ ]( ) ( )kknnnAnnnB aPbaaafPbbbf ~,~,~,~,, 11~11 == +−+−

Page 48: Patterns, universality and computational algorithmsnigel/articles/RG/Patterns... · • Example: BCS theory of superconductivity – Most successful many-body theory – Spectacular

This construction is possible only if:

Otherwise is not single valued and the coarse-graining procedure fails for the choice (N,P).

[ ]( ) [ ]( ) ( ) ( )( ).,,,,,, 321~321~ iiAA yPxPyxyyyfPxxxfP =∀=

Bf

1~

−na 1~

+nana~Af ~

1x 2x 3x

3y2y1y

)(~ xf A

)(~ yf A

1−nb nb 1+nb Bf

1z 2z 3z )( zf B

projection

Page 49: Patterns, universality and computational algorithmsnigel/articles/RG/Patterns... · • Example: BCS theory of superconductivity – Most successful many-body theory – Spectacular

When is satisfied is a coarse-graining of with a time constant .

For every step that makes

makes the move:

B A~1=T

[ ])(~),(~),(~)1(~11~ tatatafta nnnAn +−=+ A~

B [ ][ ]( )[ ]( )

( ))1(~)(~),(~),(~

)~(),~(),~()(),(),()1(

11~

11~

11

+=

=

==+

+−

+−

+−

taPtatatafPaPaPaPfP

tbtbtbftb

n

nnnA

nnnA

nnnBn

is a coarse-graining of with a time constant

because computes moves of in a single time step.

B A NT =

A~ N A

use ↓

Page 50: Patterns, universality and computational algorithmsnigel/articles/RG/Patterns... · • Example: BCS theory of superconductivity – Most successful many-body theory – Spectacular

n

t

40 80 120

20

40

60

n

t

20 40 60

10

20

30n

t

10 20 30 40 50

510152025

n

t

50 100 150

25

50

75

n

t

25 50 75

25

50

75

n

t

150 300 450

150

300

450

Page 51: Patterns, universality and computational algorithmsnigel/articles/RG/Patterns... · • Example: BCS theory of superconductivity – Most successful many-body theory – Spectacular

Results of coarse-graining elementary rules

• We were able to coarse-grain 240 out of the 256 elementary CA by trying different choices of super-cell sizes and projection operators.

• It might be possible to coarse-grain the other 16 rules (30, 45, 106, 154 and their symmetries) by trying larger super-cell sizes.

• Many elementary CA can be coarse-grained by other elementary CA.

• Coarse-grained-able CA include members of all four classes.

• At least one computationally irreducible CA can be coarse-grained by a reducible CA and is therefore predictable on a coarse level.

Page 52: Patterns, universality and computational algorithmsnigel/articles/RG/Patterns... · • Example: BCS theory of superconductivity – Most successful many-body theory – Spectacular

Example 1

n

t

50 100 150

25

50

75

n

t

10 20 30 40 50

510152025

• super-cell size N=3.

• Projection:

• Complex (potentially irreducible) on the microscopic scale, predictable on the coarse scale.

⎩⎨⎧

=≠

=aa

aP ~,

~,)~(

Rule 146 (class 3) Rule 128 (class 1)

Page 53: Patterns, universality and computational algorithmsnigel/articles/RG/Patterns... · • Example: BCS theory of superconductivity – Most successful many-body theory – Spectacular

n

t

20 40 60

10

20

30n

t

40 80 120

20

40

60

Example 2

• super-cell size N=2.

• Projection:

Rule 105 (class 2) Rule 150 (class 2)

⎩⎨⎧

==

=,~,,~,

)~(aa

aP

Page 54: Patterns, universality and computational algorithmsnigel/articles/RG/Patterns... · • Example: BCS theory of superconductivity – Most successful many-body theory – Spectacular

n

t

25 50 75

25

50

75n

t

150 300 450

150

300

450

Example 3- an irreducible case

• Super-cell size: N=6. Projection: 64 colors → 63 colors.• Rule 110 is a universal CA. Can emulate all other CA (Wolfram).• A universal CA is computationally irreducible.• More impressive information loss with larger N:

Rule 110 (class 4) 63 color CA

6 7 8 9 10 110

5

10

15

20

25

N

Alp

habe

t red

uctio

n[%

]

Page 55: Patterns, universality and computational algorithmsnigel/articles/RG/Patterns... · • Example: BCS theory of superconductivity – Most successful many-body theory – Spectacular

Coarse-graining transitions between elementary rules. with super-cell size N=2,3,4 (for rule 110 N=5)

23 77 134 146148 158 160 178182 214 232 250 254

128

200 236

51

240

15

170

85

195

60

165

90

153

102

150

105

44 100 130 132144 162 176 186190 203 217 222

242 24613 79 69 93

152 230156 198188 194196 220 140 206

5 95204

1 4 12 1929 55 68 7172 76 127 172202 205 207 216221 223 228 237

34 187 48 243

10 38 46138 139 155

174 175

52 80 116208 209 211

244 2452 42171 191

16 112241 247

252

192

238

136

255

0

3 6 7 8 9 11 14 17 18 2021 24 26 27 28 31 32 33 35 3639 40 47 49 50 53 56 58 59 6364 65 66 70 74 78 81 82 83 8487 88 92 96 98 108 111 114 115 117

119 123 125 126 129 141 143 157 159 163167 168 173 177 179 181 183 184 185 189197 199 201 213 215 219 224 226 227 229

231 234 235 239 248 249 251 253

Class 1Class 2Class 3Class 4

110 124193137 Irreducible

Coarse-graining → partial complexity order?

Page 56: Patterns, universality and computational algorithmsnigel/articles/RG/Patterns... · • Example: BCS theory of superconductivity – Most successful many-body theory – Spectacular

Why is it easy to coarse-grain CA?

Coarse-graining procedure:

elementaryCA

2N colorCA

group Ncells

project(solve an over constrained problem)

coarse-grainedCA

• When N=2 the success rate (among 256 elementary CA) is ~1/3.

• 4 color CA which are the N=2 supercell version of elementary CA are a small fraction of all possible 4 color rules (≈3·1038).

• The probability of projecting a randomly chosen 4 color CA is 10-7.

• What is special about the CA rules that allows such a high probability of success in projecting on to the original alphabet?

Page 57: Patterns, universality and computational algorithmsnigel/articles/RG/Patterns... · • Example: BCS theory of superconductivity – Most successful many-body theory – Spectacular

Algorithmic complexity of local processes

0000

3100

2200

2300

0333

1~

−na Af ~1~

+nana~

1000

2100

0200

0300

3333

1−nx Xf1+nxnx

4 color supercell version of an elementary CA.

General 4 color CA.

A short generating “program”.

random

Algorithmic complexity = stringoflengthcodegeneratingminimaloflength

Small algorithmic complexity Large algorithmic complexity

Page 58: Patterns, universality and computational algorithmsnigel/articles/RG/Patterns... · • Example: BCS theory of superconductivity – Most successful many-body theory – Spectacular

10-4 10-2 100 102 104 1060

0.2

0.4

0.6

0.8

1S=4S=8S=16S=32

proj

ectio

n pr

obab

ility

“Algorithmic complexity”

Experiment: projection probability vs. algorithmic complexity

• Use short programs to generate random multicolor CA with definite algorithmic complexity.• Measure the probability of a successful projection as a function of program length.

Program(l): 1. Randomly chose the first l entries of the rule table:2. Pick a random generating function of l arguments: f: Sl -> S3. Use the function f and the last l filled entries to fill the next entry4. Repeat until all entries filled 5. Try to find a projection.

0200

3333

0300

2100

10001−nx f1+nxnx

“Algorithmic complexity”

0 10 20 300

0.2

0.4

0.6

0.8

1S=4S=8S=16S=32

proj

ectio

n pr

obab

ility

l

3SSl l+

=

Page 59: Patterns, universality and computational algorithmsnigel/articles/RG/Patterns... · • Example: BCS theory of superconductivity – Most successful many-body theory – Spectacular

is satisfied because

Replacing x by y is irrelevant to the long time behavior of A:

The coarse-grained CA B accounts for all long time trajectories of A.A and B fall in the same complexity class.

Relevant and Irrelevant degrees of freedom

Coarse-graining can loose two different types of dynamic information, depending on how the condition

is satisfied:

1. Irrelevant DOF[ ] [ ] ( ) ( )( ).,,,,,, 321~321~ iiAA yPxPyxyyyfxxxf =∀=

[ ]( ) [ ]( )( ) ( )( )ii

AA

yPxPyx

yyyfPxxxfP

=∀

=

,

,,,, 321~321~

n

t

n

t

n

t

x in initial state y in initial state difference

Page 60: Patterns, universality and computational algorithmsnigel/articles/RG/Patterns... · • Example: BCS theory of superconductivity – Most successful many-body theory – Spectacular

2. Relevant DOFis satisfied even though for some .

Replacing x by y in the initial condition is relevant to the long time behavior of A.

[ ] [ ]321~321~ ,,,, yyyfxxxf AA ≠( ) ( )( )., ii yPxPyx =

n

t

n

t

n

t

x in initial state y in initial state difference

The coarse-grained CA B cannot account for all long time trajectories of A.

A and B may fall in different complexity classes (see example 1).

Relevant and Irrelevant degrees of freedom

Page 61: Patterns, universality and computational algorithmsnigel/articles/RG/Patterns... · • Example: BCS theory of superconductivity – Most successful many-body theory – Spectacular

Coarse-graining and undecidability

• Universal cellular automata (UCA) are subject to undecidablequestions.• This puts constraints on possible coarse-grainings of UCA.• For example, UCA cannot coarse-grain themselves (are not fixed points):

Proof sketch - Assume that a UCA U is a fixed point.Question: Will U evolve to a limit cycle on input x?Answer: Coarse-grain the input x several times to an “input

fixed point” y, and run U on y.x y y

no limit cycle limit cycle

CGCGCG

run

There is no scale invariance in UCA.

Page 62: Patterns, universality and computational algorithmsnigel/articles/RG/Patterns... · • Example: BCS theory of superconductivity – Most successful many-body theory – Spectacular

Usually in physics, we do not seek exact predictions.

What happens if we coarse-grain computationally-irreducible dynamical systems?

Conclusion: sometimes we can get good approximations to seemingly complex dynamical systems.

We do not need to abandon physics!

Page 63: Patterns, universality and computational algorithmsnigel/articles/RG/Patterns... · • Example: BCS theory of superconductivity – Most successful many-body theory – Spectacular

Renormalization group approach to multiscale modeling in materials science

Nigel GoldenfeldDepartment of Physics

University of Illinois at Urbana-Champaign

Collaborators: Badri Athreya and Jon DantzigProject also involves Ken Elder and Nik Provatas

Work supported by the US National Science Foundation and NASA

ITR: Multiscale Models for Microstructure Simulation and Process DesignNSF DMR 01-21695

Page 64: Patterns, universality and computational algorithmsnigel/articles/RG/Patterns... · • Example: BCS theory of superconductivity – Most successful many-body theory – Spectacular

Overview• Minimal models and universality

– What aspects of physical phenomena should we try to capture?– What aspects of physical phenomena should we try to predict?

• Modelling without calculus– Cell dynamical systems

• Space-time patterns during phase transitions– Universal features of pattern formation– Comparison with experiment

• Can we make simple theories of complex phenomena?– Computational irreducibility

• Multiscale modeling of patterns– Renormalization group approach to large-scale simulation of materials

processes

Page 65: Patterns, universality and computational algorithmsnigel/articles/RG/Patterns... · • Example: BCS theory of superconductivity – Most successful many-body theory – Spectacular

Scale hierarchy problem

Page 66: Patterns, universality and computational algorithmsnigel/articles/RG/Patterns... · • Example: BCS theory of superconductivity – Most successful many-body theory – Spectacular

Outline1. What should a genuine multiscale method

look like?- Scale up- Adaptive mesh refinement

2. Phase field crystal model (PFC)- Advantages over molecular dynamics- Successful computation of macroscopic quantities

3. Renormalization group- Reductive perturbation theory & rotational covariance- RG equations for the PFC

4. Preliminary results- Adaptive mesh refinement

Page 67: Patterns, universality and computational algorithmsnigel/articles/RG/Patterns... · • Example: BCS theory of superconductivity – Most successful many-body theory – Spectacular

Goal of multiscale modeling• Given underlying

dynamics, what is the effective dynamics at large scales?

• Representative behaviour of many degrees of freedom?

ψ(x,0) ψ(x,t)

Ψ(x,0) Ψ(x,t)

N

?C C

Page 68: Patterns, universality and computational algorithmsnigel/articles/RG/Patterns... · • Example: BCS theory of superconductivity – Most successful many-body theory – Spectacular

Goal of multiscale modeling

• Basic idea: model large-scale, uniform regions by some representative variable– Systematic reduction of number of degrees of freedom

• Representative variable should be predominantly spatially uniform

• Significantvariation onlyat boundaries– Adaptive mesh

refinement

Page 69: Patterns, universality and computational algorithmsnigel/articles/RG/Patterns... · • Example: BCS theory of superconductivity – Most successful many-body theory – Spectacular

Phase field models

• Coarse-grain spatial density to achieve a slowly-varying (on atomic scale) field that smoothly interpolates between phases

• Devise nonlinear PDE for the phase field so that it matches the Stefan problem for solidification

Page 70: Patterns, universality and computational algorithmsnigel/articles/RG/Patterns... · • Example: BCS theory of superconductivity – Most successful many-body theory – Spectacular

Example: dendritic growth using phase field models

More mesh points in regions where phase field is varying rapidly

Computational load ~ O(N2/3)

Page 71: Patterns, universality and computational algorithmsnigel/articles/RG/Patterns... · • Example: BCS theory of superconductivity – Most successful many-body theory – Spectacular

Phase field models

• Problem with phase field models is that they do not preserve crystallographic orientation– Merging of domains is

incorrectly captured– Modeling of complex

material microstructure impossible

Page 72: Patterns, universality and computational algorithmsnigel/articles/RG/Patterns... · • Example: BCS theory of superconductivity – Most successful many-body theory – Spectacular

Phase field crystal

• Phase field crystal (Elder et al. 2002, 2004)– Dynamical variable has periodic solutions = atomic

density

Periodic solutions

∂ρ

∂t= Γ∇2

ÃδFδρ

!+ η

Conservation law

Density

F =Z[ρ

µα∆T + λ

³q2o +∇2

´2¶ρ/2+uρ4/4]d3x

Page 73: Patterns, universality and computational algorithmsnigel/articles/RG/Patterns... · • Example: BCS theory of superconductivity – Most successful many-body theory – Spectacular

Phase field crystal

• Constant solutions or periodic– d=2 triangular lattice or stripes (smectic phase)

∂ψ/∂t= ∇2³ωψ+ ψ3

´+ ζ

ω ≡ r+ (1+∇2)2

Argon phase diagram

Page 74: Patterns, universality and computational algorithmsnigel/articles/RG/Patterns... · • Example: BCS theory of superconductivity – Most successful many-body theory – Spectacular

Phase field crystal

• Properties– Preserves crystallographic information– Representation of free boundaries– Captures defects

• Incorporates elasticity natively!

ψ = Ao sin(2πx/a)

F/a− Feq/a= (K/2)(a− aeq)2 + · · ·K ≡ −8(r+3ψ̄2)/3

Page 75: Patterns, universality and computational algorithmsnigel/articles/RG/Patterns... · • Example: BCS theory of superconductivity – Most successful many-body theory – Spectacular

Phase field crystal

• Dynamical solutions properly model grain growth

Page 76: Patterns, universality and computational algorithmsnigel/articles/RG/Patterns... · • Example: BCS theory of superconductivity – Most successful many-body theory – Spectacular

Phase field crystal describes dislocations

• Dimensionless grain boundary energy as a function of angle

• Grain boundary energy normalised by maximum value compared with expt. data

Read-Shockley equation

γ = (aY2)/(8π) θ [3/2− ln(2πθ)]

Y2 = (4√2/15(ψ̄+

q−15r − 36ψ̄2))2

Page 77: Patterns, universality and computational algorithmsnigel/articles/RG/Patterns... · • Example: BCS theory of superconductivity – Most successful many-body theory – Spectacular

PFC: Picture Gallery

K. Elder and M. Grant, Modeling elastic and plastic deformation in nonequilibrium processing using phase field crystals, Phys. Rev. E 70, 051605:1-18 (2004)

Page 78: Patterns, universality and computational algorithmsnigel/articles/RG/Patterns... · • Example: BCS theory of superconductivity – Most successful many-body theory – Spectacular

PFC: results gallery

• Quantitative scaling results in complex materials properties

Critical height for dislocation nucleation in strained epitaxial films

Yield stress of polycrystalline solid as a function of grain size

Page 79: Patterns, universality and computational algorithmsnigel/articles/RG/Patterns... · • Example: BCS theory of superconductivity – Most successful many-body theory – Spectacular

Phase field crystal

• Comparison with molecular dynamics– Fundamental time scale is diffusion time ~ DQ2

– 1 diffusion time ~ thousand time steps– For Gold, diffusion time ~ 0.26 ms, 33 µs, 5 µs at T=800,

900, 1020 oC– Molecular dynamics: time step ~ 1 fs– Conclusion: PFC ~ 106 – 109 times faster than MD!

D = 3ψ̄+ r+1 +9A2t /8

Page 80: Patterns, universality and computational algorithmsnigel/articles/RG/Patterns... · • Example: BCS theory of superconductivity – Most successful many-body theory – Spectacular

Phase field crystal

• Big problem with the phase field crystal– Modeling at the atomic scale means that it is computationally

intensive• Just like molecular dynamics

– Curse of dimensionality: N ~ Ld

• Question– How can we do multi-resolution analysis on the phase field crystal?

• Unlike the phase field model, do not have a simple kink structure describing interfaces

• Answer– Use the renormalization group …

Page 81: Patterns, universality and computational algorithmsnigel/articles/RG/Patterns... · • Example: BCS theory of superconductivity – Most successful many-body theory – Spectacular

Renormalization group for PDEs

http://guava.physics.uiuc.edu/~nigel/articles/RG

L. Chen, N. Goldenfeld, Y. Oono. Renormalization group theory for global asymptotic analysis. Phys. Rev. Lett. 73, 1311-1315 (1994).

L. Chen, N. Goldenfeld and Y. Oono. The renormalization group and singular perturbations: multiple-scales, boundary layers and reductive perturbation theory. Phys. Rev. E 54, 376-394 (1996).

• Systematic program to extract universal, i.e. asymptotic features from PDEs

Page 82: Patterns, universality and computational algorithmsnigel/articles/RG/Patterns... · • Example: BCS theory of superconductivity – Most successful many-body theory – Spectacular

How can RG help?• On scales > lattice periodicity, amplitude

and phase of density slowly varying, except near defects

• Derive dynamical equation satisfied by coarse-grained amplitude and phase– Analogue of Newell-Whitehead and

Cross-Newell equations in Rayleigh-Benard convection

– Need rotational covariant form of the equations

• Proposed by Gunaratne et al. (1994)• RG derivation by Graham (1996), Nozaki et

al (2000).

• Amplitude and phase gradient vary slowly except near defects => can solve with adaptive mesh refinement!– Then reconstruct the density

ψ(x,0) ψ(x,t)

Ψ(x,0) Ψ(x,t)

PFC

?RG

RG

Page 83: Patterns, universality and computational algorithmsnigel/articles/RG/Patterns... · • Example: BCS theory of superconductivity – Most successful many-body theory – Spectacular

Example: Swift-Hohenberg

• Swift-Hohenberg equation is a simplified model of convection in absence of mean flow– Rolls

• Rotationally-covariant complex amplitude equation near onset

∂tA =³−r − 3|A|2

´A+4k20L2A

L=Ãk̂ ·∇−

i

2k0∇2

!

∂tψ = [²− (1+∇2)2]ψ − ψ3

Page 84: Patterns, universality and computational algorithmsnigel/articles/RG/Patterns... · • Example: BCS theory of superconductivity – Most successful many-body theory – Spectacular

Example: Swift-Hohenberg

• Solve SH by finite differences• Solve RG-SH by finite differences• Compare

Initial condition SH time evolution RG-SH time evolution

ψ(x,0) ψ(x,t)

Ψ(x,0) Ψ(x,t)

PFC

?RG

RG

Page 85: Patterns, universality and computational algorithmsnigel/articles/RG/Patterns... · • Example: BCS theory of superconductivity – Most successful many-body theory – Spectacular

Example: Swift-Hohenberg

• The amplitude and phase gradient are slowly varying everywhere, except in the vicinity of grain boundaries, topological defects etc.

• Exploit this property to do adaptive mesh refinement

Order parameter Amplitude Phase

Page 86: Patterns, universality and computational algorithmsnigel/articles/RG/Patterns... · • Example: BCS theory of superconductivity – Most successful many-body theory – Spectacular

Phase field crystal

• There are three density components giving rise to a triangular lattice

• Corresponding complex amplitude equation is

ψ(~x) =Xj

Aj(t) exp(i ~kj · x) + ψ̄

∂A1∂t

= eL1A− 3A1 ³|A1| 2+ 2|A2| 2+ 2|A3|2´− 6ψ̄A2A3∂A2∂t

= eL2A− 3A2 ³2|A1|2 + |A2| 2+ 2|A3|2´− 6ψ̄A1A3∂A3∂t

= eL3A− 3A3 ³2|A1|2 + 2|A2| 2+ |A3|2´− 6ψ̄A1A2

Lj = [1−∇2− 2ikj ·∇][Γ− (∇2+ 2ikj ·∇)2] Γ ≡ −(r+3ψ̄2)

Page 87: Patterns, universality and computational algorithmsnigel/articles/RG/Patterns... · • Example: BCS theory of superconductivity – Most successful many-body theory – Spectacular

Phase field crystal

Good agreement between solution reconstructed from RG and the actual solution

RG solution of PFC Brute force solution of PFC

Page 88: Patterns, universality and computational algorithmsnigel/articles/RG/Patterns... · • Example: BCS theory of superconductivity – Most successful many-body theory – Spectacular

Phase field crystal

• The amplitude and phase gradient are constant everywhere, except in the vicinity of grain boundaries, topological defects etc.

• Exploit this property to do adaptive mesh refinement

Amplitude Phase gradient

Page 89: Patterns, universality and computational algorithmsnigel/articles/RG/Patterns... · • Example: BCS theory of superconductivity – Most successful many-body theory – Spectacular

Ongoing work• Extensions of PFC model

– Taylor PFC to specific materials• Fit correct 2-point correlations, using experimental structure factors• Fit correct dynamics, using experimentally-measured dynamic structure factor

– Multicomponent systems– Coupling to thermal and solute fields

• Adaptive mesh refinement for RG formulation of the phase field crystal, in amplitude and phase formulation

• Extensions to 3D

• RG equations far from threshold: regularized version of Cross-Newell/Passot-Newell equations derived (Sasa 1996)– Ill posed, due to enforcing of amplitude=constant.– Relaxation of constraint needed near defects– Match on to complex amplitude equation near threshold

Page 90: Patterns, universality and computational algorithmsnigel/articles/RG/Patterns... · • Example: BCS theory of superconductivity – Most successful many-body theory – Spectacular

Conclusions

1. Phase field crystal models can capture realistic phenomenology of materials

2. Renormalization group methods, developed for pattern forming instabilities, can be used to derive effective mesoscale equations of motion

3. Adaptive mesh refinement can be used to optimize the computational efficiency

4. Multiscale modeling of realistic materials processing from the nanoscale is feasible, combining PFC, RG and AMR

Page 91: Patterns, universality and computational algorithmsnigel/articles/RG/Patterns... · • Example: BCS theory of superconductivity – Most successful many-body theory – Spectacular

All my RG papers can be obtained onlinehttp://guava.physics.uiuc.edu/~nigel/articles/RG

Philosophy of science: R. Batterman has written about the use of asymptotic reasoning and RG as a scientific methodology:

The Devil in the Details: asymptotic reasoning in explanation, reduction and emergence by Robert Batterman.

Renormalization group applied to modelling and the solution of partial differential equations:

Lectures on phase transitions and the RG by NG.

Further reading