The Future of Distilling Craft Distilling Expo 2015 October 7th 2015.
Machine Science Distilling Free-Form Natural Laws from Experimental Data Hod Lipson, Cornell...
80
Machine Science Distilling Free-Form Natural Laws from Experimental Data Hod Lipson, Cornell University
-
Upload
colin-cross -
Category
Documents
-
view
215 -
download
1
Transcript of Machine Science Distilling Free-Form Natural Laws from Experimental Data Hod Lipson, Cornell...
Machine ScienceDistilling Free-Form Natural Laws from Experimental Data
Hod Lipson, Cornell University
Lipson & Pollack, Nature 406, 2000
Camera
CameraView
Adapting in simulation
Simulator
Evolve ControllerIn Simulation
Try it in reality!
Download
Crossing The
Reality Gap
Adapting in reality
Evolve ControllerIn Reality
Try it !
Too many Physical
Trials
Simulation & Reality
Evolve Controller
“Simulator”
Try it in reality!
Build
Collect Sensor Data
Evolve SimulatorEvolve Simulators Evolve Robots
ServoActuators
Tilt Sensors
Emergent Self-Model
With Josh Bongard and Victor Zykov, Science 2006
Damage Recovery
With Josh Bongard and Victor Zykov, Science 2006
RandomPredictedPhysical
System Identification
?
Perturbations
Photo: Floris van Breugel
Photo: Floris van Breugel
Structural Damage Diagnosis
With Wilkins Aquino
.com
With Jeff Clune, Jason Yosinski
Symbolic Regression
f(x)=exsin(|x|)
What function describes this data?
John Koza, 1992
Encoding Equations
*
sin–
x1 3
f(x)
+
x2 -7
Building Blocks: + - * / sin cos exp log … etc
sin(x2)
x1*sin(x2)
(x1 – 3)*sin(x2)
(x1 – 3)*sin(-7 + x2)
John Koza, 1992
Michael D. Schmidt, Hod Lipson (2006)
Models: Expression treesSubject to mutation and selection
Experiments: Data-pointsSubject to mutation and selection
{const,+,-,*,/,sin,cos,exp,log,abs}
+
sin×
–1.2
x 2x
+
sin×
–1.2
x 2x
Solution Accuracy
Entire DatasetCoevolved Dataset
15
17
19
21
23
25
27
29
31
33
35
0 5000 10000 15000
Generations
Sol
utio
n S
ize
(# o
f nod
es)
Coevolution
Exact
Solution Complexity
Entire Dataset
Coevolved Dataset
Semi-empirical mass formula
Modeling the binding energy of an atomic nucleus
ZAA
ZAa
A
ZZaAaAaE ACSVB ,
21 2
3132
odd ,
odd
even ,
0,
0
0
NZ
A
NZ
ZA
210 A
aP
R2 = 0.999915
Weizsäcker’s Formula:
A
ZN.
A
Z.A.A..E
..
B
2
260
2640 )(2917390
391243138314
R2 = 0.99944
Inferred Formula:
Systems of Differential Equations
• Regress on derivative
time x1 x2 …
0 3.4 -1.7 …0.1 3.2 -0.9 …0.2 3.1 -0.1 …0.3 2.7 1.2 …… … … …
State Variables
dx1/dt x2/dt …
-2.0 8.0 …-1.0 8.0 …-4.0 1.3 …-5.7 1.9 …… … …
Derivatives
0
1
2 6
1
2 3
3
1 3143
1 322 1 2 14
3
2*
32 1 3 2
42 3
*2.5 100
1 13.6769*
*200 6* * 12* *
1 13.6769*
6* * 16* *
16* * 100*
J
v
v v
v
v v
v
S AdS
dt A
S AdSS N S N
dt A
dSS N S A
dt
dSA S
dt
4
2 4
3 5
1
2 4
22 1 2 4
3 1 32 3 34
3 2
2*
55
*
6* * 100* *
*200 32* * 1.28*
1 13.6769
1.3*
v
v v
v v
v
J
N S
dNS N N S
dt
dA S AA S A
dt A
dSS
dt
Original Equations Inferred Equations
Inferring Biological Networks
With Michael Schmidt, John Wikswo (Vanderbilt), Jerry Jenkins (CFDRC)
0
1
2 6
1
2
1 3143
1 322 1 2 14
3
2*
32 1
*2.42114 99.2721
1 13.5956*
*199.935 5.99475* * 11.9895* *
1 13.6734*
5.99857* * 1
J
v
v v
v
v
S AdS
dt A
S AdSS N S N
dt A
dSS N
dt
3
43
2 4
3 2
42 3 2 4
22 1 2 4
3 1 3
3
43
2*
5.99606* *
15.997* * 100.0
0.01
15* *
5.99857* * 99.9963* *
*197.781
1 13.263
286
3
*
v
vv
v
extran
v
v
eous
S A
dSA S N S
dt
dNS N N S
dt
dA S
dt
S
A
A
3 5
1
2 3 3
2
55
31.9682* * 1.29659*
1.29626*
v v
J
A S A
dSS
dt
Charles Richter
Ingm
ar Z
ange
r, J
ohn
Am
end
Wet Data, Unknown System
With Michael Schmidt (Cornell) and Gurol Suel (UT Southwestern)
Wet Data, Unknown System
With Michael Schmidt (Cornell) and Gurol Suel (UT Southwestern)
Cell #1
Cell #2
Cell #3-60 …
Blue Dots = data points, Green Line = regressed fit
=
=
0
1
1
11
/ /
/ //
nK K
k Kn nK S
S kS Sp
K S
K KdKK
dt k K K S
SdSS
dt K SK k
Biologist’s Inferred Model: Gurol Suel, et. al., Science 2007
Symbolic Regression Inferred Time-Delay Model:
S
Kcba
dt
dSK
Scba
dt
dK
SSS
KKK
7423.163214.170.1582
18
51
t
tt
G
S
dt
dG
05.33019.0922.114
15
25
t
tt
S
G
dt
dS
Withheld Test Set #1 Fit
1355.10312.2192.3526
17
54
t
tt
G
S
dt
dG
9693.20178.0271.132
18
57
t
tt
S
G
dt
dS
Withheld Test Set #2 Fit
4406.67452.391.5057
21
46
t
tt
G
S
dt
dG
871.32965.0426.295
20
54
t
tt
S
G
dt
dS
Withheld Test Set #3 Fit
Looking For Invariants
Data Mining
42
42+x-x
42+1/(1000+x2)
?
From Data: x y …
0.1 2.30.2 4.50.3 9.70.4 5.10.5 3.30.6 1.0… … …
δxδy
δyδx
, , …
Calculate partial derivatives Numerically:
δfδx
δfδy
From Equation:
δx’δy’
δy’δx’
, , …
Calculate predicted partial derivatives Symbolically:
-5 0 5
-3
-2
-1
0
1
2
3
x
y
x
y
x3 + x – y2 – 1.5 = 0x2 + y2 – 16 = 0 x2 + y2 + z2 – 1 = 0
-2 -1 0 1 2-3
-2
-1
0
1
2
3
x
y
x
y
-1 -0.5 0 0.5 1-1
-0.5
0
0.5
1
x
z
x
z
Homework
Circle Elliptic Curve Sphere
Linear Oscillator
• Coefficients may have different scales and offsets each run
Pendulum
2
2.42847*cos( )
d
Ldt 2
3.52768* 9.43429*cos( ) d
Hdt
H
L
Double Linear Oscillator
2 22 2 2 11 2 1 214.691* 15.551* 21.676* 8.3808* 2.6046*
dx dxH x x x x
dt dt
would be plus for Lagrangian
-k1 ω1 – k2 ω2 + k3 ω1 cos(θ1 – θ2) + k4 ω2 cos(θ1 – θ2)
k1 ω1 ω2 – k2 cos(θ1 – θ2)
k1 ω12 + k2 ω2
2 – k3 ω1 ω2 cos(θ1 – θ2) – k4 cos(θ1) – k5 cos(θ2)
Parsimony [-nodes]
Pre
dic
tive
Ab
ilit
y [-
log
err
or]
A k1 θ2 – k2 ω12 – k3 ω2
2 + k4 ω1 ω2 cos(θ1 – k5 θ2) + k6 cos(θ2) + k7 cos(θ1) – k8 cos(k9 θ2) – k10
cos(k11 – k12 θ2)
-k1 ω12 – k1 ω2
2 + k1 ω1 ω2 cos(θ2) + k1 cos(θ2) + k1 cos(θ1)
ω2·cos(θ1 θ2) + ω1
0
-2
-0.4
-0.8
-1.2
-1.6
SimpleComplex
Less
P
red
ictiv
eM
ore
P
red
ictiv
e
1
2
3
4
K K t tK
t t
S S t tS
t t
SK
t K
KS
t S
b cda
d
b cda
d
Von Thomas Hermanowski, Dr. Andreas Rick, Dr. Jochen Weber
Particle Accelerators
angle
Distortion Correction (GE X-Ray Detectors)5x Resolution Improvement
Jay Schuren
Concluding RemarksConcluding Remarks
Correlation is enough. Faced with massive data, [the Scientific Method] is becoming obsolete. We can stop looking for models.
“”Chris Anderson
Wired 16.07
The data deluge accelerates our ability to hypothesize, model, and test.
Theoretical physicists are not yet obsolete, but scientists have taken steps toward
replacing themselves
I am worried that we have enjoyed a brief window in human history where we could actually understand things, but that
period may be coming to an end.
-- Steve Strogatz
The end of insight
