Dynamic Optimization and Automatic Differentiation
description
Transcript of Dynamic Optimization and Automatic Differentiation
![Page 1: Dynamic Optimization and Automatic Differentiation](https://reader036.fdocuments.in/reader036/viewer/2022070410/56814527550346895db1ed58/html5/thumbnails/1.jpg)
Colloquium on Optimization for Control
1
Dynamic Optimization and Dynamic Optimization and Automatic DifferentiationAutomatic Differentiation
Yi CaoSchool of EngineeringCranfield University
![Page 2: Dynamic Optimization and Automatic Differentiation](https://reader036.fdocuments.in/reader036/viewer/2022070410/56814527550346895db1ed58/html5/thumbnails/2.jpg)
Colloquium on Optimization for Control2
OutlineOutline
• Dynamic optimization problems• Parameterization• Recursive high-order Taylor series• ODE and sensitivity solver• Dynamic optimization solver• Differential recurrent neural
network• Continuous-time NMPC• Conclusions
![Page 3: Dynamic Optimization and Automatic Differentiation](https://reader036.fdocuments.in/reader036/viewer/2022070410/56814527550346895db1ed58/html5/thumbnails/3.jpg)
Colloquium on Optimization for Control3
Dynamic optimization problemDynamic optimization problem
• Controller design, =const controller parameter • Adaptive control, =variable controller
parameter• System identification, =model parameter• Predictive control, =control action, variable• State estimation, =initial state• ……
0
min ( , ( ), ( )) ( , , )
( , , , ) 0
. . ( , , ) 0
( , ( ), ( )) 0
ft
f f f t
f f f
t x t t t x dt
f t x x
s t g t x
h t x t t
![Page 4: Dynamic Optimization and Automatic Differentiation](https://reader036.fdocuments.in/reader036/viewer/2022070410/56814527550346895db1ed58/html5/thumbnails/4.jpg)
Colloquium on Optimization for Control4
Solving dynamic optimizationSolving dynamic optimization
• Optimal control theory well established in 1960s.
• Challenge for numerical solutions:– Complex / large scale problems– Efficiency for realtime optimization– Global optimization– Stability– Robustness
![Page 5: Dynamic Optimization and Automatic Differentiation](https://reader036.fdocuments.in/reader036/viewer/2022070410/56814527550346895db1ed58/html5/thumbnails/5.jpg)
Colloquium on Optimization for Control5
Differentiation and dynamic Differentiation and dynamic optimizationoptimization
• H=Φ(t,x,)+λ’f(t,x,)• Optimal conditions:
dx/dt= Hλ=f(t,x,),
dλ/dt=–Hx=–Φx(t,x,)–fx(t,x,)λ
H=Φ(t,x,)+f(t,x,)λ=0
• Efficient solution requires efficient differentiation
![Page 6: Dynamic Optimization and Automatic Differentiation](https://reader036.fdocuments.in/reader036/viewer/2022070410/56814527550346895db1ed58/html5/thumbnails/6.jpg)
Colloquium on Optimization for Control6
Differentiation approachesDifferentiation approaches
• Analytic differentiation manuallyNot a trivial task for large scale problem
• Analytic differentiation using symbolic computing softwareVery complicated results even for a small problem
• Numerical finite differenceInefficient and inaccurate
![Page 7: Dynamic Optimization and Automatic Differentiation](https://reader036.fdocuments.in/reader036/viewer/2022070410/56814527550346895db1ed58/html5/thumbnails/7.jpg)
Colloquium on Optimization for Control7
Automatic Differentiation (1)Automatic Differentiation (1)
• Techniques use computer programs to get derivatives of any functions represented in other computer programs with the same accuracy and efficiency as the function.
• Synonym: Algorithmic Differentiation• First proposed by Johnannes Joos,
1976 in his PhD Thesis, ETH, Zurich
![Page 8: Dynamic Optimization and Automatic Differentiation](https://reader036.fdocuments.in/reader036/viewer/2022070410/56814527550346895db1ed58/html5/thumbnails/8.jpg)
Colloquium on Optimization for Control8
Automatic Differentiation (2)Automatic Differentiation (2)
• Factor: every computer program, no matter how complicated, executes a sequence of elementary arithmetic operations such as additions or elementary functions such as exp().
• By applying the chain rule of derivative calculus repeatedly to these operations, derivatives of arbitrary order can be computed automatically, and accurate to working precision.
![Page 9: Dynamic Optimization and Automatic Differentiation](https://reader036.fdocuments.in/reader036/viewer/2022070410/56814527550346895db1ed58/html5/thumbnails/9.jpg)
Colloquium on Optimization for Control9
Automatic Differentiation (3)Automatic Differentiation (3)
Two modes to calculate derivatives:• Forward mode: y=f(x(t))
• Reverse mode (adjoint): y=f(x(t))
'( )dx
x y f x xdt
'( ) 'dy
v x f x y t x xdv
![Page 10: Dynamic Optimization and Automatic Differentiation](https://reader036.fdocuments.in/reader036/viewer/2022070410/56814527550346895db1ed58/html5/thumbnails/10.jpg)
Colloquium on Optimization for Control10
Automatic Differentiation (4)Automatic Differentiation (4)
Two ways to implement AD• Operator overloading
Each elementary operation is replaced by a new one, working on pairs of value and its derivative (doublet).
• Source transformationProduce new code to calculate derivative based on original code of a function
![Page 11: Dynamic Optimization and Automatic Differentiation](https://reader036.fdocuments.in/reader036/viewer/2022070410/56814527550346895db1ed58/html5/thumbnails/11.jpg)
Colloquium on Optimization for Control11
AD Example: Forward ModeAD Example: Forward Mode
x1’=1, x2’=0v1’=(x1’x2-x1x2’)/x2/
x2=2v2’=cos(v1)*v1’= –1.98 v3’=exp(x2)*x2’=0v4’=v1’-v3’=2y’=v2’+v4’=0.02
x1=1.5, x2=0.5v1=x1/x2=3 v2=sin(v1)=0.1411v3=exp(x2)=1.649v4=v1–v3=1.3513y=v2+v4=1.4924
y=sin(x1/x2)+x1/x2–exp(x2)
![Page 12: Dynamic Optimization and Automatic Differentiation](https://reader036.fdocuments.in/reader036/viewer/2022070410/56814527550346895db1ed58/html5/thumbnails/12.jpg)
Colloquium on Optimization for Control12
x1=1.5, x2=0.5v1=x1/x2=3v2=sin(v1)=0.1411v3=exp(x2)=1.649v4=v1–v3=1.3513y=v2+v4=1.4924
AD Example: Reverse ModeAD Example: Reverse Mode
x2=x2+v1*x1/x2/x2=-1.589
x1=v1/x2=0.02v1=v2*cos(v1)+v1=0.01 x2=v3*exp(x2)=-1.649v1=v4=1, v3=–v4=–1y=1, v2=y=1, v4=y=1
y=sin(x1/x2)+x1/x2–exp(x2)
![Page 13: Dynamic Optimization and Automatic Differentiation](https://reader036.fdocuments.in/reader036/viewer/2022070410/56814527550346895db1ed58/html5/thumbnails/13.jpg)
Colloquium on Optimization for Control13
Automatic Taylor expansionAutomatic Taylor expansion
• z(t)=f(x(t)), t scalar, x and z vectors• TS: x(t)=x0+x1t+x2t2+…+xdtd • TS: z(t)=z0+z1t+z2t2+…+zdtd
• AD forward (TS of z): zk = zk(x0,x1,…,xk)
• AD reverse (TS of sensitivity): zk /xj = zk-j / x0 := Ak-j
• f’x=A0+A1t+A2t2+…+Adtd
![Page 14: Dynamic Optimization and Automatic Differentiation](https://reader036.fdocuments.in/reader036/viewer/2022070410/56814527550346895db1ed58/html5/thumbnails/14.jpg)
Colloquium on Optimization for Control14
Recursively solving ODE using TSRecursively solving ODE using TS
• dx/dt=f(x)• dx/dt=z(t)• Recursive relation: xk+1=zk/(k+1).• x0=x(t0),
• x1=z0(x
0),
• x2=z1(x0,x1), …• x(t0+h)=d
i=0 xihi • Next step: t1=t0+ht0
![Page 15: Dynamic Optimization and Automatic Differentiation](https://reader036.fdocuments.in/reader036/viewer/2022070410/56814527550346895db1ed58/html5/thumbnails/15.jpg)
Colloquium on Optimization for Control15
Solving ODE with sensitivity Solving ODE with sensitivity equationequation
• Sensitivity equation:dx/dt=fx(t,x,)x+f(t,x,)
• v={x0,}
• Avk:=zk/v • Bvk:=dxk/dv• Bvk+1= (Avk+k
j=0Axk-jBvj)/(k+1)• Bv:=dx(t0+h)/dv
=Bv0+Bv1h+Bv2h2+…+Bvdhd
![Page 16: Dynamic Optimization and Automatic Differentiation](https://reader036.fdocuments.in/reader036/viewer/2022070410/56814527550346895db1ed58/html5/thumbnails/16.jpg)
Colloquium on Optimization for Control16
Solving dynamic optimization using Solving dynamic optimization using TSTS
• Convert cost to terminal cost:min F(tf,x(tf),(tf))
• Initial guass: 0={0(t0),…, 0(tf)}• Step: hi=ti+1-ti>0, i=0,1,…,n-1, tn=tf
• Integrate ODE & sensitivity from t0 to tf • x(t0), x(t1), …, x(tf) and F(tf,x(tf),0(tf))• Bx(t1),…,Bx(tf), B(t1),…,B(tf) and Fx, F • dF/d0(ti)=FxBx(tf)…Bx(ti+2)B(ti+1)• For constant , dF/d0= dF/d0(ti)
• Update i+1
=i- dF/di
![Page 17: Dynamic Optimization and Automatic Differentiation](https://reader036.fdocuments.in/reader036/viewer/2022070410/56814527550346895db1ed58/html5/thumbnails/17.jpg)
Colloquium on Optimization for Control17
Error ControlError Control
• h=step, d=order of TS• (h,d)=C(h/r)d+1
• r ≈ rd=||xd-1||/||xd|| for large d
• (h,d-1) ≈ (h,d)(rd/h)≤(h,d)+||xd||
• (h,d)≤h||xd||2/(||xd-1||-h||xd||)
• Given tolerance and d, determine h• Given tolerance, determine optimal d, h to
minimize computation.
• Global error: ≥||Bx||i and g global tolerance
• local tolerance: =g(-1)/(n-1)
![Page 18: Dynamic Optimization and Automatic Differentiation](https://reader036.fdocuments.in/reader036/viewer/2022070410/56814527550346895db1ed58/html5/thumbnails/18.jpg)
Colloquium on Optimization for Control18
NMPC using differential recurrent NNNMPC using differential recurrent NN
• Continuous-time nonlinear identification
• Efficient algorithm to train DRNN• Training performance and
efficiency• DRNN as internal model for NMPC• Efficient algorithm for NMPC• NMPC performance and efficiency
![Page 19: Dynamic Optimization and Automatic Differentiation](https://reader036.fdocuments.in/reader036/viewer/2022070410/56814527550346895db1ed58/html5/thumbnails/19.jpg)
Colloquium on Optimization for Control19
Differential recurrent neural Differential recurrent neural networksnetworks
C
W uW 2
W x
b 1 b 2
u
y
x
Input
Hidden
Output
2 1 1 1 2( )( , )
( )x ux W W x W u b bz f z u
y g z y Cx
![Page 20: Dynamic Optimization and Automatic Differentiation](https://reader036.fdocuments.in/reader036/viewer/2022070410/56814527550346895db1ed58/html5/thumbnails/20.jpg)
Colloquium on Optimization for Control20
DRNN TrainingDRNN Training
• ={b1,b2,vec(W2),vec(W1x),vec(W1u)}
• N=Nh+Nx+Nh×Nu+2Nh×Nx
• Training data: u(t), y(t) at t=0,h,…,Nh• Solving DRNN + sensitivity using TS• Assume t=0 is steady-state, apply u(t)
• y(t) at t=h,…,Nh and ek=y(kh)-y(kh)
• min keTkek/2=ETE/2, E=vec(e1,…,eN)
• Nonlinear least square (NLSQ) optimization
![Page 21: Dynamic Optimization and Automatic Differentiation](https://reader036.fdocuments.in/reader036/viewer/2022070410/56814527550346895db1ed58/html5/thumbnails/21.jpg)
Colloquium on Optimization for Control21
Continuous time NMPCContinuous time NMPC
• minu φ=∫0T(y-yr)TQ(y-yr)+(u-ur)TR(u-ur)dt/2
• s.t. dx/dt=f(x,u), y=g(x)
• umin ≤ u ≤ umax, 0=t0 ≤ … ≤ tP=T
• Parameterize u: piecewise polynomial
• u(t)=qi=0uki(t-tk)i tk ≤ t ≤ tk+1, k=0,…,P-1
• U=[uT00,…,uT
0q,…,uTP-10,…,uT
P-1q]T
• Y=[yT00,…,yT
0d,…,yTP-10,…,yT
P-1d]T
• φ=T(YeTHQYe+Ue
THRUe)/2
• φ=ETE/2 NLSQ
![Page 22: Dynamic Optimization and Automatic Differentiation](https://reader036.fdocuments.in/reader036/viewer/2022070410/56814527550346895db1ed58/html5/thumbnails/22.jpg)
Colloquium on Optimization for Control22
NMPC algorithm (with DRNN)NMPC algorithm (with DRNN)
• x0 (steady-state or estimated), ym
• d=ym-Cx0 (constant for 0 ≤ t ≤ T)
• U0 Ue=U0-Ur X Y Ye=Y-Yr+d
• Jacobian, J=E/U
• Unconstrained: Uk+1=Uk-(JTJ)-1JTE
• Input constrained: lsqnonlin
• Other constraints: fmincon/SQP
• Apply [uT00,…,uT
0q]T
• repeat
![Page 23: Dynamic Optimization and Automatic Differentiation](https://reader036.fdocuments.in/reader036/viewer/2022070410/56814527550346895db1ed58/html5/thumbnails/23.jpg)
Colloquium on Optimization for Control23
Tank Reactor ExampleTank Reactor Example
• 2-CSTR in series, reaction A+B C
• 2-output: T1 and T2 ,2-input: Qcw1 and Qcw2
• Distuabnce: Tcw
• 6 states
CSTR1 CSTR2
Mixer
Feed 1
WaterIn
WaterOut
WaterOut
Water In
Feed 2
Cooling Cooling
Product
CoolingCooling
![Page 24: Dynamic Optimization and Automatic Differentiation](https://reader036.fdocuments.in/reader036/viewer/2022070410/56814527550346895db1ed58/html5/thumbnails/24.jpg)
Colloquium on Optimization for Control24
DRNN identificationDRNN identification
• Nh=6, Nx=6, Nu=2, N=96
• 600-s data with h=0.1 s, N=6000• Total sensitivity 3456000• Validation set sampled at 0.02 s• Advantage: change sampling rate
does not need re-training
![Page 25: Dynamic Optimization and Automatic Differentiation](https://reader036.fdocuments.in/reader036/viewer/2022070410/56814527550346895db1ed58/html5/thumbnails/25.jpg)
Colloquium on Optimization for Control25
Training and ValidatingTraining and Validating
0 200 400 600350
360
370
380T O
1, o K
Time, sec
(a)
0 200 400 600355
360
365
370
T O2,
o K
Time, sec
(b)
0 200 400 600350
360
370
380
390
T O1,
o K
Time, sec
(c)
0 200 400 600350
360
370
380
T O2,
o K
Time, sec
(d)
![Page 26: Dynamic Optimization and Automatic Differentiation](https://reader036.fdocuments.in/reader036/viewer/2022070410/56814527550346895db1ed58/html5/thumbnails/26.jpg)
Colloquium on Optimization for Control26
Training efficiency (one epochTraining efficiency (one epoch))
ODE23 AD
tol t, ms error d t, ms error
103 40.61 1.590 3 2.437 0.001
106 162.28 0.074 6 4.078 1.6×10-7
108 272.66 4.6×10-
4
8 5.391 2.6×10-
10
101
0
316.37 1.9×10-
6
10 6.937 1.1×10-
12
![Page 27: Dynamic Optimization and Automatic Differentiation](https://reader036.fdocuments.in/reader036/viewer/2022070410/56814527550346895db1ed58/html5/thumbnails/27.jpg)
Colloquium on Optimization for Control27
NMPC performance, setpoint changeNMPC performance, setpoint change
0 50 100 150 200360
365
370
To1
, o K
0 50 100 150 200360
365
370
To2
, o K
0 50 100 150 2000
0.5
1
Qcw
1, m
3 /s
time, sec
0 50 100 150 2000
0.5
1
Qcw
2, m
3 /s
time, sec
![Page 28: Dynamic Optimization and Automatic Differentiation](https://reader036.fdocuments.in/reader036/viewer/2022070410/56814527550346895db1ed58/html5/thumbnails/28.jpg)
Colloquium on Optimization for Control28
NMPC performance, disturbance NMPC performance, disturbance rejectionrejection
0 10 20 30362.94
363
363.02
T o1 (
o K)
0 10 20 30362.94
363
363.02
T o2 (
o K)
0 10 20 300
0.2
0.4
0.6
0.8
Qcw
1 ( m
3 /s)
Time, sec0 10 20 30
0
0.2
0.4
0.6
0.8
Qcw
2 ( m
3 /s)
Time, sec
![Page 29: Dynamic Optimization and Automatic Differentiation](https://reader036.fdocuments.in/reader036/viewer/2022070410/56814527550346895db1ed58/html5/thumbnails/29.jpg)
Colloquium on Optimization for Control29
ConclusionsConclusions
• High-order TS using AD• Efficiently solving ODE + sensitivity• Efficient algorithm for general
dynamic optimization• Efficient algorithm for RDNN training• Efficient algorithm for continuous-
time NMPC• Demonstrated with 2-CSTR example.