1

2

t 2 [0;tf ], x(0) = x0 2 Rn

x 2 Rn , u 2 Rk, v 2 V ½R; jV j < 1

_x = f (x;u;v)

J =Z tf

0L(x;u)dt

The optimal mode-scheduling problem

3

t 2 [0;tf ], x(0) = x0 2 Rnx 2 Rn , v 2 V ½R; jVj < 1

_x = f (x;v)

J =Z tf

0L(x)dt

The autonomous problem

4

_x = f (x;v)Let v(t) = vi ; t 2 [¿i ¡ 1;¿i )i = 0;:: : ;N + 1; 0= ¿0 · ¿1; : : : ;¿N · ¿N +1 = tf

_x = f i (x) := f (x;vi )t 2 [¿i ¡ 1;¿i ); i = 1;: : :;N + 1

5

_x = f i (x)t 2 [¿i ¡ 1;¿i ); i = 1;: : :;N + 1

J =Z tf

0L(x)dt

6

Application areas

• Automotive powertrain control (Wang, Beydoun, Cook, Sun, and Kolmanovsky)• Switching circuits (Almer, Mariethoz, and Morari;

DeCarlo et al.; Kawashima et al.)• Telecommunications (Rehbinder and Sanfirdson;

Hristu-Varsakelis)• Switching control between subsystems or data

sources (Lincoln and Rantzer; Brockett)• Mobile robotics (Egerstedt)

7

Problem classifications

• Linear vs. nonlinear• Timing optimization vs. sequencing

optimization• Off line vs. on line

8

Theoretical developments

• Problem definition: Branicky, Borkar, and Mitter • Maximum principle: Piccoli; Shaikh and Caines;

Sussmann• Algorithms: Xu and Antsaklis; Shaikh and Caines;

Attia, Alamir, and Canudas de Wit; Bengea and DeCarlo; Egerstedt et al.; Caldwell and Murphy; Gonzalez, Vasudevan, Kamgarpour, Sastry, Bajcsy, and Tomlin

• Control: Bengea and DeCarlo; Almer, Mariethoz, and Morari; Kawashima et al.

9

The timing optimization problem

_x = f i (x); t 2 [¿i ¡ 1;¿i )

¿0 = 0 ¿1 ¿N +1 = tf¿2 ¿N

i = 1;: : : ;N + 1

Variable: ¹¿ = (¿1; : : : ;¿N )>

Constraints : 0 · ¿1 · ¿2 · : : : · ¿N · tf

Problem: minf J := Rtf0 L(x)dt : ¹¿ 2 ¡ g

10

r J (¹¿) =³ @J

@¿1(¹¿); : : : ; @J

@¿N(¹¿)

´>

The gradient

Define the costate equation

_p = ¡³ df i

dx(x)´>

p¡³ dL

dx (x)´>

;t 2 [¿i+1;¿i ); i = N + 1;:: :;1,with theboundary condition p(tf ) = 0.

Variational arguments:@J@¿i

(¹¿) = p(¿i )>¡f i (x(¿i )) ¡ f i+1(x(¿i ))¢; i = 1;:: : ;N

11

Steepest descent algorithm with Armijo step size

Minimize f (x) : Rn ! R

Given ®2 (0;1);¯ 2 (0;1)

xnext = x ¡ ¸(x)r f (x)

¸(x) = ¯ j (x)

12

¸

f (x ¡ ¸r f (x)) ¡ f (x) ¡ ® jjr f (x)jj2

¯ j

f (xnext) ¡ f (x) · ¡ ® j (x) jjr f (x)jj2

¸(x) = ¯ j (x)

xnext = x ¡ ¸(x)r f (x)

13

Principle of sufficient descent

If H (x) := df 2

dx2 (x) is bounded, there exists ¹ > 0

f (xnext) ¡ f (x) · ¡ ® (x)jjr f (x)jj2 · ¡ ®¹ jjr f (x)jj2

such that 8 x 2 Rn ; ¸(x) ¸ ¹ .

f (x ¡ ¸r f (x)) ¡ f (x) = ¡ ¸jjr f (x)jj2 + ¸2Z 1

0(1¡ t)hH (x)r f (x);r f (x)idt

f (x ¡ ¸r f (x)) ¡ f (x) + ® jjr f (x)jj2

= ¡ (1¡ ®)¸jjr f (x)jj2 + ¸2Z 1

0(1¡ t)hH (x)r f (x);r f (x)idt

14

The Steepest Descent Algorithm with Armijo Step size

Corollary: If jjr f (x)jj ¸ c then

f (xnext) ¡ f (x) · ¡ ®¹c2

xi+1 = xi ;next; i = 0;1;2;: : :Theorem: (Armijo, Polak) If x is an accumulationpoint of the sequence fxi g1

i=0, thenr f (x) = 0

15

Modification: descent algorithm with Armijo step size

xnext = x + ¸(x)h(x)

¸(x) = ¯ j (x)

jjh(x) + r f (x)jj < °jjr f (x)jj

j (x) : minf j = 0;1;: : : ; :f (x + ¯ j h(x)) ¡ f (x) · ® j hh(x);r f (x)ig

h(x)

r f (x)0< ° < 1

16

The timing optimization problem

_x = f i (x); t 2 [¿i ¡ 1;¿i )i = 1;: : : ;N + 1

Constraints : ¹¿ 2 ¡ := f0 · ¿1 · ¿2 · : : : · ¿N · tf g

Problem: minf J (¹¿) : ¹¿ 2 ¡ g

J (¹¿) := Rtf0 L(x)dt

17

Constrained algorithm:

¹¿next = ¹¿ + ¸(¹¿)h(¹¿)h(¹¿) = proj (¡ r J (¹¿);¡ ¡ f ¹¿g)

r J (¹¿)h(¹¿)

If ¹¿ + h(¹¿) is infeasible,start the search for ¸(¹¿)at ¹°h(¹¿), where ¹° :=maxf ° > 0: ¹¿ ¡ °h(¹¿) 2 ¡ g.

Algorithm: ¹¿i+1 = ¹¿i ;next.Convergence to Kuhn-Tucker points.

18

On-line setting

_x = f (x; ¹¿) := f i (x) given x0t 2 [¿i ;¿i+1); i = 1;: :: ;N + 1

J (¹¿) = Rtf0 L(x)dt

Given a stateobserver x(t)_~x(») = f (~x(»); ¹¿); » ¸ t; ~x(t) = x(t)J (t; x(t); ¹¿) = Rtf

t L(~x(»)d»The cost-to-go problem:min©J (t; x(t); ¹¿)ª given t; x(t)

19

Let ¿¤(t) be the (a) solution of thecost-to-go problem at (t; x(t)).

@J@¿ (t; x(t);¿¤(t)) = 0.

ddt

³@J@¿ (t; x(t);¿¤(t))

´= @2 J

@¿2 (t; x(t);¿¤(t)) _¿¤(t) +@2 J@t@¿ (t; x(t);¿¤(t)) + @2 J

@x@¿ (t; x(t);¿¤(t)) _x(t) = 0

Hence_¿¤(t) = ¡

³@2 J@¿2 (t; x(t);¿¤(t))

´ ¡ 1£³

@2 J@t@¿ (t; x(t);¿¤(t)) + @2 J

@x@¿ (t; x(t);¿¤(t)) _x(t)´

20

_¿¤(t) = ¡³

@2 J@¿2 (t; x(t);¿¤(t))

´ ¡ 1£³

@2 J@t@¿ (t; x(t);¿¤(t)) + @2 J

@x@¿ (t; x(t);¿¤(t)) _x(t)´

De ne H (t) = @J 2@¿2 (t; x(t); ¹¿(t))

Algorithm:

¹¿(t + ¢ t) = ¹¿(t) ¡ H (t)¡ 1 @J@¿ (t + ¢ t; x(t + ¢ t); ¹¿(t))

t t + ¢ t

(assuming ¹¿ lies in the interior of the feasible set)

21

Asymptotic convergence – meaningless. Instead, approach to stationary points

Consider thecase where t < ¿1

¿¤ is said to be stationary if@J@¿ (0;x0;¿¤) = 0

@J@¿ (t;x(t);¿¤) = 0

In this case,

22

Proposition: Suppose that ¿¤ is stationaryand lies in the interior of ¡ . Suppose that@2 J@¿2 (0;x0;¿¤) is positive de nite.There exist ±> 0 and K > 0 such that ifjj¹¿(t) ¡ ¿¤jj < ±; ¢ t < ±;jje(t)jj := jjx(t) ¡ x(t)jj < ±and jje(t + ¢ t)jj < ±,then

jj¹¿(t + ¢ t) ¡ ¿¤jj ·K

³jj¹¿(t) ¡ ¿¤jj2 + ¢ tjj¹¿(t) ¡ ¿¤jj + jje(t + ¢ t)jj

´:

23

Uncertainty in f i and L_x = f i (x;t)

J (¹¿) = Rtf0 L(x;t)dt

~x = ~x(s;t; ¹¿)@~x@s = ~f i (~x;s;t)

s ¸ t~x(t;t; ¹¿) = x(t)

~J (t; ¹¿) = Rtft

~L(~x;s;t)ds

Steepest descent algorithm with Armijo step size

J (t; ¹¿) = Rtft L(x;s)ds

24

25

Let ¿¤ be a local minimum for JDe ne"0(t) = j ~J (t;¢) ¡ J (t;¢)jL 1

"1(t) = jj@~J@¿ (t;¢) ¡ @J

@¿ (t;¢)jjL 1

E i (t) = maxf"i (s) : s 2 [t;tf ]g; i = 0;1

There exist constants c 2 (0;1), andConvergence result (CDC 2010):

K 1 > 0 and K 2 > 0, such that,

jj¹¿(tj ) ¡ ¿¤jj · K 1cj + K 2¡E0(tj )1=2 + E1(tj )1=2

´

26

Example: A mobile robot tracking a target (goal) while avoiding two obstacles. The robot predict the future movement of the target by linear approximation given its position and velocity

27

28

29

_x = f (x;v)_x = f 1(x) f 2(x)

J = Rtf0 L(x)dt

Minimize J as a function of ¾2 V

The sequencing optimization problem

Admissible controls: ¾2 V := fv(¢) is left continuousand has a ¯nite number of switchingsg

30

Current approaches:

• Geometric approaches (Shaikh and Caines)• Relaxation algorithms (Bengea and DeCarlo,

Caldwell and Murphy)• Gradient techniques (Xu and Antsaklis, Gonzalez

and Tomlin, Attia et al., Egerstedt et al.)

31

Sensitivity analysis and optimality function

_x = f (x;v(s))

sv(s)

_x = f (x;w)

s + ¸

D¾;s;w := dJd¸ + (0)

Gradient insertion

32

D¾;s;w = p(s)>¡f (x(s);w) ¡ f (x(s);v(s))¢

D¾;s = minfD¾;s;w : w 2 Vg Observe that D¾;s · 0

D¾ = inffD¾;s : s 2 [0;T]g D¾ · 0

D¾ acts as an optimality functionOptimality condition: D¾ = 0

_p= ¡³

@f@x (x;v)

´>¡

³dLdx (x)

´>; p(tf ) = 0

33

Optimality functions and steepest descent (Polak)

minf f (x) : x 2 X g

¢ ½X : a set where an optimality condition is satis ed

Optimality function: µ : X ! R¡ such that µ¡ 1(0) = ¢

Example: minf f (x) : x 2 Rngµ(x) = ¡ jjr f (x)jj

34

if 8±> 0 9 ´ > 0 such that, if µ(xi ) < ¡ ±thenf (xi+1) ¡ f (xi ) < ¡ ´

Proposition: If the algorithm is of su±cient descentand f (x) is bounded from below on X , then

Algorithm computing fxi g1i=1 is of su±cient-descent

limi ! 1 µ(xi ) = 0

Under certain circumstances, if x is a limit point offxi g1

i=1 then µ(x) = 0

35

s

D¾;s

D¾

´D¾

S¾;´

\ Gradient" descent algorithm: swap themodesin a subset of S¾;´

Armijo step size: based on the Lebesgue measureof the set where themodes are swappedRationale: su±cient descent

36

S(¸) ½S¾;´ such that ¹ (S(¸)) = ¸

¾next = ¾(¸ j (¾))

Mapping S : [0;¹ (S¾;´ )] ! 2S¾;´

¾(¸) - the schedule obtained by swapping v(s)by w(s) for every s 2 S(¸)

¸ j = ¹ (S¾;´ )¯ j

Given ´ 2 (0;1), ®2 (0;´), ¯ 2 (0;1)

Given ¾2 V, conmpute ¾next as follows

37

Sufficient descent

P roposition: There exists a constant c > 0 suchthat, for every ¾2 V satisfying D¾ < 0, and forevery ¸ 2 [0;¹ (S¾;´ )] satisfying ¸ · cjD¾j,

J (¾(¸)) ¡ J (¾) · ® D¾

38

Proposition: 1. The following limit is in force,limsup

k! 1D¾k = 0:

Suppose that a sequence f¾kg1k=1 of

mode-schedules is computed via the aboveprocedure.

2. If ¾¤ is a limit point of f¾kg1k=1, then

D¾¤ = 0:

39

S. Almer, S. Mriethoz, M. Morari, “Optimal Sampled Data Control of PWM Systems Using Piecewise Affine Approximations”, Proc. 49th CDC, Atlanta, 2010

C = 70=2¼farad, L = 3=2 henry, rc = 0:005 ohm, r` = 0:05 ohm

Example

40

v = 1 - switch in H positionv = 0 - switch in L position

41

PWM problem (Almer at al. [2], 2010 CDC)

Constant cyclesPWM over M cycles, initial conditions measured

Minimize J = 12

Rk+Mk

¡vc(t) ¡ vc;r ef¢2dt

Constraints: i`(t) · i`;max 8t

vs = 1:8, io shown here90 cycles

vc;r ef = 1:0;i`;max = 3:0M = 2

42

J a;c =Rk+Mk

³12¡vc(t) ¡ vc;ref

¢2dt + Lp(i`(t) ¡ imax¢ dt

43

The scheduling optimization problem

J = Rtf0

12¡vc(t) ¡ vc;ref

¢2dtConstraints: i`(t) · i`;max 8t 2 [0;T]

J p =Rtf0

³12¡vc(t) ¡ vc;r ef

¢2dt + Lp(i`(t) ¡ imax¢ dt

tf = 20;vs(t) = 1:8; i0(t) = 1:0;all other parameter as for the prevous problem

44

J (0) = 33:0, J (10) = 1:6

vint(t) =½ 0; 0 · t · 10

1; 10< t · 1

45

v 2 f1;2g

x0 = (2;2)>

tf = 20

46

J (¾1) = 70:90; J (¾100 = 4:87; J (¾200) = 4:78D¾1 = ¡ 14:92; D¾1000 = ¡ 0:23; D¾200 = ¡ 0:0062

47

Adding a switching cost

H. Kawashima, Y. Wardi, D. Taylor, and M. Egerstedt, 2012 ADHS

The PWM problem, variable number of cycles

48

The total switching energy is a function of the corrent i`and the switching times, hance a function of ½k, k = 1;:: : ;Nas well as the number of cycles.

Model for switching energy

Assume the switch is based on a transistor-diode pair.

Supose it takes ts seconds to open or close the switch.N. Mohan, T.M. Undeland, and W.P. Robbins, Power Electronics: Convert-

ers, Applications, and Design, J ohn Wiley & Sons, NY , 1995:

E = 14tsvsi`(t)

49

Nr - thenumber of switchings (Nr 2 f2N ¡ 1;2N g).¿1; : : : ;¿N r - the switchineg times.

J e = 14tsvs

P N ri=1 i`(¿i )

Energy cost:

Minimize J := (1¡ w)J p + wJ e

J e = 14tsvs

P N ri=1 i`(¿i )

Optimal control problem:

J p = 12

Rtf0

¡v0(t) ¡ vr¢2dt

Not in the form J = RT0 L(x)dt!

50

J e = 14tsvs

P N ri=1 i`(¿i )

= ts vs2

P N ¡ 1k=1

i ` (¿2k ¡ 1)+i ` (¿2k )2

= ts vs2

1Tc

P N ¡ 1k=1

i ` (¿2k ¡ 1)+i ` (¿2k )2 Tc

' ts vs2

1Tc

Rtf0 i`(¿)d¿

~J e = ts vs2

1Tc

Rtf0 i`(¿)d¿

~J := (1¡ w)J p + w ~J e

51

+wts vs2

1Tc

Rtf0 i`(t)dt

~J := 12(1¡ w) Rtf

0¡vo(t) ¡ vr

¢2dt

This is in the formRtf0 L(x(t))dt

Optimization variable: »= (Tc;½1; : : : ;½N )

Optimization problem: min ~J

Constraints: Tc ¸ ²; 0 · ½k · 1

52

»0: N = 200, ½k = 0:5»100: N = 14, ½k ' 0:66~J (»0) = 0:86; ~J (»100) = 0:024

"max = 0:0030

w=0.5

53

»0: N = 200, ½k = 0:5»100: N = 9, ½k ' 0:65~J (»0) = 0:72; ~J (»100) = 0:027

"max = 0:0028

w=0.9

54

»0: N = 200, ½k = 0:5»100: N = 22, ½k ' 0:66~J (»0) = 1:00; ~J (»100) = 0:0081

"max = 0:0030

w=0.1

55

w = 0:5; N100 = 14

w = 0:1; N100 = 22

w = 0:9; N100 = 9

56

Thank you