1

EECS 127/227AT Optimization Models in Engineering

Spring 2020 Discussion 12

1. LQR and least squares

In this question, we consider the time-dependent n-state m-input LQR problem

min~xt2Rn,~ut2Rm

T�1X

t=0

⇣~x>t Qt~xt + ~u

>t Rt~ut

⌘+ ~x

>TQT~xT

s.t. ~xt+1 = A~xt +B~ut, t = 1, . . . , T

~x0 = ~xinit,

where Qt = Q>t ⌫ 0 for all t = 0, . . . , T and Rt = R

>t ⌫ 0 for all t = 0, . . . , T � 1. Note that this

is a minor extension of the standard LQR formulation explored in class — we allow the cost asso-

ciated with each state ~xt and input ~ut to vary by time step. For clarity, note also that A 2 Rn⇥n,

B 2 Rn⇥m, Qt 2 Rn⇥n

for all t = 0, . . . , T , and Rt 2 Rm⇥mfor all t = 0, . . . , T � 1.

In this problem, we reformulate this calculation as a least squares problem, examine its prop-

erties, and compare this solution strategy with others shown in class.

(a) Concatenating variables of interest. We first make our formulation more concise by

concatenating our states and inputs into single vectors and computing the associated matrices.

i. Define full state vector ~x and input vector ~u as follows:

~x =

2

6664

~x0

~x1...

~xT

3

77752 Rn(T+1)

, ~u =

2

6664

~u0

~u1...

~uT�1

3

77752 RmT

.

Show that we can rewrite our LQR objective function as

~x>Q~x+ ~u

>R~u

for some matrices Q 2 Rn(T+1)⇥n(T+1)and R 2 RmT⇥mT

, which you determine.

Solution: The desired equivalence holds when

Q =

2

6666664

Q0 0 · · · · · · 0

0 Q1 0 · · · 0

.

.

.. . .

.

.

.

.

.

.. . . 0

0 · · · · · · 0 QT

3

7777775, R =

2

6666664

R0 0 · · · · · · 0

0 R1 0 · · · 0

.

.

.. . .

.

.

.

.

.

.. . . 0

0 · · · · · · 0 RT�1

3

7777775.

ii. Show that we can reformulate our constraints (i.e., dynamics) as

~x = G~u+H~xinit

TX x

2

for some matrices G 2 Rn(T+1)⇥mTand H 2 Rn(T+1)⇥n

, which you determine.

Solution: We can rewrite our dynamics equations iteratively as

~x0 = ~xinit

~x1 = A~x0 +B~u0 = A~xinit +B~u0

~x2 = A~x1 +B~u1 = A(A~xinit +B~u0) +B~u1 = A2~xinit +AB~u0 +B~u1

.

.

.

~xt = At~xinit +

t�1X

s=0

At�s�1

B~us

.

.

.

Compiling these equations into matrix form, we have

2

66666664

~x0

~x1

~x2...

~xT�1

~xT

3

77777775

=

2

66666664

0 · · · · · · · · · 0

B 0 · · · · · · 0

AB B 0 · · · 0

.

.

....

. . .. . .

.

.

.

AT�2

B AT�3

B · · · B 0

AT�1

B AT�2

B · · · AB B

3

77777775

| {z }G

2

666664

~u0

~u1

~u2...

~uT�1

3

777775+

2

66666664

I

A

A2

.

.

.

AT�1

AT

3

77777775

| {z }H

~xinit

as desired.

(b) Formulating the least squares problem. We have now reduced our LQR problem to

min~x2Rn(T+1),~u2RmT

~x>Q~x+ ~u

>R~u

s.t. ~x = G~u+H~xinit.

Rewrite this optimization as an unconstrained least squares problem over ~u.

Solution: Our constraint now explicitly defines the value of ~x, so we can plug it into the

objective function to get equivalent optimization problem

min~u2RmT

(G~u+H~xinit)>Q(G~u+H~xinit) + ~u

>R~u

= min~u2RmT

kQ12G~u+Q

12H~xinitk22 + kR

12~uk22

= min~u2RmT

�����

"Q

12G

R12

#~u+

Q

12H~xinit

0

������

2

2

where matrix square-roots Q12 and R

12 are

Q12 =

2

666666664

Q

120 0 · · · · · · 0

0 Q

121 0 · · · 0

.

.

.. . .

.

.

.

.

.

.. . . 0

0 · · · · · · 0 Q

12T

3

777777775

, R12 =

2

666666664

R

120 0 · · · · · · 0

0 R

121 0 · · · 0

.

.

.. . .

.

.

.

.

.

.. . . 0

0 · · · · · · 0 R

12T�1

3

777777775

.

u t 55 u eco uizyo.mn lAU bllI

QQ

3

(Recall that because each of Q’s (R’s) block diagonal entries Qt (Rt) are PSD, we can compute

a real-valued matrix square root in this manner.) This is now a standard least squares problem

over ~u as desired.

(c) Analysis. We now examine the practicality of using least squares to solve the LQR problem.

Note: This section is meant to provide intuition, not rigorous complexity analysis, and ispresented primarily to illustrate the practical utility of di↵erent LQR methods. Do not feelobligated to understand the arguments below in detail.

Recall from lecture that the LQR problem can also be solved via the following recursive

procedure:

(1) Set PT = QT , then solve iteratively backward in time for “helper” matrices PT�1, . . . , P0

via Riccati equation

Pt = A>(I + Pt+1BR

�1t B

>)�1

Pt+1A+Qt

(2) Solve iteratively forward in time for optimal ~x1, . . . , ~xT and ~u, . . . , ~uT�1 via

~ut = �R�1t B

>(I + Pt+1BR

�1t B

>)�1

Pt+1A~xt

~xt+1 = A~xt +B~ut

We will compare this strategy with the least squares formulation developed above.

i. Suppose n = 2, m = 2, and T = 10, 000, i.e., we want to solve for the optimal control of a

2-state, 2-input system over a horizon of 10,000 time steps. Which solution method would

you use? Hint: Over long time horizons, computational e�ciency is a major concern.Solution: If computational e�ciency is of primary concern, as with long time horizons,

the iterative method explored in class is the better choice. Our least squares problem is

of matrix size n(T +1)+mT ⇥mT — that’s 40, 002⇥ 20, 000, or over 800 million entries.

Computing the solution to such a problem using standard least squares techniques (QR

factorization, naive matrix inversion, etc.) would require serious computing power.

In contrast, while the iterative method requires computing matrices and vector values

at each of the 10, 000 time steps, it requires multiplying and inverted matrices no larger

than 2 ⇥ 2, so each computation is e↵ectively constant time, and the overall runtime is

several orders of magnitude smaller.

ii. Suppose n = 2, m = 2, and T = 100, and we want to impose constraints on the control

values ~u (e.g., each element of the ~u vector must remain between ±10 units).1Which of

these formulations might you choose to incorporate such constraints?

Solution: While our iterative formulation is useful for unconstrained problems, it does

not readily accommodate constraints of any kind, and is generally most useful when

operating when such constraints aren’t necessary (e.g., when you want to control a device

in an operating range that won’t require too much control input from its actuators).

Our least squares formulation, on the other hand, is just a QP, so for small enough time

horizons (resulting in small enough matrices to be e�ciently computable), we can easily

tack on arbitrary convex constraints!

2. Can we use slack variables?

So far, we’ve presented slack variables as a method of converting optimization problems to a desired

form, and it may seem like we can always use them. In this question, we take a more nuanced look

1Constraints like this are common in control problems; our motors/actuators usually can’t provide infinite power!

qgyp2 0

A4105444

2 108

g410 t

uto isW ofopramyproor

4

at when slack variables are helpful and when they are not.

For each of the following functions, consider the unconstrained optimization problem

p⇤j = min

~x2Rnfj(~x)

If possible, reformulate each problem as an SOCP using slack variables. If not possible, explain

why.2

(a) f1(~x) = kA~x� ~yk2 + k~xk1Solution: When reformulating problems using slack variables, our reformulation occurs in

two steps (a detail we often elide): first, we introduce new variables without changing

the problem (i.e., adding only equality constraints for our new variables), then we relax

the problem, being cautious to determine that this relaxation is guaranteed not to changethe optimal value. We illustrate these steps explicitly in this solution.

We first introduce our new (slack) variables to linearize our objective function, generating

equivalent optimization problem

p⇤1 = min

~x,~t2Rn,s2Rs+

nX

i=1

ti

s.t. kA~x� ~yk2 = s

|xi| = ti, i = 1, . . . , n.

Note that this problem is exactly the same as the original formulation; we have used only the

definition that k~xk1 =Pn

i=1 |xi|, and have otherwise only introduced redundant variables.

The next step is to relax our problem. Consider first a single constraint |xi| = ti, and

note that implicitly, this means that ti � 0. Now, consider an optimization problem in which

instead of enforcing |xi| = ti, we only enforce that |xi| ti. Suppose we find minimizers ~x⇤

and ~t⇤for that optimization problem in which this constraint does not hold with equality,

i.e., |x⇤i | < t⇤i . We could then decrease the value of our objective function by reducing ti by

the value of t⇤i � |x⇤i |, contradicting our assertion that ~t

⇤is a minimizer. Thus, |xi| ti is

equivalent to our equality constraint, and because this argument holds for each of our n + 1

constraints, we can rewrite our optimization problem as

p⇤1 = min

~x,~t2Rn,s2Rs+

nX

i=1

ti

s.t. kA~x� ~yk2 s

|xi| ti, i = 1, . . . , n.

while guaranteeing that the optimal value remains the same.

The last step is to rewrite our problem as an SOCP. Our objective function is linear, and

the first constraint is an SOCP constraint, so we have only to reformulate the final set of

constraints. Recall that a given constraint |xi| ti is equivalent to the constraints xi ti and

xi � �ti:

2You might notice that this problem is similar to Homework 11 problem 5, particularly part (c). Here, we’re using

this formulation to explore slack variables more precisely.

m.ienllAn Y Tis

MiwasER's

HAN y112

mainsS Kt s An

8112

ten

YzpjBecame in theinequality you

hone

at

mmsfrefdfm.tochoo

adultsPV

iv ii iHE HAE yik

proof Bycontradiction suppose s Hanky't

I Han't yy t E for some e 0

check thats e

k would be

feasible ARID hare lower objectivevalue

This 51n't cannot be optimal

Ii

mm ttrTI lait ti

M gir.ee

ad

AND

5

|xi| ti

xi tixi � �ti

ti�ti

xi

We can thus write our problem in final form as

p⇤1 = min

~x,~t2Rn,s2Rs+

nX

i=1

ti

s.t. kA~x� ~yk2 s

xi ti, i = 1, . . . , n,

xi � �ti, i = 1, . . . , n,

which is an SOCP.

(b) f2(~x) = kA~x� ~yk2 � k~xk1Solution: While this problem appears almost identical to the one in part (a), it cannot

be written as an SOCP; in fact, it is not convex. To see this, we can attempt to follow

the same procedure as above. First, we introduce our new variables, to generate equivalent

problem

p⇤2 = min

~x,~t2Rn,s2Rs�

nX

i=1

ti

s.t. kA~x� ~yk2 = s

|xi| = ti, i = 1, . . . , n.

We then relax our equality constraints to inequalities; again, we can perform this step suc-

cessfully, but we must be cautious, as the sign on each value ti in the objective function has

changed. If we require that |xi| ti as before, then our optimization problem is unbounded

below, since we can increase ti arbitrarily high to generate an arbitrarily low value. Instead,

we must require that |xi| � ti, giving us optimization problem

p⇤2 = min

~x,~t2Rn,s2Rs�

nX

i=1

ti

s.t. kA~x� ~yk2 s

|xi| � ti, i = 1, . . . , n.

While this is a valid (though nonconvex) optimization problem, each of the final constraints

|xi| � ti is not convex:

|xi| � ti

ti�ti

xi

We therefore cannot linearize them to get an SOCP.

ee ai z ti

OR ai Eti

No way towrite this

in

a convex fan

i