Lecture 18: Model Predictive Control

Constrained Optimal Control Problem

minimize J =∞∑t=0

`(xt , ut)

s.t. xt+1 = f (xt , ut), ∀t; x0 is given

ut ∈ Ut , xt ∈ Xt , ∀t

• Time-invariant or varying dynamics f (assume f (0, 0) = 0)

• Ut , Xt : feasible sets of control & state at time t (assume 0 ∈ Ut , 0 ∈ Xt)

- Example: ui ∈ [ui , ui ] (box); {x |Gx ≤ g} (polytope)

• `(·, ·) ≥ 0: running cost (assume `(0, 0) = 0)

- Example: `(x , u) = xTQx + uTRu for Q � 0 and R � 0

• Denote J∗(x0) the optimal cost

2 / 19

Convex Constrained Optimal Control Problem

minimize J =∞∑t=0

`(xt , ut)

s.t. xt+1 = Axt + But , ∀t; x0 is given

ut ∈ Ut , xt ∈ Xt , ∀t

• Affine dynamics (could be time-varying and with additive perturbations)

• Both Ut and Xt are convex sets, ∀t• Running cost `(x , u) ≥ 0 is a convex function of x and u

• Convex optimization problem (∞-dimensional)

• Optimal cost J∗(x0) is a convex function of x0

• Generally, still very difficult to solve

3 / 19

Two Solution Methods

Dynamic Programming: find value function J∗(·) via Bellman eq.

J∗(x) = infu∈U ,Ax+Bu∈X

[`(x , u) + J∗(Ax + Bu)] , ∀x ∈ X

• Assume Ut ≡ U , Xt = X• Optimal control policy u∗(·) also obtained from Bellman equation

• Difficult to solve and represent J∗(·); requires infinite number of iterations

Greedy Control: minimize immediate cost by choosing control as

ut(xt) = arg minu∈U

{`(xt , u)

∣∣Axt + Bu ∈ X}

• Very easy to compute, but often performs poorly

• Example: xt+1 = 2xt + ut , `(xt , ut) = |xt |2 + |ut |2

4 / 19

Finite Horizon Approximation

minimize JN(x0) =N−1∑t=0

`(xt , ut)

s.t. xt+1 = Axt + But , t = 0, . . . ,N − 1; x0 is given, xN = 0

ut ∈ U , xt ∈ X , t = 0, . . . ,N − 1

• Solve over a finite horizon N, with an additional terminal constraint

• Apply obtained optimal control u0, . . . , uN−1, followed by 0, 0, . . .

• Resulting in a cost J∗N(x0) that is an approximation of J∗(x0)

• Online (specific) solution: optimal u∗t for a specific x0

• Offline (policy) solution: optimal state feedback policy u∗t (·)

5 / 19

Important Special Case

minimize JN(x0) =N−1∑t=0

(xTt Qxt + uTt Rut

)s.t. xt+1 = Axt + But , t = 0, . . . ,N − 1; x0 is given, xN = 0

Gut ≤ g0, Hxt ≤ h0, t = 0, . . . ,N − 1

• Linear dynamics, quadratic cost, polytopic state and control constraints

With u =[uT0 · · · uTN−1

]Tand x =

[xT1 · · · xTN

]T, the problem is a

linearly constrained quadratic programming (QP) problem :

minu,x

xTQx + uTRu

s.t. x = Eu + Fx0, Gu ≤ g0 · 1, Hx ≤ h0 · 1, xN = 0

• An instance of the more general muliparametric programming

Multi-Parametric Programming Matlab Toolbox6 / 19

Important Special Case (cont.)

Online solution for any specific x0 is very efficient

• e.g. Matlab command quadprog

Offline (policy) solution has a nice structure

• Feasible initial set X0: set of all x0 for which solution is feasible.

• There exists a polyhedral partition of X0, X0 = ∪iXi , s.t.

• J∗N(x) is continuous, convex, and piecewise quadratic

J∗N(x0) = xT0 Pix0 + 2pTi x0 + ri , ∀x0 ∈ Xi

• u∗(x) is continuous, and piecewise affine

u∗(x0) = Kix0 + ki , ∀x0 ∈ Xi

• If `(x , u) is affine, then both J∗N(·) and u∗N(·) are piecewise affine

“The explicit linear quadratic regulator for constrained systems”, A Bemporad, MMorari, V Dua, EN Pistikopoulos, Automatica, 38 (1), 3-20

7 / 19

Model Predictive Control (MPC)

Idea: (also called receding horizon control)

• at any time t, solve the problem over a prediction horizon

minimize JN(xt) =N−1∑k=0

`(xt+k|t , ut+k|t)

s.t. xt+k+1|t = f (xt+k|t , ut+k|t), k = 0, . . . ,N − 1

ut+k|t ∈ U , xt+k|t ∈ X , k = 0, . . . ,N − 1

xt|t = xt , xt+N|t = 0

xt+k+1|t and ut+k|t , 0 ≤ k ≤ N − 1, are planned states and controls

• Implement only the first control u∗t|t at time t

ut = u∗t|t(xt), xt+1 = f (xt , ut)

• t → t + 1 and repeat the above procedure

“Tutorial overview of model predictive control”, J.B. Rawlings, IEEE Control Systems, 2002.8 / 19

9 / 19

Explicit MPC

• Linear dynamics f (x , u) = Ax + Bu

• Quadratic (or linear) running cost `(x , u)

• Polyhedral constraint sets X , U

From previous discussions, we know

• At each t, solve a quadratic/linear program

• Optimal control u∗(·) at any time is piecewise affine

• Key is to determine which polyhedral partition xt belongs to

Warning:

• Complexity (number of partitions) could still be high

• Methods exist to reduce number of partitions with sacrifice in optimality

• Online solution computed in O(N(n + m)3) time using interior point method

10 / 19

Example (Boyd)

Random 3-state 2-input LTI system (A,B) with `(x , u) = ‖x‖2 + ‖u‖2 and

constraint ‖x‖∞ ≤ 1, ‖u‖∞ ≤ 1; x0 =[0.9 −0.9 0.9

]T.

costs vs. time horizon N

• Finite horizon approximation (dashed) becomes infeasible for N ≤ 9

• MPC performance V ∗N(x0) (solid) very close to V ∗(x0) for all N ≥ 1011 / 19

Stabilization by MPC

Assumption:

• `(x , u)→ 0 implies x → 0 and u → 0

• For the given x0, the N-horizon problem at t = 0 is feasible

Fact: the closed-loop solution ut and xt under the MPC satisfy

• Persistent feasibility: ut ∈ U , xt ∈ X , ∀t• Stability: ut → 0 and xt → 0 as t →∞

Proof: show that V ∗N (xt) is a Lyapunov function

• Let u∗0|0, . . . , u∗N−1|0, and x∗1|0, . . . , x

∗N|0 = 0 be the solutions at t = 0

• After applying u0 = u∗0|0, x1 = x∗1|0

• At time t = 1, u∗1|0, . . . , u∗N−1|0, 0, and x∗2|0, . . . , x

∗N|0 = 0, 0 are feasible

⇒ V ∗N (x1) ≤ V ∗N (x0)− `(x0, u0)

• V ∗N (xt) ≥ 0 non-increasing implies `(xt , ut)→ 0 as t →∞

12 / 19

Relaxed Terminal Constraint

Imposing xt+N|t = 0 may make the problem infeasible for short horizon N

Control invariant set C ⊂ X for the dynamics x+ = f (x , u) satisfies

∀x ∈ C , ∃u ∈ U s.t. f (x , u) ∈ C

• Equivalently, C ⊂ pre(C ), the 1-step backward reachable set of C

Fact: If we replace the terminal constraint xt+N|t = 0 in the MPCproblem by xt+N|t ∈ XN , where XN is a (maximum) control invariant set,then the MPC solution is persistently feasible if x0 ∈ X0, where

Xk = pre(Xk+1) ∩ X , k = N − 1, . . . , 0

• Note that XN ⊂ X0

13 / 19

Stability Condition

With the relaxed terminal condition xt+N|t ∈ XN , how to guaranteestability of MPC solution?

Idea: Add a terminal cost ρN(xt+N|t) in JN , where ρN(·) satisfies

• It is a locally positive definite (LPD) funciton on XN

• For any x ∈ XN , there exists u ∈ U such that

ρN(x)− ρN(f (x , u)) ≥ `(x , u)

Fact: with the above terminal cost, the MPC solution is stable, ∀x0 ∈ X0

Proof: Show that V ∗N(xt+1) ≤ V ∗N(xt)− `(xt , ut), ∀t, still holds

14 / 19

MPC with Uncertainty

Optimal control problem with uncertainty

minimize J =∞∑t=0

`(xt , ut ,wt)

s.t. xt+1 = f (xt , ut ,wt), ∀t; x0 is given

ut ∈ U , xt ∈ X , ∀t

• wt models perturbation at time t

• Minimize E[J] if wt is a stochastic process with known distributions

MPC can be adapted to solve the above problem

• Assume forecast of perturbations over the lookahead horizon is available

• Most MPC practical applications follow this formulation

15 / 19

MPC Solution of Switched LQR Problem

minimize J =∞∑t=0

(xTt Qσtxt + uTt Rσtut

)s.t. xt+1 = Aσtxt + Bσtut , t = 0, 1, . . .

x0 is given

• Continuous control ut and discrete control (mode) σt

• No input or state constraints

• Optimal cost J∗(x0) can be very complicated

MPC solution:

• Solve N-horizon value function J∗N(·) and optimal control policy u∗N(·)• We know J∗N(·) is piecewise quadratic and u∗N(·) is piecewise linear

• Adopt u∗N(·) as the static state-feedback control policy: ut = u∗N(xt)

• Cost of closed-loop solution approaches J∗ as N →∞

16 / 19

MPC in Switching Stabilization of SLSs

To stabilize SLS xt+1 = Aσtxt + Bσtut , at any time t solve the problem:

minut+k|t ,σt+k|t

(xt+N|t)TP(xt+N|t)

s.t. xt+k+1|t = Aσt+k|txt+k|t + Bσt+k|tut+k|t , k = 0, . . . ,N − 1

xt|t = xt

• Minimize a quadratic function of the state N steps later• MPC solution can be applied every N time steps• V (x) = xTPx becomes a quadratic periodic Lyapunov function

“Periodic stabilization of discrete-time switched linear systems,” D.-H. Lee and J.Hu, TAC, 2016.

17 / 19

Constrained Switched LQR Problem

minimize J =∞∑t=0

(xTt Qσtxt + uTt Rσtut

)s.t. xt+1 = Aσtxt + Bσtut , t = 0, 1, . . .

Gσtut ≤ g0,σt , Hσtxt ≤ h0,σt , t = 0, 1, . . .

x0 is given

• Polyhedral state and control constraints, possibly mode dependent

• Finite horizon optimal cost J∗N(·) continuous, piecewise quadratic,but in general not convex

• Optimal control u∗(·) is piecewise affine, but not continuous

“Predictive Control for Linear and Hybrid Systems”, F. Borrelli, A. Bemporad andM. Morari, 2013.

18 / 19

References

• Tutorial overview of model predictive control, J.B. Rawlings, IEEEControl Systems, 2002.

• MPC Matlab Toobox

• Predictive Control for Linear and Hybrid Systems, F. Borrelli, A.Bemporad and M. Morari, 2013.

• Multi-Parametric Matlab Toolbox

• Stabilizing model predictive control of hybrid systems, M. Lazar, W.Heemels, S. Weiland, and A. Bemporad, TAC 2006.

19 / 19