1

Model Predictive Control for Signal TemporalLogic Specifications

Vasumathi Raman1, Alexandre Donze2, Mehdi Maasoumy3,Richard M. Murray1, Alberto Sangiovanni-Vincentelli2 and Sanjit A. Seshia2

Abstract—We present a mathematical programming-basedmethod for model predictive control of cyber-physical systemssubject to signal temporal logic (STL) specifications. We describethe use of STL to specify a wide range of properties of thesesystems, including safety, response and bounded liveness. Forsynthesis, we encode STL specifications as mixed integer-linearconstraints on the system variables in the optimization problemat each step of a receding horizon control framework. We provecorrectness of our algorithms, and present experimental resultsfor controller synthesis for building energy and climate control.

Index Terms—formal synthesis, timed logics, model predictivecontrol, cyberphysical systems

I. INTRODUCTION

Controlling a cyber-physical system (CPS) involves han-dling complex interactions between computing componentsand their physical environment, and often necessitates hierar-chies of controllers. Typically at the highest level, a supervi-sory controller is responsible for making high-level decisions,while at the lowest level traditional control laws such as PIDcontrol are used. In general, the design of these differentcontrollers is done mostly in isolation at each level, andtheir combination is implemented ad hoc. As the complexityof these systems grows, reasoning about the correctness ofinteractions between the various layers of control becomesincreasingly challenging, begging automation.

Formal methods is the subfield of computer science con-cerned with verification and synthesis, i.e., automatic andrigorous design of digital systems. It provides mathematicalformalisms for specifying behaviors and algorithms for veri-fication and synthesis of a system against properties specifiedwithin these formalisms. Methods for synthesis of correct-by-construction discrete supervisory controllers have beendeveloped and successfully used for cyber-physical systems indomains including robotics [1] and aircraft power system de-sign [2]. However, for physical systems that require constraints

This work was supported in part by TerraSwarm, one of six centersof STARnet, a Semiconductor Research Corporation program sponsored byMARCO and DARPA.

1V. Raman and R. M. Murray are with the California Instituteof Technology, Pasadena, CA 91125, USA vasu@caltech.edu,murray@cds.caltech.edu

2A. Donze, A. Sangiovanni-Vincentelli and S. A. Seshiaare with the Department of Electrical Engineering andComputer Science, UC Berkeley, Berkeley, CA 94720, USAdonze@berkeley.edu, alberto@berkeley.edu,sseshia@eecs.berkeley.edu

3M. Maasoumy is with C3 Energy Inc. Redwood City, CA 94063, USAmehdi.maasoumy@c3energy.com

Manuscript received January 14, 2016.

not just on the order of events, but on the temporal distancebetween them, simulation and testing is still the method ofchoice for validating properties and establishing guarantees;the exact exhaustive verification or synthesis of such systemsis in general undecidable [3].

Model predictive control (MPC) or receding horizon control(RHC) is based on iterative, finite horizon optimization overa model of the plant, i.e. the system to be controlled. Atany given time t, the current plant state is observed, andan optimal control strategy computed for some finite timehorizon in the future, [t, t + H]. An online calculation isperformed to explore trajectories originating from the currentstate, and an optimal control strategy computed up to timet+H . To provide robustness with respect to modelling errors,only the first step of the computed optimal control strategy isimplemented. The plant state is then sampled again, and newcalculations are performed on a horizon of H starting fromthe new current state. While the global optimality of such areceding horizon approach is not ensured, it tends to do wellin practice: in addition to reducing computational complexity,it improves the system robustness with respect to exogenousdisturbances and modeling uncertainties [4]. Another reasonMPC is particularly attractive to industry is its ability to handleconstrained dynamical systems [5].

Signal temporal logic (STL) [6] was originally developed inorder to specify and monitor the expected behavior of physicalsystems, including temporal constraints between events. STLallows the specification of properties of dense-time, real-valued signals, and the automatic generation of monitors fortesting these properties on individual simulation traces. Ithas since been applied to the analysis of several types ofcontinuous and hybrid systems, including dynamical systemsand analog circuits, where the continuous variables representquantities like currents and voltages in a circuit. STL hasthe advantage of naturally admitting quantitative semanticswhich, in addition to the yes/no answer to the satisfactionquestion, provide a real number that grades the quality of thesatisfaction or violation. Such semantics have been defined fortimed logics, including metric temporal logic (MTL) [7] andSTL [8], to assess the robustness of systems to parameter ortiming variations.

In this paper, we solve the problem of control synthesis fromSTL specifications, using a receding horizon approach. Weallow the user to specify desired properties of the system usingan STL formula, and synthesize control such that the systemsatisfies that specification, while using a receding horizonapproach to ensure practicality and robustness. We do so by

arX

iv:1

703.

0956

3v1

[cs

.SY

] 2

8 M

ar 2

017

2

decomposing the STL specifications into a series of formulasover each time horizon, such that synthesizing a controller ful-filling the formula at each horizon results in satisfaction of theglobal specification. Recent work on optimal control synthesisof aircraft load management systems [9] represented STL-like specifications as time-dependent equality and inequalityconstraints, yielding a mixed integer linear program (MILP).The MILP was then solved in an MPC framework, yieldingan optimal control policy. However, the manual transformationof specifications into equality and inequality constraints iscumbersome and problem-specific. As a key contribution, thispaper presents two automatically-generated MILP encodingsfor such STL specifications.

Our main contribution is a pair of bounded model checking-style encodings [10] for STL specifications as MILP con-straints on a cyber-physical system. We show how these en-codings can be used to generate open-loop control signals thatsatisfy finite and infinite horizon STL properties and, more-over, to generate signals that maximize quantitative (robust)satisfaction. We provide a fragment of STL, denoted SNN-STL, such that, under reasonable assumptions on the systemdynamics, the problem of synthesizing an open-loop controlsequence such that the system satisfies a provided specifica-tion is a linear program (LP), and therefore polynomial-timesolvable. We also demonstrate how our MILP formulation ofthe STL synthesis problem can be used in an MPC frameworkto compute feasible and optimal controllers for cyber-physicalsystems under timed specifications. We present experimentalresults comparing both encodings, and two case studies: one ona thermal model of an heating, ventilation and air-conditioning(HVAC) system, and another in the context of regulationservices in a micro-grid. These case studies were previouslyreported in [11], [12]. We show how the MPC schemes inthese examples can be framed in terms of synthesis from anSTL specification, and present simulation results to illustratethe effectiveness of our methodology.

II. PRELIMINARIES

A. Systems

We consider a continuous-time system Σ of the form

xt = f(xt, ut, wt)

where xt ∈ X ⊆ (Rnc × {0, 1}nl) are the continuous andbinary/logical states, ut ∈ U ⊆ (Rmc×{0, 1}ml) are the (con-tinuous and logical) control inputs, wt ∈W ⊆ (Rec×{0, 1}el)are the external environment inputs (also referred to as “dis-turbances”). As xt includes binary components, the aboveODE may contain discontinuities corresponding to switches inthese components. In general, such hybrid systems are moreaccurately modeled using differential-algebraic equations, butwe do not dwell on this point since we approximate theirbehavior with a difference equation, as follows.

Given a sampling time ∆t > 0, we assume that Σ admits adiscrete-time approximation Σd of the form

x(tk+1) = fd(x(tk), u(tk), w(tk)) (1)

where for all k > 0, tk+1−tk = ∆t. A run of Σd is a sequence

ξ = (x0u0w0)(x1u1w1)(x2u2w2)...

where xk = x(tk) ∈ X is the state of the system atindex k, and for each k ∈ N, uk = u(tk) ∈ U , wk =w(tk) ∈ W and xk+1 = fd(xk, uk, wk). We assume thatgiven an initial state x0 ∈ X , a control input sequence uN =u0u1u2 . . . uN−1 ∈ UN , and a sequence of environment inputswN = w0w1w2 . . . wN−1 ∈WN , the resulting horizon-N runof a system modeled by equation (1), which we denote by

ξ(x0,uN ,wN ) = (x0u0w0)(x1u1w1)(x2u2w2)...(xNuNwN ),

is unique. In addition, we introduce a generic cost functionJ(ξ(x0,u,w)) that maps (infinite and finite) runs to R.

B. Signal Temporal Logic

We consider STL formulas defined recursively according tothe following grammar:

ϕ ::= πµ | ¬ψ | ϕ1 ∧ ϕ2 | �[a,b] ϕ | ϕ1 U [a,b] ϕ2,

where πµ is an atomic predicate X ×U×W → B whose truthvalue is determined by the sign of a function µ : X×U×W →R and ϕ1, ϕ2 are STL formulas. The fact that a run ξ(x0,u,w)satisfies an STL formula ϕ is denoted by ξ |= ϕ. Informally,ξ |= �[a,b] ϕ if ϕ holds at some time step between a and b, andξ |= ϕ U [a,b] ψ if ϕ holds at every time step before ψ holds,and ψ holds at some time step between a and b. Additionally,we define �[a,b] ϕ = ¬ �[a,b](¬ϕ), so that ξ |= �[a,b] ϕ if ϕholds at all times between a and b. Formally, the validity of aformula ϕ with respect to the run ξ is defined inductively asfollows

ξ |= ϕ ⇔ (ξ, t0) |= ϕ(ξ, tk) |= πµ ⇔ µ(xk, yk, uk, wk) > 0(ξ, tk) |= ¬ψ ⇔ ¬(ξ, tk) |= ψ)(ξ, tk) |= ϕ ∧ ψ ⇔ (ξ, tk) |= ϕ ∧ (ξ, tk) |= ψ(ξ, tk) |= �[a,b] ϕ ⇔ ∃tk′ ∈ [tk+a, tk+b], (ξ, tk′) |= ϕ(ξ, tk) |= ϕU [a,b] ψ ⇔ ∃tk′ ∈ [tk+a, tk+b] s.t. (ξ, tk′) |= ψ

∧∀tk′′ ∈ [tk, tk′ ], (ξ, tk′′) |= ϕ.

An STL formula ϕ is bounded-time if it contains nounbounded operators; the bound of ϕ is the maximum over thesums of all nested upper bounds on the temporal operators, andprovides a conservative maximum trajectory length requiredto decide its satisfiability. For example, for �[0,10] �[1,6] ϕ, atrajectory of length N such that tN ≥ 10+6 = 16 is sufficientto determine whether the formula is satisfiable.Remark 1. Here we have defined a semantics for STL overdiscrete-time signals, which is formally equivalent to thesimpler linear temporal logic (LTL), once time and predicatesare abstracted into steps and Boolean variables, respectively.There are several advantages of still using STL over LTL,though. First, STL allows us to explicitly use real time inour specifications instead of abstract integer indices, whichimproves the readability relative to the original system’s be-haviors. Second, although in the rest of this paper we focuson the control of the discrete-time system Σd, our goal is touse the resulting controller for the control of the continuous

3

system Σ. Hence the specifications should be independentfrom the sampling time ∆t. Finally, note that the relationshipbetween the continuous-time and discrete-time semantics ofSTL, depending on discretization error and sampling time, isbeyond the scope of this paper. The interested reader can referto [13] for further discussion on this topic.

C. Quantitative semantics for STL

Quantitative or robust semantics for STL are defined byproviding a real-valued function ρϕ of signal ξ and time tsuch that ρϕ(ξ, t) > 0 ⇒ (ξ, t) |= ϕ. We define one suchfunction recursively, as follows:

ρπµ

(ξ, tk) = µ(xk, yk, uk, wk)ρ¬ψ(ξ, tk) = −ρψ(ξ, tk)ρϕ1∧ϕ2(ξ, tk) = min(ρϕ1(ξ, tk), ρϕ2(ξ, tk))ρϕ1∨ϕ2(ξ, tk) = max(ρϕ1(ξ, tk), ρϕ2(ξ, tk))

ρ �[a,b] ψ(ξ, tk) = maxtk′∈[t+a,t+b] ρψ(ξ, tk′)

ρϕ1 U [a,b] ϕ2(ξ, tk) = maxtk′∈[t+a,t+b](min(ρϕ2(ξ, tk′),mintk′′∈[tk,tk′ ] ρ

ϕ1(ξ, tk′′))

Note that if µ(xk, yk, uk, wk) = 0, neither ρπµ

(ξ, tk) > 0nor ρπ

µ

(ξ, tk) > 0. Therefore, (ξ, t) |= ϕ 6⇒ ρϕ(ξ, t) > 0. Tosimplify notation, we denote ρπ

µ

by ρµ for the remainder ofthe paper. The robustness of satisfaction for an arbitrary STLformula is computed recursively from the above semantics bypropagating the values of the functions associated with eachoperand using min and max operators corresponding to thevarious STL operators. For example, the robust satisfaction ofπµ1 where µ1(x) = x−3 > 0 at time 0 is ρµ1(ξ, 0) = x0−3.The robust satisfaction of µ1 ∧ µ2 is the minimum of ρµ1

and ρµ2 . Temporal operators are treated as conjunctions anddisjunctions along the time axis: since we deal with discretetime, the robustness of satisfaction of ϕ = �[0,2.1] µ1 is

ρϕ(x, 0) = mintk∈[0,2.1] ρµ1(x, tk)

= min{x0 − 3, x1 − 3, . . . , xK − 3},

where 0 ≤ t0 < t1 < . . . < tK ≤ 2.1 < tK+1.The robustness score ρϕ(ξ, t) can be interpreted as how

much ξ satisfies ϕ. Its absolute value can be viewed as thesigned distance of ξ from the set of trajectories satisfying orviolating ϕ, in the space of projections with respect to thefunction µ that define the predicates of ϕ [7].

III. PROBLEM STATEMENT

We now formally state the STL control synthesis problemand its model predictive control formulation. Given an STLformula ϕ, a cost function of the form J(x0,u,w, ϕ) ∈ R, aninitial state x0 ∈ X , a horizon L and a reference disturbancesignal w ∈ WN , we formulate two problems: open-loop andclosed-loop synthesis. The two scenarios are depicted as blockdiagrams in Fig. 1 and Fig. 2.

Problem 1 (open-loop). Compute u∗ = u∗0u∗1 . . . u

∗N−1 where

u∗ = argminu∈UN

J(x0,u,w, ϕ)

s.t. ξ(x0,u,w) |= ϕ

Note that we assume that the state of the plant is fullyobservable, and the environment inputs are known in advance.

Problem 2 (closed-loop). Given a horizon 0 < L < N , forall 0 ≤ k ≤ N − L, compute u∗k = uL∗k , the first element ofthe sequence uL∗k = uL∗k uL∗k+1 . . . u

L∗k+L−1 satisfying

uL∗k = argminuLK∈UL

J(xk,uLk ,wk, ϕ)

s.t. ξ(xk,uLk ,wk) |= ϕ

The closed-loop formulation corresponds to a model pre-dictive control scheme, where the reference disturbance canchange at each iteration k.

In Sections IV and V, we present both an open-loop solutionto Problem 1, and a solution to Problem 2 for a large classof STL formulas. In the absence of an objective functionJ on runs of the system, we maximize the robustness ofthe generated runs with respect to ϕ. A key component ofour solution is encoding the STL specifications as MILPconstraints, which can be combined with MILP constraintsrepresenting the system dynamics to efficiently solve theresulting state-constrained optimization problem.

IV. OPEN-LOOP CONTROLLER SYNTHESIS

To solve Problem 1, we extend the bounded model checkingencoding of [14] from finite, discrete systems to dynamicalsystems using mixed-integer programming instead of SAT. Ourpresentation and notation below follow that of [15]. For open-loop controller synthesis, we will search for a trajectory oflength N that satisfies ϕ. To admit STL formulas describinginfinite runs, we parametrize an infinite sequence of statesusing a finite sequence with a loop. Imposing this lasso-shaped structure renders our synthesis procedure conservativefor infinite-state systems, in the sense that a solution may existthat is not found when imposing such a structure. For finite-state systems, the lasso shape is without loss of generality butwe must still find an appropriate trajectory length N .

ϕ,Σd, J

Synthesisu∗ =

u∗0u∗1 . . . u

∗N−1

Plant Σ x,y

x0,w0, . . . , wN−1

x0, w

Fig. 1. Open-loop problem formulation: given an STL formula φ, the discrete-time plant model Σd, the cost function J , the goal is to generate a sequence ofcontrol inputs u∗ over a horizon of N time steps. The problem is additionallyparametrized by the the initial state x0 and disturbance vector w.

The encoding of Problem 1 as an MILP consists of systemconstraints, loop constraints and STL constraints, as definedbelow.

4

A. Constraints on system evolution

The first component of the set of constraints is provided bythe system model. Our approach applies to any system thatyields to a MILP formulation for model predictive controlover horizon N . The system constraints encode valid finite(horizon-N ) trajectories for a system with form (1) – theseconstraints hold if and only if the trajectory x(x0,uN ) satisfies(1) for t = 0, 1, ..., N . Note that this is quite general, andaccommodates any system for which the resulting constraintsand objectives form a mixed integer-linear program. An ex-ample is the smart grid regulation control system presented in[16]. Other useful examples include mixed logical dynamicalsystems such as those presented in [17]. Other cost functionsand system dynamics can also be included by using appropri-ate solvers.

B. Loop constraints for trajectory parametrization

As described above, to synthesize open-loop control forunbounded (infinite-horizon) specifications, we parametrizethe trajectory as a lasso, i.e. constrain it to contain a loop. Thisloop encoding is again inspired by the basic idea of boundedmodel checking [10], which is to consider only a finite prefixof a path when looking for a solution to an existential modelchecking problem. A crucial observation is that although theconsidered path prefix is finite, it can still represent an infinitepath if there is a loop back from the last state to any of theprevious states.

To enforce the existence of a loop in the finite systemtrajectory, we introduce N binary variables l1, ..., lN , whichdetermine where the loop forms. These are constrained suchthat only one can be high at a time, and if lj = 1, thenxj−1 = xN . The following constraints enforce these require-ments:•

∑Nj=1 lj = 1

• xN ≤ xj−1 +Mj(1− lj), j = 1, ..., N ,• xN ≥ xj−1 +Mj(1− lj), j = 1, ..., N ,

where Mj are sufficiently large positive numbers, picked basedon X .

C. Boolean encoding of STL constraints

Given a formula ϕ, we introduce a variable zϕt , whose valueis tied to a set of mixed integer linear constraints required forthe satisfaction of ϕ at position t in the state sequence ofhorizon N . In other words, zϕt has an associated set of MILPconstraints such that zϕt = 1 if and only if ϕ holds at positiont. We recursively generate the MILP constraints correspondingto zϕ0 – the value of this variable determines whether a formulaϕ holds in the initial state.

1) Predicates: The predicates are represented by con-straints on system state variables. For each predicate µ ∈ P ,we introduce binary variables zµt ∈ {0, 1} for times t =0, 1, ..., N . The following constraints enforce that zµt = 1 ifand only if µ(xt) > 0:

µ(xt) ≤ Mtzµt − εt

−µ(xt) ≤ Mt(1− zµt )− εt

where Mt are sufficiently large positive numbers, and εtare sufficiently small positive numbers that serve to boundµ(xt) away from 0. This encoding restricts the set of STLformulas that can be encoded using our approach to thoseover linear predicates, but admits arbitrary STL formulas oversuch predicates.

2) Boolean operations on MILP variables: As describedin Section IV-C1, each predicate µ has an associated binaryvariable zµt which equals 1 if µ holds at time t, and 0otherwise. In fact, by the recursive definition of our MILPconstraints on STL formulas, each operand ϕ in a Booleanoperation has a corresponding variable zϕt which is 1 if ϕ holdsat t and 0 otherwise. Here we define Boolean operations onthese variables: these are the building blocks of our recursiveencoding. The definitions in this subsection are consistent withthose in [15].

Logical operations on variables zψt ∈ [0, 1] are defined asfollows:Negation: zψt = ¬zϕt zψt = 1− zϕt

Conjunction: zψt = ∧mi=1zϕiti

zψt ≤ zϕiti , i = 1, ...,m,

zψt ≥ 1−m+∑mi=1 z

ϕiti

Disjunction: zψt = ∨mi=1zϕiti

zψt ≥ zϕiti , i = 1, ...,m,

zψt ≤∑mi=1 z

ϕiti

Given a formula ψ containing a Boolean operation, we addnew continuous variables zψt ∈ [0, 1], and set zψt = ¬zµt ,zψt = ∧mi=1z

ϕiti , and zψt = ∨mi=1z

ϕiti for ψ = ¬µ, ψ = ∧mi=1ϕi

and ψ = ∨mi=1ϕi, respectively. These constraints enforce thatzψt = 1 if ψ holds at time t and zψt = 0 otherwise.

3) Temporal constraints: We first present encodings forthe � and � operators. We will use these encodings to definethe encoding for the U[a,b] operator.

Always: ψ = �[a,b] ϕ

Let aNt = min(t+ a,N) and bNt = min(t+ b,N)

Define zψt = ∧bNt

i=aNtzϕi ∧ (

∨Nj=1 lj ∧

∧bNji=aNj

zϕi )

The logical operation ∧ on the variables zϕi here is as definedin Section IV-C2. Intuitively, this encoding enforces that theformula ϕ is satisfied at every time step on the interval [a, b]relative to time step t.

Eventually: ψ = �[a,b] ϕ

Define zψt = ∨bNt

i=aNtzϕi ∧ (

∨Nj=1 lj ∧

∨bNji=aNj

zϕi ) This encodingenforces that the formula ϕ is satisfied at some time step onthe interval [a, b] relative to time step t.

Until: ψ = ϕ1 U[a,b] ϕ2

The bounded until operator U[a,b] can be defined in terms ofthe unbounded U (inherited from LTL) as follows [18]:

ϕ1 U[a,b] ϕ2 = �[0,a] ϕ1 ∧ �[a,b] ϕ2 ∧ �[a,a](ϕ1 U ϕ2)

We will use the encoding of the unbounded U from[10]. When encoding over infinite trajectories, this requires

5

an auxiliary encoding that prevents the pitfalls of circularreasoning on the finite parametrization of the infinitesequences. The interested reader is referred to [10] forthe details of the encoding. The auxiliary encoding of theunbounded until is〈〈ϕ1 U ϕ2 〉〉t ={

zϕ2

t ∨ (zϕ1

t ∧ 〈〈ϕ1 U ϕ2 〉〉t+1), t = 1, ..., N − 1

zϕ2

N .

With this definition in place, we define

zϕ1 U ϕ2

t = zϕ2

t ∨ (zϕ1

t ∧ zϕ1 U ϕ2

t+1 )

for t = 1, ..., N − 1, and

zϕ1 U ϕ2

N = zϕ2

N ∨ (zϕ1

N ∧ (

N∨j=1

(lj ∧ 〈〈ϕ1 U ϕ2 〉〉j))).

Given this encoding of the unbounded until and the encodingsof �[a,b] and �[a,b] above, we can encode

zϕ1 U[a,b] ϕ2

t = z�[0,a] ϕ1

t ∧ z �[a,b] ϕ2

t ∧ z �[a,a](ϕ1 U ϕ2)

t .

By induction on the structure of STL formulas ϕ, zϕt = 1if and only if ϕ holds on the system at time t. With thismotivation, given a specification ϕ, we add a final constraint:

zϕ0 = 1. (2)

For a bounded horizon formula, the union of the STLconstraints, loop constraints and system constraints gives theMILP encoding of Problem 1; this enables checking feasibilityof this set of constraints and finding a solution using an MILPsolver. Given an objective function on runs of the system, thisapproach also enables finding the optimal open-loop trajectorythat satisfies the STL specification. Algorithm 1 reviews theprocedure for solving Problem 1.

Algorithm 1 Algorithm for Problem 11: procedure OPEN LOOP(f, x0,w, N, ϕ, J)2: LOOP CONSTRAINTS ← Sec. IV-B3: SYSTEM CONSTRAINTS ← Sec. IV-A4: STL CONSTRAINTS ← Sec. IV-C2 OR Sec. IV-D5:

u∗ ← argminu∈UN

J(x0,u,w, ϕ)

s.t. LOOP CONSTRAINTSSYSTEM CONSTRAINTSSTL CONSTRAINTS

Return u∗

6: end procedure

D. Quantitative Encoding

The robustness of satisfaction of the STL specification, asdefined in II-C, provides a natural objective for the MILPdefined in Section IV-C, either in the absence of, or asa complement to domain-specific objectives on runs of thesystem. The robustness can be computed recursively on thestructure of the formula in conjunction with the generation of

constraints. Moreover, since max and min operations can beexpressed in an MILP formulation using additional binary vari-ables, this does not add complexity to the encoding, althoughthe additional variables do make it more computationallyexpensive in practice.

In this section, we sketch the MILP encoding of thepredicates and Boolean operators using the quantitative se-mantics; the encoding of the temporal operators builds onthese encodings, as in Section IV-C. Given a formula ϕ,we introduce a variable rϕt , and an associated set of MILPconstraints such that rϕt > 0 if and only if ϕ holds at positiont. We recursively generate the MILP constraints, such thatrϕ0 determines whether a formula ϕ holds in the initial state.Additionally, we enforce rϕt = ρϕ(x, t).

For each predicate µ ∈ P , we now introduce variables rµtfor time indices t = 0, 1, ..., N , and set rµt = µ(xt). To definerψt , where ψ is a Boolean formula, we inductively assume thateach operand ϕ has a corresponding variable rϕt = ρϕ(x, t).Then the Boolean operations are defined as:

Negation: rψt = ¬rφt rψt = −rφtConjunction: rψt = ∧mi=1r

ϕiti

m∑i=1

pϕiti = 1 (3)

rψt ≤ rϕiti , i = 1, ...,m (4)

rϕiti − (1− pϕiti )M ≤ rψt ≤ rϕiti +M(1− pϕiti ) (5)

where we introduce new binary variables pϕiti for i = 1, ...,m,and M is a sufficiently large positive number. Then equation(3) enforces that there is one and only one j ∈ {1, ...,m}such that pϕjti = 1, equation (4) ensures that rψt is smallerthan all rϕiti , and equation (5) enforces that rψt = r

ϕjtj if and

only if pϕjtj = 1. Together, these constraints enforce that rψt =mini(r

ϕiti ).

Disjunction: ψ = ∨mi=1rϕiti is encoded similarly to conjunc-

tion, replacing (4) with rψt ≥ rϕiti , i = 1, ...,m. Using a similar

reasoning to that above, this enforces rψt = maxi(rϕiti ).

The encoding for bounded temporal operators is defined asin Section IV-C; robustness for the unbounded until is definedusing sup and inf instead of max and min, but these areequivalent on our finite trajectory representation with discretetime. By induction on the structure of STL formulas ϕ, thisconstruction yields rϕt > 0 if and only if ϕ is satisfied at timet. Therefore, we can replace the constraints over zϕt in SectionIV-C by these constraints that compute the value of rϕt , andinstead of (2), add the constraint rϕ0 > 0.

Since we consider only the discrete time semantics ofSTL in this work, the Boolean encoding in Section IV-Ccould be achieved by converting each formula to LTL, andusing existing encodings such as that in [15]. However, therobustness-based encoding we presented in this section has nonatural analog for LTL. The advantage of this encoding is thatit allows us to maximize the value of rϕ0 , obtaining a trajectorythat maximizes robustness of satisfaction. Additionally, anencoding based on robustness has the advantage of allowingthe STL constraints to be softened or hardened as necessary.

6

For example, if the original problem is infeasible, we canallow ρϕ0 > −ε for some ε > 0, thereby easily modifyingthe problem to allow a limited violation of the STL property.

The disadvantage is that it is more expensive to compute,due the the additional binary variables introduced during eachBoolean operation. Additionally, including robustness as anobjective makes the cost function inherently non-convex, withpotentially many local minima, and harder to optimize. On theother hand, the robustness constraints are more easily relaxed,allowing us to use a simpler cost function, which can makethe problem more tractable.

E. ComplexityIn general, our synthesis algorithm has the same com-

plexity as MILPs, which are NP-hard, hence computationallychallenging when the dimensions of the problem grow. Itis nevertheless appropriate to characterize the computationalcosts of our encoding and approach in terms of the numberof variables and constraints in the resulting MILP. In practice,one measure of problem size is the number of binary variablesrequired to indicate the satisfaction of the predicates µ. Thisdepends directly on the number of predicates used in the STLformula ϕ.

For the Boolean encoding, if P is the set of predicatesused in the formula, then O(N · |P |) binary variables areintroduced. In addition, continuous variables are introducedduring the MILP encoding of the STL formula. The numberof continuous variables used is O(N · |ϕ|), where |ϕ| is thelength (i.e. the number of operators) of the formula.

For the robustness-based encoding, O(N · |P |) continuousvariables are introduced (one per predicate per time step).In addition, binary variables are introduced during the MILPencoding of each operator in the STL formula. The numberof binary variables used is thus O(N · |ϕ|), where |ϕ| is thenumber of operators of the formula.

Our synthesis algorithm also has polynomial runtime for thefollowing fragment of STL.

Definition 1 (SNN-STL). Safe Negation-Normal STL (SNN-STL) is the fragment of STL generated by the recursivegrammar

ϕ ::= πµ | ¬πµ | ϕ1 ∧ ϕ2 | �[a,b] ϕ

SNN-STL has the following properties:• All negations appear only on atomic propositions (pushed

down to the leaf nodes of the formula abstract syntaxtree).

• The only temporal operators are � (with unbounded andbounded intervals).

• Only conjunctions are allowed, no disjunctions.Such specifications are expressive enough to enforce, e.g.,

safety specifications in environments where the system stateis confined to a conjunction of polyhedra.

LetOPEN LOOP NO STL(f, x0, N, ϕ, J)

denote the procedure that is identical to Algorithm 1, ex-cept that the optimization problem in Step 5 is solved withSTL CONSTRAINTS = ∅.

u∗ =u∗0u

∗1 . . . u

∗N−1

Synthesis

ϕ,Σd, J

Controller

Plant Σ

x0

w0

w0w1 . . . wN−1

u∗0 x1

x0

Fig. 2. Closed-loop (MPC) problem formulation. As in the open loop scenario,a sequence of control inputs is synthesized from the specifications, dynamicsand cost function. However, at each time step, only the first computed inputis used by the plant.

Theorem 1 (Polynomial-time Synthesis for SNN-STL). Sup-pose that ϕ is in SNN-LTL and has linear predicates.Then if OPEN LOOP NO STL(f, x0,w, N, ϕ, J) is convex, sois OPEN LOOP(f, x0, N, ϕ, J).

Proof. The proof proceeds by induction on the structureof the formula, showing that the constraints added bySTL CONSTRAINTS for each operator restrict the solution toa convex set. First note that since the predicates πµ are linear,negation of predicates preserves convexity, since ¬πµ is alsolinear. Because the intersection of convex sets is convex, theconjunction of a set of convex constraints is also convex.Finally, since the � operator is implemented in terms ofconjunctions, the constraints imposed by � also preserveconvexity of the resulting optimization problem.

Informally, Theorem 1 states that our encoding of SNN-STLconstraints into an MILP preserves convexity in the resultingoptimization problem. The resulting optimization problem istherefore encodable as an LP, i.e. without the use of integervariables.

Corollary 1. Algorithm 1 is polynomial-time for SNN-STLspecifications ϕ.

V. MODEL PREDICTIVE CONTROL SYNTHESIS

In this section, we will describe a solution to Problem 2by adding STL constraints to an MPC problem formulation.At each step t of the MPC computation, we will search fora finite trajectory of fixed horizon length H , such that theaccumulated trajectory satisfies ϕ.

A. Synthesis for bounded-time STL formulas

The length of the horizon H is chosen to be at least thebound of formula ϕ. At time step 0, we will synthesize controluH,0 using the open-loop formulation in Section IV, including

7

the STL constraints on the length-H trajectory, but without theloop constraints. We will then execute only the first time stepuH,00 . At the next step of the MPC, we will solve for uH,1,while constraining the previous values of x0, u0 in the MILP,and the STL constraints on the trajectory up to time H . In thismanner, we will keep track of the history of states in order toensure that the formula is satisfied over the length-H prefixof the trajectory, while solving for uH,t at every time step t.

B. Extension to unbounded formulas

For certain types of unbounded formulas, we can stitchtogether trajectories of length H using a receding horizonapproach, to produce an infinite computation that satisfies theSTL formula. An example of this is safety properties, i.e.ϕ = �(ϕMPC) for bounded STL formulas ϕMPC . For suchformulas, at each step of the MPC computation, we will searchfor a finite trajectory of horizon length H (determined fromϕMPC as above) that satisfies ϕMPC .

We now describe this approach in more detail. At each stept of the receding horizon control computation, we will employthe open-loop approach in Section IV to find a finite trajectoryof fixed horizon length H , such that the trajectory accumulatedover time satisfies ϕ. Given a specification ϕ = �ϕMPC ,where ϕMPC is a bounded-time formula with bound H . Inthis case, we can stitch together trajectories of length Husing a receding horizon approach to produce an infinitecomputation that satisfies the STL formula. At each step of thereceding horizon computation, we search for a finite trajectoryof horizon length 2H , keeping track of the past values androbustness constraints necessary to determine satisfaction ofϕ at every time step in the trajectory. Note that we omit theloop constraints in this approach, because at each step wesearch for a finite trajectory, rather than an infinite trajectorywith a finite parametrization.

First we define a procedure

OPEN LOOP∗(f, x0,w, N,�ϕMPC , J,PH ,utold)

that takes additional inputs P = {P0, P1, ..., PH−1} andutold = u0, u1, ..., ut−1, and is identical to Algorithm 1, exceptthat the optimization problem posed in Step 5 is solved withoutthe loop constraints, and with the added constraints:

ρϕ(f(x0,u,w), i) > Pi ∀i ∈ [0, H − 1]u[i...t] = utold

We then define a receding horizon control procedure as inAlgorithm 2. At each step, we are optimizing over a horizonof 2H . We assume available a method PREDICT W(t) forpredicting the sequence of 2H environment inputs starting attime step t.

Algorithm 2 has two phases, a transient phase (Lines 4-10) and a stationary phase (Lines 11-14). The transient phaseapplies until an initial control sequence of length H has beencomputed, and the stationary phase follows. In the transientphase, the number of stored previous inputs (utold) as well asthe number of time steps at which formula ϕMPC is enforced(i.e. time steps for which Pi = 0) grows by one at eachiteration, until they both attain a maximum of H at iteration H .

Algorithm 2 MPC Algorithm for Problem 21: procedure MPC(f, x0, φ = �ϕMPC , J)2: Let M be a large positive constant.3: Let H be the bound of ϕMPC .4: Set P0 = 0 and Pi = −M ∀0 < i ≤ H .5: wt ← PREDICT W(0).6: Compute u0 = u00, u

01, ...., u

02H−1 as:

u0 ← OPEN LOOP∗(f, x0,w0, 2H,�[0,H] ϕMPC , J,P

H , ∅)

7: for t=1; t¡=H;t=t+1 do8: Set utold = u00, u

11, u

22, ..., u

t−1t−1.

9: Set Pi = 0 for 0 ≤ i ≤ t, Pi = −M ∀t < i ≤ H .10: wt ← PREDICT W(t).11: Compute ut = ut0, u

t1, ...., u

t2H−1 as:

ut ← OPEN LOOP∗(f, xt,wt, 2H,�[0,H] ϕMPC , J,P

H ,utold)

12: end for13: while True do14: Set utold = ut−11 , ut−12 , ut−13 , ..., ut−1t .15: Set Pi = 0 for 0 ≤ i ≤ H .16: wt ← PREDICT W(t).

ut ← OPEN LOOP∗(f, xt,wt, 2H,�[0,H] ϕMPC , J,P

H ,utold)

17: end while18: end procedure

Every following iteration uses a window of size H for storedprevious inputs, and sets all Pi = 0. The size-H windowof previously-computed inputs advances forward one step intime at each iteration after step H . In this manner, we keepa record of the previously computed inputs required to ensuresatisfaction of ϕMPC up to H time steps in the past.

We now show that if Algorithm 2 does not terminate,then the resulting infinite sequence of control inputs enforcessatisfaction of the specification φ = �ϕMPC .

Theorem 2. Let φ = �ϕMPC , and assume that u∗ isan infinite sequence of control inputs generated by settingu∗[t] = ut0, where ut = ut0u

t1...u

t2H−1 is the control input

sequence of length 2H generated by Algorithm 2 at time t.Then f(x0,u

∗,w) |= ϕ.

Proof. Since H is the bound of ϕMPC , the satisfaction ofϕMPC at time t is established by the control inputs u∗[t :t+H − 1]. At time t+H ,

ut+Hold = ut+H0 , ut+H1 , ut+H2 , ..., ut+Ht+H−1= ut+H−11 , ut+H−12 , ut+H−13 , ..., ut+H−1t+H

= utt, ut+1t+1, u

t+2t+2, ..., u

t+H−1t+H−1

= u∗[t : t+H − 1],

and so all the inputs required to determine satisfaction of ϕat time t have been fixed. Moreover, if ut+H is successfullycomputed, then by the correctness of Algorithm 1, ut+Hold hasthe property that f(xt,ut+Hold ,wH) |= ϕMPC . Since u∗[t : t+H−1] = ut+Hold , we see that f(xt,u∗[t : t+h],wH) |= ϕMPC .

It follows that f(x0,u∗,w) |= ϕMPC .

We have therefore shown how a control input can be

8

synthesized for infinite sequences satisfying ϕ, by repeatedlysynthesizing control for sequences of length 2H . A similarapproach applies for formulas �ϕMPC and ϕMPC U ψMPC ,where ϕMPC , ψMPC are bounded-time.

Note that we assumed that PREDICT W(t) returns an exactprediction of the disturbance signal over the next 2H timesteps. The correctness of our approach relies on this assump-tion. An interesting direction of future work is to relax thisrequirement, demanding only an uncertain prediction of thedisturbance signal.

Remark 2. The control objective for MPC is usually to steerthe state to the origin or to an equilibrium state. Questions thatarise include those of ensuring feasibility at each time step,closed-loop stability and near-optimal performance [19]. Thereis a mature theory of stability for MPC, where the essentialingredients are terminal costs, terminal constraint sets, andlocal stabilizing controller that ensure closed-loop stability [5].

In this work, our control objective is not closed-loop sta-bility, but satisfaction of an STL formula. We achieve this, asdetailed above, through choice of a sufficiently large predictionhorizon H . This can be compared with the manner in whichautomatic satisfaction of a terminal constraint is sometimesattained by prior choice of a sufficiently large horizon.

VI. EXPERIMENTAL COMPARISON OF ENCODINGS

We implemented the Boolean and robust encodings usingthe tools Breach [20] and YALMIP [21], and now presentresults obtained with the following formulas:

• ϕ1 = �[0,0.1] x(1)t > 0.1

• ϕ2 = �[0,0.1](x(1)t > 0.1) ∧�[0,0.1](x

(2)t < −0.5)

• ϕ3 = �[0,0.5] �[0,0.1](x(1)t > 0.1)

• ϕ4 = �[0,0.2](x(1)t > 0.1 ∧ ( �[0,0.1](x

(2)t > 0.1)

∧ �[0,0.1](x(3)t > 0.1)))

In this study, we used the trivial system x = u, where x

is a 3-dimensional signal (i.e. xt = x(1)t x

(2)t x

(3)t ), so that

no constraint is generated for the system dynamics, and thecost function J(x,u) =

∑Nk=1 ‖utk‖1. Note that the output

of this procedure for a formula ϕ is a signal of minimalnorm which satisfies ϕ when using the Boolean encoding andwhich satisfies ϕ with a specified robustness ρϕ(x) = 0.1for the robust encoding. For each formula we computed theBoolean and robust encodings for an horizon N = 30 andsampling time τ = 0.025s and report the number of constraintsgenerated by each encoding, the time to create the resultingMILP with YALMIP and the time to solve it using the solverGurobi.1 All experiments were run on a laptop with an IntelCore i7 2.3 GHz processor and 16 GB of memory.

A first observation is that for both encodings, most of thetime is spent creating the MILP, while solving it is donein a fraction of a second. Also, while the robust encodinggenerates 3 to 5 times more constraints, the computationaltime to create and solve the corresponding MILPs is hardlytwice more. The exception is solving the MILP for ϕ4, whichtakes significantly more time for the robust encoding than for

1http://www.gurobi.com/

Formula #constraints YALMIP Time (s) Solver time (s)B R B R B R

ϕ1 154 488 1.71 2.04 0.0070 0.0085ϕ2 364 897 1.94 2.69 0.0115 0.0229ϕ3 244 1282 1.84 3.15 0.0064 0.1356ϕ4 574 1330 2.29 3.37 0.2167 238.6

TABLE IBOOLEAN (B) VS ROBUST (R) ENCODINGS. YALMIP TIME REPRESENTSTHE TIME TAKEN BY THE TOOL YALMIP IN ORDER TO GENERATE THE

MILP AND SOLVER TIME IS THE TIME TAKEN BY THE SOLVER GUROBI TOACTUALLY SOLVE IT.

the Boolean encoding. The reason is hard to pinpoint withouta more thorough investigation, but we can already note thatsolving a MILP is NP-hard, and while solvers use sophisticatedheuristics to mitigate this complexity, instances for which theseheuristics fail are bound to appear.

VII. CASE STUDY: BUILDING CLIMATE CONTROL

A. Mathematical Model of a Building

Next we consider the problem of controlling building indoorclimate, using the model proposed by Maasoumy et al [22]. Inthis section we present a summary of the building’s thermalmodel.

As shown in Fig. 3, the building is modeled as a resistor-capacitor circuit with n nodes, m of which are rooms andthe remaining n−m are walls. We denote the temperature ofroom ri by Tri . The wall and temperature of the wall betweenrooms i and j are denoted by wi,j and Twi,j , respectively.The temperature of wall wi,j and room ri are governed by thefollowing equations:

Cwi,jdTwi,jdt

=∑

k∈Nwi,j

Trk − Twi,jRi,jk

+ ri,jαi,jAwi,jQradi,j

(6)

CridTridt

=∑k∈Nri

Tk − TriRi,ki

+ mrica(Tsi − Tri)+

wiτwiAwiniQradi + Qinti , (7)

where Cwi,j , αi,j and Awi,j are heat capacity, a radiative heatabsorption coefficient, and the area of wi,j , respectively. Ri,jkis the total thermal resistance between the centerline of wall(i, j) and the side of the wall on which node k is located.Qradi,j is the radiative heat flux density on wi,j . Nwi,j is theset of all neighboring nodes to wi,j . ri,j is a wall identifier,which equals 0 for internal walls and 1 for peripheral walls,where either i or j is the outside node. Tri , C

ri and mri are

the temperature, heat capacity and air mass flow into room i,respectively. ca is the specific heat capacity of air, and Tsi isthe temperature of the supply air to room i. wi is a windowidentifier, which equals 0 if none of the walls surroundingroom i have windows, and 1 if at least one of them does. τwiis the transmissivity of the glass of window i, Awini is thetotal area of the windows on walls surrounding room i, Qradiis the radiative heat flux density per unit area radiated to roomi, and Qinti is the internal heat generation in room i. Nri is

9

Fig. 3. Resistor-capacitor representation of a typical room with a window.

the set of neighboring room nodes for room i. Further detailson this thermal model can be found in [22].

The heat transfer equations for each wall and room yieldthe system dynamics:

xt = f(xt, ut, wt).

Here xt ∈ Rn is the state vector representing the temperatureof the nodes in the thermal network, and ut ∈ Rlm is theinput vector representing the air mass flow rate and dischargeair temperature of conditioned air into each thermal zone (withl being the number of inputs to each thermal zone, e.g. twofor air mass flow and supply air temperature). The HVACsystem of the building considered for this study operates witha constant supply air temperature, while air mass flow is thetime varying control input. Hence, in the following simulationswe consider supply air temperature constant and treat air massflow as the control signal. Vector wt stores the estimateddisturbance values, aggregating various unmodelled dynamicssuch as Tout, Qint and Qrad, and can be estimated usinghistorical data [23]. yt ∈ Rm is the output vector, representingthe temperature of the thermal zones. The building modelwas trained using historical data, and the result of the systemidentification is shown in Fig. 4.

B. MPC for Building Climate Control

We consider the problem of controlling the above building’sHVAC system using an MPC scheme. We adopt the MPCformulation proposed by Maasoumy et al. [24], with theobjective of minimizing the total energy cost (in dollar value).τ and H denote the length of each time slot and the predictionhorizon (in number of time slots) of the MPC, respectively.Assume that the system dynamics are also discretized with

12:00 am 6:00 am 12:00 pm 6:00 pm 12:00 am18

19

20

21

22

23

Ro

om

te

mp

era

ture

[oC

]

12:00 am 6:00 am 12:00 pm 6:00 pm 12:00 am−1

0

1

2

3

Dis

turb

an

ce

[oC

/hr]

Time of day [hr]

Simulation

Measurement

Fig. 4. Simulated temperature, measured temperature and unmodelled dy-namics of a thermal zone in Bancroft library on UC Berkeley campus.

a sampling time of τ . Here we consider τ = 0.5 hr andH = 24. At each time t, the predictive controller solves anoptimal control problem to compute ~ut = [ut, . . . , ut+H−1],and minimizes the cumulative norm of ut:

∑H−1k=0 ‖ut+k‖. We

assume known an occupancy function occt which is equal to1 when the room is occupied and to 0 otherwise. The purposeof the MPC is to maintain a comfort temperature given byT comf whenever the room is occupied while minimizing thecost of heating. This problem can be expressed as follows:

min~ut

H−1∑k=0

‖ut+k‖ s.t.

xt+k+1 = f(xt+k, ut+k, wt+k),

xt |= ϕ with ϕ = �[0,H]((occt > 0)⇒ (Tt > T comft )

ut+k ∈ Ut+k, k = 0, ...,H − 1

The STL formula was encoded using the robust MILP en-coding and results are presented in Fig. 5. Again we observedthat creating the MILP structure was longer than solving aninstance of it (4.1s versus 0.15s). However, by using a properparametrization of the problem in YALMIP, the creation of theMILP structure can be done once offline and reused online foreach step of the MPC, which makes the approach promisingand potentially applicable even for real-time applications.

VIII. CASE STUDY II: REGULATION CONTROL FORSMART GRID

A. Mathematical Model

The second case study we consider is the n-areas smartgrid model presented in [16] and depicted in Fig. 6. Theinterconnection of power system components, including agovernor, turbine and generator in each area is shown in theblock diagram in Fig. 7. In the diagram, δPC is a controlinput which acts against an increase or decrease in powerdemand to regulate the system frequency ω, and δPD denotesfluctuations in power demand, modeled as an exogenous input(disturbance). Under steady state, we have: ω = ωo andPM = PG = P oM , where ωo, V ot , and P oM are the nominalvalues for rated frequency, terminal voltage and mechanicalpower input.

10

6 12 18 24

66

67

68

69

70

Time of day (Hours)

TemperatureComfort Limit

6 12 18 240

0.5

1

Occupancy

6 12 18 240

20

40

60

Time of day (Hours)

Air Flow

Fig. 5. Room temperature control with constraints based on occupancy,expressed in STL.

Fig. 6. Power system grid with n areas. The dynamics in each area is depictedin Fig.7.

Next, we present the mathematical model for one area i(note that superscripts refer to the control area, and subscriptsindex states in each area).

dxi1dt

=(−Dixi1 + δP iM − δP

iD − δP

itie + δP ianc)

M, (8a)

dxi2dt

=(xi3 − x

i2)

T i7,

dxi3dt

=(xi4 − x

i3)

T i6,

dxi4dt

=(xi5 − x

i4)

T i5, (8b)

dxi5dt

=(P iGV − x

i5)

T i4,

dxi6dt

=(xi7 − x

i6)

T i3, (8c)

dxi7dt

=(−xi7 + δP iC − x

i1/R

i)

T i1,

dxi8dt

= xi1 (8d)

where δP iM and P iGV are given by δP iM = Ki1xi5 + Ki

3xi4 +

Ki5xi3 + Ki

7xi2, and P iGV = (1 − T2/T3)xi6 + (T2/T3)xi7. D

is the damping coefficient, M is the machine inertia constant,R is the speed regulation constant, Ti’s are time constantsfor power system components, and Ki’s are fractions of totalmechanical power outputs associated with different operatingparts of the turbine. δP itie represents power transfer from areai to other areas. In equation (8), the first state represents thefrequency increment, xi1 = δωi. It can be shown that P itie canbe obtained from

δP ijtie =

n∑j=1

νij(xi8 − x

j8), (9)

Fig. 7. Block diagram of power system and its relation to governor, turbine,generator, and the AGC signal for each control area. More details on thepower grid model can be found in [16].

where νij is the transmission line stiffness coefficient, andthe state variable xi8 is the integral of xi1.

The classical automatic generation control (AGC) imple-ments a simple PI control to regulate the grid frequency. In amulti-area power system, in addition to regulating frequencywithin each area, the auxiliary control should maintain thenet interchange power with neighboring areas at scheduledvalues [25]. This is generally accomplished by adding a tie-line flow deviation to the frequency deviation in the auxiliaryfeedback control loop. A suitable linear combination of thefrequency and tie-line deviations for area i, is known asthe Area Control Error (ACE): this measures the differencebetween the scheduled and actual electrical generation withina control area while taking frequency bias into account. TheACE of area i is thus defined as ACEi = δP itie+βixi1, and βi

is the bias coefficient of area i. The standard industry practiceis to set the bias βi at the so-called Area Frequency ResponseCharacteristic (AFRC), which is defined as βi = Di + 1/Ri.The integral of ACE is used to construct the speed changerposition feedback control signal (δP iC), i.e., δP iC = −Kixi9,where Ki is the feedback gain and dxi9

dt = ACEi.The resulting state space model can be discretized and

written in compact form as

x(tk+1) = Ax(tk) +B1uanc(tk) +B2w(tk). (10)

Where uanc = [δP 1anc . . . δP

nanc]

T are the ancillary inputs,and the exogenous inputs (i.e. disturbances or variations indemands) are denoted by w = [δP 1

D . . . δPnD]T . We propose

controller synthesis for the ancillary services, complementingthe primary control of AGC.

B. MPC for Ancillary Services

We require that uanc be bounded and satisfies a maximumramp constraint, i.e., uanc ≤ uanc(tk) ≤ uanc with uanc >0 and |uanc(tk+1) − uanc(tk)| ≤ λ, for some λ > 0. At eachtime step k, we thus solve the following problem:

minUanc(k)

J(ACE, Uanc) (11)

s.t. x(tk+1) = Ax(tk) +B1uanc(tk) +B2w(tk)

uanc ≤ uanc(tk+j) ≤ uanc|uanc(tk+j+1)− uanc(tk+j)| ≤ λ

where Uanc(k) = (uanc(tk), uanc(tk+1), . . . , uanc(tk+H)) isthe vector of inputs from k to k+H and H is the prediction

11

0 10 20 30 40 50 60 70 80 90 100−0.4

−0.3

−0.2

−0.1

0

0.1

0.2

0.3

0 10 20 30 40 50 60 70 80 90 100−0.6

−0.4

−0.2

0

0.2

0.4

0.6

0.8

ACE1, τ=5

ACE1, τ=10

uanc1 , τ=5

uanc1 , τ=10

Fig. 8. Comparison of ACE1 (Area Control Error of Control Area 1)stabilization using ϕt in (12) with τ = 5 and τ = 10. The controller enforcesthe stabilization delay in both cases.

horizon. All the constraints of problem (11) that depend on jshould hold for j = 0, 1, . . . ,H − 1.

The cost function proposed in [16] minimizes the `2 normof the ACE signal in areas i = 1, . . . , n, by exploiting theancillary service available in each area, while taking intoaccount the system dynamics and constraints. We propose toconstrain the ACE signal to satisfy a specified set of STLproperties, while minimizing the ancillary service used byeach area. Thus we defined J(ACE,Uanc) = ‖Uanc‖`2 =∑2i=1

∑H−1j=0 (U ianc[k + j])2, and an STL formula ϕ which

says that whenever |ACEi| is larger than 0.01, it shouldbecome less than 0.01 in less than τ s. More precisely weused ϕ = �(ϕt) with

ϕt = ¬(|ACE1| < .01))⇒ ( �[0,τ ](|ACE1| < .01)

∧ (¬(|ACE2| < .01))⇒ ( �[0,τ ](|ACE2| < .01)(12)

We encoded this formula and added the resulting constraintsto the MPC problem as described in the previous sections, andsolved it for different values of τ . Results are shown in Fig. 8,and demonstrate that the STL constraint is correctly enforcedin the stabilization of the ACE signal.

IX. RELATED WORK

Receding horizon control for temporal logic has been con-sidered before in the context of LTL [26], where the authorspropose a reactive synthesis scheme for specifications withGR(1) goals. The authors in [27] also propose an MPC schemefor specifications in synthetically co-safe LTL – our approachextends synthesis capabilities to a wider class of temporallogic specifications. In [28], the authors consider full LTLbut use an automata-based approach, involving potentiallyexpensive computations of a finite state abstraction of thesystem and a Buchi automaton for the specification. Wecircumvent these expensive operations using a bounded modelchecking (BMC) approach to synthesis. In [17], the authors

present a model predictive control scheme to stabilize mixedlogical dynamical systems on desired reference trajectories,while fulfilling propositional logic constraints and heuristicrules. A major contribution of this work is to extend theconstraint specification language for such systems to STLspecifications, which allow expression of complex temporalproperties including safety, liveness, and response.

Our work extends the standard BMC paradigm for finitediscrete systems [14] to STL, which accommodates continuoussystems. In BMC, discrete state sequences of a fixed length,representing counterexamples or plans, are obtained as satis-fying assignments to a Boolean satisfiability (SAT) problem.The approach has been extended to hybrid systems, either bycomputing a discrete abstraction of the system [29], [30] or byextending SAT solvers to reason about linear inequalities [31],[32]. Similarly, MILP encodings inspired by BMC have beenused to generate trajectories for continuous systems with LTLspecifications [15], [33], [34], and for a restricted fragmentof MTL without nested operators [35]. While we draw muchinspiration from these early efforts, ours is the first work toconsider a BMC approach to synthesis for full STL.

X. CONCLUDING REMARKS

The main contribution of this paper is a pair of boundedmodel checking style encodings for signal temporal logicspecifications as mixed integer linear constraints. We showedhow our encodings can be used to generate control for systemsthat must satisfy STL properties, and additionally to ensuremaximum robustness of satisfaction. Our formulation of theSTL synthesis problem can be used as part of existing con-troller synthesis frameworks to compute feasible and optimalcontrollers for cyber-physical systems. We presented experi-mental results for controller synthesis on simplified models ofa smart micro-grid and HVAC system, and showed how theMPC schemes in these examples can be framed in terms ofsynthesis from an STL specification, with simulation resultsillustrating the effectiveness of our proposed synthesis.

We have demonstrated the ability to synthesize control forsystems on both the demand and supply sides of a smart grid.We view this as progress toward a contract-based frameworkfor specifying and designing components of the smart gridand their interactions using STL specifications. Future workincludes a reactive synthesis approach to synthesizing controlinputs for systems operating in uncertain environments: wehave already demonstrated preliminary results in this directionin [36]. We will also further explore synthesis in an MPCframework for unbounded STL properties. As mentioned inSection V-B, this is an easy extension of our approach forcertain types of properties. Extending this to arbitrary prop-erties has ties to online monitoring of STL properties [37],which is another direction of further exploration.

REFERENCES

[1] G. E. Fainekos, A. Girard, H. Kress-Gazit, and G. J. Pappas,“Temporal logic motion planning for dynamic robots,” Automatica,vol. 45, no. 2, pp. 343 – 352, 2009. [Online]. Available:http://www.sciencedirect.com/science/article/pii/S000510980800455X

12

[2] P. Nuzzo, H. Xu, N. Ozay, J. Finn, A. Sangiovanni-Vincentelli, R. Mur-ray, A. Donze, and S. Seshia, “A contract-based methodology for aircraftelectric power system design,” Access, IEEE, vol. PP, no. 99, pp. 1–1,2013.

[3] R. Alur, T. A. Henzinger, G. Lafferriere, and G. J. Pappas, “Discreteabstractions of hybrid systems,” Proceedings of the IEEE, vol. 88, no. 7,pp. 971–984, 2000.

[4] R. M. Murray, J. Hauser, A. Jadbabaie, M. B. Milam, N. Petit, W. B.Dunbar, and R. Franz, “Online control customization via optimization-based control,” in In Software-Enabled Control: Information Technologyfor Dynamical Systems. Wiley-Interscience, 2002, pp. 149–174.

[5] D. Q. Mayne, J. B. Rawlings, C. V. Rao, and P. O. M. Scokaert,“Constrained model predictive control: Stability and optimality,”Automatica, vol. 36, no. 6, pp. 789–814, 2000. [Online]. Available:http://dx.doi.org/10.1016/S0005-1098(99)00214-9

[6] O. Maler and D. Nickovic, “Monitoring temporal properties of contin-uous signals,” in FORMATS/FTRTFT, 2004, pp. 152–166.

[7] G. E. Fainekos and G. J. Pappas, “Robustness of temporallogic specifications for continuous-time signals,” Theor. Comput.Sci., vol. 410, no. 42, pp. 4262–4291, 2009. [Online]. Available:http://dx.doi.org/10.1016/j.tcs.2009.06.021

[8] A. Donze and O. Maler, “Robust satisfaction of temporal logic overreal-valued signals,” in FORMATS, 2010, pp. 92–106.

[9] M. Maasoumy, P. Nuzzo, F. Iandola, M. Kamgarpour, A. Sangiovanni-Vincentelli, and C. Tomlin, “Optimal load management system foraircraft electric power distribution,” in IEEE Conference on Decisionand Control (CDC), 2013.

[10] A. Biere, K. Heljanko, T. A. Junttila, T. Latvala, and V. Schuppan,“Linear encodings of bounded LTL model checking,” Logical Methodsin Computer Science, vol. 2, no. 5, 2006.

[11] V. Raman, M. Maasoumy, and A. Donze, “Model predictive controlfrom signal temporal logic specifications: A case study,” in Proceedingsof the 4th ACM SIGBED International Workshop on Design, Modeling,and Evaluation of Cyber-Physical Systems, ser. CyPhy’14. NewYork, NY, USA: ACM, 2014, pp. 52–55. [Online]. Available:http://doi.acm.org/10.1145/2593458.2593472

[12] V. Raman, M. Maasoumy, A. Donze, R. M. Murray, A. Sangiovanni-Vincentelli, and S. A. Seshia, “Model predictive control with signaltemporal logic specifications,” in Proc. of the IEEE Conf. on Decisionand Control, 2014.

[13] G. E. Fainekos and G. J. Pappas, “Robust sampling for MITLspecifications,” in Formal Modeling and Analysis of Timed Systems, 5thInternational Conference, FORMATS 2007, Salzburg, Austria, October3-5, 2007, Proceedings, 2007, pp. 147–162. [Online]. Available:http://dx.doi.org/10.1007/978-3-540-75454-1 12

[14] A. Biere, A. Cimatti, E. M. Clarke, and Y. Zhu, “Symbolic modelchecking without BDDs,” in TACAS, 1999, pp. 193–207.

[15] E. M. Wolff, U. Topcu, and R. M. Murray, “Optimization-basedtrajectory generation with linear temporal logic specifications,” in 2014IEEE International Conference on Robotics and Automation, ICRA2014, Hong Kong, China, May 31 - June 7, 2014, 2014, pp. 5319–5325.[Online]. Available: http://dx.doi.org/10.1109/ICRA.2014.6907641

[16] M. Maasoumy, B. M. Sanandaji, A. Sangiovanni-Vincentelli, andK. Poolla, “Model predictive control of regulation services from com-mercial buildings to the smart grid,” in IEEE American Control Confer-ence (ACC), 2014.

[17] A. Bemporad and M. Morari, “Control of systems integrating logic,dynamics, and constraints,” Automatica, vol. 35, no. 3, pp. 407–427,1999. [Online]. Available: http://dx.doi.org/10.1016/S0005-1098(98)00178-2

[18] A. Donze, T. Ferrere, and O. Maler, “Efficient robust monitoring forstl,” in CAV, 2013, pp. 264–279.

[19] M. Morari and J. Lee, “Model predictive control: Past, present andfuture,” Computers & Chemical Engineering, vol. 23, no. 4, pp.667–682, 1999. [Online]. Available: https://control.ee.ethz.ch/index.cgi?page=publications;action=details;id=1641

[20] A. Donze, “Breach, a toolbox for verification and parameter synthesisof hybrid systems,” in CAV, 2010, pp. 167–170.

[21] J. Lfberg, “YALMIP : A toolbox for modeling and optimization inMATLAB,” in Proceedings of the CACSD Conference, Taipei, Taiwan,2004. [Online]. Available: http://users.isy.liu.se/johanl/yalmip

[22] M. Maasoumy Haghighi, “Controlling energy-efficient buildings inthe context of smart grid: A cyber physical system approach,”Ph.D. dissertation, University of California, Berkeley, Dec 2013.[Online]. Available: http://www.eecs.berkeley.edu/Pubs/TechRpts/2013/EECS-2013-244.html

[23] M. Maasoumy and A. Sangiovanni-Vincentelli, “Total and peak energyconsumption minimization of building hvac systems using model pre-dictive control,” Design Test of Computers, IEEE, vol. 29, no. 4, pp. 26–35, aug. 2012.

[24] M. Maasoumy, M. Razmara, M. Shahbakhti, and A. Sangiovanni-Vincentelli, “Selecting building predictive control based on model un-certainty,” in IEEE American Control Conference (ACC 2014), Portland,USA, June 2014.

[25] H. Bevrani, Robust Power System Frequency Control, ser. Power Elec-tronics and Power Systems. Springer, 2009.

[26] T. Wongpiromsarn, U. Topcu, and R. M. Murray, “Receding horizontemporal logic planning,” IEEE Trans. Automat. Contr., vol. 57, no. 11,pp. 2817–2830, 2012.

[27] E. A. Gol and M. Lazar, “Temporal logic model predictive controlfor discrete-time systems,” in Proceedings of the 16th internationalconference on Hybrid systems: computation and control, HSCC 2013,April 8-11, 2013, Philadelphia, PA, USA, 2013, pp. 343–352. [Online].Available: http://doi.acm.org/10.1145/2461328.2461379

[28] X. C. Ding, M. Lazar, and C. Belta, “LTL receding horizon control forfinite deterministic systems,” Automatica, vol. 50, no. 2, pp. 399–408,2014. [Online]. Available: http://dx.doi.org/10.1016/j.automatica.2013.11.030

[29] G. J. P. Nicolo Giorgetti and A. Bemporad, “Bounded model checkingof hybrid dynamical systems,” in Decision and Control, 2005 and 2005European Control Conference. CDC-ECC ’05. 44th IEEE Conferenceon, Dec 2005, pp. 672–677.

[30] S. Jha, B. A. Brady, and S. A. Seshia, “Symbolic reachability analysisof lazy linear hybrid automata,” in Proc. 5th International Conferenceon Formal Modeling and Analysis of Timed Systems (FORMATS), ser.Lecture Notes in Computer Science, vol. 4763, 2007, pp. 241–256.

[31] G. Audemard, M. Bozzano, A. Cimatti, and R. Sebastiani, “Verifyingindustrial hybrid systems with MathSAT,” Electronic Notes in Theoret-ical Computer Science, vol. 119, no. 2, pp. 17 – 32, 2005, proceedingsof the 2nd International Workshop on Bounded Model Checking(BMC 2004) Bounded Model Checking 2004. [Online]. Available:http://www.sciencedirect.com/science/article/pii/S1571066105000885

[32] M. Franzle and C. Herde, “Efficient proof engines for boundedmodel checking of hybrid systems,” Electronic Notes in TheoreticalComputer Science, vol. 133, no. 0, pp. 119 – 137, 2005,proceedings of the Ninth International Workshop on Formal Methodsfor Industrial Critical Systems (FMICS 2004) Formal Methodsfor Industrial Critical Systems 2004. [Online]. Available: http://www.sciencedirect.com/science/article/pii/S1571066105050279

[33] S. Karaman, R. G. Sanfelice, and E. Frazzoli, “Optimal control of mixedlogical dynamical systems with linear temporal logic specifications,” inDecision and Control, 2008. CDC 2008. 47th IEEE Conference on, Dec2008, pp. 2117–2122.

[34] Y. Kwon and G. Agha, “Ltlc: Linear temporal logic for control,” inHSCC, M. Egerstedt and B. Mishra, Eds., 2008, pp. 316–329.

[35] S. Karaman and E. Frazzoli, “Vehicle routing problem with metrictemporal logic specifications,” in Proceedings of the 47th IEEEConference on Decision and Control, CDC 2008, December 9-11,2008, Cancun, Mexico, 2008, pp. 3953–3958. [Online]. Available:http://dx.doi.org/10.1109/CDC.2008.4739366

[36] V. Raman, A. Donze, D. Sadigh, R. M. Murray, and S. A. Seshia,“Reactive synthesis from signal temporal logic specifications,” in HybridSystems: Computation and Control, HSCC 2015, Seattle, WA, USA, April14-16, 2015, 2015, pp. 239–248.

[37] J. V. Deshmukh, A. Donze, S. Ghosh, X. Jin, G. Juniwal, and S. A.Seshia, “Robust online monitoring of signal temporal logic,” in RuntimeVerification - 6th International Conference, RV 2015 Vienna, Austria,September 22-25, 2015. Proceedings, 2015, pp. 55–70. [Online].Available: http://dx.doi.org/10.1007/978-3-319-23820-3 4

Vasumathi Raman received the B.A. degree inComputer Science and Mathematics from WellesleyCollege in 2007 and the M.S. and Ph.D. degreesin Computer Science from Cornell University in2011 and 2013, respectively. She was a postdoctoralscholar in the Department of Computing & Mathe-matical Sciences at the California Institute of Tech-nology from 2013-2015, and is currently a SeniorScientist at the United Technologies Research Centerin Berkeley, CA. Her research explores algorithmicmethods for designing and controlling autonomous

systems, guaranteeing correctness with respect to user-defined specifications.

13

physical systems and systems biology.

Mehdi Maasoumy received his PhD in ElectricalEngineering & Computer Sciences in 2013 from UCBerkeley, where he was advised by Prof. AlbertoSangiovanni-Vincentelli. He received his MSc de-gree in 2010 from Mechanical Engineering Depart-ment at UC Berkeley with major in Controls andminor in Optimization with Designated Emphasisin Energy Systems and Technology (DEEST). Hegraduated form Sharif University of Technology in2008 with a BSc degree in Mechanical Engineering.

Richard M. Murray received the B.S. degree inElectrical Engineering from California Institute ofTechnology in 1985 and the M.S. and Ph.D. degreesin Electrical Engineering and Computer Sciencesfrom the University of California, Berkeley, in 1988and 1991, respectively. He is currently the ThomasE. and Doris Everhart Professor of Control & Dy-namical Systems and Bioengineering at Caltech.Murray’s research is in the application of feedbackand control to networked systems, with applicationsin biology and autonomy. Current projects include

analysis and design biomolecular feedback circuits; specification, design andsynthesis of networked control systems; and novel architectures for controlusing slow computing.

Alberto Sangiovanni Vincentelli holds the EdgarL. and Harold H. Buttner Chair of Electrical Engi-neering and Computer Sciences at the University ofCalifornia at Berkeley. He has been on the Faculty ofthe Department since 1976. He obtained an electricalengineering and computer science degree (“Dottorein Ingegneria”) summa cum laude from the Politec-nico di Milano, Milano, Italy in 1971. In 1980-1981, he spent a year as a Visiting Scientist at theMathematical Sciences Department of the IBM T.J.Watson Research Center. In 1987, he was Visiting

Professor at MIT. He has held a number of visiting professor positions atItalian Universities, including Politecnico di Torino, Universita di Roma, LaSapienza, Universita di Roma, Tor Vergata, Universita di Pavia, Universitadi Pisa, Scuola di Sant’Anna. He was awarded the IEEE/RSE WolfsonJames Clerk Maxwell Medal “for groundbreaking contributions that havehad an exceptional impact on the development of electronics and electricalengineering or related fields” and the Kaufman Award of the Electronic DesignAutomation Council for “pioneering contributions to EDA” . He is HonoraryProfessor at Politecnico di Torino and has Honorary Doctorates from theUniversity of Aalborg and KTH. He helped founding Cadence and Synopsys,the two leading companies in EDA. He is the author of over 850 papers, 18books and 2 patents in the area of design tools and methodologies, large scalesystems, embedded systems, hybrid systems and innovation.

Alexandre Donze is a research scientist at theUniversity of California, Berkeley in the departmentof Electrical Engineering and Computer Science. Hereceived his Ph.D. degree in Mathematics and Com-puter Science from the University of Joseph Fourierat Grenoble in 2007. He worked as a post-doctoralresearcher at Carnegie Mellon University in 2008,and at Verimag in Grenoble from 2009 to 2012. Hisresearch interests are in simulation-based design andverification techniques using formal methods, SignalTemporal Logic (STL) with applications to cyber-Sanjit A. Seshia received the B.Tech. degree inComputer Science and Engineering from the IndianInstitute of Technology, Bombay in 1998, and theM.S. and Ph.D. degrees in Computer Science fromCarnegie Mellon University in 2000 and 2005 re-spectively. He is currently an Associate Professorin the Department of Electrical Engineering andComputer Sciences at the University of California,Berkeley. His research interests are in dependablecomputing and computational logic, with a currentfocus on applying automated formal methods to em-

bedded and cyber-physical systems, electronic design automation, computersecurity, and synthetic biology. He has served as an Associate Editor of theIEEE Transactions on Computer-Aided Design of Integrated Circuits andSystems. His awards and honors include a Presidential Early Career Awardfor Scientists and Engineers (PECASE) from the White House, an Alfred P.Sloan Research Fellowship, the Prof. R. Narasimhan Lecture Award, and theSchool of Computer Science Distinguished Dissertation Award at CarnegieMellon University.