Nonlinear Analysis: Hybrid Systems 2 (2008) 74–80www.elsevier.com/locate/nahs

An error analysis for execution traces of hybrid automata usingJacobi’s equation

Aaron Diaz

Department of Math and Computer Science, Santa Clara University, 500 El Camino Real, Santa Clara, CA 95053, United States

Received 11 October 2006; accepted 9 November 2006

Abstract

We present an error analysis for execution traces of hybrid automata. We describe an interpretation of Jacobi’s variationalequation as the infinitesimal separation between geodesics in the plant state space. This allows us to estimate the distance betweenthe plant state evolution produced by a hybrid control automata and geodesic curves with respect to some Lagrangian cost function.We provide an analysis for both control automata extracted from convex problems and chattering controllers extracted from measurevalued control laws.c© 2007 Elsevier Ltd. All rights reserved.

Keywords: Error estimation; Hybrid control automata; Jacobi equation; Measure valued control; Chattering control

1. Introduction

1.1. Setting

We consider here suboptimal control policies implemented by finite state controllers and the correspondingtrajectories in the plant’s state space M . We view these suboptimal trajectories as arising from the flow of a sprayfield which is a perturbation of a known geodesic field. In the convex case this exact geodesic field arises as the sprayfield on the tangent bundle T M determined by the equations of parallel transport, in local coordinates:

xk + Γ ki j xi x j = 0 (1)

where the Christoffel symbols Γ ki j are given by:

Γ ki j =

12

glk(

∂gl j

∂ui−

∂gi j

∂ul−

∂gil

∂u j

). (2)

Here gi j =d2 L2

∂ xi ∂ x jdenotes the Finsler metric tensor for a given homogeneous Lagrangian cost function L and gi j

its inverse. In the non-convex case trajectories produced by the control automata are real approximations to measurevalued control laws, while geodesics of convex problems solve

E-mail address: adiaz1@scu.edu.

1751-570X/$ - see front matter c© 2007 Elsevier Ltd. All rights reserved.doi:10.1016/j.nahs.2006.11.005

A. Diaz / Nonlinear Analysis: Hybrid Systems 2 (2008) 74–80 75

min∫ t1

t0L(x(t), x(t))dt (3)

x(t0) = x0 (4)

x(t1) = x1. (5)

Measured valued control laws are generalized curves γ = 〈y(t), µt 〉, where y(t) ∈ M , µt is a probability measureon Ty(t)M , and t ∈ [t0, t1], which solve

min∫ t1

t0

∫Tx(t) M

L(x(t), v)dµt (v)dt (6)

x(t) =

∫Tx(t) M

vdµt (v) (7)

x(t0) = x0 (8)

x(t1) = x1. (9)

1.2. Small topologies

We will partition the smooth manifold of plant states into discrete regions. This reflects the finite precision ofa hybrid system’s sensors, states, and actuators. Points in the interior of one of the regions in such a partition areindistinguishable from the point of view of the controller. Let R ⊂ M be a compact set of admissible states. We viewa partition as a many to one map from R to the finite set {1, 2, 3, . . . , n}. Since we would want points which are closetogether in R to be mapped to the same symbol, we want to construct the partition so that the inverse image of anysymbol i is open.

We construct partitions by first covering the compact space R with a finite number of open sets from its topology.The topological closure of this collection constitutes a small topology, or a sub-topology of R, which will generallynot be Hausdorff. This small topology does, however, form a lattice under union and intersection. The join irreducibleare those lattice elements which satisfy: if x = y

∨z then x = y or x = z. The join irreducible of the aforementioned

open cover constitute the elements of our partition. A complete description of this methodology can be found in [4].

1.3. Models of computation

We formulate a hybrid control automata as a non-deterministic finite state machine whose input and outputalphabets are associated with the tangent bundle of the plant’s state space. Formally, a hybrid control automata isa structure 〈{Ri }, C, s, δ, goal〉.

(1) {Ri } is a finite collection of input symbols which we associate with the join irreducible of an open cover of acompact region S in state space. With each region Ri we associate a point ri ∈ Ri as its representative.

(2) C ⊂ T R is the finite output alphabet. C has the additional structure that there exist sets Ci ⊂ T R such thatC = ∪

ni=1 Ci , Ci ⊂ Tri S, and ∀c ∈ Ci ∃εi s.t. |c − v(ri )| < εi for some small number εi . This represents the fact

that the output symbols of the automata are control directives which lie in cones containing the vector tangent tothe exact mathematical control law.

(3) s is a finite set of automata states.(4) δ : Pow(s) × {Ri } → Pow(s) × Pow(C) is the transition relation. δ has the additional structure that if, for some

T ,U ∈ Pow(s), δ(T, Ri ) = 〈U, V 〉, then V ⊂ Ci .(5) goal = R j , for some j , is a termination symbol. Execution halts upon reading input symbol goal.

The regions {Ri } represent the controller’s finite sensing capacity. The output symbols are tangent vectors by whichto steer the plant. They are clustered in cones Ci applied to region representatives ri . The transition map producing anoutput contained in Ci upon input Ri models the fact that the controller produces some direction applied to the currentstate and that that direction is non-deterministically chosen from a set of possible control values.

We allow non-deterministic models for several reasons. Computational routines used to compute control directivesare often iterative, producing different, if perhaps nearby, results from run to run. Computation may need to be abortedand default values substituted in order to meet real time constraints.

76 A. Diaz / Nonlinear Analysis: Hybrid Systems 2 (2008) 74–80

2. Infinitesimal separation in the convex case

2.1. Introduction

In this section we analyze the error accrued from implementing a continuous control law, derived from aconvex calculus of variations problem, with a finite state hybrid control automata as described above. Plant controlimplemented using a finite automata introduces error due to the limited capacity of the controller to distinguishbetween nearby states (finite input alphabet), finite machine precision (finite memory/states), and the limited capacityto actuate plant control (finite output alphabet). If the machine precision of the controller and the resolution ofthe sensors and actuators are known then one can bound the difference between a geodesic of the system and anapproximating curve produced by the controller in terms of the infinitesimal geodesic separation. We provide thatanalysis now firstly for the convex case.

This analysis is inherently local and in this section we work within a compact region R of the state manifold M .Fix a goal point B. We assume that there exists a convex calculus of variations problem

min∫ t1

t0L(x(t), x(t))dt (10)

x(t1) = B (11)

F(x, x) = 0 (12)

whose geodesics govern the evolution of the plant and which has been, in principle, solved. With this solution weassociate a geodesic field v : M → T M , which associates to each plant state the direction of the geodesic connectingthat point to the goal point B. We term this the exact control law.

Note that solving the above calculus of variations problem amounts to solving a second order differential equationusing for boundary conditions the current plant state and the terminal desired state B. We could have equally viewedthe terminal point as being free and solved the same ODE with an initial condition such as the current plant state andcurrent plant velocity 〈x, x〉 ∈ T M . Using this perspective our control law is of the form v : M × T M → T M . Wemention this because we bound the error introduced by the automata by bounding the infinitesimal difference betweenthe geodesic connecting our current state to B and the geodesic the controller chooses by setting the plant velocity.

2.2. Separation analysis after one control cycle

We begin by analyzing the error accrued during one control cycle, that is, one application of a control directive.We start at some initial state A, at some initial time t0. We would like to travel along the geodesic connecting A toB, which we label x(t). The automata will try to estimate the direction of this geodesic, x(t0), and will produce someapproximate control directive c. This initial condition 〈A, c〉 will cause the plant to evolve along a nearby path y(t).At time t1 we will have traveled to some point B ′

:= y(t1), which we hope is close to B. How far is B ′ from B?The answer to this question can be given by considering Jacobi’s equation. Jacobi’s equation is the variational

equation for the Euler–Lagrange equations. The equation was first investigated to establish when a curve x(t), whichwas a critical value of J , was indeed an extrema. In this context one looks for a sufficient condition by examiningthe second variational derivative of J in the direction of an arbitrary curve h(t). This expression is quadratic in h andconsequently has a unique extrema which can be computed from necessary conditions, that is, from looking at itsvariational derivative with respect to h. This derivative is exactly the Jacobi expression.

Another interpretation of Jacobi’s equation is that of the infinitesimal separation between two nearby geodesics.We can derive these equations in the following fashion. Let

E :=∂L

∂x−

∂

∂t

∂L

∂ x(13)

denote the Euler–Lagrange operator associated with Lagrangian L and write y(t) = x(t) + h(t) for some unknownfunction h. Formally expand E in a power series about x .

E(y) = E(x + h) = E(x) + E ′(x)h + O(2). (14)

A. Diaz / Nonlinear Analysis: Hybrid Systems 2 (2008) 74–80 77

Here O(2) denotes terms of quadratic degree and up in h. Notice that E(x) = 0 since x is a geodesic curve. Thecondition that y also be geodesic, up to first order, is:

E ′(x)h = 0. (15)

h is known as the infinitesimal separation between x and y. It is straightforward to compute E ′ in terms of the partialderivatives of L and it can be shown that the Jacobi equation has the following form.

Qh −∂

∂t

(P

∂h

∂t

)= 0, (16)

where Q =12 ( ∂2 L

∂2x−

∂∂t

∂2 L∂x∂ x ) and P =

12

∂2 L∂2 x

. Note that P and Q are functions of the geodesic x and its derivative xand are consequently given. The derivation of these quantities and a rigorous yet simple exposition of these facts canbe found in most classical calculus of variations texts such as [2]. The infinitesimal separation between the geodesicwe would like to follow x and the geodesic determined by the control directive c can thus be computed by solving theinitial value problem:

Qh −∂

∂t

(P

∂h

∂t

)= 0 (17)

h(t0) = 0 (18)

h(t0) = c − x(t0). (19)

This is a linear homogeneous second order ordinary differential equation for the unknown separation function h.If the control directive c is tangent to the geodesic curve passing through B, i.e. if c = x(t0), then the separation isidentically zero. Otherwise, we can integrate Eq. (17) to compute the infinitesimal separation at time t1.

We can apply classical ODE theory to prove what we already know intuitively, that h tends to 0 in norm as c tendsto x(t0). Indeed, we now obtain an explicit bound on the growth of h. Introducing a new variable l = h, the system(17) is equivalent to the system[

lh

]=

[−P−1 P ′

−P−1 QI 0

] [l h

]. (20)

Note that

P−1(x(t), x(t)) =12

L−1x x (x(t), x(t)) (21)

exists for all t ∈ [t0, t1] by the Weierstrass condition [2,3] since x(t) is geodesic.P ′, P , and Q are continuous functions of t on the compact region [t0, t1] by hypothesis. Consequently, there exists

a Lipschitz constant k such that[−P−1 P ′

−P−1 QI 0

](22)

is bounded by k in matrix norm on [t0, t1]. All norms on R2n×2n are equivalent and a different choice of matrix normwill determine a different Lipschitz constant. A fundamental result in the theory of ODEs is that solutions to Lipschitzcontinuous homogeneous differential equations have at most exponential growth. For our problem this amounts to thefollowing

|h(t)|1,p ≤

∣∣∣∣[l(t0)0

]∣∣∣∣1,p

ek|t−t0| (23)

where

| f (t)|1,p := | f (t)|p + | f (t)|p (24)∣∣∣∣[uv

]∣∣∣∣1,p

:= |u|p + |v|p (25)

78 A. Diaz / Nonlinear Analysis: Hybrid Systems 2 (2008) 74–80

is the H1,p norm on curves in the tangent bundle and | · |p is the L p norm for 1 ≤ p < ∞. Thus,

|h(t)|p ≤ |c − x0|pek|t−t0|. (26)

This gives us our estimate of the infinitesimal separation in terms of the difference between the control direction c andthe direction of the desired geodesic x0.

In what ways can the control directive c be off? The first way in which c can be off is due to error in computation.This comes from the finite number of automata states and finite computation time and in practice stems from finitemachine precision and the use of iterative algorithms. In practice we will know the machine precision and have anestimate of the accuracy of iterative algorithms used in terms of residual information etc. Thus, if the plant is in stateA, then we have some estimate, ε1, such that

|c − v(A)| < ε1 (27)

where as before v(A) is the direction of the geodesic connecting A to B.The second way in which c can be off comes from the finite automata input alphabet and finite sensor resolution.

Recall that this limited capacity of the controller to differentiate between nearby plant states was modeled byassociating the automata input symbols with the open regions in the plant’s state space {Ri }. Moreover, given aninput symbol Ri , computation proceeds by using a representative point ri ∈ Ri . For example, the ri could be thefloating point numbers representable by the machine. Then Ri is the set of all real numbers rounded to ri by thecontroller. Thus, if the current and goal plant state are A and B respectively, then there exist i , j such that A ∈ Ri andB ∈ R j . If i = j then we are at the goal as far as the controller can tell and execution halts. Otherwise, then c is notactually an estimate of the geodesic direction v(A) of where we are, but rather is an estimate of v(ri ), the direction ofthe geodesic connecting ri to r j .

The error introduced by this approximation is thus |v(ri ) − v(A)|. We can bound this error if we have a bound onthe diameter of the set {v(p)|p ∈ Ri }. Alternatively, we can note that v(A) is obtained by solving the boundary valueproblem

E(z) = 0 (28)

z(t0) = A (29)

z(t1) = B (30)

and v(ri ) is obtained as a solution to

E(z) = 0 (31)

z(t0) = ri (32)

z(t1) = r j . (33)

Thus, the question becomes how do solutions to the geodesic equations vary as the boundary values vary? The answerto both of those questions is problem dependent, but it is clear firstly that v(ri ) → v(A) as ri → A and r j → B andsecondly that for any particular problem we have some estimate ε2 such that

|v(ri ) − v(A)| < ε2. (34)

Combining these two estimates we are able to make the bound

|c − x0| (35)

= |c − v(A)| (36)

≤ |c − v(ri )| + |v(ri ) − v(A)| (37)

ε1 + ε2. (38)

Combining this analysis we reach the conclusion that the infinitesimal separation error introduced by the controllerhas the following bound, where ε = ε1 + ε2

|h(t)| ≤ εek|t−t0|. (39)

A. Diaz / Nonlinear Analysis: Hybrid Systems 2 (2008) 74–80 79

2.3. Separation analysis after multiple control cycles

Infinitesimal separation after one control cycle can be computed by solving Jacobi’s equation with a zero initialseparation and an initial rate of separation equal to the difference between the control directive and the tangent tothe desired geodesic. We can determine the total separation after multiple control cycles by compounding our localanalysis. The actual separation after k control cycles can be computed by solving a sequence of k initial value problemswhile a bound on its growth can be determined from local exponential bounds using single cycle analysis.

Consider a trace of execution where the controller issues control directive ci at time t0 + i∆, for i = 0 . . . k − 1and ∆ = (t1 − t0)/k. Let y(t) denote the piecewise geodesic curve determined by this trace of execution. Denote byy(i)(t) the value of y(t) on the interval [t0 + i∆, t0 + (i + 1)∆]. Then, by definition, y(i)(t) satisfies

y(i)p = Γ p

lm y(i)l y(i)

m (40)

y(i)(t0 + i∆) = ci (41)

y(i)(t0 + i∆) = y(i−1)(t0 + i∆) (42)

for i = 1 . . . k − 1 which are the geodesic equations of parallel transport whose initial condition is determined bycontrol directive ci . For i = 0 the equation is the same but with the initial conditions y(0)(t0) = A and y(0)(t0) = c0.Let Bi = y(i)(t0 + i∆) be the plant states where the controls are applied. Let x i (t) denote the local geodesic from ourcurrent point Bi to the goal B. i.e. x i (t) satisfies

x (i)p = Γ p

lm x (i)l x (i)

m (43)

x (i)(t0 + i∆) = Bi (44)

x (i)(t1) = B. (45)

Let h(t) denote the infinitesimal separation between y(t) and x i (t). Denote by h(i)(t) the value of h(t) on the interval[t0 + i∆, t0 + (i +1)∆]. Then the infinitesimal separation, h(i)(t) on the i th sub-interval is given by solving the Jacobiequation

−∂

∂t

(P

∂h(i)

∂t

)+ Qh(i)

= 0 (46)

h(i)(t0 + i∆) = ci − xi (t0 + i∆) (47)

h(i)(t0 + i∆) = 0 (48)

for i = 1 . . . k − 1. For i = 0 the initial condition is the same except h(0)(t0) = 0. According to these definitions theseparation after r control cycles is given by h(r−1)(t1). However if we compare Eq. (46) to Eq. (17) we see that theerror accrued after r cycles does not depend upon position directly and consequently the global error does not havememory. In other words the global error is just the local error accrued in the last step. The only position dependenceoccurs in the matrix 22. Thus, if the Lipschitz constant k is chosen large enough to bound the matrix 46 in operatornorm in the compact region R then we have that

|hr−1|1,p ≤ εekδ. (49)

This gives us a formula for choosing a sensor resolution and machine precision to achieve a given accuracy.

3. Infinitesimal separation in the non-convex case

3.1. Introduction

In this section we consider the case where the Lagrangian cost function is not convex. We will be consideringgeneralized solutions to the relaxed problem given by time dependent finite control measures µt . We will considerthe error accrued from control automata extraction based on Kohn–Nerode chattering control. JL(·) will denote therelaxed cost functional of the non-convex problem with underlying Lagrangian L .

80 A. Diaz / Nonlinear Analysis: Hybrid Systems 2 (2008) 74–80

3.2. The method of chattering control

The method of chattering control is an algorithm for producing a convex linear combination of Dirac measures. Theaction of which approximates the action of the optimal control measure µt to arbitrary accuracy. The mathematicalfoundation of this algorithm is based on the fact that non-convex problems can be convexified. This convexifiedproblem always achieves its minimum. This minimum value is equal to the infimum of the non-convex problem.Moreover, the minimizer of the convex problem is a cluster point, in the standard topology, of minimizing sequencesof the non convex problem. We survey in this section a method for producing near optimal controls which cause theplant to chatter near geodesics of the convexified Lagrangian.

Given a function f : Rn→ R, denote by epi( f ) the epigraph of f defined as {〈x, a〉| f (x) ≤ a}. A set S ⊂ Rn is

convex if for all u, v ∈ S the point λu + (1 − λ)v is also in S for 0 ≤ λ ≤ 1. A function is convex if its epigraph is aconvex set. The convex hull of a set T ⊂ Rn is the intersection of all convex sets containing T . Given these conceptswe can define the Γ -regularization of a function f , which we denote by f , as the function whose epigraph is theconvex hull of f .

Theorem 1. f is well defined.

Theorem 2. If x(t) minimizes JL and {xi (t)}∞i=1 is any sequence of simplicial curves s.t. JL(xi ) > JL(xi+1) then• JL(x) = infJL• xi → x point wise.

At the beginning of each control cycle the chattering control algorithm first estimates the direction of the geodesicof the convexified Lagrangian x . Call the geodesic chosen by this estimate y. The controller then produces a curvey(t) such that for any optimality tolerance δ > 0 there exists an εδ > 0 such that |y(t)− y(t)| < εδ . This constructionhas the property that εδ → 0 as δ → 0. This algorithm works by using a linear program to produce a time schedulefor the application of Dirac measures concentrated on extreme points of the epigraph of L . The mean value of theresulting measure valued control determines y(t) via integration. A complete description of this algorithm with proofsof correctness can be found in [1].

3.3. Infinitesimal analysis

The chattering controller produces a curve y(t) which is within εc of y(t). y is a geodesic of the convexifiedLagrangian L which approximates the goal geodesic x(t). We can use the convex analysis of the previous section todetermine the infinitesimal separation between the geodesics of the convexified Lagrangian and tack on the chatteringtolerance εc. If ε is a bound on the sum of the sensor resolution and machine precision and k is a Lipschitz constant,both determined as in the previous chapter and if B ′ is the state of the plant after r chattering control cycles then afirst order estimate of the error accrued by finite automata implementation of chattering control for ∆ units of time isgiven by εek∆

+ εc.

3.4. Conclusion

We have described a method for estimating the distance from a plant state trajectory generated by a control automatato the systems geodesics with respect to a given Lagrangian cost function. This analysis is based on an interpretationof Jacobi’s variational equation as an infinitesimal separation. This method is applicable to convex problems as wellgeneralized problems and can be used for open loop controllers as well as multiple control cycles applied by a closedloop controller.

References

[1] Xiaolin Ge, Wolf Kohn, Anil Nerode, Algorithms for chattering approximations to relaxed optimal controls, Technical Report, MathematicalSciences Institute, 1994.

[2] Izrail Moiseevich Gelfand, Sergei Vasilevich Fomin, Calculus of Variations, Prentice-Hall, New Jersey, 1963.[3] Andrew D. Lewis, Richard M. Murray, Variational principles for constrained systems: Theory and experiment, The International Journal of

Nonlinear Mechanics (1994).[4] Anil Nerode, Wolf Kohn, Models for hybrid systems: Automata, topologies, controllability, observability, Technical Report, Mathematical

Sciences Institute, 1993.