COSTATE ESTIMATION FOR OPTIMAL CONTROL PROBLEMS...

COSTATE ESTIMATION FOR OPTIMAL CONTROL PROBLEMS USINGORTHOGONAL COLLOCATION AT GAUSSIAN QUADRATURE POINTS

By

CAMILA CLEMENTE FRANCOLIN

A DISSERTATION PRESENTED TO THE GRADUATE SCHOOLOF THE UNIVERSITY OF FLORIDA IN PARTIAL FULFILLMENT

OF THE REQUIREMENTS FOR THE DEGREE OFDOCTOR OF PHILOSOPHY

UNIVERSITY OF FLORIDA

2013

c© 2013 Camila Clemente Francolin

2

To the ones who supported me: Mamae and Carol.

And to the ones who inspired me: Papai and Nicolas.

3

ACKNOWLEDGMENTS

Getting a PhD has been, hands down, the hardest thing I have accomplished thus

far in my life. It goes without saying I did none of it by myself, and I owe many thanks to

the people who, in many ways, helped me through this process. For obvious reasons,

none of this would have been possible without my faculty advisor, Dr. Anil Rao. I thank

you for always holding me to the highest standards. You were tough when you needed

to be, and encouraging the rest of the time. You helped me evolve and gain confidence

as a researcher, and I am a much better scientist for it. I would also like to thank my

committee members for helping me through the doctoral process: Dr. William Hager,

Dr. Richard Lind, and Dr. Warren Dixon. I am deeply grateful to Dr. William Hager for

patiently taking the time to meet and discuss my research, and kindly correcting me

when I was wrong.

I would also like to thank the Office of Naval Research, especially Dr. Maria

Medeiros and Dr. David Drumheller, for their financial support. The time I spent working

at the Naval Undersea Warfare Center was highly instructive. Thank you to Chris Duarte

and Gerry Martel for their mentorship during the time I spent there.

A PhD is not just about academic growth, but also about personal development. I

have a lot of people to thank for the latter part of my formation. I first thank those who,

through their love of science and discovery, inspired me to go down this path, as it is

not an easy one to pick. My first inspiration was my Dad, whom I watched go through

this process so many years ago. He was in a foreign country with two small children

and he still made it look easy. He taught me by example at a very young age to always

question things, and to never lose a sense of curiosity; it is still this sense curiosity that

propels me to keep learning. Nick, you were my second source of inspiration. I would

never have had the courage to take this path if you hadn’t been there, forging ahead with

no fear and showing me the way. You showed me this process was possible by taking

one step at a time, and I thank you for you inspiring me with your enthusiasm and love

4

of science. Next, I want to thank all those who encouraged me and helped me to keep

going when quitting would have been so much easier. Greg, you always believed in me,

even when I didn’t. Ed, thank you for keeping me motivated toward graduation by asking

me when I would be done each and every time you saw me. I’m so lucky to have you

in my life. Thomas, thank you for moving my futon all the eight times it took to get me

graduated. Thanks to all my family in Brazil: tia Terezinha, Thais, Luluca, Erika, Celma,

Joao Matheus, Maria Alice; every time I went to see you over the summers I came back

renewed. Maxie, you were always sitting at my feet through the ups and downs of the

research process. And always had healing licks when things didn’t go according to plan

(which, as it turned out, happened quite a lot).

I would like to thank all the members of VDOL. Especially Chris and Divya in the

beginning, and Begum, Matt, and Brian toward the end (yes Brian, you are an honorary

VDOL member). Matt, how could I ever thank you for all your support. Thank you for

patiently listening when I overindulged in telling you every gory detail of my research,

kindly telling me it would be okay when my results didn’t turn out as expected, and

sharing in my excitement when it finally did turn out as expected. Thank you for putting

a fence up in my backyard just so I could write this dissertation with no distractions from

my dog. You’re my lifeboat.

Finally, to my Mom and my Sister. Getting a PhD is just one of the things that I could

never have accomplished without you both by my side. You supported me emotionally,

financially, and any other way that is possible. Mom, thank you for each and every time

you helped me move, cleaned my house, or ran my errands just so I could have more

time to finish a piece of work. I hope to always make you proud. Carol, you showed

me that it was possible to succeed, and it was okay to fail, because the first follows the

latter. You are always there when I need someone to talk to (or when I have someone to

sue). Thank you.

5

TABLE OF CONTENTS

page

ACKNOWLEDGMENTS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

LIST OF TABLES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

LIST OF FIGURES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

ABSTRACT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

CHAPTER

1 INTRODUCTION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

2 MATHEMATICAL BACKGROUND . . . . . . . . . . . . . . . . . . . . . . . . . 25

2.1 Continuous-Time Bolza Optimal Control Problem . . . . . . . . . . . . . . 252.1.1 First-Order Optimality Conditions . . . . . . . . . . . . . . . . . . . 262.1.2 First-Order Optimality Conditions of Integral Formulation . . . . . . 272.1.3 Control Inequality Path Constraints . . . . . . . . . . . . . . . . . . 292.1.4 State Inequality Path Constraints . . . . . . . . . . . . . . . . . . . 31

2.2 Differential-Algebraic Equations . . . . . . . . . . . . . . . . . . . . . . . . 322.2.1 Index-Reduction for Differential-Algebraic Equations . . . . . . . . 342.2.2 Solutions of High-Index Differential-Algebraic Equations . . . . . . 37

2.3 State Inequality Path Constrained Optimal Control Problems . . . . . . . 402.3.1 Indirect Adjoining . . . . . . . . . . . . . . . . . . . . . . . . . . . . 412.3.2 Direct Adjoining . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 452.3.3 Indirect Adjoining With Continuous Multipliers . . . . . . . . . . . . 47

2.4 Numerical Properties of Orthogonal Collocation Methods . . . . . . . . . 492.4.1 Function Approximation and Interpolation . . . . . . . . . . . . . . 49

2.4.1.1 Family of Legendre-Gauss points . . . . . . . . . . . . . 512.4.2 Numerical Integration . . . . . . . . . . . . . . . . . . . . . . . . . . 54

2.4.2.1 Low-order integrators . . . . . . . . . . . . . . . . . . . . 542.4.2.2 Gaussian quadrature . . . . . . . . . . . . . . . . . . . . 57

2.5 Orthogonal Collocation for the Solution of Optimal Control Problems . . . 592.5.1 Global Collocation at LG Points . . . . . . . . . . . . . . . . . . . . 612.5.2 Global Collocation at LGR Points . . . . . . . . . . . . . . . . . . . 632.5.3 Global Collocation at Flipped LGR Points . . . . . . . . . . . . . . 652.5.4 Variable-Order Collocation at LG Points . . . . . . . . . . . . . . . 662.5.5 Variable-Order Collocation at LGR Points . . . . . . . . . . . . . . 682.5.6 Variable-Order Collocation at Flipped LGR Points . . . . . . . . . . 70

3 COSTATE ESTIMATION USING THE INTEGRAL FORMULATION . . . . . . . 72

3.1 Continuous-Time Bolza Optimal Control Problem . . . . . . . . . . . . . . 733.1.1 Differential and Integral Forms of Optimal Control Problem . . . . . 743.1.2 First-Order Optimality Conditions of Differential and Integral Forms 75

6

3.2 Costate Estimation Using Integral Legendre-Gauss Collocation . . . . . . 763.2.1 Differential Form of LG Collocation . . . . . . . . . . . . . . . . . . 763.2.2 KKT Conditions Using Differential LG Collocation . . . . . . . . . . 783.2.3 Integral Form of LG Collocation . . . . . . . . . . . . . . . . . . . . 803.2.4 KKT Conditions Using Integral LG Collocation . . . . . . . . . . . . 823.2.5 A Relationship Between Integral and Differential Costate Estimates 85

3.3 Costate Estimation Using Integral Legendre-Gauss-Radau Collocation . . 863.3.1 Differential Form of LGR Collocation . . . . . . . . . . . . . . . . . 873.3.2 KKT Conditions Using Differential LGR Collocation . . . . . . . . . 883.3.3 Integral Form of LGR Collocation . . . . . . . . . . . . . . . . . . . 913.3.4 KKT Conditions Using Integral LGR Collocation . . . . . . . . . . . 933.3.5 A Relationship Between Integral and Differential Costate Estimates 97

3.4 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 973.5 Concluding Remarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98

4 MOTIVATION FOR NEW COSTATE ESTIMATE . . . . . . . . . . . . . . . . . . 100

4.1 Continuous-Time Bolza Optimal Control Problem . . . . . . . . . . . . . . 1014.1.1 First-Order Optimality Conditions of Continuous Problem . . . . . . 102

4.2 Variable-Order Collocation at Legendre-Gauss Points . . . . . . . . . . . 1034.2.1 KKT Conditions of Variable-Order LG Collocation Method . . . . . 1044.2.2 Costate Estimate and Transformed Adjoint System . . . . . . . . . 105

4.3 Variable-Order Collocation at Flipped Legendre-Gauss-Radau Points . . . 1084.3.1 KKT Conditions of Variable-Order Flipped LGR Collocation Method 1094.3.2 Costate Estimate and Transformed Adjoint System . . . . . . . . . 110

4.4 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113

5 COSTATE ESTIMATION FOR STATE CONSTRAINED PROBLEMS . . . . . . 116

5.1 Continuous-Time State-Constrained Optimal Control Problem . . . . . . . 1175.1.1 First-Order Optimality Conditions . . . . . . . . . . . . . . . . . . . 119

5.2 Costate Estimation Using Legendre-Gauss Collocation . . . . . . . . . . . 1205.2.1 Variable-Order Collocation at Flipped LG Points . . . . . . . . . . . 1205.2.2 Costate Estimate and Transformed Adjoint System . . . . . . . . . 122

5.3 Costate Estimation Using Flipped Legendre-Gauss-Radau Collocation . . 1275.3.1 Variable-Order Collocation at Flipped LGR Points . . . . . . . . . . 1275.3.2 Costate Estimate and Transformed Adjoint System . . . . . . . . . 129

5.4 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137

6 EXAMPLES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139

6.1 Example 1: Mayer Optimal Control Problem . . . . . . . . . . . . . . . . . 1406.1.1 Solution Using Collocation at LG Points . . . . . . . . . . . . . . . 1406.1.2 Solution Using Collocation at LGR Points . . . . . . . . . . . . . . 144

6.2 Example 2: Lagrange Optimal Control Problem . . . . . . . . . . . . . . . 1476.2.1 Solution Using Collocation at LG Points . . . . . . . . . . . . . . . 1486.2.2 Solution Using Collocation at LGR Points . . . . . . . . . . . . . . 151

7

6.3 Example 3: First-Order State Inequality Path Constraint Problem . . . . . 1546.3.1 Solution Using Collocation at LG Points . . . . . . . . . . . . . . . 154

6.3.1.1 Previously derived costate estimate . . . . . . . . . . . . 1576.3.1.2 Indirect adjoining with continuous multipliers . . . . . . . 160

6.3.2 Solution Using Collocation at Flipped LGR Points . . . . . . . . . . 1626.3.2.1 Previously derived costate estimate . . . . . . . . . . . . 1626.3.2.2 Indirect adjoining with continuous multipliers . . . . . . . 164

6.4 Example 4: Second-Order State Inequality Path Constraint Example . . . 1686.4.1 Solution Using Collocation at LG Points . . . . . . . . . . . . . . . 169

6.4.1.1 Previously derived costate estimate . . . . . . . . . . . . 1726.4.1.2 Indirect adjoining with continuous multipliers . . . . . . . 175

6.4.2 Solution Using Collocation at Flipped LGR Points . . . . . . . . . . 1786.4.2.1 Previously derived costate estimate . . . . . . . . . . . . 1816.4.2.2 Indirect adjoining with continuous multipliers . . . . . . . 184

7 CONCLUSIONS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 187

REFERENCES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 191

BIOGRAPHICAL SKETCH . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197

8

LIST OF TABLES

Table page

2-1 Absolute maximum error in v(t) and u(t) for problems A and B. . . . . . . . . 40

9

LIST OF FIGURES

Figure page

2-1 Error in the solution of DAE system . . . . . . . . . . . . . . . . . . . . . . . . . 39

2-2 Function approximation using uniformly spaced points . . . . . . . . . . . . . . 52

2-3 Distribution of Gaussian quadrature points . . . . . . . . . . . . . . . . . . . . . 53

2-4 Function approximation using LG points . . . . . . . . . . . . . . . . . . . . . . 55

2-5 Function approximation using LGR points . . . . . . . . . . . . . . . . . . . . . 56

2-6 Error associated with function approximation using uniform, LG, and LGR points 57

2-7 Approximation of integral using Trapezoid rule. . . . . . . . . . . . . . . . . . . 58

2-8 Error in approximation of integral using Trapezoid rule as a function of N . . . . 58

2-9 Error in approximation of integral using Gaussian quadrature as a function of N 60

2-10 Distribution of LG points for global collocation . . . . . . . . . . . . . . . . . . . 62

2-11 Distribution of LGR points for global collocation . . . . . . . . . . . . . . . . . . 64

2-12 Distribution of flipped LGR points for global collocation . . . . . . . . . . . . . . 65

2-13 Distribution of LG points for variable-order collocation . . . . . . . . . . . . . . 67

2-14 Distribution of LGR points for variable-order collocation . . . . . . . . . . . . . 69

2-15 Distribution of flipped LGR points for variable-order collocation . . . . . . . . . 71

4-1 Relationship Between the Direct and Indirect Methods . . . . . . . . . . . . . . 115

5-1 Equivalence of the Direct and Indirect Methods . . . . . . . . . . . . . . . . . . 138

6-1 Primal solution for Example 1 obtained using integral collocation at LG points. . 141

6-2 Costate solutions for Example 1 obtained using collocation at LG points. . . . . 142

6-3 Costate errors for Example 1 obtained using collocation at LG points. . . . . . 143

6-4 Primal solution for Example 1 obtained using integral collocation at LGR points. 145

6-5 Costate solutions for Example 1 obtained using collocation at LGR points. . . . 146

6-6 Costate errors for Example 1 obtained using collocation at LGR points. . . . . 146

6-7 State and control for Example 2 obtained using integral LG collocation. . . . . 148

6-8 Costate solutions for Example 2 obtained using collocation at LG points. . . . . 149

10


6-10 State and control for Example 2 obtained using integral LGR. . . . . . . . . . . 152

6-11 Costate solutions for Example 2 obtained using collocation at LGR points. . . . 153


6-13 Primal solution for Example 3 obtained using collocation at LG points. . . . . . 155

6-14 Errors in state and control for Example 3 obtained using LG collocation. . . . . 156

6-15 Costate estimate as derived by Ref. [1] for Example 3. . . . . . . . . . . . . . . 158

6-16 Costate errors for estimate derived in Ref.[1] for Example 3. . . . . . . . . . . . 159

6-17 Dual variables for Example 3 obtained using collocation at LG points. . . . . . 161


6-19 Primal solution for Example 3 obtained using collocation at LGR points. . . . . 163

6-20 Errors for Example 3 obtained using collocation at LGR points. . . . . . . . . . 164

6-21 Costate Estimate as derived by Ref. [1] for Example 3. . . . . . . . . . . . . . . 165


6-23 Costate estimate for Example 3 obtained using collocation at LGR points. . . . 167


6-25 Primal solution for Example 4 obtained using LG collocation. . . . . . . . . . . 170

6-26 State and control errors for Example 4 using collocation at LG points. . . . . . 171

6-27 Costate Estimate as derived by Ref. [1] for Example 4. . . . . . . . . . . . . . . 173


6-29 Costate Estimate for Example 4 obtained using collocation at LG points. . . . . 176


6-31 Primal solution for Example 4 obtained using LGR collocation. . . . . . . . . . 179

6-32 State and control errors for Example 4 using collocation at LGR points. . . . . 180

6-33 Costate estimate as derived by Ref. [1] for Example 4. . . . . . . . . . . . . . . 182

6-34 Costate Errors using estimate derived in Ref.[1] for Example 4. . . . . . . . . . 183

6-35 Costate Estimate for Example 4 obtained using collocation at LGR points. . . . 185

11


12

Abstract of Dissertation Presented to the Graduate Schoolof the University of Florida in Partial Fulfillment of theRequirements for the Degree of Doctor of Philosophy

COSTATE ESTIMATION FOR OPTIMAL CONTROL PROBLEMS USINGORTHOGONAL COLLOCATION AT GAUSSIAN QUADRATURE POINTS

By

Camila Clemente Francolin

August 2013

Chair: Anil V. RaoMajor: Aerospace Engineering

Computing the costate in an optimal control problem is important for verifying

the optimality of the solution and performing sensitivity analysis. This dissertation is

concerned with the problem of estimating the costate in an optimal control problem

using orthogonal collocation at Legendre-Gauss (LG) and Legendre-Gauss-Radau

(LGR) points. First, methods are presented for estimating the costate using orthogonal

collocation at the LG or LGR points when the dynamic constraints of the optimal control

problem are formulated in integral form. A new continuous-time dual variable called

the integral costate is introduced, where the integral costate is the Lagrange multiplier

of the integral dynamic constraint. The first-order optimality conditions of the integral

form of the optimal control problem are derived in terms of the integral costate. The

integral form of the optimal control problem is then discretized using the integral LG

and LGR collocation methods and relationship between the discrete form of the integral

costate and the costate of the original differential optimal control problem are developed.

It is shown that the LGR integration matrix that relates the differential costate to the

integral costate is singular while the corresponding LG integration matrix is full rank. The

approach developed in this research then provides a way to estimate the costate of the

original optimal control problem using the Lagrange multipliers of the integral form of the

LG and LGR collocation methods. Furthermore, the costate estimates presented in this

research result in a set of Karuhn-Kush-Tucker conditions of the nonlinear programming

13

problem which are a discrete approximation of the first-order optimality conditions of the

continuous-time optimal control problem both in differential and integral forms.

The second part of this research focuses on state inequality path constrained

optimal control problems. Problems with active state-inequality path constraints are

difficult to solve due to the high-index differential-algebraic equations (DAE) that result

from the constraint activity. This DAE index fluctuation in the solution domain results in

possible discontinuities in the dual variables which are hard to approximate numerically.

Due to these discontinuities, previous costate estimates for direct transcription methods

using collocation at LG or LGR points resulted in a transformed adjoint system

which was not a discrete approximation to the first-order optimality conditions in the

presence of state inequality path constraints. In this research a different set of costate

estimates are developed which result in a transformed adjoint system that is a discrete

approximation of the first-order optimality conditions of the continuous-time optimal

control problem. Specifically, a costate estimate using the method of indirect adjoining

with continuous multipliers is derived. The equivalence between the first-order optimality

conditions of the finite-dimensional nonlinear program and the first-order optimality

conditions of the continuous-time optimal control problem ensures convergence of the

discrete problem to a local minimum which satisfies the optimality conditions of the

original problem. This costate estimate can thus be used to verify the extremality of the

approximated solution.

14

CHAPTER 1INTRODUCTION

Many problems in engineering, economics, and biology can be modeled as

differential-algebraic systems. In addition, it is often desired to optimize the performance

of such systems. The goal of an optimal control problem is to determine the state and

control that optimize a given performance index subject to a set of differential-algebraic

constraints. In aerospace engineering, optimal control applications include trajectory

optimization, parameter estimation, and vehicle guidance. As alluded to earlier, the

constraints in an optimal control problem include differential equations that describe the

motion of the dynamical system, path constraints that define limits on the process, and

event constraints that define way points that must be met during the motion.

Optimal control problems that involve inequality path constraints are common in

aerospace engineering. Such constraints can be purely a function of the control (for

example, control limits such as maximum allowable thrust), purely a function of the

state (for example, no-fly zone constraints), or more generally a function of both the

control and the state (for example, maximum heating rate constraints). Quoting Ref. [2],

“Solving an optimal control or estimation problem is not easy”. Optimal control problems

with inequality path constraints are particularly challenging to solve because the optimal

trajectory may contain regions where the inequality constraint is active. Even more

challenging are problems with inequality path constraints that are purely a function of

the state, leading to high-index differential-algebraic equation (DAE) constraints [3–6].

Systems with state inequality path constraints of index one or less can generally be

solved numerically using numerical integrators. Systems with state inequality path

constraints of index greater than one, however, pose computational challenges for

numerical integration methods [3]. In the context of an optimal control problem, a state

inequality path constrained high-index differential-algebraic system have a non-smooth

state and possibly a discontinuous costate, while a control inequality constrained

15

problem can have a discontinuous optimal control [7, 8]. Such discontinuities can be

difficult to approximate accurately using numerical methods.

Methods for approximating solutions to optimal control problems fall into two broad

categories: indirect and direct methods. In an indirect method the first-order optimality

conditions are derived using the calculus of variations, resulting in a Hamiltonian

boundary-value problem (HBVP) [9]. In the case when the inequality path constraints

are inactive on the optimal solution, the HBVP is a two-point boundary value problem.

When the solution domain contains active/inactive switches in state inequality path

constraint activity, however, the HBVP will have interior-point constraints, resulting in a

multi-point boundary value problem [7].

A great deal of research has been done on solving optimal control problems with

state inequality path constraints using indirect methods [8, 10–13]. This research has

yielded a number of different ways to derive the necessary conditions for optimality,

each resulting in a different set of conditions. In the method of direct adjoining, the state

inequality path constraint is augmented to the Hamiltonian, and the first-order optimality

conditions are derived using the calculus of variations. This method results on a set

of “jump conditions” on the optimal costate which must be applied at the entrance and

exit of the constrained arc. In the aerospace engineering literature, state inequality

constraints have historically been handled through index-reduction of the high-index

differential-algebraic equation (DAE) system that results from the state constraint activity

[2]. The necessary conditions for optimality are derived from the calculus of variations

using an approach termed indirect adjoining in which the state inequality constraint is

differentiated before being adjoined to the Lagrangian [7]. Using this approach, Ref. [10]

develops a set of tangency conditions that are enforced at the entrance of a constrained

arc, often leading to discontinuities in the costate. The control along the constrained

arc is then defined by setting to zero the lowest derivative of the inequality constraint

that is an explicit function of the control variable. The costate discontinuities that arise

16

from the necessary conditions for optimality then become a function of the tangency

conditions. In Ref. [13] a modified problem is posed where the original path constraint

is augmented to the cost functional and the tangency conditions are applied at both

the entrance and exit of the constraint activity. The formulation of Ref. [13] leads to a

reduction in the dimension of the state space in the region of active constraint activity. In

[14] a numerical technique for dealing with these problems is developed using steepest

descent.

Another technique for solving state inequality path constrained optimal control

problems is the method of indirect adjoining approach with continuous multipliers

[15]. In this method, the discontinuity in the costate is “subtracted out,” leading to

a set of optimality conditions that yield a continuous costate even if if the solution

lies on a constrained arc [16–19]. Reference [15] summarizes the methods of direct

adjoining, indirect adjoining, and indirect adjoining with continuous multipliers used

in the derivation of the necessary conditions for optimality of a state inequality path

constrained optimal control problem.

Indirect methods are attractive because the solution of the HBVP is an extremal

and thus must satisfy the first-order optimality conditions from the calculus of variations.

Consequently, a solution obtained using an indirect method can be accurate. The

HBVP, however, generally does not have an analytic solution. Therefore, numerical

methods must be employed. Common numerical approaches for solving the HBVP are

shooting, multiple shooting, and collocation [20]. Numerical implementations of Indirect

methods pose a number of computational challenges. First, the first-order optimality

conditions are often difficult to derive. Second, the radius of convergence of the resulting

Hamiltonian boundary value problem can be notably small due to instabilities in the

Hamiltonian dynamics [21]. As a result, an indirect method often requires a good initial

guess for both the state and the costate [2, 7, 9]. However, providing an initial guess for

the costate is often difficult because the costate has no physical interpretation. Finally, in

17

the case when the optimal solution has constrained and unconstrained arcs, it becomes

necessary to estimate the constrained arc sequence [2]. Estimating switches in path

constraint activity is often difficult when no a-priori knowledge of the solution structure is

available.

The second class of numerical methods in optimal control are direct methods.

Different from indirect methods, direct methods parametrize the control and/or the state,

and the continuous-time problem is discretized and transcribed into a finite-dimensional

nonlinear programming problem (NLP). The resulting NLP can then be solved using

well developed optimization software [22–25]. Direct methods have gained a great deal

of popularity as they avoid a number of the pitfalls associated with indirect methods.

Specifically, because a direct method directly transcribes the optimal control problem

into a NLP, the lengthy derivations of the first-order optimality conditions are avoided.

Also, direct methods do not require an initial guess for the costate, and the problem

can be modified relatively easily without having to re-derive the optimality conditions

[2, 26, 27]. Many direct methods, however, are not as accurate as indirect methods and

they require further analysis to verify optimality once a solution is achieved.

Direct methods can employ either a sequential or a simultaneous optimization

approach. In a sequential approach the control is parametrized and the dynamics

are integrated over the trajectory domain. One example of a sequential optimization

method is the direct shooting method [28–30]. In a direct shooting method the

control is parametrized and the dynamics are integrated using numerical integration

methods. Direct shooting methods are useful when the control can be parametrized

using few parameters, keeping the problem size small. As the number of variables

needed to parametrize the control increases, however, convergence to a solution

using direct shooting methods becomes difficult. Direct multiple-shooting methods

improve convergence by subdividing the solution domain into multiple intervals [28]. The

shooting method is then applied in each interval, and continuity of the state is enforced

18

at the interval boundaries. Multiple-shooting methods have better convergence than

shooting methods because the integration of the state dynamics is done over shorter

intervals. Both direct shooting and direct multiple shooting methods, however, are not

computationally efficient due to the sequential numerical integration technique used to

integrate the dynamics. Furthermore, convergence still depends on a-priori knowledge

of the constrained and unconstrained arc sequence.

A particular direct methods known as a collocation method, employ a simultaneous

optimization approach [2, 30–36]. Collocation methods parametrize both the control and

the state, and the differential-algebraic equations are enforced at a set of discrete points

in the domain [2, 26, 37]. Direct collocation methods are attractive because they require

no a priori knowledge of the solution structure [38]. Furthermore, direct collocation

methods are less sensitive to the initial guess than the sequential approach of shooting

methods [2]. Well-known software implementations of direct collocation methods include

SOCS, DIDO, DIRCOL, and GPOPS [39–42].

Direct collocation methods can employ local h-method collocation, or global

p-method collocation. Often the class of Runge-Kutta methods is used to collocate

and integrate the system dynamics [2, 33, 43–45]. Runge-Kutta methods are usually

employed as h-methods in which the solution domain is subdivided into many intervals

and a fixed low-degree approximation is used in each interval. This type of scheme

is computationally efficient as it has a sparse structure that can be exploited by NLP

solvers [46]. Convergence of the numerical discretization using h-methods is then

achieved by increasing the number of intervals in the domain. Due to the polynomial

convergence rate of this kind of scheme, however, h-methods can lead to extremely

large NLP’s [45, 47, 48].

In contrast to local h-methods, global p-methods use a single polynomial to

approximate the state over the entire domain [26, 27, 49]. Convergence in a p-method

is then obtained by increasing the degree of the approximating polynomial. A p-method

19

has the advantage that it converges exponentially for problems for problems whose

solutions are smooth. In the case when the solution is not smooth (as often happens in

the presence of active inequality constraints) the convergence rate is significantly slower.

Furthermore, the NLP arising from a p-method is less sparse than the NLP arising from

an h-method.

This research will employ an hp-method using collocation at the LG and LGR points

[50, 51]. In an hp−method, or variable-order method, the solution domain is divided

into a mesh, and the degree of the approximating polynomial (that is, the number of LG

or LGR collocation points) in each interval is allowed to vary. Using an hp-method it is

possible to divide the problem into intervals such that the solution in each interval is

smooth. Thus convergence is achieved by increasing the degree of the approximating

polynomial in each interval. In this manner it is possible to achieve a high accuracy

solution solution while keeping the NLP smaller than what might be possible using an

h-method.

Over the last decade, one class of direct collocation methods which has risen

to prominence in the numerical solution of optimal control problems is the class of

orthogonal collocation methods [26, 27, 34–36, 42, 49, 52–65].Orthogonal collocation

methods parametrize the state using global polynomials and collocate the differential-algebraic

equations using nodes obtained from a Gaussian quadrature. The three most commonly

used sets of collocation points are Legendre-Gauss (LG), Legendre-Gauss-Radau

(LGR), and Legendre-Gauss-Lobatto (LGL) points. These three sets of points are

obtained from the roots of a Legendre polynomial and/or linear combinations of a

Legendre polynomial and its derivatives. All three sets of points are defined on the

domain [−1, 1], but differ significantly in that the LG points include neither of the

endpoints, the LGR points include one of the endpoints, and the LGL points include

both of the endpoints. In addition, the LGR points are asymmetric relative to the

origin and are not unique in that they can be defined using either the initial point or

20

the terminal point. Although collocation at the LGL points provides state and control

approximations at the endpoints, it was shown by Refs. [64, 65] that the control and

costate approximations using LGL points tends to be innacurate due to a rank-deficient

differentiation matrix. Furthermore Ref. [36] also shows that using collocation at the

LG and LGR points yields a highly accurate approximation to the optimal state, control,

and costate. Because collocation at the LG and LGR points provide similar accuracy

whereas collocation at LGL points can provide erroneous solutions, this research will

focus on using collocation at the LG and LGR points.

When approximating the solution to an optimal control problem using any numerical

method, it is important to analyze the solution in an attempt to verify the convergence of

the discrete problem to a local minima of the continuous-time problem [4, 5, 66, 67]. One

key advantage of an orthogonal collocation method is the elegant transformations of

the KKT conditions of the NLP to the first-order optimality conditions derived analytically

from the calculus of variations [36, 64, 65, 68]. Such transformations have previously

been derived for optimal control problems with no active state inequality path constraints

and when the dynamic constraints are formulated in their differential form. When

available, such transformations show that the first-order optimality conditions of the

discrete NLP are equivalent to the discrete form of the first-order optimality conditions

of the continuous-time optimal control problem derived from the calculus of variations.

Therefore, in this research a gap of costate estimation theory is closed using collocation

at LG and LGR points by deriving a mapping for the costate estimate for the case when

the dynamic constraints are expressed in integral form and in the presence of state

inequality path constraints.

While the LG and LGR methods are equivalent regardless of whether collocation

is performed in either differential or integral form, the differential form of either method

has been predominantly used. Recently, however, more practical work has been done

in implementing both the differential and integral forms of LG and LGR collocation using

21

so called variable-order methods where the time interval is partitioned into a mesh and

mesh refinement techniques are used to determine an appropriate mesh that meets a

specified solution accuracy tolerance [69, 70]. This research indicates strongly that their

may be computational advantages to using the integral form of LG and LGR collocation

over the differential form. In fact, the most current implementation of LGR collocation is

the MATLAB optimal control software GPOPS− II [69]. GPOPS− II uses the integral

form of LGR collocation as the default because it has been found through a variety

of examples that the integral form provides more consistent results. Moreover, the

use of the implicit integral form of LG and LGR collocation is most consistent with the

implementations used by many established optimal control software packages such as

SOCS [39], DIRCOL [41], OTIS [71], ICLOCS [72], and ACADO [73].

While the differential and integral forms of the LG and LGR methods are mathematically

equivalent with regard to the primal variables (that is, the state and control), the

two formulations produce completely different dual variables. In particular, the

relationship between the Lagrange multipliers of the collocation conditions of the

dynamic constraints and the costate of the optimal control problem has been well

documented [27, 36, 64, 65]. On the other hand, the corresponding relationship

between the Lagrange multipliers associated with the integral forms of LG and LGR

collocation and the costate of the optimal control problem has not been established.

When employing the integral forms of LG and LGR collocation, however, it may be of

interest to either verify optimality or perform sensitivity analysis in a manner consistent

with that which would be performed when using variational methods. In such cases it

is useful to obtain a costate estimate when using the integral forms of the LG and LGR

methods.

In this research a methods for estimating the optimal control costate using the

integral forms of LG and LGR collocation is developed. Specifically, transformations

are derived that relate the Lagrange multipliers of the integral forms of the LG and

22

LGR collocation methods to the costate of the original optimal control problem. These

transformations are derived by writing the original continuous-time optimal control

problem in integral form. A new continuous-time dual variable called the integral costate

is then introduced, where the integral costate is the Lagrange multiplier of the integral

dynamic constraint. The first-order optimality conditions of the integral form of the

optimal control problem are derived in terms of the integral costate. The integral form of

the optimal control problem is then discretized using the integral LG and LGR collocation

methods and the relationships between the discrete form of the integral costate and the

costate of the original differential optimal control problem are developed. It is shown

that the LGR integration matrix that relates the differential costate to the integral costate

is singular while the corresponding LG integration matrix is full rank. The approach

developed in this research then provides a way to estimate the costate of the original

optimal control problem using the Lagrange multipliers of the integral form of the LG and

LGR collocation methods.

Next, inequality path constrained optimal control problems are analyzed. Although

previous research has successfully derived a high-accuracy estimate of the costate from

the KKT multipliers of the NLP for the case of a problem with no active state inequality

path constraints, Ref. [1] subsequently showed that in the case when the costate is

discontinuous (as is the case in the presence of active state inequality path constraints),

this costate estimate leads to a set of first-order optimality conditions of the NLP that

are not equivalent to the discrete form of the variational optimality conditions. This lack

of equivalence leads to an inaccurate approximation of the costate. Therefore, in this

research this inaccuracy is rectified by developing a new approach for costate estimation

using the method of indirect adjoining with continuous multipliers [15, 19]. The costate

estimate derived in this research leads to a transformed adjoint system which is a

discrete approximation of the first-order optimality conditions of the continuous-time

problem.

23

The contributions of this research are as follows. First, costate estimates are

derived using collocation at Legendre-Gauss and Legendre-Gauss-Radau points for the

case when the dynamic constraints of the optimal control problem are formulated

in integral form. Second, it is demonstrated that the costate mapping derived for

collocation at the LG and LGR points leads to a set of transformed optimality conditions

of the NLP that are shown to be a discrete representation of the necessary conditions

for optimality of the continuous-time problem. Third, a relationship between the integral

and the differential forms of the costate estimate is given and it is shown that the

two sets of optimality conditions are equivalent. Fourth, a new costate estimate for

collocation at LG and LGR points is derived for problems with active state inequality

path constraints. This costate estimate is shown to lead to a transformed adjoint system

of the NLP which is a discrete approximation of the necessary conditions for optimality

of the continuous-time optimal control problem. Finally, examples are presented that

characterize the accuracy of the costate estimates presented in this research.

24

CHAPTER 2MATHEMATICAL BACKGROUND

In this chapter the mathematical background necessary to understand the scope of

the research is provided. First, a general continuous-time Bolza optimal control problem

is defined and the first-order optimality conditions of this continuous-time Bolza optimal

control problem arising from the calculus of variations are derived. Second, an overview

of methods for solving differential-algebraic equations (DAE) is presented. In particular,

it is shown that optimal control problems with active state inequality path constraints lead

to high-index DAEs which are inherently difficult to solve using numerical methods. A

method of index-reduction is then presented to overcome the numerical difficulties that

arise from high-index DAE systems. Third, various methods are presented to derive the

necessary conditions for optimality of state inequality path constrained continuous-time

optimal control problems. Fourth, methods for transcribing a general continuous-time

optimal control problem to a nonlinear program (NLP) using orthogonal collocation at

Legendre-Gauss and Legendre-Gauss-Radau points are described. Finally, in order

to explain and legitimize the use of orthogonal collocation methods to solve optimal

control problems, a brief background is provided in function interpolation and numerical

integration.

2.1 Continuous-Time Bolza Optimal Control Problem

Without loss of generality, consider the following optimal control problem in Bolza

form. Determine the state, y(t) ∈ Rn, and the control, u(t) ∈ R

m, that minimize the cost

functional

J = Φ(y(tf )) +

∫ tf

t0

g(y(t), u(t))dt (2–1)

subject to the dynamic constraint

dy

dt= y(t) = f(y(t), u(t)) ∈ R

n, (2–2)

25

the boundary condition

φ(y(t0)) = 0 ∈ Rq, (2–3)

and the state and control inequality path constraint

C(y(t), u(t)) ≤ 0 ∈ Rc . (2–4)

The cost functional of Eq. (2–1) consists of a Mayer cost, which is evaluated purely at

the end points of the domain, and a Lagrange, or integral, cost. The optimal control

problem of Eqs. (2–1)–(2–4) will be referred to as the continuous Bolza problem.

2.1.1 First-Order Optimality Conditions

The first-order necessary conditions for an extremal solution of the continuous

Bolza problem can be derived using the calculus of variations [9]. First, using Lagrange

multipliers, the constraints of the optimal control problem are augmented to the cost

functional to generate the augmented cost functional

Ja =Φ(y(tf ))−ψ⊤φ(y(t0)) (2–5)

+

∫ tf

t0

[

g(y(t), u(t))− λ⊤(t)(y(t)− f(y(t), u(t)))− µ⊤(t)C(y(t), u(t))]

dt,

where ψ, λ, and µ are the Lagrange multipliers associated, respectively, with the

boundary conditions of Eq. (2–3), the dynamic constraints of Eq. (2–2), and the

inequality path constraints of Eq. (2–4).

Next, taking the first variation of the augmented cost with respect to all free

variables (i.e., y(t),u(t), ψ,λ(t), and µ(t)), we obtain

δJa =∂Φ

∂y(tf )δyf − δψ⊤φ−ψ⊤

[

∂φ

∂y(t0)δy0

]

+

∫ tf

t0

[

∂g

∂yδy +

∂g

∂uδu

−δλ⊤(y− f) + λ⊤

(

δfδy

δy+δfδu

δu− δy

)

− δµ⊤C− µ⊤

(

δCδy

δy+δCδu

δu

)]

dt. (2–6)

26

The term λ⊤δy in Eq. (2–6) can be integrated by parts as follows

∫ tf

t0

λ⊤δydt = λ⊤(tf )δy(tf )− λ⊤(t0)δy(t0) +

∫ tf

t0

λ⊤δydt. (2–7)

Applying the relationship of Eq. (2–7) to Eq. (2–6), the first variation of the augmented

cost can then be rewritten as a function of the augmented Hamiltonian

H(y, u,λ,µ) = g + λ⊤f− µ⊤C (2–8)

as

δJa = δψ⊤φ+

(

−ψ⊤ ∂φ

∂y(t0)+ λ⊤(t0)

)

δy(t0) +

(

∂Φ

∂y(tf )− λ⊤(tf )

)

δy(tf )

+

∫ tf

t0

[(

∂H

∂y+ λ

)

δy+

(

∂H

∂u

)

δu− δλ⊤(y− f)− δµ⊤C]

dt.

An extremal solution will satisfy the condition δJa = 0. Because the variations of the

free variables are not zero, the only way to obtain an extremal solution is to satisfy the

following set of first-order optimality conditions:

y =f(y, u), 0 = φ(y(t0)), (2–9)

λ⊤(t0) =ψ⊤ ∂φ

∂y(t0), (2–10)

λ⊤(tf ) =∂Φ

∂y(tf ), (2–11)

∂H

∂u=∂g

∂u+ λ⊤ ∂f

∂u− µ⊤∂C

∂u= 0, (2–12)

∂H

∂y=∂g

∂y+ λ⊤ ∂f

∂y− µ⊤∂C

∂y= −λ⊤

, (2–13)

µ ≤0, µ⊤S(y) = 0. (2–14)

2.1.2 First-Order Optimality Conditions of Integral Formulation

The continuous Bolza problem given by Eqs. (2–1)–(2–4) can be reformulated such

that the dynamic constraint of Eq. (2–2) are written in integral form. Reformulating the

problem in this way will be of interest to this research so that a relationship between the

27

Lagrange multipliers of the integral form can be related to the Lagrange multipliers of the

original differential form. In integral form, the optimal control problem is to determine the

state, y(t) ∈ Rn, and the control, u(t) ∈ Rm, that minimize the cost functional

J = Φ(y(tf )) +

∫ tf

t0

g(y(t), u(t))dt (2–15)

subject to the integral constraint

y(t) = y(t0) +

∫ t

t0

f(y(t), u(t)) dt, (2–16)

and the boundary condition

φ(y(t0)) = 0 ∈ Rq. (2–17)

The optimal control problem given by Eq. (2–15), along with the constraints of

Eqs. (2–16) and (2–17), will be referred to as the integral Bolza Problem.

The first-order necessary conditions for an extremal solution of the integral Bolza

problem can again be derived through the calculus of variations. First, the constraints of

Eqs. (2–16) and (2–17) are augmented to the cost such that

Ja = Φ(tf ) +ψ⊤φt0 +

∫ tf

t0

(

g(y, u)− p⊤(

y− y(t0)−∫ τ

t0

f(y, u) dt

)

dτ , (2–18)

where p(t) and ψ are the Lagrange multipliers associated with the dynamic constraints

of Eq. (2–16) and the boundary conditions of Eq. (2–17), respectively. Next, the first

variation is taken with respect to all free variables (y, u, p, and ψ), such that

δJa =∂Φ

∂y(tf )δyf −ψ⊤

[

∂φ

∂y(t0)δy0

]

− δψ⊤φ+

∫ tf

t0

[

∂g(y, u)

∂yδy

+∂g(y, u)

∂uδu− δp⊤

(

y − y(t0)−∫ t

t0

f(y, u) dτ

)

− p⊤δy

+p⊤δy(t0) +

∫ tf

t

p⊤ dτ · ∂f(y, u)∂y

δy+

∫ tf

t

p⊤ dτ · ∂f(y, u)∂u

δu

]

dt.

(2–19)

28

It is noted that the following relationship was used in Eq. (2–19):

∫ tf

t0

[

q(t)

∫ t

t0

p(τ) dτ

]

dt =

∫ tf

t0

[

p(t)

∫ tf

t

q(τ) dτ

]

dt. (2–20)

Furthermore, note that the variation of the final state is not independent, but depends on

the variations of the initial states and the state and control at intermediate points. Thus,

the variation of the final state is given as

δyf = y0 +

∫ +1

−1

[

∂f

∂yδy +

∂f

∂uδu

]

dt. (2–21)

Eq. 2–21 can be substituted into Eq. (2–19) to obtain an expression for the first variation

of the cost with respect to the independent variables.

An extremal solution will satisfy the condition δJa = 0. Because the variations of the

free variables are not zero, the only way to obtain an extremal solution is by satisfying

the following set of first-order optimality conditions:

y = y(t0) +

∫ t

t0

f(y, u) dt, φ(y(t0)) = 0, (2–22)

0 =∂g

∂u+

(∫ tf

t

p⊤ dτ +∂Φ

y(tf )

)

· ∂f(y, u)∂u

, (2–23)

p⊤ =∂g

y+

(∫ tf

t

p⊤ dτ +∂Φ

∂y

)

· ∂f(y, u)∂y

, (2–24)

ψ⊤ ∂φ

y(t0)=

∫ tf

t0

p⊤ dt +∂Φ

∂y(tf ). (2–25)

2.1.3 Control Inequality Path Constraints

Now consider an optimal control problem with an active control inequality path

constraint. When a control constraint is inactive, the optimal control is determined using

the strong form of Pontryagin’s Minimum Principle, given by Eq. (2–12) [7, 9]. When the

inequality constraint is active, however, a subset of the optimal control is determined by

the relation

Ck(y(t), u(t)) = 0, t ∈ [t1, t2], (2–26)

29

where the subscript k denotes the subset of active constraints in the time interval

[t1, t2] ⊆ [t0, tf ].

A special case of an active control inequality constraint is one that results in a bang-

bang control. If the control appears linearly in the Hamiltonian defined by Eq. (2–8),

the strong form of Pontryagin’s Minimum Principle given by Eq. (2–12) provides no

information about the optimal control, and the weak form of Pontryagin’s Minimum

Principle must be used instead [9]. Denoting u∗ as the control that minimizes the cost

functional of Eq. (2–1), by definition the following must hold true:

J(u)− J(u∗) ≥ 0,

for all trajectories with the admissible control u sufficiently close to u∗. Furthermore, for

small variations around the optimal trajectory

J(u)− J(u∗) = δJ(u∗, δu) ≥ 0.

It is known that the variation in the cost is related to the Hamiltonian by

δJ(u∗, δu) =

∫ tf

t0

[

∂H

∂u

]

u∗δudt,

δJ(u∗, δu) =

∫ tf

t0

[H(y∗, u∗ + δu,λ∗)− H(y∗, u∗,λ∗)] dt.

Therefore,

H(y∗, u∗ + δu,λ∗)−H(y∗, u∗,λ∗) ≥ 0,

H(y∗, u∗ + δu,λ∗) ≥H(y∗, u∗,λ∗),

for all admissible variations in u. From this discussion it can be concluded that the

optimal control minimizes the Hamiltonian given in Eq. (2–8). This optimality condition is

called the weak form of Pontryagin’s Minimum Principle and is given as

u∗ = argminuH(y∗, u,λ∗). (2–27)

30

Active control inequality path constraints may cause finite discontinuities in the

control at the entering and exit corners of the constraint activity. Activity in these control

constraints, however, only produces discontinuities in the time derivative of the state and

costate; and in general the state and costate themselves are continuous across such

corners. Furthermore, the Hamiltonian will also be continuous in the presence of these

constraints.

2.1.4 State Inequality Path Constraints

Of primary interest in this research is the case when the inequality path constraint

of Eq. (2–4) is purely a function of the state and the independent variable (that is, the

inequality path constraint is not a function of the control) such as

S(y(t)) ≤ 0, t ∈ [t0, tf ]. (2–28)

For those time intervals where an extremal solution lies on the constraint boundary (and

contrary to the case of a control inequality path constraint), the optimal control cannot be

determined by the active constraint because it is not an explicit function of the control.

While on the constraint boundary, the DAEs describing the system will then have the

form

y(t) =f(y(t), u(t)), (2–29)

0 =Sk(y(t)), t ∈ [t1, t2], (2–30)

where k denotes the active constraint in the time interval [t1, t2] ⊆ [t0, tf ]. Equation (2–30)

is an algebraic constraint that, when satisfied, removes a degree of freedom from the

differential equations defined by Eq. (2–29). Removal of this degree of freedom results

in what the DAE literature refers to as a high-index differential-algebraic equation. In

general, high-index DAE are difficult to solve numerically. Methods for solving high-index

DAEs will be discussed in the following section.

31

Active state inequality path constraint will cause trouble not only when solving an

optimal control problem numerically, but also when solving an optimal control problem

analytically. The constraint activity introduces additional unknowns that cannot be

determined by applying the first-order optimality conditions of Eqs. (2–9)–(2–13).

Specifically, the additional unknowns include the times of the entrance and exit of the

constrained arc and the path constraint multiplier µ(t). Therefore, when solving these

problem analytically, the first-order optimality conditions derived in the previous section

must be modified to account for the constrained arcs. Active state inequality path

constraints will often produce discontinuities in the costate and the Hamiltonian at the

entrance and/or exit of the constraint activity. Furthermore, it is often the case that the

time derivative of the state will be discontinuous at the corners of a constrained arc. The

state and control, however, will generally not be discontinuous even in the presence of

active state inequality path constraints.

2.2 Differential-Algebraic Equations

The numerical solution of DAEs can be far more complicated than the numerical

solution of ordinary differential equations (ODEs). The accuracy of a numerical

method for the solution of a DAE depends upon on the DAE’s solvability, index, and

the consistency of initial conditions [3, 4]. The most general representation of a DAE is

given in the nonlinear implicit form

F(y(t), y(t), u(t)) = 0. (2–31)

The DAE is said to be solvable if a family of unique solutions exists locally. Furthermore,

the index of the DAE is defined as the minimum number of times that all or subsets of

the DAE given by Eq. (2–31) need to be differentiated with respect to time in order to

determine y(t) as a continuous function of the state and control. It is noted that the DAE

index can vary along a solution trajectory of a nonlinear DAE. Finally, the consistency

32

of initial conditions for a DAE system is defined by a set of initial conditions (y0, y0) that

satisfies the extended system (2–31) at time t0.

In order to better understand the difference between a DAE and an ODE, suppose

the DAE given by Eq. (2–31) has the form

F(y, y, u) = 0 = A(t)y+ B(t)y+D(t)u+ e(t). (2–32)

The Jacobian of Eq. (2–32) is defined as

∂F∂y= A(t). (2–33)

If the matrix A(t) is full rank, it is possible to solve Eq. (2–32) for the state as

[A(t)]−10 = [A(t)]−1 (A(t)y+ B(t)y+ D(t)u+ e(t)) ,

y = −[A(t)]−1B(t)y− [A(t)]−1D(t)u− [A(t)]−1e(t), (2–34)

Equation (2–34) is an ordinary differential equation. Therefore, in general, if the system

Jacobian given in Eq. (2–33) is invertible, Eq. (2–31) can be transformed into an ODE of

the form y = f(y, u). If, however, the Jacobian of Eq. (2–33) is singular, then Eq. (2–31)

is a differential-algebraic system which can be written in the semi-explicit form

y(t) =f(y(t), u(t)), (2–35)

0 =C(y(t), u(t)). (2–36)

Furthermore, when the DAE system is in the semi-explicit form of Eqs. (2–35)–(2–36)

and the matrix

G =∂C∂u

is singular, then the system is said to be a high-index DAE system.

One way to understand the solution of a DAE system, and why high-index systems

can be problematic, is by viewing the algebraic Eq. (2–36) as a way of “eliminating” the

control variable such that a standard ODE integration method can be used to obtain

33

a numerical solution. In that case one degree of freedom is removed from the ODEs

for each algebraic equation that is satisfied. For instance, using Newton’s method,

Eq. (2–36) can be solved iteratively for the control, u, as

uk = uk −G−1C, (2–37)

where uk is the new approximation at the k th iterative step. The resulting control could

then be substituted into the ODE of Eq. (2–35) such that

y(t) = f(y, u).

This last ODE could then be solved by employing available numerical ODE solvers.

Although this approach would be time consuming and is not a practical way of obtaining

a solution, it does illustrate the computational challenges associated with high-index

constraints. Namely, if C−1u is rank deficient then not only can the operation given by

Eq. (2–37) not be performed, but the algebraic constraints will also not uniquely specify

all of the degrees of freedom of the system.

2.2.1 Index-Reduction for Differential-Algebraic Equations

Differential Algebraic Equations of index at most one can generally be solved

numerically using methods developed for the solution of ODEs. For systems of index

higher than one, however, such methods may have poor convergence, may converge

to the wrong solution, or may not converge at all [3]. Two general approaches accepted

in the DAE literature exist for obtaining numerical solutions to these types of high-index

systems. The first is through the use of presently available numerical methods and

codes that are modified ODE solvers designed specifically for high-index DAE systems

such as backward-differentiation formulas (BDF). The second is through index reduction

of the system by symbolic manipulation of the DAE equations.

In the scope of this research, it is desired to not only find a numerical solution

to the DAE system but also perform the optimization described by Eqs. (2–1)–(2–4).

34

This optimization is done by transcribing the optimal control problem into a nonlinear

programming problem (NLP). Therefore, specialized codes for solving high-index

systems are not desirable as they would result in a computationally inefficient NLP

structure. Furthermore, the solution structure of the resulting optimal control problem

will often consist of both constrained and unconstrained arcs, making the DAE index

fluctuate throughout the solution. It is desirable, however, to have one DAE solver for

both constrained and unconstrained segments of the trajectory. For these reasons, the

method of index reduction might be preferred over specialized approaches for solving

the high-index DAE that result from state inequality constrained problems.

The method of index reduction is formulated as follows. Given the DAE

y(t) =f(y(t), u(t)), (2–38)

0 =S(y(t)), (2–39)

it is clear that the matrix

G =∂S∂u

is rank deficient. Therefore the system given by Eqs. (2–38)–(2–39) is a high-index DAE

system. Now if S = 0, then it must also be true that all its time derivatives are zero.

Therefore, Eq. (2–39) can be differentiated with respect to time such that

0 =∂S∂yy +

∂S∂uu,

0 =∂S∂yf(y, u) +

∂S∂uu.

(2–40)

This differentiation and back-substitution procedure can be repeated until the control

appears explicitly in the algebraic constraints. If r time derivatives are needed for the

derivative of the constraint with respect to the control

G(r) =∂rS∂ur

35

to be full rank, then the constraints are said to have index r. Once G(r) is full rank, the

DAE system of Eqs. (2–38)–(2–39) can be rewritten equivalently as

y(t) =f(y(t), u(t)),

u(t) =[G(r)]−1(

∂S∂yf(y, u)

)

.(2–41)

Equation (2–41) is an ODE system that can be solved using well-known algorithms.

Therefore, in general, the DAE index is the minimum number of times that the original

constraints must be differentiated with respect to time in order to obtain an ODE.

In the context of dynamic optimization, index reduction is used only to the point

where the optimal control can be explicitly defined by the active algebraic constraint.

Therefore, one less time derivative need be taken, such that the resulting DAE is defined

by

y(t) =f(y(t), u(t)),

0 =dq

dtqSk(y(t), u(t)),

(2–42)

where k denotes the active constraint. In the dynamic optimization literature, the

algebraic constraint given in Eq. (2–42) is termed a qth order constraint. Therefore, in

general, the order of a state inequality path constraint is the minimum number of times

that constraint must be differentiated with respect to time before the control appears

explicitly in the expression. Furthermore, the order of a constraint and the index of a

DAE are related by the expression q = r − 1. Finally, as the algebraic constraints are

differentiated, the intermediate q − 1 time derivatives are not discarded. Instead, they

are evaluated at the initial time t0 and used as a set of consistent initial conditions for the

new DAE given by Eq. (2–42):

S(y(t))

d

dtS(y(t))

...

d (q−1)

dt(q−1)S(y(t))

t=t0

= 0. (2–43)

36

2.2.2 Gaussian Quadrature Collocation Method for Solutions of High-Index DAEs

Suppose the state y from Eqs. (2–35)–(2–36) is approximated as

y(τ) ≈ Y(τ) =N+1∑

i=1

YiLi(τ), (2–44)

where τi , (i = 1, ... ,N + 1) are the discretization points in the domain τ ∈ [−1,+1],

Yi ∈ Rn is a row vector of the state approximated at τi , and the Lagrange polynomials

Li(τ) are defined as

Li(τ) =N+1∏

j=1

j 6=i

τ − τjτi − τj

. (2–45)

It is noted that the domain t ∈ [t0, tf ] can be transformed to τ ∈ [−1,+1] through the

affine transformation

t =tf − t02

τ +tf + t02. (2–46)

The time derivative of the state approximation given in Eq. (2–44) is then given as

y(τ) ≈ Y(τ) =N+1∑

i=1

Li(τ)Yi . (2–47)

While any set of points τi can be used as support points for the state approximation,

it has been shown that non-uniform discretization points obtained from the roots of

orthogonal polynomials such as Chebyshev or Legendre polynomials will minimize

the interpolation error associated with Runge phenomenon. In this research, the

Legendre-Gauss-Radau (LGR) points plus the initial point −1 are used as the support

points. Applying the time derivative of Eq. (2–47) at the N LGR points, (τ1, ... , τN), gives

y(τj) ≈ Y(τj) =N+1∑

i=1

Lj(τi)Yi =N+1∑

i=1

DjiYi , (j = 1, ... ,N), (2–48)

where Dji , (j = 1, ... ,N); (i = 1, ... ,N +1) is a (N)× (N + 1) matrix of known coefficients

known as the LGR differentiation matrix.

37

The DAE given by Eqs. (2–35)–(2–36) can then be approximated by the set of

algebraic equations

N+1∑

i=1

DjiYi −tf − t02f(Yj ,Uj) =0, (j = 1, ... ,N), (2–49)

C(Yj ,Uj) =0, (j = 1, ... ,N), (2–50)

where Uj ∈ Rm is a row vector of the control approximation at τj . Eqs. (2–49)–(2–50) are

a set of nonlinear equations that can be solved for the unknowns Yi , (i = 1, ... ,N + 1),

Uj , (j = 1, ... ,N) using known numerical methods.

Example: In order to illustrate the benefits of index reduction, consider the following

simple example where a double integrator has its state constrained:

ProblemA

x(t) =v(t),

v(t) =u(t),

x(t) =ℓ,

(2–51)

where (x , v) is the state, u is the control and t ∈ [−1,+1]. Because the algebraic

constraint must be differentiated twice with respect to time before the control appears

explicitly in the expression, Eq. (2–51) is an example of an index-3 DAE and a second-

order constraint. Index reduction results in the system

ProblemB

x(t) =v(t) , x(−1) =ℓ,

v(t) =u(t) , v(−1) =0,

u(t) =0,

(2–52)

which is equivalent to the original system given in Eq. (2–51). The analytic solution to

the DAE in Eq. (2–51) is given as

x(t) = ℓ, v(t) = 0,

u(t) =0.

(2–53)

38

Because the analytic solution to Eq. (2–51) is constant in the entire domain, it is

reasonable to expect small errors in the solution using low-degree polynomial approximations

(that is, a small number of collocation points).

ag

2 4 6 8 10 12 14 16 18

-2

-4

-6

-8

-10

-12

-14

-16

-18

Number of Collocation Points

log10

Abs

olut

eE

rror

inx(t)

Problem AProblem B

Figure 2-1. Base ten logarithm of the maximum absolute error in x(t) as a function ofthe number of LGR collocation points for problems A and B.

Both problems A and B, defined by Eqs. (2–51) and (2–52), respectively, were

solved using the method described by Eqs. (2–49)–(2–50). Figure 2-1 shows the

base ten logarithm of the maximum absolute error in the x(t) component of the state

as a function of the number of LGR collocation points. It can be seen that using the

formulation given in Eq. (2–52) results in much smaller errors as compared with

the formulation in Eq. (2–51). Furthermore, Eq. (2–51) requires a large number of

collocation points to achieve a reasonable accuracy tolerance of 10−8, whereas the

numerical solution of Eq. (2–52) has a much smaller error of approximately 10−16

regardless of the degree of the approximating polynomial. The errors found for the

second component of the state, v(t), and the control, u(t), are shown in Table 2-1. From

these results it can be seen that the disparity in the accuracy of the solution between

39

the formulations of Eqs. (2–51) and (2–52) is even larger for the variables that are

not explicitly defined by the algebraic constraint in the original problem formulation.

Furthermore, it can be seen that an accuracy of O(10−5) can still be attained without

performing index-reduction. Therefore it can be concluded that index reduction greatly

reduces the numerical error when solving high-index DAE systems, but when no

index-reduction is performed it is still possible to achieve a reasonable level of accuracy

when solving high-index DAE of order three.

Table 2-1. Absolute maximum error in v(t) and u(t) for problems A and B.

Number of LGR Pointsv(t) Max Absolute Error u(t) Max Absolute ErrorProblem A Problem B Problem A Problem B

3 3.38× 10−3 0 7.97× 10−3 06 1.29× 10−4 0 9.79× 10−4 09 1.80× 10−5 0 2.57× 10−4 0

12 5.49× 10−6 0 8.08× 10−5 015 2.57× 10−6 0 3.98× 10−5 018 1.13× 10−6 0 2.57× 10−5 0

2.3 State Inequality Path Constrained Optimal Control Problems

From the discussion of Section 2.2, it is clear that optimal control problems with

active state inequality constraints lead to high-index DAEs and are difficult to solve

numerically. Furthermore, it was shown that when solving these problems analytically

the first-order optimality conditions of Section 2.1 must be modified in order to account

for the extra unknowns introduced by the constraint activity. Many methods are available

in the literature for deriving the necessary conditions for optimality of a state-inequality

constrained problem. Of these methods, three of them will be discussed here [7, 8, 15].

The first method is called indirect adjoining [15]. In indirect adjoining the index-reduction

of the high-index system of DAEs is taken into account, as first derived by [7]. Indirect

adjoining results in a costate that has discontinuities at the entrance of the constraint

activity. The second method presented for solving state inequality path constrained

problems is called direct adjoining [15]. In direct adjoining, the state inequality path

constraint is directly augmented to the cost and the first-order optimality conditions are

40

derived. Using direct adjoining the costate may be discontinuous at the entrance or

exit of a constrained arc because of jumps in the state constraint multiplier. The third

method presented is called indirect adjoining with continuous multipliers [15]. In indirect

adjoining with continuous multiplers, the costate discontinuity is “subtracted out” from

the costate dynamics, yielding a costate that is continuous even in the presence of state

constraint activity.

Consider again the optimal control problem of Section 2.1, restated here such

that the inequality path constraint is a function of only the state. Determine the state,

y(t) ∈ Rn, and the control u(t) ∈ Rm, that minimize the cost functional

Φ(y(tf )) +

∫ tf

t0

g(y(t), u(t))dt (2–54)


y(t) = f(y(t), u(t)), (2–55)


φ(y(t0)) = 0, (2–56)

and the state inequality path constraint

S(y(t)) ≤ 0. (2–57)

2.3.1 Indirect Adjoining

The first-order optimality conditions of the state inequality path constrained problem

of Eqs. (2–54)–(2–57) are now derived by applying the method of indirect adjoining.

It was previously shown in Section 2.2.1 that high-index DAE are difficult to solve

numerically. Thus, in the indirect adjoining method of deriving the first-order optimality

conditions, index reduction is performed on the high-index DAE system that results from

the state constraint activity, resulting in a modified optimal control problem formulation.

In order to simplify the analysis presented here, a scalar inequality path constraint is

41

considered. Generality is not lost, however, because index reduction can be applied

to a vector inequality path constraint by considering each component individually.

Furthermore, it is assumed the state constraint is active in an interval [t1, t2] such that

S(y(t)) = 0 ∈ [t1, t2] ⊆ [t0, tf ]. On the constrained arc, the state constraint must be

satisfied such that

S(y(t)) = 0, t ∈ [t1, t2]. (2–58)

Because Eq. (2–58) must be satisfied on the optimal solution, all derivatives of the

path constraint in [t1, t2] must also be zero. Performing index-reduction as described in

Section 2.2.1 , Eq. (2–58) is differentiated q times, where q is the lowest derivative of

S that is an explicit function of the control. The intermediate time derivatives are then

defined as

π(y(t)) ≡

S(y(t))

d

dtS(y(t))

...

d (q−1)

dt(q−1)S(y(t))

, t ∈ [t1, t2]. (2–59)

These intermediate time derivatives are evaluated at the entrance of the constrained

arc, t1, and act as consistent initial conditions for the modified DAE. Reference [7]

denotes these constraints as tangency conditions. Physically, the interpretation of these

conditions is that since the constraint function is controllable only by changing its qth

time derivative, there does not exist a finite control for which the system will remain on

the constraint boundary unless the tangency conditions are also satisfied [10].

Consequently, the state inequality path constraint given by Eq. (2–57) can be

replaced by the tangency conditions along with the following state and control inequality

path constraint:

d (q)

dt(q)S(y(t), u(t)) = C(y(t), u(t)) ≤ 0, t ∈ [t1, t2]. (2–60)

42

The optimal control problem of Eqs. (2–54)–(2–57) can then be modified as follows to

account for the DAE index reduction as follows. Minimize the cost functional

Φ(y(tf )) +

∫ tf

t0

g(y(t), u(t))dt (2–61)


y(t) = f(y(t), u(t)), (2–62)


φ(y(t0)) = 0, (2–63)

the tangency conditions

π(y(t1)) = 0, (2–64)


C(y(t), u(t)) ≤ 0, t ∈ [t1, t2]. (2–65)

The first-order optimality conditions of the modified problem of Eqs. (2–61)–(2–65) can

be obtained using the calculus of variations as previously described in Section 2.1. They

are given as the original constraints of Eqs. (2–62)–(2–65) along with the conditions

0 =∂H(y, u, λ, µ)

∂u, (2–66)

˙λ⊤ = −∂H(y, u, λ, µ)∂y

, (2–67)

λ⊤(t0) = ψ⊤ ∂φ

∂y(t0), (2–68)

λ⊤(t+1 ) = λ

⊤(t−1 ) + η

⊤(t1)∂π

∂y(t1), (2–69)

λ(t+2 ) = λ(t−2 ), (2–70)

λ⊤(tf ) =

∂Φ

∂y(tf ), (2–71)

S(y) ≤ 0, µ ≤ 0, µ⊤S(y) = 0. (2–72)

43

In the conditions of Eqs. (2–66)–(2–72), λ(t) ∈ Rn is the costate, µ(t) ∈ R is the

Lagrange multiplier associated with the path constraint of Eq. (2–65), ψ ∈ Rp is the

Lagrange multiplier associated with the boundary condition of Eq. (2–63), η ∈ Rq is

the Lagrange multiplier associated with the tangency conditions of Eq. (2–64), and the

augmented Hamiltonian is defined as

H = g(y, u) + λ⊤

f(y, u)− µ⊤C(y, u). (2–73)

The aforementioned approach is called indirect adjoining because the qth derivative of

the state constraint is adjoined to the Hamiltonian rather than the state constraint itself.

It can be seen that by applying these conditions, active state inequality path

constraints may lead to a discontinuous costate at the entrance, but not at the exit, of

the constrained arc, as can be seen by Eqs. (2–95) and (2–96). Furthermore, note that

the costate discontinuities at the entrance of the constrained arc are quantified by the

costate jump conditions of Eq. (2–95). Thus, Eq. (2–95) acts as terminal conditions for

the interval preceding the constrained arc, and the optimal control problem can be seen

as a three-point boundary value problem.

Although the method of indirect adjoining offers the obvious benefits of index-reduction,

as stated in Section 2.2.1, in practice this technique may be cumbersome to apply.

In particular, index-reduction requires a reformulation of the problem in which the

derivatives of the state inequality path constraint must be taken analytically. Furthermore,

the solution structure of the problem (that is, the sequence of constrained and

unconstrained arcs) must be known a priori. Because solutions to optimal control

problems are often obtained through an automated process (such as a mesh-refinement

technique), indirect adjoining is an impractical solution method.

It was seen in Section 2.2.1 that it is still possible to obtain an accuracy of O(10−5)

when estimating solutions to high-order DAE of up to order three. Because most

aerospace engineering applications generally do not formulate state inequality path

44

constraints of order higher than two, it is reasonable to explore alternate methods of

solving state inequality path constrained optimal control problems that do not involve

index-reduction. In Sections 2.3.2 and 2.3.3 two such methods will be discussed:

the method of direct adjoining and the method of indirect adjoining with continuous

multipliers.

2.3.2 Direct Adjoining

Using the method of direct adjoining, the first-order optimality conditions of the state

inequality path constrained problem of Eqs. (2–54)–(2–57) can be derived in the same

manner as was done in Section 2.1. These conditions are given as

y =f(y, u), 0 = φ(y(t0)), (2–74)

0 =∂H(y, u,λ,µ)

∂u, (2–75)

λ⊤=∂H(y, u,λ,µ)

∂y, (2–76)

λ⊤(t0) =ψ⊤ ∂φ

∂y(t0), (2–77)

λ⊤(tf ) =∂Φ

∂y(tf ), (2–78)

S(y) ≤0, µ ≤ 0, µ⊤S(y) = 0, (2–79)

where the augmented Hamiltonian is given as

H(y, u,λ,µ) = g(y, u) + λ⊤f(y, u)− µ⊤S(y). (2–80)

The first-order optimality conditions of Eqs. (2–74)–(2–79) are necessary conditions

for optimality of a pure state inequality path constrained problem. They are, however,

insufficient to fully determine an extremal solution. In particular, the state constraint

multipliers given by Eq. (2–79) may be discontinuous at the entrance and exit of

a constrained arc. Because the state constraint multipliers and the costate are

related through the costate dynamics of Eq. (2–76), the discontinuities in the state

constraint multiplier will in turn cause discontinuities in the costate. To quantify these

45

discontinuities, the costate dynamics of Eq. (2–76) can be integrated as

λ⊤(t+1 ) = λ⊤(t0)−

∫ t1

t0

[

∂g(y, u)

∂y+ λ⊤(t)

∂f(y, u)∂y

]

dt +

∫

(t0,t1]

µ⊤(t)∂S(y)∂ydt, (2–81)

where t0 ≤ t1 ≤ tf . Now, define the variable ν(t) = −µ(t). The integral of Eq. (2–81)

can then be re-written as

λ⊤(t+1 ) = λ⊤(t0)−

∫ t1

t0

[

∂g(y, u)

∂y+ λ⊤(t)

∂f(y, u)∂y

]

dt −∫

(t0,t1]

∂S⊤(y)

∂ydν(t). (2–82)

Because ν(t) is a function of a bounded variation, it can be decomposed uniquely as

ν(t) = ν1(t) + ν2(t), (2–83)

where ν1(t) is absolutely continuous with respect to t and ν2(t) is singular with

respect to t. Therefore, the second integral on the right-hand side of Eq. (2–82) can

be expressed as

∫

[t0,t1]

∂S⊤(y)

∂ydν(t) =

∫ t1

t0

∂S⊤(y)

∂yν1(t)dt +

∫

(t0,t1]

∂S⊤(y)

∂ydν2(t). (2–84)

Also, ν(t) is monotone so that it can have at most countably many jumps. Moreover,

it is reasonable to assume that ν(t) is sufficiently well behaved to have a piecewise

continuous derivative. In that case the second integral on the right-hand side of

Eq. (2–84) can be expressed as

∫

[t0,t1]

∂S⊤(y)

∂ydν2(t) =

∑

τi∈(t0,t1]

∂S⊤(y(τi))

∂y[ν(τ+1 )− ν(τ−i )], (2–85)

where τi are the points of discontinuity of ν(t). Next, defining the costate discontinuity

as

η(τ) = ν(τ−)− ν(τ+), (2–86)

the integral of Eq. (2–81) can be expressed as

λ⊤(t+1 ) = λ⊤(t0)−

∫ t1

t0

[

∂g(y, u)

∂y+ λ⊤(t)

∂f(y, u)∂y

− µ⊤(t)∂S(y)∂y

]

dt+∑

τi∈(t0,t1]

η(τ)⊤∂S(y(τi))

∂y.

46

Therefore, the costate dynamics can be expressed as

− λ⊤=

∂g

∂y+ λ⊤ ∂f

∂y− µ⊤∂S

∂y. (2–87)

Furthermore, let t = τ denote an entry time into a constrained arc, an exit time from

a constrained arc, or a single contact point in which S(y(τ)) = 0. Then the costate

trajectory may have a discontinuity given by the following jump conditions

λ⊤(τ+) = λ⊤(τ−) + η(τ)⊤∂S(y(τi))

∂y. (2–88)

Because ν(t) ≥ 0, the function ν(t) is non-decreasing in the solution domain,

and thus η(k) ≤ 0. Furthermore, the condition 〈η(k),S(Y(Tk))〉 = 0 must hold

true. Thus, using direct adjoining, the first-order optimality conditions for the state

inequality path constrained optimal control problem of Eqs. (2–54)–(2–57) are given by

Eqs. (2–74)–(2–76) along with the costate discontinuity conditions of Eq. (2–88).

The first-order optimality conditions derived here are very similar to the conditions

derived in Section 2.3.1. However, it is noted that in the conditions derived in Section

2.3.1 the qth derivative of the state inequality path constraint is enforced on the

constrained arc, whereas the original undifferentiated path constraint was enforced

in the problem formulation of this section. Moreover, the optimality conditions stated in

Section 2.3.1 normalize the jump conditions such that the costate is discontinuous only

at the entry time of the constrained arc, while the conditions stated in this section make

no such distinction.

2.3.3 Indirect Adjoining With Continuous Multipliers

The third method for deriving the necessary conditions for optimality of a state

inequality path constrained optimal control problem is the method of indirect adjoining

with continuous multipliers. Using the method of indirect adjoining with continuous

multipliers yields a costate that is continuous even in the presence of state inequality

path constraints. Because discontinuities are difficult to approximate numerically, the

47

method of indirect adjoining with continuous multipliers offers an advantage over the

methods of direct and indirect adjoining which approximate a discontinuous costate.

The first-order optimality conditions for the state inequality path constrained optimal

control problem given by Eqs. (2–54)–(2–57) are now derived by using the method of

indirect adjoining with continuous multipliers. Similar to the approach used in Section

2.3.2, the costate dynamics of Eq. (2–76) can be integrated to give

λ⊤(t+1 ) = λ⊤(t0)−

∫ t1

t0

[

∂g(y, u)

∂y+ λ⊤(t)

∂f(y, u)∂y

]

dt +

∫

(t0,t1]

µ⊤(t)∂S(y)∂ydt,

where t0 ≤ t1 ≤ tf . Now, let ν(t) = −µ(t). Furthermore, because µ(t) ≤ 0, ν(t) must

be a non-decreasing function of time. The costate discontinuity can now be “subtracted”

by defining a new costate p(t) such that

p⊤(t) = λ⊤(t) + ν⊤(t)∂S(y)∂y. (2–89)

The Hamiltonian can be defined in terms of the continuous costate p(t) such that

H(y, u, p,ν) = g(y, u) + p⊤f(y, u)− ν⊤S(y, u). (2–90)

It can be seen that this expression can be decomposed to the Hamiltonian by substituting

the relationship

S(y) ≡ ∂S(y)

∂y· y = ∂S(y)

∂y· f(y, u).

into Eq. (2–90) such that

H(y, u,λ) = g(y, u) + λ⊤f(y, u)←→ H(y, u, p,ν). (2–91)

48

The first-order optimality conditions are then given in terms of the continuous

costate as Eqs. (2–55)–(2–57) along with the conditions

0 =∂H(y, u, p,ν)

∂u, (2–92)

p⊤ = −∂H(y, u, p,ν)∂y

, (2–93)

p⊤(t0) = ψ⊤ ∂φ

∂y(t0)+ ν(t0)

⊤∂S(y(t0))∂y

, (2–94)

λ⊤(tf ) =∂Φ

∂y(tf )+ ν(tf )

⊤∂S(y(tf ))∂y

. (2–95)

ν(tf ) ≤ 0, ν ≥ 0, C(y) ∈ N (ν), (2–96)

Furthermore, let C(Rq) denote the space of continuous functions mapping [t0, tf ] to Rq.

Assuming ν is Lipschitz continuous and non-decreasing with ν(tf ) ≤ 0, the set-valued

map N (ν) is defined as

N (ν) = {z ∈ C(Rq) : z ≤ 0, ν⊤z = 0,ν⊤(tf )z(tf ) = 0}.

2.4 Numerical Properties of Orthogonal Collocation Methods

Numerically solving an optimal control problems require knowledge of a number of dif-

ferent concepts. In particular, two concepts are important in constructing a discretized finite-

dimensional optimization problem from a continuous-time optimal control problem: polynomial

approximation and numerical integration. Polynomial approximation is important because the

infinite-dimensional continuous functions (that is, the state components) of the optimal control

problem are approximated by a finite-dimensional Lagrange polynomial basis. Numerical integra-

tion methods are important because the dynamic constraints and the cost must be integrated as

part of the optimization. In this section, these important mathematical concepts that are used to

transcribe a continuous-time optimal control problem to a nonlinear programming problem (NLP)

using orthogonal collocation at Gaussian quadrature points are reviewed.

2.4.1 Function Approximation and Interpolation

Collocation methods for solving optimal control problems approximate the continuous

functions of time at a set of support points. In this research, Lagrange polynomials are used to

49

interpolate the state and the costate. Specifically, given a continuous function y(t), there exists

a unique polynomial Y (t) of degree N − 1 which uses N arbitrary support points (t1 ... , tN) ∈

[t0, tf ], such that

Y (ti) = y(ti), (i = 1, ... ,N). (2–97)

Furthermore, the unique polynomial can be described by Lagrange interpolation such that

Y (t) =

N∑

i=1

yiLi(τ), (2–98)

where yi = y(ti) and Li(t) are the Lagrange polynomials

Li(t) =

N∏

j=1

j 6=i

τ − τj

τi − τj. (2–99)

One important property of Lagrange interpolating polynomials is that they satisfy the isolation

property

Li(tj) =

1 for i = j ,

0 for i 6= j .(2–100)

Eq. (2–100) is important for this research because it leads to a sparse nonlinear program

transcription of the optimal control problem being solved. Thus the isolation property leads to a

transcription which can be efficiently solved by a nonlinear program solver. The error associated

with the Lagrange approximation of a Nth times differentiable function is given by

y(t)− Y (t) = (t − t1) ... (t − tN)N!

yN(ζ), (2–101)

where yN(ζ) is the Nth derivative of the function y(t) evaluated at a point ζ ∈ [t0, tf ]. It is

seen from this error formula that the error is exactly zero at any of the support points of the

interpolating polynomial. Furthermore, since the error is a direct function of the Nth derivative

of y(t), the Lagrange interpolation approximation using N support points will be exact for

polynomials of degree at most N − 1.

Although smooth functions can be accurately approximated as stated above, the behavior

of the interpolation error as N approaches infinity for non-smooth functions becomes erratic, a

behavior called Runge phenomenon. Runge phenomenon is characterized by large amplitude

50

oscillations in the interpolating polynomials near the domain boundaries when the support points

are uniformly distributed and N becomes large. In order to understand Runge phenomenon

better, consider the following function defined in the domain τ ∈ [−1,+1]

y(τ) =1

1 + 50τ2. (2–102)

The function in Eq. (2–102) was approximated using N uniformly distributed support points for

a basis of Lagrange interpolating polynomials and the result for N = 11 and N = 41 is shown

in Fig. (2-2). The approximations for N = 11 and N = 41 points correspond to polynomials

of degree 10 and 40. It can be seen that as the number of support points is increased, the

error in the interpolation becomes larger near the end points due to the large oscillations in

the polynomial approximations. Thus the interpolation does not converge to the function being

approximated as N is increased.

2.4.1.1 Family of Legendre-Gauss points

One way to rectify the effect of Runge phenomenon is to use a non-equally spaced set

support points. In this research the support points used are the points obtained from the roots of

a Legendre polynomial and/or linear combinations of a Legendre polynomial and its derivatives.

These points are known to have a distribution that minimizes Runge phenomenon. In particular,

two sets of points are of interest: the Ledendre-Gauss (LG) points, and the Legende-Gauss-

Radau (LGR) points. Both these sets of points are defined on the domain τ ∈ [−1,+1], but

differ significantly in that the LG points include neither of the endpoints whereas the LGR points

include one of the endpoints. In addition, the LGR points are asymmetric relative to the origin

and are not unique because they can be defined using either the initial or the terminal point. The

LGR points that include the terminal point are often called the flipped LGR points. The flipped

LGR points are a mirror image of the LGR points that include the initial point in the domain.

These points can be calculated as follows. Denoting the Nth degree Legendre polynomial by

PN(τ) =1

2NN!

dN

dτN

[

(τ2 − 1)N]

,

51

0

0 0.2 0.4

0.5

0.6 0.8

1

1-0.8 -0.6 -0.4 -0.2

-0.5

-1-1

2

3

2.5

3.5

1.5

τ

y(τ)

y(τ)

Y (τ)

(A) Approximation of the function given by Eq. (2–102) using 11uniformly spaced support points.

0

0 0.2 0.4

0.5

0.6 0.8 1-0.8 -0.6 -0.4 -0.2

-0.5

-1.5

-2

-1

-1-2.5

6x10

τ

y(τ)

y(τ)

Y (τ)

(B) Approximation of the function given by Eq. (2–102) using 41uniformly spaced support points.

Figure 2-2. Approximation of the function given by Eq. (2–102) using 11 and 41uniformly spaced support points.

52

0 0.2 0.4 0.6 0.8 1-0.8 -0.6 -0.4 -0.2-1τ

LGR

LGR-f

LG

Figure 2-3. Distribution of Legendre-Gauss, Legendre-Gauss-Radau, and FlippedLegendre-Gauss-Radau Points in the domain τ ∈ [−1,+1].

the LG points are defined as the roots of PN(τ) and the LGR points are defined as the roots

obtained from PN−1(τ) + PN(τ). Fig. (2-3) is a schematic representation of the LG, LGR, and

flipped LGR points in the domain [−1,+1] for N = 5.

The function given by Eq. (2–102) is now approximated using a basis of Lagrange interpo-

lating polynomial with LG and LGR support points. A function y(τ) can be approximated in the

domain [−1,+1] using a basis of Lagrange interpolating polynomials and the LG points as

Y (τ) =

N+1∑

i=0

y(τi)L)i(τ) (2–103)

where (τ1, ... , τN) are the N LG points, τ0 = −1, and τN+1 = +1. Similarly, the same function

y(τ) can be approximated in the domain [−1,+1] using a basis of Lagrange interpolating

polynomials and the LGR points as

Y (τ) =

N+1∑

i=1

y(τi)L)i(τ) (2–104)

where (τ1, ... , τN) are the N LG points, and τN+1 = +1. Fig. (2-4) shows the results of the

function approximation when N = 11 and N = 41 LG support points are used. Furthermore,

Fig. (2-5) shows the results of the function approximation when N = 11 and N = 41 LGR support

53

points are used. It can be seen that as the number of support points is increased the polynomial

approximation converges to the true function. In order to understand the behavior of the error,

Fig. (2-6) plots the base ten logarithm of the maximum absolute error defined as

Ey = log10 ||Y (τ)− y(τ)||∞

as a function of N for an approximation using Lagrange interpolating polynomials with uni-

formly spaced, LG, and LGR support points. It can be seen that as the number of support

points increases, the polynomial approximation which uses uniformly spaced support points

diverges from the function being approximated, whereas using LG and LGR support points the

approximation converges to the true function.

2.4.2 Numerical Integration

Numerical integration plays a key role when numerically approximating the solution to a

continuous-time optimal control problem. In particular, an optimal control problem requires that a

cost be minimized (or maximized), and this cost is integrated across the time domain of interest.

Furthermore, the dynamic constraints must be integrated. Therefore, an optimal control problem

optimizes and integrates simultaneously. A summary of numerical integration methods will now

be given in order to better understand the methods used in this research.

2.4.2.1 Low-order integrators

A common technique used to approximate the integral of a function is to use low-degree

polynomial approximations. The approximation of the integral is then found by summing the low-

order method integral approximations of each subinterval. One commonly used technique that

uses low-order methods is called the composite trapezoid rule [2]. The composite trapezoid rule

divides the domain of interest into many uniformly distributed sub-intervals and approximates

the function to be integrated with a straight line that passes through the function at the endpoints

of the subinterval. Therefore, for N approximating subintervals, the composite trapezoid rule is

given by

∫ tf

t0

f (t) dt ≈ tf − t02N

[f (t0) + 2f (t1) + 2f (t2) + ... + 2f (tN−1) + f (tN)] , (2–105)

54

1.2

0

0

0.2

0.2

0.4

0.4

0.6

0.6

0.8

0.8

1

1-0.8 -0.6 -0.4-0.2

-0.2-1τ

y(τ)

y(τ)

Y (τ)

(A) Approximation of the function given by Eq. (2–102) using 11Legendre-Gauss points.

00

0.1

0.2

0.2

0.3

0.4

0.4

0.5

0.6

0.6

0.7

0.8

0.8

0.9

1

1-0.8 -0.6 -0.4 -0.2-1τ

y(τ)

y(τ)

Y (τ)

(B) Approximation of the function given by Eq. (2–102) using 41Legendre-Gauss points.

Figure 2-4. Approximation of the function given by Eq. (2–102) using 11 and 41Legendre-Gauss points.

55

1.2

0

0

0.2

0.2

0.4

0.4

0.6

0.6

0.8

0.8

1

1-0.8 -0.6 -0.4-0.2

-0.2-1τ

y(τ)

y(τ)

Y (τ)

(A) Approximation of the function given by Eq. (2–102) using 11Legendre-Gauss-Radau points.

00

0.1

0.2

0.2

0.3

0.4

0.4

0.5

0.6

0.6

0.7

0.8

0.8

0.9

1

1-0.8 -0.6 -0.4 -0.2-1τ

y(τ)

y(τ)

Y (τ)

(B) Approximation of the function given by Eq. (2–102) using 41Legendre-Gauss-Radau points.

Figure 2-5. Approximation of the function given by Eq. (2–102) using 11 and 41Legendre-Gauss-Radau points.

56

0

0 30 40 50 60 70 80 90 100

-5

10

10-10

20

20

15

5

N

Ey

Uniform

LG

LGR

Figure 2-6. Base ten logarithm of infinity norm error as a function of number of supportpoints, N , for approximating the function given by Eq. (2–102).

where (t0, ... , tN) are the subinterval boundaries, or grid points, at which the function is being

evaluated. In order to demonstrate the use of the composite trapezoid rule, consider approximat-

ing the integral of the function f (τ) in the domain τ ∈ [−1,+1]:

f (τ) =

∫ +1

−1exp(τ) dτ . (2–106)

Figure (2-7) shows a graphical representation of the trapezoid rule approximation using three

intervals. Furthermore, Fig. (2-8) shows the base ten logarithm error of this integration as a

function of the base ten logarithm number of approximating intervals. It can be seen that the

convergence of this method is slow as 105 subintervals are necessary to reach an error of

O(10−10).

2.4.2.2 Gaussian quadrature

In contrast with low-order integrators such as the composite trapezoid rule, a Gaussian

quadrature is a high accuracy integrator which displays exponential convergence when approxi-

mating the integral of smooth functions. Gaussian quadrature rules approximate the integral of a

57

00 0.2 0.4

0.5

0.6 0.8

1

1-0.8 -0.6 -0.4 -0.2-1

2

3

2.5

1.5

f (τ)

f(τ)

Approx.

τ

Figure 2-7. Approximation of the integral of the function given by Eq. (2–106) using a 4interval Trapezoid rule.

−4

2 4 4.5 5

-5

32.5 3.5

-6

-7

-8

-9

-10

-11

log10

Err

or

log10 Number of Approximating Intervals

Figure 2-8. Base ten logarithm error of the integral of the function given by Eq. (2–106)as a function of the base ten logarithm number of approximating intervals.

58

function by evaluating the expression

∫ tf

t0

f (t) dt ≈N∑

i=1

wi f (τi), (2–107)

where wi are the quadrature weights associated with the set of points chosen to approximate the

integration. The three sets of points defined by Gaussian quadrature are the Legendre-Gauss-

Lobatto (LGL) points, the Legendre-Gauss-Radau (LGR) points, and the Legendre-Gauss (LG)

points. The LGL, LGR, and LG quadrature rules are exact for polynomials of degree at most

2N − 3, 2N − 2, and 2N − 1, respectively. In this research the LG and LGR points are used.

The N LG points are the roots of the Nth degree Legendre polynomial PN(τ), and the

corresponding LG quadrature weights are given as

wi =2

1− τ2i[PN(τi)]2

, (i = 1, ... ,N).

Similarly, the N LGR points are computed from the roots of PN(τ) + PN−1(τ), and the corre-

sponding LGR quadrature weights are given as

w1 =2

N2,

wi =1

(1− τi)[(PN−1(τi)]2, (i = 2, ... ,N).

Finally, the flipped LGR points and weights are simply the negative of the LGR points.

The accuracy of the LG and LGR quadrature methods can be seen from the function

f (τ) = exp(τ) in the domain τ ∈ [−1,+1]. Figure 2-9 shows the base ten log of the error

for the approximation of the integral in Eq. (2–106) as a function of N. It can be seen that the

convergence rate using Gaussian quadrature is exponential. Furthermore, it is seen that N = 6

LG or LGR points results in an error of less than O(10−10). For comparison, this same error

of O(10−10) required 105 subintervals using the composite Trapezoid rule. Thus, the benefits

of using Gaussian quadrature over low-order integrators when approximating the integral of a

smooth function is evident.

2.5 Orthogonal Collocation for the Solution of Optimal Control Problems

It is now possible to combine the concepts described in Section 2.4 in order to develop a

method to approximate the solution of the continuous-time optimal control problem of Section

59

6 7 8 9 10-16

-14

-4

-2

4 53

-6

-8

-10

-12

N

LG Points

LGR Points

log10

Err

or

Figure 2-9. Base ten logarithm of the error in the integration of the function given byEq. (2–106) as a function of number of LG and LGR Points Used.

2.1. In this section the methods of orthogonal collocation at both Legendre-Gauss (LG) and

Legendre-Gauss-Radau (LGR) points are described. Both these methods for approximating

solutions to optimal control problems can be used as either global collocation methods or

variable-order collocation methods. Global collocation methods use one single polynomial

approximation to collocate the differential-algebraic equations over the entire domain. Global

collocation at LG and LGR points is advantageous when solving problems whose solutions are

smooth, because the LG and LGR methods converge exponentially. When the solution is not

smooth, however, the convergence rate is significantly lower. In the case where the optimal

solution lies on a constrained arc for a subset of the solution domain, non-smooth features in the

solution state and/or control can occur. For such problems, it will thus be beneficial to employ

variable-order LG or LGR collocation. In a variable-order collocation scheme the solution domain

is divided into a mesh, and the degree of the approximating polynomial (that is, the number of

LG or LGR collocation points) in each mesh interval is allowed to vary. This method is useful

because it allows for capturing non-smoothness in the solution domain at interval boundaries.

Because both the LG and the LGR set of points are defined on the domain τ ∈ [−1,+1], the

following affine transformation will be used to map the time domain t ∈ [t0, tf ] to τ ∈ [−1,+1]

60

when using global collocation:

t =tf − t02

τ +tf + t02. (2–108)

Furthermore, it is noted that

d t

dτ=tf − t02

≡ h2, h = tf − t0.

When using variable-order collocation, the time domain t ∈ [t0, tf ] is divided into a mesh

consisting of K mesh intervals where the mesh points are t0 = T0 < T1 < · · · < TK−1 < TK =

tf , and the corresponding mesh intervals are [Tk−1,Tk ], (k = 1, ... ,K). Therefore each mesh

interval can be mapped to the domain τ ∈ [−1,+1] through the affine transformation

t =tk − tk−12

τ +tk + tk−12

, (k = 1, ... ,K).

It is also noted thatd t

dτ=tk − tk−12

≡ h(k)

2, h(k) = tk − tk−1.

The following notation and conventions will be used in the discussion that ensues. First, all

vector functions of time are denoted as row vectors, that is, if y(τ) ∈ Rn is a vector function of the

scalar variable τ , then y(τ) = [y1(τ), · · · , yn(τ)]. Next, any capital boldface character, Y, denotes

a matrix of size M × n, where each row of Yi corresponds to the evaluation of a function y(τ) at

a particular value τ = τi . Next, the notation Yi:j denotes rows i through j of the matrix Y, except

when referring to a differentiation matrix D, in which case Di refers to the i th column of D. Finally,

D⊤ denotes the transpose of matrix D, and D⊤i denotes the transpose of the i th column of D.

2.5.1 Global Collocation at Legendre-Gauss Points

The method for approximating solutions to optimal control problems using global orthogonal

collocation at Legendre-Gauss (LG) points is now described [27]. The LG points are defined

in the domain (−1,+1) which does not include either of the endpoints. However, the method

derived here for collocation at the LG points still approximates, but does not collocate, the state

at both endpoints τ0 = −1 and τN+1 = +1. Figure 2-10 shows the LG collocation points as well

as the endpoints at which the state is approximated but not collocated for various values of N.

61

0 0.2 0.4 0.6 0.8 1-0.8 -0.6 -0.4 -0.2-13

4

5

6

7

8

9

τ

Discretization Points

Collocation Points

Num

ber

ofLG

Poi

nts,N

Figure 2-10. Distribution of Legendre-Gauss discretization and collocation points in thedomain τ ∈ [−1,+1].

The state is then approximated using a Lagrange polynomial with support points at the N

LG points plus the noncollocated point τ0 = −1, such that

y(τ) ≈ Y(τ) =N∑

i=0

YiLi(τ), (2–109)

where the Lagrange polynomials Li(τ) are defined as

Li(τ) =

N∏

j=0

j 6=i

τ − τj

τi − τj; (i = 0, ... ,N). (2–110)

The state approximation is then differentiated at τ = τj , (j = 1, ... ,N) as

Y(τj) ≈N∑

i=0

Yi Li(τj) = [DY0:N ]j , (2–111)

where Dij = Li (τj) , (i = 1, ... ,N, j = 0, ... ,N) are the components of the N × (N + 1)

Legendre-Gauss (LG) differentiation matrix.

62

Next, the cost functional of Eq. (2–1) is approximated with a Gaussian quadrature. The

finite-dimensional transcription of the continuous-time optimal control problem of Eqs. (2–1)–

(2–4) then becomes to minimize the cost function

J = Φ(YN+1) +h

2

N∑

j=1

wjg(Yj ,Uj) (2–112)

subject to the algebraic constraints

DY0:N =h

2f(Y1:N,U1:N), (2–113)

YN+1 = Y0 +h

2

N∑

j=1

wj f(Yj ,Uj), (2–114)

φ(Y0) = 0, (2–115)

C(Y1:N ,U1:N) ≤ 0, (2–116)

where w = (w1, ... ,wN) are the LG quadrature weights. It is noted for LG collocation that

Eq. (3–24) provides an LG quadrature approximation of the state, YN+1, at the final noncollo-

cated point τN+1 = +1.

2.5.2 Global Collocation at Legendre-Gauss-Radau Points


collocation at Legendre-Gauss-Radau (LGR) points is now described [64]. The LGR points are

defined in the domain [−1,+1) such that τ1 = −1 is a LGR collocation point but τN+1 = +1 is

a noncollocated point. However, the method derived here for collocation at the LGR points still

approximates, but does not collocate, the state at the terminal point τN+1 = +1. Figure 2-11

shows the LGR collocation points as well as the noncollocated terminal point for various values

of N.


LGR points plus the noncollocated point τN+1 = +1 such that

y(τ) ≈ Y(τ) =N+1∑

i=1

YiLi(τ), (2–117)

63

0 0.2 0.4 0.6 0.8 1-0.8 -0.6 -0.4 -0.2-13

4

5

6

7

8

9

τ


Collocation Points

Num

ber

ofLG

RP

oint

s,N



Li(τ) =

N+1∏

j=1

j 6=i

τ − τj

τi − τj; (i = 1, ... ,N + 1). (2–118)


Y(τj) ≈N+1∑

i=1

Yi Li(τj) = [DY1:N+1]j , (2–119)

where Dij = Li (τj) , (i = 1, ... ,N, j = 1, ... ,N + 1) are the components of the N × (N + 1)

Legendre-Gauss-Radau (LGR) differentiation matrix. Next, the cost functional of Eq. (2–1) is

approximated by an LGR quadrature. The finite-dimensional approximation of the continuous-

time optimal control problem of Eqs. (2–1)–(2–4) is then given as follows. Minimize the cost

function

J = Φ(YN+1) +h

2

N∑

j=1

wjg(Yj ,Uj) (2–120)

64

0 0.2 0.4 0.6 0.8 1-0.8 -0.6 -0.4 -0.2-13

4

5

6

7

8

9

τ


Collocation Points

Num

ber

ofF

lippe

dLG

RP

oint

s,N



DY1:N+1 =h

2f(Y1:N ,U1:N), (2–121)

φ(Y1) = 0, (2–122)

C(Y1:N ,U1:N) ≤ 0, (2–123)

where w = (w1, ... ,wN) are the LGR quadrature weights.

2.5.3 Global Collocation at Flipped Legendre-Gauss-Radau Points


collocation at the flipped Legendre-Gauss-Radau (LGR) points is now described [64]. The LGR

points are defined in the domain (−1,+1] such that τN = +1 is a LGR collocation point but

τ0 = −1 is a noncollocated point. However, the method derived here for collocation at the LGR

points still approximates, but does not collocate, the state at the initial point τ0 = −1. Figure

(2-12) shows the flipped LGR collocation points as well as the initial point at which the state is

approximated but not collocated for various values of N.

65


LGR points plus the noncollocated point τ0 = −1, such that

y(τ) ≈ Y(τ) =N∑

i=0

YiLi(τ), (2–124)


Li(τ) =

N∏

j=0

j 6=i

τ − τj

τi − τj; (i = 0, ... ,N). (2–125)


Y(τj) ≈N∑

i=0

Yi Li(τj) = [DY0:N ]j , (2–126)

where Dij = Li (τj) , (i = 1, ... ,N, j = 0, ... ,N) are the components of the N × (N + 1)

Legendre-Gauss-Radau (LGR) differentiation matrix.

Next, the cost functional of Eq. (2–1) is approximated by an LGR quadrature. The finite-

dimensional approximation of the continuous-time optimal control problem of Eqs. (2–1)–(2–4) is

then given as follows. Minimize the cost function

J = Φ(YN) +h

2

N∑

j=1

wjg(Yj ,Uj) (2–127)


DY0:N =h

2f(Y1:N ,U1:N), (2–128)

φ(Y0) = 0, (2–129)

C(Y1:N ,U1:N) ≤ 0, (2–130)

where w = (w1, ... ,wN) are the flipped LGR quadrature weights.

2.5.4 Variable-Order Collocation at Legendre-Gauss Points

The method for approximating solutions to optimal control problems using variable-order

collocation at the Legendre-Gauss (LG) points is now described. When implementing the

variable-order LG method, a single variable is used for the value of the state at the end of mesh

interval k and the start of mesh interval k + 1, that is, Y(k)Nk+1

≡ Y(k+1)0 , 1 ≤ k ≤ K − 1 such that

66

3

4

5

6

7

8

9

tT0 T1 T2 T3


Collocation PointsNum

ber

ofLG

poin

tsP

erIn

terv

al,Nk

Figure 2-13. Distribution of multiple-interval Legendre-Gauss discretization andcollocation points for various values of Nk . The domain [t0, tf ] is split into K=3 intervalssuch that t0 = T0 and tf = T3.

continuity in the state is enforced. Hence, redundant variables defining the state at the interior

mesh points are eliminated. Figure 2-13 shows the LG collocation points as well as the mesh

points at which the state is approximated but not collocated for when K = 3 and for various

values of N.

In multiple-interval LG collocation, the state is approximated in each mesh interval k as

y(k)(τ) ≈ Y(k)(τ) =Nk∑

i=0

Y(k)i L

(k)i (τ), L

(k)i (τ) =

Nk∏

j=0

i 6=j

τ − τ(k)j

τi − τ(k)j

, (2–131)

Differentiating Y(k)(τ) in Eq. (2–131) with respect to τ , yields

Y(k)(τj) ≈Nk∑

i=0

Y(k)iL(k)i(τj) = [D

(k)Y(k)0:Nk]j , (2–132)

where D(k)ij= L

(k)i(τ)j , (i = 1, ... ,Nk , j = 0, ... ,Nk) are the components of the Nk × (Nk + 1)

Legendre-Gauss (LG) differentiation matrix in the kth mesh interval.

Next, the cost functional of Eq. (2–1) is approximated using a multiple-interval LG quadra-

ture. The finite-dimensional approximation of the continuous-time optimal control problem of

67

Eqs. (2–1)–(2–4) is then given as follows. Minimize the cost function

J ≈ Φ(Y(K)NK+1) +K∑

k=1

Nk∑

j=1

h(k)

2w(k)j g(Y

(k)j ,U

(k)j ), (2–133)


D(k)Y(k)0:Nk

=h(k)

2f(Y

(k)1:Nk,U(k)1:Nk), (k = 1, ... ,K), (2–134)

Y(k+1)0 = Y

(k)0 +

h(k)

2

Nk∑

j=1

w(k)j f(Y

(k)j ,U

(k)j ), (k = 1, ... ,K − 1), (2–135)

Y(K)N+1 = Y

(K)0 +

h(K)

2

Nk∑

j=1

w(K)j f(Y

(K)j ,U

(K)j ), (2–136)

φ(Y(1)0 ) = 0, (2–137)

C(Y(k)1:Nk,U(k)1:Nk ) ≤ 0, (k = 1, ... ,K), (2–138)

where w(k) = (w (k)1 , ... ,w(k)N k) are the LG quadrature weights in interval k . It is noted for LG

collocation that Eq. (2–135) provides an LG quadrature approximation, Y(k)0 , of the state at the

final noncollocated point τ (k)N+1 = +1 in interval (k = 1, ... ,K − 1), while Eq. (2–136) provides an

LG quadrature approximation, Y(K)N+1, of the state at the final noncollocated point of the domain,

tf = τ(k)N+1 = +1.

2.5.5 Variable-Order Collocation at Legendre-Gauss-Radau Points


collocation at the Legendre-Gauss-Radau (LGR) points is now described. When implementing

the variable-order LGR method, a single variable is used for the value of the state at the end of

mesh interval k and the start of mesh interval k + 1, that is, Y(k)Nk+1

≡ Y(k+1)1 , 1 ≤ k ≤ K − 1

such that continuity in the state is enforced. Hence, redundant variables defining the state at the

interior mesh points are eliminated. It is noted that the LGR points are particularly conducive

to this type of collocation; because only one of the domain endpoints is collocated, there is no

“double collocation” at the boundaries. Also, the only noncollocated point is the last point of the

final interval, tf = τ(K)NK+1

= +1. Figure (2-14) shows the LGR collocation points as well as the

terminal point at which the state is approximated but not collocated for when N = 3 and for

various values of K .

68

3

4

5

6

7

8

9

tT0 T1 T2 T3


Collocation PointsNum

ber

ofLG

RP

oint

sP

erIn

terv

al,Nk

Figure 2-14. Distribution of multiple-interval Legendre-Gauss-Radau discretization andcollocation points for various values of Nk . The domain [t0, tf ] is split into K=3 intervalssuch that t0 = T0 and tf = T3.

In multiple-interval LGR collocation, the state is approximated in each mesh interval k as

y(k)(τ) ≈ Y(k)(τ) =Nk+1∑

i=1

Y(k)iL(k)i(τ), L

(k)i(τ) =

Nk+1∏

j=1

i 6=j

τ − τ(k)j

τi − τ(k)j

, (2–139)


Y(k)(τj) ≈Nk+1∑

i=1

Y(k)i L

(k)i (τj) = [D

(k)Y(k)1:Nk+1

]j , (2–140)

where D(k)ij = L(k)i (τ)j , (i = 1, ... ,Nk , j = 1, ... ,Nk + 1) are the components of the Nk × (Nk + 1)

Legendre-Gauss-Radau (LGR) differentiation matrix in the kth mesh interval.




J ≈ Φ(Y(K)NK+1) +K∑

k=1

Nk∑

j=1

h(k)

2w(k)j g(Y

(k)j ,U

(k)j ), (2–141)

69


D(k)Y(k)1:Nk+1

=h(k)

2f(Y

(k)1:Nk,U(k)1:Nk), (k = 1, ... ,K), (2–142)

φ(Y(1)1 ) = 0, (2–143)

C(Y(k)1:Nk,U(k)1:Nk ) ≤ 0, (k = 1, ... ,K), (2–144)

where w(k) = (w (k)1 , ... ,w(k)N k) are the LGR quadrature weights in interval k .

2.5.6 Variable-Order Collocation at Flipped Legendre-Gauss-Radau Points


collocation at the flipped Legendre-Gauss-Radau (LGR) points is now described. When

implementing the flipped variable-order LGR method, a single variable is used for the value

of the state at the end of mesh interval k and the start of mesh interval k + 1, that is, Y(k)Nk ≡

Y(k+1)0 , 1 ≤ k ≤ K − 1 such that continuity in the state is enforced. Hence, redundant

variables defining the state at the interior mesh points are eliminated. It is noted that the flipped

LGR points are particularly conducive to this type of collocation; since only one of the domain

endpoints are collocated, there is no “double collocation” at the boundaries. Also, the only

noncollocated point is the first point of the first interval, t0 = τ(1)0 = −1. Figure (2-15) shows the

flipped LGR collocation points as well as the initial point at which the state is approximated but

not collocated for when N = 3 and for various values of K .

In multiple-interval flipped LGR collocation, the state is approximated in each mesh interval

k as

y(k)(τ) ≈ Y(k)(τ) =Nk∑

i=0

Y(k)i L

(k)i (τ), L

(k)i (τ) =

Nk∏

j=0

i 6=j

τ − τ(k)j

τi − τ(k)j

, (2–145)


Y(k)(τj) ≈Nk∑

i=0

Y(k)i L

(k)i (τj) = [D

(k)Y(k)0:Nk]j , (2–146)

where D(k)ij = L(k)i (τ)j , (i = 1, ... ,Nk , j = 0, ... ,Nk) are the components of the Nk × (Nk + 1)

flipped Legendre-Gauss-Radau (LGR) differentiation matrix in the kth mesh interval.

70

3

4

5

6

7

8

9

t

T0 T1 T2 T3


Collocation Points

Num

ber

ofF

lippe

dLG

RP

oint

sP

erIn

terv

al,Nk

Figure 2-15. Distribution of multiple-interval Flipped Legendre-Gauss-Radaudiscretization and collocation points for various values of Nk . The domain [t0, tf ] is splitinto K=3 intervals such that t0 = T0 and tf = T3.




J ≈ Φ(Y(K)NK) +

K∑

k=1

Nk∑

j=1

h(k)

2w(k)jg(Y

(k)j,U(k)j), (2–147)


D(k)Y(k)0:Nk

=h(k)

2f(Y

(k)1:Nk,U(k)1:Nk), (k = 1, ... ,K), (2–148)

φ(Y(1)0 ) = 0, (2–149)

C(Y(k)1:Nk,U(k)1:Nk ) ≤ 0, (k = 1, ... ,K), (2–150)

where w(k) = (w (k)1 , ... ,w(k)N k) are the flipped LGR quadrature weights in interval k .

71

CHAPTER 3COSTATE ESTIMATION USING THE INTEGRAL FORMULATION

As was previously discussed, the Legendre-Gauss (LG) and Legendre-Gauss-Radau

(LGR) methods for approximating solutions to optimal control problems are equivalent

regardless of whether the collocation is performed in differential or integral form.

Typically, however, the differential form of either method has been used. Therefore, the

relationship between the Lagrange multipliers of the differential form of the collocation

methods and the costate of the optimal control problem has been well documented.

On the other hand, the corresponding relationship between the Lagrange multipliers

associated with the integral forms of LG and LGR collocation and the costate of the

optimal control problem has not been established. In this chapter methods for estimating

the optimal control costate using the integral forms of LG and LGR collocation are

developed. Specifically, transformations are derived that relate the Lagrange multipliers

of the integral forms of the LG and LGR collocation methods to the costate of the

original optimal control problem. A new continuous-time dual variable called the inte-

gral costate is introduced, where the integral costate is the Lagrange multiplier of the

integral dynamic constraint. The first-order optimality conditions of the integral form of

the optimal control problem are derived in terms of the integral costate. The integral

form of the optimal control problem is then discretized using the integral LG and LGR

collocation methods and relationship between the discrete form of the integral costate

and the costate of the original differential optimal control problem are developed. The

approach developed in this research then provides a way to estimate the costate of the

original optimal control problem using the Lagrange multipliers of the integral form of the

LG and LGR collocation methods.

The following notation and conventions will be used throughout this chapter. First,

all vector functions of time are denoted as row vectors, that is, if y(τ) ∈ Rn is a vector

function of the scalar variable τ , then y(τ) = [y1(τ), · · · , yn(τ)]. Next, any capital

72

boldface character, Y, denotes a matrix of size M×n, where each row of Yi corresponds

to the evaluation of a function y(τ) at a particular value τ = τi . Next, the notation Yi:j

denotes rows i through j of the matrix Y, except when referring to a differentiation matrix

D or the integration matrix A, in which case Di and Ai refers to the i th column of D and

A. Finally, D⊤ denotes the transpose of matrix D, and D⊤i denotes the transpose of the

i th column of D. Given vectors x and y ∈ Rn, the notation 〈x, y〉 is used to denote the

standard inner product between x and y. Furthermore, if f : Rn −→ Rm, then ∇f is the

m by n Jacobian matrix whose i th row is ∇fi . In particular, the gradient of a scalar-valued

function is a row vector. If φ : Rm×n −→ R and Y is an m by n matrix, then ∇φ denotes

the m by n matrix whose (i , j) element is (∇φ(Y))ij = ∂φ(Y)/∂Yij .

This remainder of this chapter is organized as follows. In Section 3.1 the continuous-time

optimal control problem is presented with the dynamic constraints formulated in both

differential and integral form, and the first-order optimality conditions of each formulation

are given. In Sections 3.2 and 3.3, the Legendre-Gauss and Legendre-Gauss-Radau

collocation methods in both differential and integral forms are presented, and the

first-order optimality conditions of each form are derived. Furthermore, the transformed

adjoint system of the integral form is derived, and a costate estimate is presented in

terms of the Lagrange multipliers of the integral forms. Finally, Section 3.4 provides a

discussion of the differences between the collocation schemes at LG and LGR points.


In order to make this exposition clearer, in this chapter the Bolza continuous-time

optimal control problem of Section 2.1 is formulated in the domain τ ∈ [−1,+1]. It is

noted that the time interval τ ∈ [−1,+1] can be transformed to the interval [t0, tf ] via the


t =tf − t02

τ +tf + t02. (3–1)

Furthermore, this problem formulation is stated with no inequality path constraints.

Inequality path constrained optimal control problems will be discussed in Chapter 5.

73

3.1.1 Differential and Integral Forms of Optimal Control Problem

Consider again the Bolza continuous-time optimal control problem from Section

2.1 defined on the interval τ ∈ [−1,+1]. Determine the state y(τ) ∈ Rn and the control

u(τ) ∈ Rm that minimize the cost functional

J = Φ(y(+1)) +

∫ +1

−1

g(y, u)dτ (3–2)


y(τ)− f(y, u) = 0. (3–3)


φ(y(−1)) = 0. (3–4)

Henceforth, Eqs. (3–2)–(3–4) will be referred to as the differential optimal control

problem.

The differential optimal control problem given in Eqs. (3–2)–(3–4) can be re-written

in the following integral form. In particular, integrating the dynamics given in Eq. (3–3),

yields

y(τ) = y(−1) +∫ τ

−1

f(y, u) dτ .

The optimal control problem in integral form is then stated as follows. Determine the

state, y(τ) ∈ Rn, and the control, u(τ) ∈ Rm, that minimize the cost functional

J = Φ(y(+1)) +

∫ +1

−1

g(y(τ), u(τ))dτ (3–5)

subject to the integral constraint

y(τ)− y(−1)−∫ τ

−1

f(y, u) dt = 0, (3–6)


φ(y(−1)) = 0. (3–7)

74

Henceforth, Eqs. (3–5)–(3–7) will be referred to as the integral optimal control problem.

3.1.2 First-Order Optimality Conditions of Differential and Integral Forms

The first-order optimality conditions of the differential optimal control problem

obtained from the calculus of variations were derived in Section 2.1.1 and are given as

y(τ) = f(y, u), φ(y(−1)) = 0, (3–8)

0 = ∇uH(y, u,λ), (3–9)

−λ = ∇yH(y, u,λ), (3–10)

λ(−1) = ∇y 〈ψ,φ(−1)〉, (3–11)

λ(+1) = ∇yΦ(+1), (3–12)

Next, the first-order optimality conditions of the integral optimal control problem

obtained from applying the calculus of variations were derived in Section 2.1.2 are given

as

y = y(−1) +∫ τ

−1

f(y, u) dτ , φ(y(−1)) = 0, (3–13)

0 = ∇ug + 〈(∫ +1

τ

p dt +∇yΦ(+1))

, ∇uf(y, u) 〉, (3–14)

p = ∇yg + 〈(∫ +1

τ

p dt +∇yΦ(+1))

, ∇yf(y, u)〉, (3–15)

∇y〈ψ,φ(−1)〉 =∫ +1

−1

p dτ +∇yΦ(+1), (3–16)

where

H(y, u,λ) = g(y, u) + 〈λ, f(y, u)〉, (3–17)

is the Hamiltonian, λ(τ) is the costate of the differential optimal control problem (and

will be referred to henceforth as the differential costate), p(τ) is the costate of integral

optimal problem (and will be referred to henceforth as the integral costate), and ψ is the

Lagrange multiplier associated with the boundary condition of Eq. (3–4). It is noted that

the differential costate and the integral costate are different in that λ(τ) is the Lagrange

75

multiplier associated with the differential equation constraint of Eq. (3–3) while p(τ) is

the Lagrange multiplier associated with the integral equation constraint of Eq. (3–6).

The differential and integral costate are related as

λ(τ) = ∇yΦ(+1) +∫ +1

τ

p dt. (3–18)

In particular, substituting Eq. (3–18) together with the relationship

λ(τ) = −p(τ) (3–19)

into the first-order optimality conditions of the integral optimal problem as given in

Eqs. (3–13)–(3–16) yields the the first-order optimality conditions of the differential

optimal control problem as given in Eqs. (3–8)–(3–12). The remainder of this chapter is

devoted to deriving two discrete approximations of the differential costate, λ(τ), using

discrete approximations of the integral costate, p(τ).

3.2 Costate Estimation Using Integral Legendre-Gauss Collocation

In this section a costate estimate for the differential optimal control problem is

developed via an estimate of the integral costate obtained using the integral form of

the Legendre-Gauss orthogonal collocation method. In Section 3.2.1 the differential

form of the Legendre-Gauss collocation method is described. In Section 3.2.2 the

first-order optimality conditions of the nonlinear programming problem described in

Section 3.2.1 are provided. In Section 3.2.3 the integral form of the Legendre-Gauss

collocation method is described. In Section 3.2.4 the first-order optimality conditions of

the nonlinear programming problem described in Section 3.2.1 are provided. Finally, in

Section 3.2.5 a differential costate estimate using the integral costate estimate derived in

Section 3.2.4 is developed.

3.2.1 Differential Form of Legendre-Gauss Collocation

The differential optimal control problem of Eqs. (3–2)–(3–4) can now be approximated

using collocation at Legendre-Gauss (LG) points as was described in Section 2.5.1. The

76

LG points are defined in the domain (−1,+1) which does not include either of the

endpoints. The state is approximated using a Lagrange polynomial with support points

at the N LG points plus the noncollocated point τ0 = −1, such that

y(τ) ≈ Y(τ) =N∑

i=0

YiLi(τ), Li(τ) =N∏

j=0

j 6=i


; (i = 0, ... ,N). (3–20)

where Li(τ), (i = 0, ... ,N) is a basis of Lagrange polynomials of degree N with support

points (τ0, ... , τN). The time derivative of the state at τ = τj is then approximated as

Y(τj) ≈N∑

i=0

Yi Li(τj) = [DY0:N ]j , (3–21)

where D is the N × (N + 1) Legendre-Gauss differentiation matrix whose elements are

given as Dij = Li (τj) , (i = 1, ... ,N, j = 0, ... ,N). Furthermore, the cost functional of

Eq. (3–2) is approximated using a Legendre-Gauss quadrature as

J = Φ(YN+1) +

N∑

j=1

wjg(Yj ,Uj). (3–22)

The differential optimal control problem is then approximated via the following finite-dimensional

nonlinear programming problem. Minimize the cost function of Eq. (3–22) subject to the

algebraic constraints

DY0:N = f(Y1:N ,U1:N), (3–23)

YN+1 = Y0 +

N∑

j=1

wj f(Yj ,Uj), (3–24)

φ(Y0) = 0, (3–25)

where w = (w1, ... ,wN) is a vector of Legendre-Gauss quadrature weights. It is

noted for LG collocation that Eq. (3–24) provides an LG quadrature approximation,

YN+1, of the state at the final noncollocated point τN+1 = +1. The NLP described by

Eqs. (3–22)–(3–25) will be referred to as the differential Legendre-Gauss collocation

method.

77

3.2.2 KKT Conditions Using Differential Legendre-Gauss Collocation

The Karush-Kuhn-Tucker (KKT) first-order optimality conditions of the differential

Legendre-Gauss collocation method are now derived [27, 64]. First the Lagrangian is

formed such that

L = Φ(YN+1) + 〈w, g(Y1:N,U1:N)〉 − 〈ψ,φ(Y0)〉

− 〈Λ1:N ,DY0:N − f(Y1:N,U1:N)〉

− 〈ΛN+1,YN+1 −Y0 −w⊤f(Y1:N,U1:N)〉,

where (ψ, Λ1:N , ΛN+1) are the KKT multipliers associated with the constraints of

Eqs. (3–25), (3–23), and (3–24), respectively. Next, the KKT conditions are obtained by

differentiating the Lagrangian with respect to all variables in the NLP. They are given as

DY0:N = f(Y1:N,U1:N), φ(Y0) = 0, (3–26)

YN+1 = Y0 + w⊤f(Y1:N,U1:N), (3–27)

0 = ∇UHG(Y1:N ,U1:N,Λ1:N+1), (3–28)

W−1D⊤1:NΛ1:N = ∇YHG(Y1:N,U1:N ,Λ1:N+1) (3–29)

D⊤0 Λ1:N = ΛN+1 −∇Y〈ψ,φ(Y0)〉, (3–30)

ΛN+1 = ∇YΦ(YN+1). (3–31)

where

HG(Y1:N ,U1:N,Λ1:N+1) = 1⊤g(Y1:N,U1:N) + 〈W−1Λ1:N + 1ΛN+1, f(Y1:N,U1:N)〉 (3–32)

is the discrete Hamiltonian. Furthermore, 1 is an N × 1 column vector of all ones, andW

is a N × N diagonal matrix of LG weights. Suppose now that the following transformed

78

dual variables are introduced:

λ1:N =W−1Λ1:N + 1ΛN+1, (3–33)

λN+1 = ΛN+1, (3–34)

λ0 = ΛN+1 −D⊤0 Λ1:N . (3–35)

In addition, consider the N × (N + 1) matrix D†,

D†ij = −

wj

wiDji , and D

†i,N+1 =

N∑

j=1

D†ij (3–36)

for (i , j = 1, ... ,N). It was shown in Ref. [36] that D† is a differentiation matrix for the

space of polynomials of degree N . In other words, if b is a polynomial of degree at most

N and b ∈ RN+1 is the vector whose i th element is bi = b(τi), then

(D†b)i = b(τi). (3–37)

Using the transformations described in Eqs. (3–33)–(3–35) along with Eq. (3–36), the

KKT conditions of the differential Legendre-Gauss collocation method of Eqs. (3–26)–(3–31)

can be written as [64]

DY0:N = f(Y1:N,U1:N), φ(Y0) = 0, (3–38)

YN+1 = Y0 + wT f(Y1:N,U1:N), (3–39)

0 = ∇UH(Y1:N,U1:N ,λ1:N), (3–40)

D†λ1:N+1 = −∇YH(Y1:N ,U1:N,λ1:N), (3–41)

λ0 = ∇Y〈ψ,φ(Y0)〉, (3–42)

λN+1 = ∇YΦ(YN+1), (3–43)

where H is a discrete form of the Hamiltonian given by Eq. (3–17). It is seen by

examination that Eqs. (3–38)–(3–43) is a discrete form of Eqs. (3–8)–(3–12).

79

It is noted in this transcription that the state is being differentiated by a matrix D

[given by Eq. (3–21)] which is based on the derivatives of polynomials of degree N with

coefficients at the N LG points plus the initial noncollocated point τ0 = −1, whereas the

costate is being differentiated by a matrix D† [given by Eq. (3–36)] which is based on

the derivatives of polynomials of degree N with coefficients at the N LG points plus the

terminal noncollocated point τN+1 = +1.

3.2.3 Integral Form of Legendre-Gauss Collocation

The integral optimal control problem is now discretized using the integral form of

Legendre-Gauss collocation. It has been shown in Ref. [64] that the differential LG

transcription method given in Section 3.2.1 can equivalently be expressed as an implicit

integration scheme. In particular, let p be any polynomial of degree at most N . By the

construction of the N × (N + 1) matrix D, then Dp = p where

pi = p(τi), (i = 0, ... ,N),

pi = p(τi), (i = 0, ... ,N).(3–44)

Now, let 1 be a N × 1 column vector composed of ones. since D is a differentiation

matrix, it follows that the components of the vector D1 are the derivatives at the

collocation points of the constant polynomial p(τ) = 1. Therefore, D1 = 0, which

implies that D1 = D0 +D1:N1 = 0. Rearranging,

D0 = −D1:N1. (3–45)

It has been shown by Ref. [64] that the matrix D1:N is full rank. Therefore, multiplying by

D−11:N gives the relationship

−D−11:ND0 = 1. (3–46)

Furthermore, the expression p can equivalently be written as

p = Dp = D0p0 +D1:Np1:N. (3–47)

80

premultiplying by D−11:N and utilizing relationship given by Eq. (3–46), the following is

obtained

pi = p0 +(

D−11:N p

)

i, (i = 1, ... ,N). (3–48)

Now, to show that the N × N matrix D−11:N is an integration matrix, a different

expression for pi − p0 can be obtained based on the integration of the interpolant of the

derivative. Let L†j (τ) be the Lagrange polynomial basis given by

L†j =

N∏

i=1i 6=j

τ − τiτj − τi

, (j = 1, ... ,N). (3–49)

Notice that the Lagrange polynomials Li defined in the differential problem formulation,

given by Eq. (3–20) are degree N while the Lagrange polynomials L†j are degree N − 1.

Then because p is a polynomial of degree at most N − 1, it can be interpolated exactly

by the Lagrange polynomials L†j :

p =

N∑

j=1

pjL†j (τ) (3–50)

Integrating p from −1 to τi , the following relationship is obtained

p(τi) = p(−1) +N∑

j=1

pjAij ,

Aij =

∫ τi

−1

L†j (τ)dτ , (i = 1, ... ,N),

(3–51)

which can equivalently be written as

pi = p0 + (Ap)i , (i = 1, ... ,N). (3–52)

The relations (3–48) and (3–52) are satisfied for any polynomial of degree at most N . By

equating (3–48) and (3–52) the following becomes true

Ap = D−11:Np.

81

Thus, the dynamic constraints of Eq. (3–6) can be approximated as

Y1:N = 1Y0 + Af(Y1:N,U1:N), (3–53)

where 1 denotes a N × 1 column vector of the constant 1 for every component, and

A = D−11:N is the N × N Legendre-Gauss integration matrix defined by Eq. (3–51).

Thus the state at each Legendre-Gauss point is approximated via quadrature using the

Legendre-Gauss integration matrix. Next, the state at τ = +1 is approximated using a

Legendre-Gauss quadrature as

YN+1 = Y0 +

N∑

i=1

wi f(Yi ,Ui).

Using these properties of LG collocation, the integral optimal control problem of

Eqs. (3–5)–(3–7) can be approximated via the following finite-dimensional nonlinear

programming problem. Minimize the cost function of Eq. (3–22) subject to the algebraic

constraints

Y1:N = 1Y0 + Af(Y1:N,U1:N), (3–54)

YN+1 = Y0 +wT f(Y1:N,U1:N), (3–55)

φ(Y0) = 0. (3–56)

The NLP described by Eqs. (3–22) and (3–54)–(3–56) will be referred to as the integral

Legendre-Gauss collocation method.

3.2.4 KKT Conditions Using Integral Legendre-Gauss Collocation

Similarly as was done for the differential form of the differential LG transcription, the

Karush-Kuhn-Tucker (KKT) first-order optimality conditions of the integral Legendre-Gauss

82

collocation method are now derived. First, the Lagrangian is defined as

L = Φ(YN+1) + 〈w, g(Y1:N,U1:N)〉 − 〈ψ,φ(Y0)〉

− 〈P1:N ,Y1:N − 1Y0 − Af(Y1:N,U1:N)〉

− 〈ΛN+1,YN+1 −Y0 −w⊤f(Y1:N,U1:N)〉,

where (ψ, P1:N , ΛN+1) are the KKT multipliers associated with the constraints of

Eqs. (3–56), (3–54), and (3–55), respectively. Next, the KKT conditions are obtained by

differentiating the Lagrangian with respect to all free variables in the NLP. They are given

as

Y1:N = 1Y0 + Af(Y1:N,U1:N), φ(Y0) = 0, YN+1 = Y0 + w⊤f(Y1:N,U1:N), (3–57)

0 = ∇U(

w⊤g(Y1:N,U1:N) + 〈A⊤P, f(Y1:N,U1:N)〉+ 〈ΛN+1,w⊤f(Y1:N,U1:N)〉)

, (3–58)

P1:N = ∇Y(

w⊤g(Y1:N,U1:N) + 〈A⊤P, f(Y1:N,U1:N)〉+ 〈ΛN+1,w⊤f(Y1:N,U1:N)〉)

, (3–59)

ΛN+1 = ∇Y〈ψ,φ(Y0)〉 − 1⊤P1:N, (3–60)

ΛN+1 = ∇YΦ(YN+1), (3–61)

Suppose now that the following transformed dual variables are introduced:

p1:N =W−1P1:N, λN+1 = ΛN+1. (3–62)

In addition, consider the N × N matrix A†,

A†ij = −

wj

wiAji , (3–63)

for (i , j = 1, ... ,N). It will now be proven that A† is a backward integration matrix for the

space of polynomials of degree N − 1.

Theorem 1. The matrix A† described by Eq. (3–63) is a backwards integration matrix for

the space of polynomials of degree N−1. More specifically, if p is a polynomial of degree

83

at most N − 1 and p ∈ RN is the vector with i th component pi = p(τi), (i = 1, ... ,N), then

(A†p)i =

∫ +1

τi

p(t)dt (3–64)

Proof. Let p, q denote polynomials of degree at most N − 1 such that pj = p(τj) and

qj = q(τj), (j = 1, ... ,N). The order of integration in a double integral can be switched by

changing the limits of integration such that

∫ +1

−1

[

q(τ) ·∫ τ

−1

p(t) dt

]

dτ =

∫ +1

−1

[

p(τ) ·∫ +1

τ

q(t) dt

]

dτ . (3–65)

Now, since p and q are polynomials of degree at most N − 1, then p ·∫ +1

τq dt and

q ·∫ τ

−1p dt are polynomials of degree at most 2N − 1. Since Gauss quadrature is exact

for polynomials of degree 2N − 1, the integrals in Eq. (3–65) can be replaced by their

quadrature equivalents to obtain

N∑

j=1

wjqj ·∫ τ

−1

p(t) dt =

N∑

j=1

wjpj ·∫ +1

τ

q(t) dt. (3–66)

Substituting∫ τ

−1p(t)dt = Ap and

∫ +1

τq(t) dt = A†q the following expression is obtained

(Wq)⊤Ap = (A†q)⊤Wp,

q⊤(A⊤W −WA†)p = 0.

(3–67)

Since p and q are arbitrary vectors, it must be true that

A⊤W −WA† = 0, (3–68)

which implies that

A†ij =wj

wiAji , (i , j = 1, ... ,N).

Furthermore, a polynomial of degree N − 1 is uniquely defined by its value at N points,

and can thus be exactly interpolated by a Lagrange interpolating polynomial of degree

84

N − 1. Therefore:

∫ +1

τi

q(τ)dτ =N∑

j=1

A†ijqj , A

†ij =

∫ +1

τi

L†j (τ)dτ , (3–69)

where L† is the basis of interpolating polynomials of degree N − 1 defined in Eq. (3–51).

Using the transformations described in Eq. (3–62) along with the definition of A† in

Eq. (3–63), the KKT conditions of the integral form of the Legendre-Gauss collocation

method as shown in Eqs. (3–57)–(3–61) can be written as

Y1:N = 1Y0 +Af(Y1:N ,U1:N), φ(Y0) = 0, YN+1 = Y0 + w⊤f(Y1:N,U1:N), (3–70)

0 = ∇Ug(Y1:N ,U1:N) + 〈(

A†p1:N +∇YΦ(YN+1))

, ∇Uf(Y1:N ,U1:N)〉, (3–71)

p1:N = ∇Yg(Y1:N,U1:N) + 〈(

A†p1:N +∇YΦ(YN+1))

, ∇Yf(Y1:N,U1:N)〉, (3–72)

w⊤p1:N = ∇Y〈ψ,φ(Y0)〉 − ∇YΦ(YN+1), (3–73)

λN+1 = ∇YΦ(YN+1). (3–74)

It is seen by examination that Eqs. (3–70)–(3–74) are a discrete form of the necessary

conditions for optimality of the integral optimal control problem given in Eqs. (3–13)–(3–16).

It must be noted that in the integral Legendre-Gauss collocation method this

transcription that the state, y(τ), and the dual variable, p(τ), are approximated using

an integration matrix for the space of polynomials of degree N − 1, the difference being

that the state is approximated using a forward quadrature using the matrix A given in

Eq. (3–51) while the integral costate, p(τ), is approximated using a backward quadrature

using the matrix A† given in Eq. (3–69).

3.2.5 Differential Costate Estimate Using Integral Legendre-Gauss Collocation

The results of Sections 3.2.1–3.2.4 can now be used to define an estimate

for the differential costate using the estimate of the integral costate. In particular,

the transformed necessary conditions of Eq. (3–38)–(3–43) are equivalent to the

85

transformed necessary conditions of Eqs. (3–70)–(3–73) if the discrete approximations

of λ(τ) and p(τ) are related as

D†λ1:N+1 = −p1:N , (3–75)

where D† is as defined in Eq. (3–36).

It has been shown by Ref.[64] that the matrix D† has the following properties

that are similar to the properties the matrix D: (a) the square matrix D†1:N obtained by

removing the last column of D† is full-rank, and (b) −[D†1:N ]

−1D†N+1 = 1. Using these

properties, Eq. (3–75) can be rewritten as

λ1:N = 1λN+1 − [D†1:N ]

−1p1:N ,

λ1:N = 1λN+1 + A†p1:N,

(3–76)

where the matrix A† is the integration matrix defined by Eq. (3–69). From this relationship,

it can be seen that the differential and integral dual variable estimates are related

by [D†1:N ]

−1 = −A†. Furthermore, the costate at the initial noncollocated point τ0 is

approximated through a Gaussian quadrature such that

λ0 = λN+1 + w⊤p1:N. (3–77)

It can be seen that applying the transformations of Eqs. (3–75)–(3–77) to the transformed

first-order optimality conditions given in Eqs. (3–70)–(3–74) will result in the transformed

first-order optimality conditions given in Eqs. (3–38)–(3–43).

3.3 Costate Estimation Using Integral Legendre-Gauss-Radau Collocation

In this section a costate estimate for the differential optimal control problem is

developed via an estimate of the integral costate obtained using the integral form

of the Legendre-Gauss-Radau orthogonal collocation method. In Section 3.3.1 the

differential form of the Legendre-Gauss collocation method is revisited. In Section 3.3.2

the first-order optimality conditions of the nonlinear programming problem described

86

in Section 3.3.1 are derived. In Section 3.3.3 the integral form of the Legendre-Gauss

collocation method is described. In Section 3.3.4 the first-order optimality conditions of

the nonlinear programming problem described in Section 3.3.1 are derived. Finally, in

Section 3.3.5 a differential costate estimate using the integral costate estimate derived in

Section 3.3.4 is presented.

3.3.1 Differential Form of Legendre-Gauss-Radau Collocation

The differential optimal control problem of Eqs. (3–2)–(3–4) can now be approximated

using collocation at Legendre-Gauss-Radau (LGR) points as was described in Section

2.5.2. The LGR points are defined in the domain [−1,+1) such that τ1 = −1 is a LGR

collocation point but τN+1 = +1 is a noncollocated point. The state is approximated

using a Lagrange polynomial with support points at the N LGR points plus the

noncollocated point τN+1 = +1, such that

y(τ) ≈ Y(τ) =N+1∑

i=1

YiLi(τ), Li(τ) =

N+1∏

j=1

j 6=i


; (i = 1, ... ,N + 1). (3–78)

where Li(τ), (i = 1, ... ,N + 1) is a basis of Lagrange polynomials of degree N

with support points (τ1, ... , τN+1). The time derivative of the state at τ = τj is then

approximated as

Y(τj) ≈N+1∑

i=1

Yi Li(τj) = [DY1:N+1]j , (3–79)

where D is the N × (N + 1) Legendre-Gauss-Radau differentiation matrix whose

elements are given as Dij = Li (τj) , (i = 1, ... ,N, j = 1, ... ,N + 1). Furthermore, the

cost functional of Eq. (3–2) is approximated using a Legendre-Gauss-Radau quadrature

as

J = Φ(YN+1) +

N∑

j=1

wjg(Yj ,Uj). (3–80)

The differential optimal control problem is then approximated via the following finite-dimensional

nonlinear programming problem. Minimize the cost function of Eq. (3–80) subject to the

87

algebraic constraints

DY1:N+1 = f(Y1:N,U1:N), (3–81)

φ(Y1) = 0, (3–82)

where w = (w1, ... ,wN) is a vector of Legendre-Gauss-Radau quadrature weights. The

NLP described by Eqs. (3–80)–(3–82) will be referred to as the differential Legendre-

Gauss-Radau method.

3.3.2 KKT Conditions Using Differential Legendre-Gauss-Radau Collocation

Similarly as was done for collocation at the LG points, the Karush-Kuhn-Tucker

(KKT) first-order optimality conditions of the differential Legendre-Gauss-Radau

collocation method are now derived [64]. First the Lagrangian is formed such that

L = Φ(YN+1) + 〈w, g(Y1:N,U1:N)〉 − 〈ψ,φ(Y1)〉

− 〈Λ1:N ,DY1:N+1 − f(Y1:N ,U1:N)〉,

where (ψ, Λ1:N) are the KKT multipliers associated with the constraints of Eqs. (3–82)

and (3–81), respectively. Next, the KKT conditions are obtained by differentiating the

Lagrangian with respect to all variables in the NLP. They are given as

DY1:N+1 = f(Y1:N,U1:N), φ(Y1) = 0, (3–83)

0 = ∇UHR(Y1:N,U1:N,Λ1:N), (3–84)

D⊤1:NΛ1:N = ∇YHR(Y1:N,U1:N,Λ1:N)− e1∇Y〈ψ,φ(Y1)〉, (3–85)

D⊤N+1Λ1:N = ∇YΦ(YN+1), (3–86)

where

HR(Y1:N ,U1:N,Λ1:N) = w⊤g(Y1:N ,U1:N) + 〈Λ1:N , f(Y1:N,U1:N)〉 (3–87)

88

is the discrete Hamiltonian, and e1 is the first column of the identity matrix. Suppose now

that the following transformed dual variables are introduced:

λ1:N =W−1Λ1:N, (3–88)

λN+1 = D⊤N+1Λ1:N. (3–89)

In addition, consider the N × N matrix D†,

D†11 = −D11 −

1

w1and D

†ij = −

wj

wiDji otherwise (3–90)

for (i , j = 1, ... ,N). It was shown in Ref. [36] that D† is a differentiation matrix for the

space of polynomials of degree N − 1. In other words, if b is a polynomial of degree at

most N − 1 and b ∈ RN is the vector i th element bi = b(τi), then

(D†b)i = b(τi). (3–91)

Using the transformations described in Eqs. (3–88)–(3–89) along with Eq. (3–90),

the KKT conditions of differential Legendre-Gauss-Radau collocation method of

Eqs. (3–83)–(3–86) can be written as[36]

DY1:N+1 = f(Y1:N,U1:N), φ(Y1) = 0, (3–92)

0 = ∇UH(Y1:N,U1:N ,λ1:N), (3–93)

D†λ1:N = −∇YH(Y1:N ,U1:N,λ1:N) +e1

w1(∇Y〈ψ,φ(Y1)〉 − λ1) , (3–94)

λN+1 = ∇YΦ(YN+1), (3–95)

These equations are incomplete because a new variable λN+1 was introduced without

adding a new equation. An equation for this new variable can be developed by

manipulating the matrix D. Because D is a differentiation matrix, it has the property

that D1 = 0, where 1 is a vector whose components are all constant and equal to 1. This

89

implies that

DN+1 = −N∑

j=1

D1:N, j ,

D⊤N+1Λ =

N∑

i=1

Di,N+1Λi = −N∑

i=1

N∑

j=1

Di, jΛi ,

λN+1 = λ1 +

N∑

i=1

N∑

j=1

wiλjD†i,j = λ1 +

N∑

j=1

wj(D†λ)j , (3–96)

where the relationships in (3–88) and (3–91) were used to obtain Eq. (4–70). It can be

seen that this relationship approximates the integral of the costate dynamics across the

domain via a Radau quadrature. Combining Eqs. (3–92)–(3–95) with Eq. (4–70), the

complete transformed adjoint system can then be written as

DY1:N+1 = f(Y1:N,U1:N), φ(Y1) = 0, (3–97)

0 = ∇UH(Y1:N,U1:N ,λ1:N), (3–98)

D†λ1:N = −∇YH(Y1:N ,U1:N,λ1:N) +e1

w1(∇Y〈ψ,φ(Y1)〉 − λ1) , (3–99)

λN+1 = ∇Y〈ψ,φ(Y0)〉 −N∑

i=1

wi∇YH(Yi ,Ui ,λi), (3–100)

λN+1 = ∇YΦ(YN+1), (3–101)

where H is a discrete form of the Hamiltonian given by Eq. (3–17). It is seen in

Eq. (3–100) that the costate at the noncollocated final point τ = +1 is approximated

via a Legendre-Gauss-Radau quadrature of the costate dynamics across the solution

domain. Consequently, Eq. (3–100) is a subtle way of enforcing the relationship λ1 =

∇Y〈ψ,φ(Y1)〉 and it is expected that the last term of Eq. (3–99) will be small while the

remaining terms in Eq. (3–99) are a collocation collocation scheme for the continuous

adjoint equation. The transformed optimality conditions of Eqs. (3–97)–(3–101) are,

thus, a discrete form of the necessary conditions for optimality of the differential optimal

control problem described by Eqs. (3–8)–(3–12). It is noted in these conditions that

90

the time derivative of the state is being approximated using the differentiation matrix D

for the space of polynomials of degree N [see Eq. (3–79)], while the costate is being

differentiated by a differentiation matrix D† for the space of polynomials of degree N − 1

[see Eq. (3–90)].

3.3.3 Integral Form of Legendre-Gauss-Radau Collocation

The integral optimal control problem is now discretized using the integral form of

Legendre-Gauss-Radau collocation. It has been shown in Ref. [64] that the differential

LG transcription method given in Section 3.2.1 can equivalently be expressed as an

implicit integration scheme. In particular, suppose that p is any polynomial of degree

at most N . Then, by the construction of the N × (N + 1) differentiation matrix D, then

Dp = p where

pi = p(τi), (i = 1, ... ,N + 1),

i = p(τi), (i = 1, ... ,N + 1).(3–102)Now, let 1 be a N × 1 column vector composed

of ones. since D is a differentiation matrix, it follows that the components of the vector

D1 are the derivatives at the collocation points of the constant polynomial p(τ) = 1.

Therefore, D1 = 0, which implies that D1 = D1:N1+DN+1 = 0. Rearranging,

DN+1 = −D1:N1. (3–103)

It has been shown by Ref. [64] that the matrix D1:N is full rank. Therefore, pre-multiplying

by D−11:N gives the relationship

−D−11:NDN+1 = 1. (3–104)

Furthermore, the expression p can equivalently be written as

p = Dp = D1:Np1:N +DN+1pN+1. (3–105)

premultiplying by D−11:N and utilizing the relationship given by Eq. (3–104), the following is

obtained

pi = pN+1 +(

D−11:Np

)

i, (i = 1, ... ,N). (3–106)

91

Next, let Lj(τ) be Lagrange polynomial basis given by

Lj(τ) =N∏

i=1i 6=j


, (j = 1, ... ,N).

Then p can be interpolated exactly as

p(τ) =

N∑

j=1

pj Lj(τ).

Integrating from +1 to τi ,

pi = pN+1 + (Ap)i , (i = 1, ... ,N), (3–107)

Aij =

∫ τi

+1

Lj(τ) dτ , (i , j = 1, ... ,N). (3–108)

By (3–106) and (3–107), it can be seen that A = D−11:N . Thus, the dynamic constraints of

Eq. (3–6) can be approximated as

Y1:N = 1YN+1 + Af(Y1:N,U1:N), (3–109)

where 1 denotes a N × 1 column vector of the constant 1 for every component, and

A = D−11:N is the N × N Legendre-Gauss-Radau integration matrix given by Eq. (3–108).

The integral optimal control problem can then be approximated via the following

finite-dimensional nonlinear programming problem. Minimize the cost function of

Eq. (3–80) subject to the algebraic constraints

Y1:N = 1YN+1 +Af(Y1:N ,U1:N), (3–110)

φ(Y1) = 0, (3–111)

The NLP described by Eqs. (3–80), (3–110), and (3–111) will be referred to as the

integral Legendre-Gauss-Radau collocation method.

92

3.3.4 KKT Conditions Using Integral Legendre-Gauss-Radau Collocation

The Karush-Kuhn-Tucker (KKT) first-order optimality conditions of the integral

Legendre-Gauss-Radau collocation method are given as

Y1:N = 1YN+1 + Af(Y1:N,U1:N), φ(Y1) = 0, (3–112)

0 = ∇U(

w⊤g(Y1:N ,U1:N) + 〈A⊤P1:N , f(Y1:N ,U1:N)〉)

(3–113)

P1:N = ∇Y(

w⊤g(Y1:N,U1:N) + 〈A⊤P1:N , f(Y1:N,U1:N)〉)

− e1∇Y〈ψ , φ(Y1)〉 (3–114)

N∑

i=1

Pi = −∇YΦ(YN+1). (3–115)

where (P1:N, ψ) are the KKT multipliers associated with the constraints of Eqs. (3–110)

and (3–111), respectively. Suppose now that we introduce the following transformed

dual variables:

p2:N =W−12:N,2:NP2:N, λN+1 = −

N∑

i=1

Pi . (3–116)

In addition, suppose we introduce the N × N matrix A†,

A†ij =wj

wiAji + wj , (3–117)

for (i , j = 1, ... ,N). We will now prove that A† is a backward integration matrix for the

space of polynomials of degree N − 2.

Theorem 2. The matrix A† described by Eq. (3–117) is a backwards integration matrix

for the space of polynomials of degree N − 2. More specifically, let p be a polynomial of

degree at most N − 2 and p ∈ RN−1 be the vector with i th component pi = p(τi), (i =

2, ... ,N). Since a polynomial of degree N − 2 is uniquely defined by its values at N − 1

points, p can be expressed exactly as

p(τ) =

N∑

j=2

pi · L†j (τ), L†j (τ) =N∏

i=2j 6=i


. (3–118)

93

Furthermore, let p1 = p(τ1) be the extrapolated value of p(τ) evaluated at τ1 such that

p(τ1) ≈ p1 =N∑

j=2

pjL†j (τ1). (3–119)

Then the matrix A† approximates the integral

(A†p)i =

∫ +1

τi

p(t) dt. (3–120)

Proof. Let p denote the polynomials of degree at most N − 2 defined in the statement

of the theorem. Furthermore, let q denote a polynomial of degree N − 1 which satisfies

qj = q(τj), (j = 1, ... ,N). If p and q are smooth, real-valued functions then integration by

parts gives

∫ +1

−1

[

p(τ) ·∫ τ

+1

q(t) dt

]

dτ = −∫ −1

+1

p(τ) dτ ·∫ −1

+1

q(τ) dτ +

∫ +1

−1

[

q(τ) ·∫ +1

τ

p(t) dt

]

dτ .

(3–121)

Now, since p is a polynomial of degree at most N − 2 and q is a polynomial of degree at

most N − 1, then q ·∫ +1

τp dt and p ·

∫ τ

+1q dt are polynomials of degree at most 2N − 2.

Since Radau quadrature is exact for polynomials of degree 2N − 2, the integrals in

Eq. (3–121) can be replaced by their quadrature equivalents to obtain

N∑

j=1

wjpj ·∫ τ

+1

q(t) dt = −N∑

i=1

wipi ·N∑

j=1

wjpj +

N∑

j=1

wjqj ·∫ +1

τ

p(t) dt. (3–122)

Substituting∫ τ

+1q(t)dt = Aq and

∫ +1

τp(t) dt = A†p with the first column of A† consisting

of a N × 1 column vector of zeros, the following expression is obtained

(Aq)⊤Wp = −(Wq)⊤1N×NWp+ (Wq)⊤A†p,

q⊤(A⊤W +W1N×NW −WA†)p = 0.

(3–123)

Since p and q are arbitrary vectors, then

A⊤W +W1N×NW −WA† = 0, (3–124)

94

which implies that

A†ij =wj

wiAji + wj , (i , j = 1, ... ,N).

It has previously been shown that the matrix A defined as

Aij =wj

wiAji , (i , j = 1, ... ,N) (3–125)

has the form

Aij = −∫ τi

−1

L†j (τ)dτ + w1L

†j (τ1), (i , j = 2, ... ,N),

Ai1 = −w1, (i = 1, ... ,N),

A1j = w1L†j (τ1), (j = 2, ... ,N).

(3–126)

Since A†ij = Aij + wj , for (i , j = 1, ... ,N), then A† has the form

A†ij = −

∫ τi

−1

L†j (τ)dτ + w1L

†j (τ1) + wj , (i , j = 2, ... ,N),

A†i1 = 0, (i = 1, ... ,N),

A†1j = w1L

†j (τ1) + wj , (j = 2, ... ,N).

(3–127)

To better understand the structure of the matrix A†, suppose the N × 1 vector p is

multiplied by A† where pi = p(τi) and p(τ) is the polynomial of degree N − 2 defined in

Eq. (3–118). The resulting operation yields

(A†p)i =

N∑

j=2

w1pjL†j (τ1) +

N∑

j=2

wjpj −N∑

j=2

pj

∫ τi

−1

L†j (τ). (3–128)

The summation∑N

j=2 pjL†j (τ1) extrapolates the (N − 2)th-degree polynomial p(τ) to

the initial point τ1. Therefore the sum of the first and second terms in Eq. (3–128) is

a Legendre-Gauss-Radau quadrature approximation of∫ +1

−1p(τ)dτ . Furthermore,

because p(τ) is a polynomial of degree N − 2 it can be interpolated exactly as defined

in Eq. (3–118). Therefore the final term of Eq. (3–128) approximates∫ τ

−1p(τ)dτ . This

shows that A† is thus an integration matrix for the space of polynomials of degree N − 2

95

which approximates

(A†p)i =

∫ +1

τi

p(τ)d(τ).

Recall that the first column of A† is all zeros (that is, A†1 = 0), and that p is being

approximated by a polynomial of degree N − 2 which is uniquely defined by its values at

N − 1 points. Using the transformations described in Eq. (3–116) along with Eq. (3–117),

the KKT conditions of the integral Legendre-Gauss-Radau collocation method as shown

in Eqs. (3–112)–(3–115) can be written as

Y1:N = 1Y1 + Af(Y1:N,U1:N), φ(Y1) = 0, (3–129)

0 = ∇Ug(Y1:N,U1:N) + 〈(

A†2:Np2:N +∇YΦ(YN+1)

)

, ∇Uf(Y1:N ,U1:N)〉, (3–130)

p2:N = ∇Yg(Y2:N ,U2:N) + 〈(

A†2:N,2:Np2:N +∇YΦ(YN+1)

)

, ∇Yf(Y2:N,U2:N)〉, (3–131)

1

w1(P1 +∇Y〈ψ , φ(Y1)〉)

= ∇Yg(Y1,U1) + 〈(

A†1,2:Np2:N +∇YΦ(YN+1)

)

, ∇Yf(Y1,U1)〉, (3–132)

P1 = −N∑

i=2

wipi −∇YΦ(YN+1). (3–133)

Substituting the values of pi , (i = 2, ... ,N), and P1 obtained from Eqs. (3–131) and

(3–132), respectively, into Eq. (3–133) yields

N∑

i=1

wi

(

∇Yg(Yi ,Ui) + 〈(

A†i,2:Np2:N+∇YΦ(YN+1) , ∇Yf(Yi ,Ui)〉

=∇Y〈ψ , φ(Y1)〉 − ∇YΦ(YN+1),(3–134)

Equation (3–134) is a Legendre-Gauss-Radau quadrature approximation to the

continuous-time condition of Eq. (3–16). It can be seen that Eqs. (3–130) and (3–131)

are approximations to the continuous-time optimality conditions of Eqs. (3–14) and

(3–15), respectively. Therefore, it has been shown that Eqs. (3–129)–(3–131) along

with Eq. (3–134) form a set of necessary conditions for optimality and are a discrete

96

approximation to the necessary conditions for optimality of the integral form of the

optimal control problem as defined in Eqs. (3–5)–(3–7).

3.3.5 Differential Costate Estimate Using Integral Legendre-Gauss-RadauCollocation

The results of Sections 3.3.1–3.3.4 can now be used to define an estimate for the

differential costate using the integral costate estimate from the Legendre-Gauss-Radau

collocation method. In particular, the transformed necessary conditions of Eq. (3–97)–(3–101)

are equivalent to the transformed necessary conditions of Eq. (3–129)–(3–131) along

with Eq. (3–134) if the discrete approximations of λ(τ) and p(τ) are related as

λ1:N = λN+1 + A†p2:N , (3–135)

where the matrix A† is the integration matrix for the space of polynomials of degree N−2

defined in Eq. (3–117). Equation (3–135) can equivalently be written in differential form

such that

D†λ1:N = −p1:N (3–136)

where the matrix D† is a differentiation matrix for the space of polynomials of degree N −

1 defined by Eq. (3–90). It can be seen that applying the transformation of Eq. (3–135)

to the optimality conditions of the integral Legendre-Gauss-Radau collocation method

as given in Eqs. (3–129)–(3–131) along with Eq. (3–134) will result in the first-order

optimality conditions of problem the differential Legendre-Gauss-Radau collocation

method as given in Eqs. (3–97)–(3–101).

3.4 Discussion

While it may appear at first glance as if the costate estimate using either the integral

Legendre-Gauss or integral Legendre-Gauss-Radau method is the same, the estimate

obtained using either of the methods has nuances that distinguish it from the estimate

obtained using the other method. First, the N ×N Legendre-Gauss-Radau differentiation

matrix of Eq. (3–90) is singular for the space of polynomials of degree N − 1, while the

97

N × (N + 1) Legendre-Gauss differentiation matrix of Eq. (3–36) is full rank for the space

of polynomials of degree N . Second, the N × N matrix A† in Eq. (3–117) associated with

the integral Legendre-Gauss-Radau method is singular and simultaneously integrates

and interpolates a polynomial of degree N − 2, while the matrix N × N matrix A† in

Eq. (3–69) is an integration matrix for the space of polynomials of degree N − 1. Finally,

although both integral collocation methods provide estimates of the differential costate,

λ(τ), at all discretization points in the domain (including both endpoints), the integral

Legendre-Gauss collocation method produces an approximation of integral costate,

p(τ), at only the N Legendre-Gauss points while the integral Legendre-Gauss-Radau

collocation method provides estimates of p(τ) at N − 1 interior Legendre-Gauss-Radau

points and extrapolates the value of p(τ) to the initial (τ = −1) Legendre-Gauss-Radau

point.

3.5 Concluding Remarks

A method was presented for costate estimation of an optimal control problem

using orthogonal collocation at Legendre-Gauss and Legendre-Gauss-Radau points

when the dynamic constraints are presented in integral form. A key feature of these

collocation schemes is that the inverse of the matrix associated with an implicit LG (or

LGR) integration scheme is the LG (or LGR) differentiation matrix. Hence, the methods

presented in this chapter can be thought of as either an implicit integration method or

a differential method. It was shown that the KKT multipliers stemming from the implicit

integration transcription can be related to the costate of the differential form of the

problem via an integration matrix. The LG collocation scheme yielded a costate estimate

which was approximated by a polynomial of degree N , whereas the costate estimate for

the LGR collocation scheme was approximated by a polynomial of degree N − 1. The

relationship between the costate of the differential formulation and the dual multipliers

of the implicit integral formulation provided an equivalence between the first-order

98

optimality conditions of the optimal control problem when posed with the constraints in

either form.

99

CHAPTER 4MOTIVATION FOR NEW COSTATE ESTIMATE

In this chapter a motivation is given for developing new costate estimates for

variable-order collocation at the Legendre-Gauss (LG) and flipped Legendre-Gauss-Radau

(LGR) points when solving problems with active state inequality path constraints. In

particular, a previously derived costate estimate for variable-order collocation at the LG

and flipped LGR points will be presented [1]. It will be shown that in the case when the

costate is discontinuous (as is the case in the presence of active state inequality path

constraints), this costate estimate leads to a set of first-order optimality conditions of the

NLP that are not equivalent to the discrete form of the variational optimality conditions.

This lack of equivalence leads to an inaccurate approximation of the costate.

The following notation and conventions will be used throughout this chapter in

order to make the exposition more clear. First, all vector functions of time are denoted

as row vectors, that is, if y(τ) ∈ Rn is a vector function of the scalar variable τ , then

y(τ) = [y1(τ), · · · , yn(τ)]. Next, any capital boldface character, Y, denotes a matrix of

size M × n, where each row of Yi corresponds to the evaluation of a function y(τ) at a

particular value τ = τi . Next, the notation Yi:j denotes rows i through j of the matrix Y,

except when referring to a differentiation matrix D or the integration matrix A, in which

case Di and Ai refers to the i th column of D and A. Finally, D⊤ denotes the transpose

of matrix D, and D⊤i denotes the transpose of the i th column of D. Given vectors x and

y ∈ Rn, the notation 〈x, y〉 is used to denote the standard inner product between x

and y. Furthermore, if f : Rn −→ Rm, then ∇f is the m by n Jacobian matrix whose

i th row is ∇fi . In particular, the gradient of a scalar-valued function is a row vector. If

φ : Rm×n −→ R and Y is an m by n matrix, then ∇φ denotes the m by n matrix whose

(i , j) element is (∇φ(Y))ij = ∂φ(Y)/∂Yij .

The remainder of this chapter is organized as follows. Section 4.1 reformulates the

continuous-time Bolza optimal control problem of Section 2.1 such that the domain is

100

divided into a mesh and each mesh interval is defined in the domain τ ∈ [−1,+1]. The

transformation of variables is done to facilitate comparison of the first-order optimality

conditions of the continuous-time problem with the modified optimality conditions

of the variable-order collocation methods. Next, Section 4.2 presents a previously

derived costate estimate for variable-order collocation at the LG points, develops the

transformed adjoint system, and shows that the costate estimate is only valid for the

case when the costate is continuous across interval boundaries in the domain. Similarly,

in Section 4.3 a previously derived costate estimate for variable-order collocation at

the flipped LGR points is presented, the transformed adjoint system is developed, and

it is shown that the costate estimate does not lead to an accurate approximation of

the continuous-time optimal costate in the case when the costate is discontinuous.

Finally, Section 4.4 discusses the implications of the inaccuracy in the costate estimates

presented.


Consider again the continuous Bolza problem that was presented in Section 2.1. To

simplify comparisons with the transformed adjoint system, the domain t ∈ [t0, tf ] = I

is now divided into K sub-intervals Sk = [Tk−1,Tk ] ⊆ [t0, tf ], (k = 1, ... ,K), where

T0 = t0, TK = tf , Tk−1 < Tk , (k = 1, ... ,K), and⋃K

k=1 Sk = I. Furthermore,

without loss of generality the optimal control problem can be scaled by transforming the

independent variable in each sub-interval from t ∈ [Tk−1,Tk ] to τ (k) ∈ [−1,+1] via the


t =Tk −Tk−12

τ (k) +Tk + Tk−12

(4–1)

such that

dt =Tk − Tk−12

dτ (k) ≡ h(k)

2, where h(k) ≡ Tk − Tk−1. (4–2)

The optimal control problem problem then becomes to determine the state y(k)(τ) ∈ Rn,

and the control u(k)(τ) ∈ Rm in each sub-interval (k = 1, ... ,K), to minimize the cost

101

functional

J = Φ(y(K)(+1)) +

K∑

k=1

h(k)

2

∫ +1

−1

g(y(k)(τ), u(k)(τ))dτ (4–3)

subject to the dynamic constraints

y(k)(τ) =h(k)

2f(y(k)(τ), u(k)(τ)), (k = 1, ... ,K), (4–4)

the boundary conditions

φ(y(1)(−1)) = 0, (4–5)


C(y(k)(τ), u(k)(τ)) ≤ 0, (k = 1, ... ,K). (4–6)

4.1.1 First-Order Optimality Conditions of Continuous Problem

The first-order optimality conditions of the optimal control problem given by

Eqs. (4–3)–(4–6) can be derived from the calculus of variations in the manner described

by Section 2.1. They are given as

y(k) = f(y(k), u(k)), φ(y(1)(−1)) = 0, (k = 1, ... ,K), (4–7)

0 = ∇uH(y(k), u(k),λ(k),µ(k)), (k = 1, ... ,K), (4–8)

−λ(k) = h(k)

2∇yH(y(k), u(k),λ(k),µ(k)), (k = 1, ... ,K), (4–9)

λ(1)(−1) = ∇y 〈ψ,φ(y(1))〉|τ=−1, (4–10)

λ(K)(+1) = ∇yΦ(y(K))|τ=+1, (4–11)

C(y(k), u(k)) ≤ 0, µ(k)(τ) ≤ 0, 〈µ(k),C(y(k), u(k))〉 = 0, (k = 1, ... ,K), (4–12)

where (λ(k),µ(k)) are the Lagrange multipliers associated with the constraints of

Eqs. (4–4) and (4–6) in interval k , and ψ are the Lagrange multipliers associated with

the boundary conditions of Eq. (4–5). Furthermore, the Hamiltonian in interval k is given

as

H(y(k), u(k),λ(k),µ(k)) = g(k) + 〈λ(k), f(k)〉 − 〈µ(k),C(k)〉. (4–13)

102

4.2 Variable-Order Collocation at Legendre-Gauss Points

The optimal control problem of Eqs. (4–3)–(4–6) is now discretized using variable-order

collocation at the Legendre-Gauss (LG) points as described in Section 2.5.4. First, recall

that the LG points (τ0, ... , τN+1) are defined in the domain τ ∈ (−1,+1) such that

τ0 = −1 and τN+1 = +1 are noncollocated points. The state is then approximated in

each mesh interval k as

y(k)(τ) ≈ Y(k)(τ) =Nk∑

i=0

Y(k)i L

(k)i (τ), L

(k)i (τ) =

Nk∏

j=0

i 6=j

τ − τ (k)j

τi − τ (k)j, (4–14)


Y(k)(τj) ≈Nk∑

i=0

Y(k)i L

(k)i (τj) = [D

(k)Y(k)0:Nk]j , (4–15)

where D(k)ij = L(k)i (τ)j , (i = 1, ... ,Nk, j = 0, ... ,Nk) are the components of the

Nk × (Nk + 1) Legendre-Gauss (LG) differentiation matrix in the k th mesh interval. It is

noted that when implementing the variable-order LG method, a single variable is used

for the value of the state at the end of mesh interval k and the start of mesh interval

k + 1, that is, Y(k)Nk+1 ≡ Y(k+1)0 , 1 ≤ k ≤ K − 1 such that continuity in the state is enforced.

Next, the cost functional of Eq. (2–1) is approximated using a multiple-interval LG

quadrature. The finite-dimensional approximation of the continuous-time optimal control

problem of Eqs. (2–1)–(2–4) is then given as follows. Minimize the cost function

J ≈ Φ(Y(K)NK+1) +K∑

k=1

Nk∑

j=1

h(k)

2w(k)j g(Y

(k)j ,U

(k)j ), (4–16)

103


D(k)Y(k)0:Nk

=h(k)

2f(Y

(k)1:Nk,U(k)1:Nk), (k = 1, ... ,K) (4–17)

Y(k+1)0 = Y

(k)0 +

h(k)

2

Nk∑

j=1

w(k)j f(Y

(k)j ,U

(k)j ), (k = 1, ... ,K − 1) (4–18)

Y(K)N+1 = Y

(K)0 +

h(K)

2

Nk∑

j=1

w(K)j f(Y

(K)j ,U

(K)j ), (4–19)

φ(Y(1)0 ) = 0, (4–20)

C(Y(k)1:Nk,U(k)1:Nk ) ≤ 0, (k = 1, ... ,K), (4–21)

where w(k) = (w (k)1 , ... ,w(k)N k) are the LG quadrature weights in interval k . It is noted

for LG collocation that Eq. (4–18) provides an LG quadrature approximation, Y(k)0 , of

the state at the final noncollocated point τ (k)N+1 = +1 in interval (k = 1, ... ,K − 1), while

Eq. (4–19) provides an LG quadrature approximation, Y(K)N+1, of the state at the final

noncollocated point of the domain, tf = τ (k)N+1 = +1.

4.2.1 KKT Conditions of Variable-Order Legendre-Gauss Collocation Method

The first-order optimality conditions of the discrete problem given by Eqs. (4–16)–(4–21),

also called the KKT conditions of the NLP, are now derived. First, the Lagrangian is

defined as

L = Φ(Y(K)NK+1)− 〈ψ,φ(Y(1)0 )〉+

K∑

k=1

Nk∑

i=1

(

h(k)

2w(k)i g

(k)i − 〈Γ(k)i ,C(k)i 〉

)

−K∑

k=1

Nk∑

i=1

(

〈Λ(k)i ,D(k)i,0:NkY(k)0:Nk− h

(k)

2f(k)i 〉

)

−K−1∑

k=1

Nk∑

i=1

(

〈Λ(k)Nk+1,Yk+10 − Y(k)0 −

h(k)

2w(k)i f(k)i 〉

)

−NK∑

i=1

(

〈Λ(K)NK+1,Y(K)NK+1

− Y(K)0 − h(K)

2w(K)i f(K)i 〉

)

(4–22)

where (Λ(k)i ,Λ(k)Nk+1,Γ(k)i ) are the Lagrange multipliers associated with the dynamic

constraints of Eq. (4–17), the quadrature constraints of Eqs. (4–18)–(4–19), and the

104

inequality path constraint of Eq. (4–21), respectively, in interval k at the LGR point

τi . Furthermore ψ denotes the Lagrange multipliers associated with the boundary

conditions of Eq. (4–20). Note that function dependencies have been omitted for clarity,

such that g(k)i ≡ g(Y(k)i ,U(k)i ), and similarly f(k)i ≡ f(Y(k)i ,U(k)i ) and C(k)i ≡ C(Y(k)i ,U(k)i ).

The KKT conditions of the NLP are then given as

D(k)0:NkY(k)0:Nk=h(k)

2f(Y

(k)1:Nk,U(k)1:Nk ), φ(Y

(1)0 ) = 0, (k = 1, ... ,K), (4–23)

Y(k+1)0 = Y

(k)0 +

Nk∑

i=1

w(k)i f(Y

(k)i ,U

(k)i ), (k = 1, ... ,K − 1), (4–24)

Y(k)NK+1

= Y(K)0 +

NK∑

i=1

w(K)i f(Y

(K)i ,U

(K)i ), (4–25)

0 = ∇UH(Y(k)1:Nk,U(k)1:Nk ,Λ

(k)1:Nk+1

,Γ(k)1:Nk), (k = 1, ... ,K), (4–26)

D(k)⊤1:NkΛ(k)1:Nk=h(k)

2∇YH(Y(k)1:Nk ,U

(k)1:Nk,Λ(k)1:Nk+1

,Γ(k)1:Nk), (k = 1, ... ,K), (4–27)

D(1)⊤0 Λ(1)1:N1= Λ

(1)N1+1

−∇Y〈ψ,φ(Y(1)0 )〉, (4–28)

D(k)⊤0 Λ(k)1:N1= Λ

(k)Nk+1

− Λ(k−1)Nk+1, (k = 2, ... ,K), (4–29)

Λ(K)NK+1

= ∇YΦ(Y(K)NK+1), (4–30)

C(Y(k)1:Nk ,U(k)1:Nk) ≤ 0, Γ(k)1:Nk ≤ 0, 〈Γ

(k)1:Nk,C(Y(k)1:Nk ,U

(k)1:Nk)〉 = 0, (k = 1, ... ,K). (4–31)

The discrete Hamiltonian in interval k is given as

H(Y(k)1:Nk,U1:Nk ,Λ

(k)1:Nk+1

,Γ(k)1:Nk) = w (k)⊤g

(k)1:Nk+ 〈Λ(k)1:Nk +W

(k)1Λ(k)Nk+1, f(k)1:Nk〉−〈 2h(k)Γ(k)1:Nk,C(k)1:Nk〉,

whereW(k) is a N × N diagonal matrix of LG quadrature weights in interval k , and 1 is a

N × 1 column vector of ones.

4.2.2 Costate Estimate and Transformed Adjoint System

Consider the following costate estimate, first derived by Ref. [1], that relates the

KKT conditions multipliers of Eqs. (4–23)–(4–31) to the dual variables of the first-order

105

optimality [given by Eqs. (4–7)–(4–12)] of the continuous-time optimal control problem:

µ(k)i =

2

h(k)Γ(k)i

w(k)i

, (i = 1, ... ,Nk), (k = 1, ... ,K), (4–32)

λ(k)i = Λ

(k)Nk+1

+Λ(k)i

w(k)i

, (i = 1, ... ,Nk), (k = 1, ... ,K), (4–33)

λ(k)0 = Λ

(k)Nk+1

− D(k)⊤

0 Λ(k)1:Nk, (k = 1, ... ,K), (4–34)

λ(k)Nk+1

= Λ(k)Nk+1, (k = 1, ... ,K). (4–35)

Next, let D† be an Nk × (Nk + 1) matrix defined as follows:

D†ij = −

wj

wiDji , and D

†i,N+1 = −

N∑

j=1

D†ij (4–36)

for i = 1, ... ,N . Based on the theory developed in [36], D† is a differentiation matrix for

the space of polynomials of degree N . That is, if b is a polynomial of degree at most N

and b ∈ RN is the vector with i-th element bi = b(τi), then

(D†b)i = b(τi).

106

Using the adjoint differentiation matrix defined in Eq. (4–36) along with the transformations

given by Eqs. (4–32)–(4–35), the KKT system of Eqs. (4–23)–(4–31) can be rewritten as


2f(Y

(k)1:Nk,U(k)1:Nk ), φ(Y

(1)0 ) = 0, (k = 1, ... ,K), (4–37)

Y(k+1)0 = Y

(k)0 +

Nk∑

i=1

w(k)i f(Y

(k)i ,U

(k)i ), (k = 1, ... ,K − 1), (4–38)

Y(K)NK+1

= Y(K)0 +

NK∑

i=1

w(K)i f(Y

(K)i ,U

(K)i ), (4–39)

0 = ∇UH(Y(k)1:Nk ,U(k)1:Nk,λ(k)1:Nk,µ(k)1:Nk), (k = 1, ... ,K), (4–40)

D†(k)1:Nkλ(k)1:Nk= −h

(k)

2∇YH(Y(k)1:Nk ,U

(k)1:Nk,λ(k)1:Nk,µ(k)1:Nk), (k = 1, ... ,K), (4–41)

λ(1)0 = ∇Y〈ψ,φ(Y(1)0 )〉, (4–42)

λ(k)0 = λ

(k−1)Nk+1, (k = 2, ... ,K), (4–43)

λ(K)NK+1

= ∇YΦ(Y(K)NK+1), (4–44)

C(Y(k)1:Nk ,U(k)1:Nk) ≤ 0, µ

(k)1:Nk≤ 0, 〈µ(k)1:Nk ,C(Y

(k)1:Nk,U(k)1:Nk )〉 = 0, (k = 1, ... ,K). (4–45)

where H(Y(k)1:Nk ,U(k)1:Nk,λ(k)1:Nk+1

,µ(k)1:Nk) is a discrete form of the Hamiltonian given by

Eq. (4–13). It can be seen that the transformed optimality conditions of Eqs. (4–37)–(4–42)

are a discrete form of the first-order optimality conditions of the continuous-time optimal

control problem given by Eqs. (4–7)–(4–10). Furthermore, the costate terminal condition

given by Eq. (4–11) is satisfied in the discrete transformed adjoint system by Eq. (4–44),

and the complementary slackness conditions of Eq. (4–12) are satisfied by the discrete

condition of Eq. (4–45). From the relationship given in the transformed adjoint system

by Eq. (4–43), however, it is seen that in the discrete problem the costate across

interval boundaries must be continuous. It was previously shown in Section 2.3

that discontinuities in the costate stem from inequality path constraint activity in the

solution domain. Therefore in the presence of state inequality constraints, the costate

becomes discontinuous, and the transformed adjoint system of the NLP is not a discrete

approximation of the first-order optimality conditions of the continuous-time problem.

107

4.3 Variable-Order Collocation at Flipped Legendre-Gauss-Radau Points


collocation at the flipped Legendre-Gauss-Radau points as described in Section

2.5.6. First, recall that the flipped LGR points (τ0, ... , τN) are defined in the domain

τ ∈ (−1,+1] such that τ0 = −1 is a noncollocated point. The state in each mesh interval

k is approximated as

y(k)(τ) ≈ Y(k)(τ) =Nk∑

i=0

Y(k)i L

(k)i (τ), L

(k)i (τ) =

Nk∏

j=0

i 6=j

τ − τ(k)j

τ (k)i − τ (k)j, (4–46)


Y(k)(τj) ≈Nk∑

i=0

Y(k)i L

(k)i (τj) = [D

(k)Y(k)0:Nk]j , (4–47)

where D(k)ij = L(k)i (τ)j , (i = 1, ... ,Nk, j = 0, ... ,Nk) are the components of the

Nk×(Nk+1) flipped Legendre-Gauss-Radau (LGR) differentiation matrix in the k th mesh

interval.

The optimal control problem then becomes to minimize the cost

J ≈ Φ(Y(K)NK ) +K∑

k=1

Nk∑

j=1

h(k)

2w(k)j g(Y

(k)j ,U

(k)j ), (4–48)


D(k)Y(k)0:Nk

=h(k)

2f(Y

(k)1:Nk,U(k)1:Nk), (k = 1, ... ,K), (4–49)

φ(Y(1)0 ) = 0, (4–50)

C(Y(k)1:Nk,U(k)1:Nk ) ≤ 0, (k = 1, ... ,K), (4–51)

where w(k) = (w (k)1 , ... ,w(k)N k) are the flipped LGR quadrature weights in interval k .

When implementing the flipped variable-order LGR method, a single variable is used for

the value of the state at the end of mesh interval k and the start of mesh interval k + 1,

that is, Y(k−1)Nk≡ Y(k)0 , 2 ≤ k ≤ K such that continuity in the state is enforced.

108

4.3.1 KKT Conditions of Variable-Order Flipped Legendre-Gauss-Radau Colloca-tion Method

The first-order optimality conditions of the discrete problem given by Eqs. (4–48)–(4–51),

also called the KKT conditions of the NLP, are now derived. First, the Lagrangian is

defined as

L = Φ(Y(K)NK )− 〈ψ,φ(Y(1)0 )〉+

K∑

k=1

Nk∑

i=1

(

h(k)

2w(k)i g

(k)i − 〈Γ(k)i ,C(k)i 〉

)

−N1∑

i=1

(

〈Λ(1)i ,D(1)i,0 Y(1)0 + D(1)i,1:N1Y(1)1:N1− h

(k)

2f(1)i 〉

)

−K−1∑

k=1

Nk∑

i=1

(

〈Λ(k)i ,D(k)i,0 Y(k−1)N + D(k)i,1:NkY(k)1:Nk− h

(k)

2f(k)i 〉

)

(4–52)

where (Λ(k)i ,Γ(k)i ) are the Lagrange multipliers associated with the dynamic constraints

of Eq. (4–49) and the inequality path constraint of Eq. (4–51) in interval k at the LGR

point τi . Furthermore ψ denotes the Lagrange multipliers associated with the boundary

conditions of Eq. (4–50). Note that function dependencies have been omitted for clarity,

such that g(k)i ≡ g(Y(k)i ,U(k)i ), and similarly f(k)i ≡ f(Y(k)i ,U(k)i ) and C(k)i ≡ C(Y(k)i ,U(k)i ).



2f(Y

(k)1:Nk,U(k)1:Nk ), φ(Y

(1)0 ) = 0, (k = 1, ... ,K), (4–53)

0 = ∇UH(Y(k)1:Nk,U(k)1:Nk ,Λ

(k)1:Nk,Γ(k)1:Nk), (k = 1, ... ,K), (4–54)


2∇YH(Y(k)1:Nk ,U

(k)1:Nk,Λ(k)1:Nk,Γ(k)1:Nk)

− eNkD(k+1)0 Λ

(k+1)1:Nk, (k = 1, ... ,K − 1),

(4–55)

D(K)⊤1:NKΛ(K)1:NK=h(k)

2∇YH(Y(K)1:NK ,U

(K)1:NK,Λ(K)1:NK,Γ(K)1:NK) + eNK∇YΦ(Y(K)NK ), (4–56)

D(1)⊤0 Λ(1)1:N1= −∇Y〈ψ,φ(Y(1)0 )〉, (4–57)

C(Y(k)1:Nk ,U(k)1:Nk) ≤ 0, Γ(k)1:Nk ≤ 0, 〈Γ


(k)1:Nk)〉 = 0, (k = 1, ... ,K), (4–58)

109

where eN denotes the N th column of the identity matrix and the discrete Hamiltonian in

interval k is


(k)1:Nk) = w (k)⊤g

(k)1:Nk+ 〈Λ(k)1:Nk , f

(k)1:Nk〉 − 〈 2

h(k)Γ(k)1:Nk,C(k)1:Nk〉. (4–59)


Consider the following costate estimate, first derived by Ref. [1], that relates the

KKT conditions multipliers of Eqs. (4–53)–(4–58) to the dual variables of the first-order

optimality [given by Eqs. (4–7)–(4–12)] of the continuous-time optimal control problem:

µ(k)i =

2

h(k)Γ(k)i

w(k)i

, (i = 1, ... ,Nk), (k = 1, ... ,K), (4–60)

λ(k)i =

Λ(k)i

w(k)i

, (i = 1, ... ,Nk), (k = 1, ... ,K), (4–61)

λ(k)0 = −D(k)

⊤

0 Λ(k)1:Nk, (k = 1, ... ,K). (4–62)

Next, let D† be an Nk × Nk matrix defined as follows:

D†ij =

−DNN + 1wN, i = j = N

−wjwiDji , i , j = 2, ... ,N.

(4–63)

Based on the theory developed in [36], D† is a differentiation matrix for the space of

polynomials of degree N − 1. That is, if b is a polynomial of degree at most N − 1 and

b ∈ RN is the vector with i-th element bi = b(τi), then

(D†b)i = b(τi).

110

Using the adjoint differentiation matrix defined in Eq. (4–63) along with the transformations

given by Eqs. (4–60)–(4–62), the KKT system of Eqs. (4–53)–(4–58) can be rewritten as


2f(Y

(k)1:Nk,U(k)1:Nk ), φ(Y

(1)0 ) = 0, (k = 1, ... ,K), (4–64)


D† (k)1:Nkλ(k)1:Nk= −h

(k)

2∇YH(Y(k)1:Nk ,U

(k)1:Nk,λ(k)1:Nk,µ(k)1:Nk)

− eNkwNk

(

λ(k+1)0 − λ(k)N

)

, (k = 1, ... ,K − 1), (4–66)

D† (K)1:NK

λ(K)1:NK= −h

(k)

2∇YH(Y(K)1:NK ,U

(K)1:NK,λ(K)1:NK,µ(K)1:NK)

− eNKwNK

(

∇YΦ(Y(K)N )− λ(K)N)

, (4–67)

λ(1)0 = ∇Y〈ψ,φ(Y(1)0 )〉, (4–68)

C(Y(k)1:Nk ,U(k)1:Nk) ≤ 0, µ

(k)1:Nk≤ 0, 〈µ(k)1:Nk ,C(Y

(k)1:Nk,U(k)1:Nk )〉 = 0, (k = 1, ... ,K), (4–69)

where H(Y(k)1:Nk ,U(k)1:Nk,λ(k)1:Nk,µ(k)1:Nk) is a discrete form of the Hamiltonian given by

Eq. (4–13). These equations are incomplete because a new variable λ(k)0 was

introduced without adding a new equation. An equation for this new variable can be

developed by manipulating the matrix D(k). Consider now a (N + 1) × 1 column vector

composed of ones. The components of the vector D(k)1 are the derivatives at the

collocation points of the polynomial whose value is 1 at τi , (i = 1, ... ,N + 1) in interval

k . The derivative of the constant polynomial is zero everywhere. Thus, D(k)1 = 0, which

implies that

D(k)0 = −Nk∑

j=1

D(k)1:Nk , j ,

D(k)⊤0 Λ(k) =

Nk∑

i=1

D(k)i, 0Λ(k)i = −

Nk∑

i=1

Nk∑

j=1

D(k)i, j Λ(k)i ,

−λ(k)0 = −λ(k)Nk +Nk∑

i=1

Nk∑

j=1

w(k)i λ

(k)j D†(k)

i,j = −λ(k)Nk +Nk∑

j=1

w(k)j [D

†(k)λ(k)]j ,

(4–70)

111

where the relationships in (4–61)–(4–62) and (4–63) were used to obtain Eq. (4–70).

It can be seen that this relationship approximates the integral of the costate dynamics

across the interval k via a Radau quadrature. That is, it approximates the relationship

−λ(−1) = −λ(+1) +∫ +1

−1

λ(τ)dτ .

Combining Eqs. (4–66)–(4–67) with Eq. (4–70), the complete transformed adjoint

system can then be written as


2f(Y

(k)1:Nk,U(k)1:Nk ), φ(Y

(1)0 ) = 0, (k = 1, ... ,K), (4–71)


D† (k)1:Nkλ(k)1:Nk= −h

(k)

2∇YH(Y(k)1:Nk ,U

(k)1:Nk,λ(k)1:Nk,µ(k)1:Nk)

− eNkwNk

(

λ(k+1)0 − λ(k)N

)

, (k = 1, ... ,K − 1), (4–73)

λ(k+1)0 = λ

(k)0 −

h(k)

2

Nk∑

j=1

w(k)j ∇YH(Y(k)j ,U(k)j ,λ(k)j ,µ(k)j ), (k = 1, ... ,K − 1), (4–74)

D† (K)1:NK

λ(K)1:NK= −h

(k)

2∇YH(Y(K)1:NK ,U

(K)1:NK,λ(K)1:NK,µ(K)1:NK)

− eNKwNK

(

∇YΦ(Y(K)N )− λ(K)N)

, (4–75)

∇YΦ(Y(K)N ) = λ(K)0 −h(K)

2

NK∑

j=1

w(K)j ∇YH(Y(K)j ,U(K)j ,λ(K)j ,µ(K)j ), (4–76)

λ(1)0 = ∇Y〈ψ,φ(Y(1)0 )〉, (4–77)

C(Y(k)1:Nk ,U(k)1:Nk) ≤ 0, µ(k)1:Nk ≤ 0, 〈µ


(k)1:Nk)〉 = 0, (k = 1, ... ,K), (4–78)

It is seen that Eq. (4–76) is a Legendre-Gauss-Radau quadrature of the costate

dynamics across interval K . Consequently, the right-hand side of Eq. (4–76) approximates

the costate at the final point in the domain. Eq. (4–76) is thus a subtle way of enforcing

the relationship λ(K)N = ∇YΦ(YNk) and it is expected that the last term of Eq. (4–75)

will be small while the remaining terms in Eq. (4–75) are a collocation collocation

scheme for the continuous adjoint equation in the final interval K . Similarly, the

112

right-hand side of Eq. (4–74) approximates the costate, λ(k)N , at the terminal point

in interval k via a Legendre-Gauss-Radau quadrature of the costate dynamics for

(k = 1, ... ,K − 1). Equation (4–74) is therefore a subtle way of enforcing the relationship

λ(k)N = λ

(k+1)0 , (k = 1, ... ,K − 1). If the costate is continuous across an interval

boundary, the relationship λ(k)N = λ(k+1)0 holds true and the last term of Eq. (4–73)

will be small while the remaining terms in Eq. (4–73) are a collocation scheme for the

continuous adjoint equation. Therefore in cases when the costate is continuous across

an interval boundary, the transformed optimality conditions of Eqs. (4–71)–(4–78) are

a discrete form of the first-order optimality conditions [given by Eqs. (4–7)–(4–12)] of

the continuous-time optimal control problem. However, it has previously been shown in

Section 2.3 that the presence of active state inequality path constraints in the solution

domain may cause discontinuities in the costate. Therefore, in the presence of active

state inequality path constraints in the solution domain, λ(k)N 6= λ(k+1)0 at the entrance

or exit of a constrained arc, and the last term of Eq. (4–73) will not be small. Therefore

Eq. (4–73) will not be a collocation scheme for the continuous adjoint equation using this

costate estimate.

4.4 Discussion

In this chapter a method first derived by Ref. [1] for obtaining costate estimates

from the KKT multipliers of the NLP was presented. This derivation showed that if

the costate is continuous, variable-order collocation at the LG and LGR points yields

a set of transformed optimality conditions of the KKT system which are a discrete

representation of the continuous-time first-order necessary conditions of the optimal

control problem, as can be seen in Fig. (4-1). If the costate is discontinuous, however,

variable-order collocation at the LG and LGR points yields a set of transformed

optimality conditions of the KKT system which are an inexact discrete representation

of the continuous-time first-order necessary conditions. This result was first shown by

113

Ref. [74], who suggested that the costate estimate must be modified in order to account

for the costate discontinuities:

“. . . high-accuracy approximations are achieved by the proposed

hp-method if the costate is continuous. If mesh points are at the location

of discontinuity in the costate, the transformed adjoint system is an inexact

discrete representation of the continuous-time first-order necessary

conditions. For a dynamic refinement algorithm that will exactly locate

the switch in activity of inequality path constraints, it is likely that the costate

may be discontinuous at mesh points. It is necessary to determine how

to make the transformed adjoint system a discrete representation of the

continuous-time first-order necessary conditions for a discontinuous costate

solution if mesh points are at the location of discontinuity in the costate”.

It was previously shown in Section 2.3 that discontinuities in the costate stem from

inequality path constraint activity in the solution domain. Therefore in the presence

of state inequality constraints, the costate becomes discontinuous, and the first-order

optimality conditions of the NLP are not a discrete approximation of the first-order

optimality conditions of the continuous-time problem. Therefore, in this research a new

method of costate estimation for variable-order collocation at LG and LGR points will be

derived using three different methods. Specifically, a costate estimate using the method

indirect adjoining with continuous multipliers will be presented. It will be shown that

this method for costate estimation using variable-order collocation at the LG and LGR

points leads to a transformed adjoint system which is a discrete representation of the

continuous-time first-order necessary conditions even in the presence of state inequality

path constraints.

114

Figure 4-1. Relationship between the direct and indirect methods for solving an optimalcontrol problem. In the indirect method, the problem is first optimized through thecalculus of variations, leading to a set of conditions which can then be discretized andsolved. In the direct method, the problem is first discretized and transcribed to an NLP,then it is optimized by solving the KKT system. The two systems are equivalent onlywhen the costate is continuous in the solution domain.

115

CHAPTER 5COSTATE ESTIMATION FOR STATE CONSTRAINED PROBLEMS

As was shown in Chapter 4, previous research has successfully derived a

high-accuracy estimate of the costate using collocation at the Legendre-Gauss and

Legendre-Gauss-Radau points for the case of a problem with no active state inequality

path constraints. However, Ref. [1] showed that in the case when the costate is

discontinuous (as is the case in the presence of active state inequality path constraints),

this previously derived costate estimate leads to a set of first-order optimality conditions

of the NLP that are not equivalent to the discrete form of the variational optimality

conditions. This non-equivalence leads to an inaccurate approximation of the costate. In

order to rectify this inacuracy, in this chapter a method for estimating the costate of state

inequality path constrained optimal control problems using collocation at LG and LGR

points is developed using the method of indirect adjoining with continuous multipliers.

The method of indirect adjoining with continuous multipliers was chosen to

develop a costate estimate over the direct and indirect adjoining methods for two

reasons. First, the method of indirect adjoining requires a modification of the original

problem formulation through index-reduction of the differential-algebraic equations. The

reformulation of the problem must be done analytically, and requires prior knowledge

of the solution structure. Thus, when using an automated solution process (such as

a mesh refinement technique), this procedure might be cumbersome to implement.

Second, both the methods of indirect and direct adjoining result in a discontinuous

costate. Because discontinuities are difficult to approximate numerically, both these

methods may yield large errors in the costate estimate if the location of the discontinuity

is not exact. Thus, because the method of indirect adjoining with continuous multipliers

yields a continuous costate, it offers an advantage over the methods of direct and

indirect adjoining which approximate a discontinuous costate.

116

Similar to the approach of Chapter 4, the following notation and conventions will

be used throughout this chapter to make the exposition more clear. First, all vector

functions of time are denoted as row vectors, that is, if y(τ) ∈ Rn is a vector function of

the scalar variable τ , then y(τ) = [y1(τ), · · · , yn(τ)]. Next, any capital boldface character,

Y, denotes a matrix of size M × n, where each row of Yi corresponds to the evaluation

of a function y(τ) at a particular value τ = τi . Next, the notation Yi:j denotes rows

i through j of the matrix Y, except when referring to a differentiation matrix D or the

integration matrix A, in which case Di and Ai refers to the i th column of D and A. Finally,

D⊤ denotes the transpose of matrix D, and D⊤i denotes the transpose of the i th column

of D. Given vectors x and y ∈ Rn, the notation 〈x, y〉 is used to denote the standard inner

product between x and y. Furthermore, if f : Rn −→ Rm, then ∇f is the m by n Jacobian

matrix whose i th row is ∇fi . In particular, the gradient of a scalar-valued function is a row

vector. If φ : Rm×n −→ R and Y is an m by n matrix, then ∇φ denotes the m by n matrix

whose (i , j) element is (∇φ(Y))ij = ∂φ(Y)/∂Yij .

The remainder of this chapter is organized as follows. First, Section 5.1 formulates

the continuous-time state inequality path constrained optimal control problem and

states the first order optimality conditions of the continuous problem. Next, in Sections

5.2 and 5.3 a new costate estimate is derived using variable-order collocation at the

Legendre-Gauss and flipped Legendre-Gauss-Radau points, respectively, through the

method of indirect adjoining with continuous multipliers. It is shown for each of these

derived costate estimates that the transformed first-order optimality conditions of the

NLP are a discrete form of the first-order optimality conditions of the continuous-time

optimal control problem. Finally, in Section 5.4 the derived costate estimates are

discussed, and conclusions are given.

5.1 Continuous-Time State Inequality Path Constrained Optimal Control Problem

The state inequality path constrained optimal control problem to be studied in

the remainder of this chapter is now presented. To simplify comparisons with the

117

transformed adjoint system, the domain t ∈ [t0, tf ] = I is divided into K intervals

Sk = [Tk−1,Tk ] ⊆ [t0, tf ], (k = 1, ... ,K), where T0 = t0, TK = tf , Tk−1 < Tk , (k =

1, ... ,K), and⋃K

k=1 Sk = I. Furthermore, without loss of generality the optimal control

problem can be scaled by transforming the independent variable in each interval from

t ∈ [Tk−1,Tk ] to τ (k) ∈ [−1,+1] via the affine transformation

t =Tk −Tk−12

τ (k) +Tk + Tk−12

(5–1)

such that

dt =Tk − Tk−12

dτ (k) ≡ h(k)

2, where h(k) ≡ Tk − Tk−1. (5–2)

The state inequality constrained optimal control problem problem is stated as

follows. Determine the state y(k)(τ) ∈ Rn, and the control u(k)(τ) ∈ Rm in each interval

(k = 1, ... ,K), to minimize the cost functional

J = Φ(y(K)(+1)) +

K∑

k=1

h(k)

2

∫ +1

−1

g(y(k)(τ), u(k)(τ))dτ (5–3)

subject to the dynamic constraints

y(k)(τ) =h(k)

2f(y(k)(τ), u(k)(τ)), (k = 1, ... ,K), (5–4)

the boundary conditions

φ(y(1)(−1)) = 0, (5–5)

and the state inequality path constraint

S(y(k)(τ)) ≤ 0, (k = 1, ... ,K). (5–6)

The continous-time optimal control problem of Eqs. (5–3)–(5–6) will be the topic of the

remainder of this chapter.

118

5.1.1 First-Order Optimality Conditions Using Method of Indirect Adjoining withContinuous Multipliers

The first-order optimality conditions of the state inequality path constrained optimal

problem given by Eqs. (5–3)–(5–6) were derived in Section 2.3.3 using the method of

indirect adjoining with continuous multipliers. These conditions are repeated here as

y(k) = f(y(k), u(k)), φ(y(1)(−1)) = 0, (k = 1, ... ,K), (5–7)

0 = ∇uH(y(k), u(k), p(k),ν(k)), (k = 1, ... ,K), (5–8)

−p(k) = h(k)

2∇yH(y(k), u(k), p(k),ν(k)), (k = 1, ... ,K), (5–9)

p(1)(−1) = ∇y(〈ψ,φ(y(1))〉+ 〈ν(k),S(y(k))〉)∣

∣

τ=−1(5–10)

p(K)(+1) = ∇y(Φ(y(K)) + 〈ν(K),S(y(K))〉)∣

∣

τ=+1(5–11)

ν(K)(+1) ≤ 0, ν(k) ≥ 0, S(y(k)) ∈ N (ν(k)), (k = 1, ... ,K), (5–12)

where p(k)(τ) and ν(k)(τ) are the Lagrange multipliers associated with the dynamic

constraints of Eq. (5–4) and the state inequality path constraints of Eq. (5–6), respectively,

in interval k . Furthermore, ψ is the Lagrange multiplier associated with the boundary

conditions of Eq. (5–5). The Hamiltonian in interval k , H(y(k), u(k), p(k),ν(k)), is defined

as

H(y(k), u(k), p(k),ν(k)) = g(y(k), u(k)) + 〈p(k), f(y(k), u(k))〉 − 〈ν(k), S(y(k))〉, (5–13)

where S(k) ≡ ∇S(y(k))f(y(k), u(k)). Let S(Rq) denote the space of continuous functions

mapping [t0, tf ] to Rq. Assuming ν(k) is Lipschitz continuous and nondecreasing with

ν(K)(+1) ≤ 0, the set-valued map N (ν(k)) is defined as

N (ν(k)) = {z(k) ∈ S(Rq) : z(k) ≤ 0, 〈ν(k), z(k)〉 = 0, 〈ν(K)(+1), z(K)(+1)〉 = 0},

for (k = 1, ... ,K).

119

5.2 Costate Estimation Using Legendre-Gauss Collocation


collocation at the Legendre-Gauss points as described in Section 2.5.4. Unlike previous

implementations of the LG collocation method, the state inequality path constraint is

enforced at all LG points and all interior mesh points (T1, ... ,TK−1) but is not enforced

at the endpoints T0 = −1 and TK = +1. LG collocation provides an approximation to

the state, but not the control, at the mesh points. Therefore it is impossible to enforce

inequality path constraints that are a function of the control at any of the mesh points.

However, because this research is concerned with inequality path constraints that are

a function of purely the state, it is possible to enforce the inequality path constraint at

all interior mesh points. Indeed, it will be seen that an accurate approximation of the

costate using variable-order collocation at the LG points can only be achieved when the

state inequality path constraint is enforced at all the interior mesh points.

5.2.1 Variable-Order Collocation at Legendre-Gauss Points

Recall that the LG points (τ1, ... , τN) are defined in the domain τ ∈ (−1,+1)

such that τ0 = −1 and τN+1 = +1 are noncollocated points. When implementing the

variable-order LG method, a single variable is used for the value of the state at the end

of mesh interval k and the start of mesh interval k + 1, that is, Y(k)Nk+1 ≡ Y(k+1)0 , (k =

1, ... ,K − 1) such that continuity in the state is enforced.

The NLP is then given as follows. Minimize the cost function

J ≈ Φ(Y(K)NK+1) +K∑

k=1

Nk∑

j=1

h(k)

2w(k)j g(Y

(k)j ,U

(k)j ), (5–14)

120


D(k)Y(k)0:Nk=h(k)

2f(Y

(k)1:Nk,U(k)1:Nk), (k = 1, ... ,K), (5–15)

Y(k+1)0 = Y

(k)0 +

h(k)

2

Nk∑

j=1

w(k)j f(Y

(k)j ,U

(k)j ), (k = 1, ... ,K − 1), (5–16)

Y(K)N+1 = Y

(K)0 +

h(K)

2

Nk∑

j=1

w(K)j f(Y

(K)j ,U

(K)j ), (5–17)

φ(Y(1)0 ) = 0, (5–18)

S(Y(k)0:Nk) ≤ 0, (k = 2, ... ,K), (5–19)

S(Y(1)1:N1) ≤ 0. (5–20)

It is seen that the quadrature constraints of Eqs. (5–16) and (5–17) provide an

approximation to the final state in intervals (k = 1, ... ,K − 1) and K , respectively,

through a Gaussian quadrature approximation to the integral of the state dynamics

across that interval.

The first-order optimality (KKT) conditions of the discrete problem given by

Eqs. (5–14)–(5–19) are derived in the same manner of Section 4.2. First, the Lagrangian

is defined as

L = Φ(Y(K)NK+1)− 〈ψ,φ(Y(1)0 )〉+

K∑

k=1

Nk∑

i=1

(

h(k)

2w(k)i g

(k)i − 〈Γ(k)i ,S(k)i 〉

)

−K∑

k=1

Nk∑

i=1

(

〈Λ(k)i ,D(k)i,0:NkY(k)0:Nk− h

(k)

2f(k)i 〉

)

−K−1∑

k=1

(

〈Λ(k)Nk+1,Yk+10 − Y(k)0 −

h(k)

2w(k)i f(k)i 〉 − 〈Γ(k)Nk+1,S

(k+1)0 〉

)

−NK∑

i=1

(

〈Λ(K)NK+1,YKNK+1

− Y(K)0 − h(K)

2w(K)i f(K)i 〉

)

(5–21)

where Λ(k)1:Nk+1 and Γ(k)1:Nk+1 are the Lagrange multipliers associated with the dynamic

constraints of Eq. (5–15), the quadrature constraints of Eqs. (5–16)–(5–17), and the

inequality path constraint of Eq. (5–19), respectively, in interval k . Furthermore ψ

121

denotes the Lagrange multipliers associated with the boundary conditions of Eq. (5–18).



2f(Y

(k)1:Nk,U(k)1:Nk ), φ(Y

(1)0 ) = 0, (k = 1, ... ,K), (5–22)

Y(k+1)0 = Y

(k)0 +

Nk∑

i=1

w(k)i f(Y

(k)i ,U

(k)i ), (k = 1, ... ,K − 1), (5–23)

Y(k)NK+1

= Y(K)0 +

NK∑

i=1

w(K)i f(Y

(K)i ,U

(K)i ), (5–24)

0 = ∇UH(Y(k)1:Nk,U(k)1:Nk ,Λ

(k)1:Nk+1

,Γ(k)1:Nk), (k = 1, ... ,K), (5–25)


2∇YH(Y(k)1:Nk ,U

(k)1:Nk,Λ(k)1:Nk+1

,Γ(k)1:Nk), (k = 1, ... ,K), (5–26)

D(1)⊤0 Λ(1)1:N1= Λ

(1)N1+1

−∇Y〈ψ,φ(Y(1)0 )〉, (5–27)

D(k)⊤0 Λ(k)1:N1= Λ

(k)Nk+1

− Λ(k−1)Nk+1−∇Y〈Γ(k−1)N+1 ,S(Y

(k)0 )〉, (k = 2, ... ,K), (5–28)

Λ(K)NK+1

= ∇YΦ(Y(K)NK+1), (5–29)

S(Y(k)1:Nk ) ≤ 0, Γ(k)1:Nk≤ 0, 〈Γ(k)1:Nk ,S(Y

(k)1:Nk)〉 = 0, (k = 1, ... ,K), (5–30)

S(Y(k)0 ) ≤ 0, Γ(k−1)N+1 ≤ 0, 〈Γ(k−1)N+1 ,S(Y(k)0 )〉 = 0, (k = 2, ... ,K). (5–31)

The discrete Hamiltonian in interval k is given as


(k)1:Nk+1

) = w (k)⊤g(k)1:Nk+ 〈Λ(k)1:Nk +W

(k)1Λ(k)Nk+1, f(k)1:Nk〉 − 〈 2

h(k)Γ(k)1:Nk,S(k)1:Nk〉,

whereW(k) is a N × N diagonal matrix of LG quadrature weights in interval k , and 1 is a

N × 1 column vector of ones.


In order to relate the necessary conditions for optimality of the continuous problem

[given by Eqs. (5–7)–(5–12)] to the discrete KKT conditions of the NLP [given by

122

Eqs. (5–22)–(5–31)], consider the following transformed dual variables

p(k)1:Nk= [W(k)]−1Λ

(k)1:Nk+ 1Λ

(k)N+1 +∇Y〈ν(k)1:Nk ,S(Y

(k)1:Nk)〉, (k = 1, ... ,K), (5–32)

p(k)Nk+1

= Λ(k)Nk+1

+∇Y 〈ν(k)Nk+1,S(Y(k)Nk+1)〉, (k = 1, ... ,K), (5–33)

Γ(k)1:N = −W(k)D†(k)ν1:N+1, (k = 1, ... ,K), (5–34)

ν(K)NK+1

= Γ(K)NK+1

= 0, (5–35)

ν(k)Nk+1

= Γ(k)Nk+1

+ ν(k+1)0 , (k = 1, ... ,K), (5–36)

p(k)0 = p

(k)Nk+1

−D(k)⊤0 W(k)[p(k)1:Nk− 1 · p(k)Nk+1], (k = 1, ... ,K), (5–37)

ν(k)0 = ν

(k)Nk+1

+

Nk∑

i=1

Γ(k)i , (k = 1, ... ,K). (5–38)

where D† is a N × (N + 1) matrix defined by

D†ij = −

wj

wiDji , and D

†i,N+1 =

N∑

j=1

Dij (5–39)

for i = 1, ... ,N . Based on the theory developed in [36], D† is a differentiation matrix for

the space of polynomials of degree N . That is, if b is a polynomial of degree at most N

and b ∈ RN+1 is the vector with i-th element bi = b(τi), then

(D†b)i = b(τi). (5–40)

Furthermore, it can be shown that D† has similar properties to the Gauss differentiation

matrix D. Specifically, as was seen in Chapter 3, the following hold true: (a) the

square matrix D†1:N obtained by removing the last column of D† is full-rank, and (b)

−(D†1:N)

−1D†N+1 = 1. Using these properties, Eq. (5–34) can be rewritten as

ν(k)1:Nk= ν

(k)N+1 − [W(k)D†(k)

1:N ]−1Γ

(k)1:Nk

= ν(k)Nk+1

− A†(k)[W(k)]−1Γ(k)1:Nk,

(5–41)

where the matrix A† is a backward integration matrix for the space of polynomials of

degree N − 1. Specifically, let L†i (τ) be a basis of Lagrange interpolating polynomials of

123

degree N − 1,

L†i (τ) =

N∏

j=1

j 6=i


. (5–42)

Then if q is a polynomial of degree at most N − 1 with q(τi) = qi , it can be interpolated

exactly by the Lagrange polynomials L†i such that

q(τ) =N∑

i=1

qiL†i (τ). (5–43)

Integrating this expression backwards yields

q(τj) = q(+1) +N∑

i=1

A†ji qi , A

†ji =

∫ τj

+1

L†i (τ)dτ . (5–44)

Furthermore, a constant of integration (in this case a terminal condition) is needed for

the integration of the KKT multipliers Γ. Since the constraint S(Y(k)1:N) ≤ 0 is not enforced

at the final point τ (K)NK+1 = +1 in the final interval K , by definition its associated multiplier

is zero. Thus, ν(K)NK+1 = Γ(K)NK+1

= 0. The terminal condition for the remainder of the

intervals, (k = 1, ... ,K − 1), is defined by Eq. (5–36).

Expressions are now developed that justify Eqs. (5–37) and (5–38) as approximations

of the state constraint multiplier and the costate, respectively, at τ (k) = −1, (k =

1, ... ,K). First, let ν(k)(τ) be the polynomial of degree N that satisfies ν(k)(τi) = ν(k)i for

(i = 1, ... ,N + 1) in interval k , (k = 1, ... ,K). Since the Legendre-Gauss quadrature is

exact for a polynomial of degree N − 1, then

ν(k)0 = ν

(k)N+1 −

∫ +1

−1

ν(k)(τ)dτ = ν(k)N+1 −N∑

i=1

w(k)j ν(k)(τi). (5–45)

Furthermore, D† is a differentiation matrix for the space of polynomials of degree N , as

seen by Eq. (5–40). Therefore, Eq. (5–45) becomes

ν(k)0 = ν

(k)N+1 −

Nk∑

i=1

w(k)j D

†(k)i,1:N+1ν

(k)1:N+1 = ν

(k)N+1 +

Nk∑

i=1

Γi(k),

where Eq. (5–34) was used in the last substitution.

124

Next, it is shown that the costate approximation given by Eq. (5–37) is equivalent to

applying the Fundamental Theorem of Calculus. Let 1 ∈ RN+1 denote the vector whose

elements are all unity. Because the components of the vector D(k)1 are the derivatives

of the constant polynomial q(τ) = 1 at the LG points, D(k)1 = 0, which implies that

D(k)0 = −

Nk∑

j=1

D(k)j . (5–46)

Taking the transpose of Eq. (5–46) and substituting the rows of D†(k) for the rows of D(k),

the following expression is obtained:

D(k)0 =

N∑

i=1

N∑

j=1

wi

wjD

†(k)ij (5–47)

Finally, post-multiplying the result byW(k)[p(k)1:N−1 ·p(k)N+1], and subtracting p(k)N+1 from both

sides yields

−p(k)N+1 +D(k)⊤0 W(k)[p(k)1:N − 1 · p(k)N+1]

= −p(k)N+1 +Nk∑

j=1

w(k)j D

†(k)j,1:N+1p

(k)1:N+1,

(5–48)

for each interval (k = 1, ... ,K). Now let p(k)(τ) be the polynomial of degree N that

satisfies p(k)(τi) = p(k)i for (i = 1, ... ,N + 1), (k = 1, ... ,K), then using the same logic as

was done for Eq. (5–45)

−p(k)0 = −p(k)N+1 +∫ +1

−1

p(k)(τ)dτ = −p(k)N+1 +Nk∑

i=1

w(k)i p

(k)(τi).

Comparing this expression with Eq. (5–48), it is seen that p(k)0 , (k = 1, ... ,K) given by

Eq. (5–37) is consistent with the Fundamental Theorem of Calculus.

125

Using the transformations described in Eqs. (5–32)–(5–38) along with Eq. (5–39),

the KKT conditions of the NLP given by Eqs. (5–22)–(5–31) can be written as

D(k)Y(k)0:N = f(Y

(k)1:N,U

(k)1:N), φ(Y

(1)0 ) = 0, (k = 1, ... ,K), (5–49)

Y(k+1)0 = Y

(k)0 + w

(k)⊤f(Y(k)1:N ,U

(k)1:N), (k = 1, ... ,K − 1), (5–50)

Y(K)N+1 = Y

(K)0 +w(k)⊤f(Y

(k)1:N,U

(k)1:N), (5–51)

0 = ∇UH(Y(k)1:N,U(k)1:N, p(k)1:N,ν(k)1:N), (k = 1, ... ,K), (5–52)

D†(k)p(k)1:N+1 = −∇YH(Y(k)1:N,U(k)1:N, p(k)1:N ,ν(k)1:N), (k = 1, ... ,K), (5–53)

p(k)0 = ∇Y

(

〈ψ,φ(Y(1)0 )〉+ 〈ν(1)0 ,S(Y(1)0 )〉)

, (5–54)

p(k+1)0 = p

(k)N+1, (k = 1, ... ,K − 1), (5–55)

p(K)N+1 = ∇Y

(

Φ(Y(K)N+1) + 〈ν(K)N+1,S(YN+1)〉

)

, (5–56)

0 = 〈D†(k)ν(k)1:N+1,S(Y

(k)1:N)〉, (k = 1, ... ,K), (5–57)

S(Y(k)1:N) ≤ 0, D†(k)ν

(k)1:N+1 ≥ 0, ν

(K)N+1 = 0, (k = 1, ... ,K), (5–58)

where H(k) is a discrete form of the Hamiltonian in interval k given by Eq. (5–13).Furthermore,

the partial differentials of the Hamiltonian in interval k given in Eqs. (5–52) and (5–53)

are given as

∇UH(k) =∇Ug(Y(k)i ,U(k)i ) + 〈p(k)i ,∇Uf⊤(Y(k)i ,U(k)i )〉 − 〈ν(k)i ,∇YS⊤(Y(k)i )∇Uf⊤(Y(k)i ,U(k)i )〉,

and

∇YH(k) =∇Yg(Y(k)i ,U(k)i ) + 〈p(k)i ,∇Yf⊤(Y(k)i ,U(k)i )〉 − 〈ν(k)i ,∇YS⊤(Y(k)i )∇Yf⊤(Y(k)i ,U(k)i )〉

− 2

h(k)D

†(k)i,1:N+1 · ∇Y〈ν(k)1:N+1,S(Y(k)1:N+1)〉+ 〈

2

h(k)D

†(k)i,1:N+1ν

(k)1:N+1,∇YS(Y(k)i )〉,

for i = 1, ... ,N . Note that the product rule was used to differentiate the state inequality

constraint, that is, the following identity was used:

〈ν, ddt∇yS⊤(y)〉 =

d

dt〈ν,∇yS⊤(y)〉 − 〈

d

dtν,∇yS⊤(y)〉.

126

Careful comparison of the necessary conditions for optimality of the discrete and

continuous problems [given by Eqs. (5–49)–(5–58) and Eqs. (5–7)–(5–12), respectively]

reveals their equivalence. Note the costate is continuous across interval boundaries, as

given by Eq. (5–55). Furthermore, it is reinforced that the state is being differentiated by

a matrix D which is based on the derivatives of polynomials of degree N with coefficients

at the N LG points plus the initial uncollocated point τ0 = −1, whereas the costate and

the state constraint multipliers are being differentiated by a matrix D† which is based

on the derivatives of polynomials of degree N with coefficients at the N LG points plus

the terminal uncollocated point τN+1 = +1. Finally, note that the integration matrix

A associated with the state dynamics integrates the state forward in the domain, and

requires an initial condition, whereas the integration matrix A† associated with the

costate and the state constraint multipliers is a backward integration matrix which

requires a terminal condition.

5.3 Costate Estimation Using Flipped Legendre-Gauss-Radau Collocation


collocation at the flipped Legendre-Gauss-Radau points as described in Section 2.5.6.

It is noted that the flipped LGR points are particularly conducive to variable-order

collocation; since only one of the domain endpoints are collocated, there is no “double

collocation” at the boundaries. Also, the only noncollocated point is the first point of the

first interval, t0 = τ (1)0 = −1. Thus, the flipped LGR points are an improvement over

the LG points, which provided no information on the optimal control at any of the mesh

points.

5.3.1 Variable-Order Collocation at Flipped Legendre-Gauss-Radau Points

Recall that the flipped LGR points are defined on the domain (−1,+1] such that

τN = +1 is a LGR collocation point but τ0 = −1 is a noncollocated point. When

implementing the flipped variable-order LGR method, a single variable is used for the

value of the state at the end of mesh interval k and the start of mesh interval k + 1, that

127

is, Y(k)Nk ≡ Y(k+1)0 , 1 ≤ k ≤ K − 1 such that continuity in the state is enforced. Hence,

redundant variables defining the state at the interior mesh points are eliminated.


collocation at the flipped Legendre-Gauss-Radau points as described in Section 2.5.6.

The NLP is then given as follows. Minimize the cost function

J ≈ Φ(Y(K)NK ) +K∑

k=1

Nk∑

j=1

h(k)

2w(k)j g(Y

(k)j ,U

(k)j ), (5–59)


D(k)Y(k)0:Nk=h(k)

2f(Y

(k)1:Nk,U(k)1:Nk), (k = 1, ... ,K), (5–60)

φ(Y(1)0 ) = 0, (5–61)

S(Y(k)1:Nk) ≤ 0, (k = 1, ... ,K). (5–62)

The first-order optimality (KKT) conditions of the discrete problem given by

Eqs. (5–59)–(5–62) are derived in the same manner of Chapter 4. First, the Lagrangian

is defined as

L = Φ(Y(K)NK )− 〈ψ,φ(Y(1)0 )〉+

K∑

k=1

Nk∑

i=1

(

h(k)

2w(k)i g

(k)i − 〈Γ(k)i ,S(k)i 〉

)

−N1∑

i=1

(

〈Λ(1)i ,D(1)i,0 Y(1)0 + D(1)i,1:N1Y(1)1:N1− h

(k)

2f(1)i 〉

)

−K−1∑

k=1

Nk∑

i=1

(

〈Λ(k)i ,D(k)i,0 Y(k−1)N +D(k)i,1:NkY(k)1:Nk− h

(k)

2f(k)i 〉

)

,

(5–63)

where Λ(k)i and Γ(k)i are the Lagrange multipliers associated with the dynamic constraints

of Eq. (5–60) and the inequality path constraint of Eq. (5–62),respectively, in interval

k at the LGR point τi . Furthermore ψ is the Lagrange multipliers associated with the

boundary conditions of Eq. (5–61).

128



2f(Y

(k)1:Nk,U(k)1:Nk ), φ(Y

(1)0 ) = 0, (k = 1, ... ,K), (5–64)

0 = ∇UH(Y(k)1:Nk,U(k)1:Nk ,Λ

(k)1:Nk,Γ(k)1:Nk), (k = 1, ... ,K), (5–65)


2∇YH(Y(k)1:Nk ,U

(k)1:Nk,Λ(k)1:Nk,Γ(k)1:Nk)

− eNkD(k+1)0 Λ

(k+1)1:Nk, (k = 1, ... ,K − 1), (5–66)

D(K)⊤1:NKΛ(K)1:NK=h(k)

2∇YH(Y(K)1:NK ,U

(K)1:NK,Λ(K)1:NK,Γ(K)1:NK) + eNK∇YΦ(Y(K)NK ), (5–67)

D(1)⊤0 Λ(1)1:N1= −∇Y〈ψ,φ(Y(1)0 )〉, (5–68)

S(Y(k)1:Nk ) ≤ 0, Γ(k)1:Nk≤ 0, 〈Γ(k)1:Nk ,S(Y

(k)1:Nk)〉 = 0, (k = 1, ... ,K), (5–69)

where eN denotes the N-th column of the identity matrix and the discrete Hamiltonian in

interval k is


(k)1:Nk) = w (k)⊤g

(k)1:Nk+ 〈Λ(k)1:Nk , f

(k)1:Nk〉 − 〈h

(k)

2Γ(k)1:Nk,S(k)1:Nk〉. (5–70)


In order to relate the necessary conditions for optimality of the continuous problem

[given by Eqs. (5–7)–(5–12)] to the discrete KKT conditions of the NLP [given by

Eqs. (5–64)–(5–69)], consider the following transformed dual variables

p(k)1:N = [W

(k)]−1Λ(k) +∇Y〈ν(k)1:N,S(Y(k)1:N)〉, (k = 1, ... ,K), (5–71)

Γ(K)1:N = −W(K)D(K)ν(K)1:N , (5–72)

p(k)0 = −D(k)⊤0 W(k)p

(k)1:N, (k = 1, ... ,K), (5–73)

ν(k)0 = −D(k)⊤0 W(k)ν

(k)1:N , (k = 1, ... ,K), (5–74)

whereW(k) is the diagonal matrix with the quadrature weights w on the diagonal in

interval k and D is defined by

D = −W−1DT1:NW. (5–75)

129

Next, let D† be an Nk × Nk matrix defined as follows:

D†ij =

−DNN + 1wN, i = j = N

−wjwiDji , i , j = 2, ... ,N.

(5–76)

Based on the theory developed in [36], D† is a differentiation matrix for the space of

polynomials of degree N − 1. That is, if b is a polynomial of degree at most N − 1 and

b ∈ RN is the vector with i-th element bi = b(τi), then

(D†b)i = b(τi).

The matrix D is identical to the differentiation matrix D† introduced in Eq. (5–76) except

for the (N,N) element:

DNN = −DNN = D†NN −

1

wN. (5–77)

Because D equals D† except for the single element in row N given in Eq. (5–77), it

follows that

(Db)i =

b(τi), 1 ≤ i ≤ N − 1,

b(τN)− b(τN)/wN, i = N.(5–78)

Hence, D behaves like a differentiation matrix except for the last row that both

differentiates and evaluates. Note that D† is singular while D is invertible since D1:N

is invertible. In particular, by Eq. (5–75), D can be inverted as

D−1 =W−1D−T1:NW.

Now let A denote D−1, by Eq. (5–72), the following representation for ν(K) in terms of

µ(K) in the last mesh interval K is obtained:

ν(K)1:N = −[W(K)D(K)]−1Γ(K)1:N = −A(K)[W(K)]−1Γ(K)1:N . (5–79)

130

It can be shown that A is an integration matrix which also extrapolates the final value of

the vector it operates on. Specifically, the elements of A are given as

Aij =

∫ τi

+1

Lj(τ)dτ + wN Lj(τN), (i , j = 1, ... ,N − 1),

AiN = −wN , (i = 1, ... ,N),

ANj = wN Lj(τN), (j = 1, ... ,N − 1),

(5–80)

where the N − 1 Lagrange interpolating polynomials Lj(τ) are defined as

Lj (τ) =

N−1∏

i=1j 6=i


, j = 1, ... ,N − 1. (5–81)

It is known that the KKT multiplier Γ is related to the dual variables of the continuous-time

problem as follows

Γ(k)i =

µ(k)i

w(k)i

, (i = 1, ... ,N − 1),

Γ(k)N =

µ(k)N + η

(k)

w(k)N

,

(5–82)

where it is recalled from Chapter 2 that µ(k) = ν(k), and η is the multiplier associated

with the “jump”, or discontinuity of the state constraint multiplier. Next, let the continuous-time

state constraint multiplier µ be a polynomial of degree at most N−2 such that µi = µ(τi)

for (i = 1, ... ,N − 1). This polynomial can be described exactly using the Lagrange

interpolating bases of Eq. (5–81) such that

µ(τ) =

N−1∑

j=1

µj Lj . (5–83)

Substituting the expressions from Eq. (5–82) into Eq. (5–80) it is seen that

A(k)i Γ

(k)1:N =

∫ τi

+1

µ(k) dτ +N−1∑

j=1

µj L(k)j (τN)− w (k)N Γ(k)N , (i = 1, ... ,N − 1),

N−1∑

j=1

µj L(k)j (τN)− w (k)N Γ(k)N , (i = N).

(5–84)

131

Thus, it is seen that the matrix A integrates and extrapolates the multiplier µ for (i =

1, ... ,N − 1). Furthermore, the expression for (i = N) amounts to an approximation of

the jump multiplier η(k). Consequently, the right hand side of Eq. (5–79) is equivalent to

integrating the state inequality constraint multipliers backward from τ (K) = +1 in the last

mesh interval K . Furthermore, because Γ(k) is being integrated backward across each

mesh interval, the value of the integration in interval k + 1 must be added as an initial

condition of the integration in interval k , yielding:

ν(k)1:N = 1ν(k+1)0 − A(k)[W(k)]−1Γ(k)1:N, (k = 1, ... ,K − 1), (5–85)

where 1 is a N × 1 column vector composed of ones.

To justify that Eq. (5–74) is an approximation of the state constraint multiplier

at τ (k) = −1 for (k = 1, ... ,K), it is now shown that this definition is equivalent to

applying the Fundamental Theorem of Calculus. Let 1 ∈ RN+1 denote the vector whose

elements are all unity. Because the components of the vector D1 are the derivatives of

the constant polynomial q(τ) = 1 at the LGR points, D1 = 0, which implies that

D0 = −N∑

j=1

Dj . (5–86)

Taking the transpose of Eq. (5–86), substituting the rows of D for the rows of DT , and

multiplying the result on the right byW(K)ν(K)1:N yields

D(K)0

⊤W(K)ν(K)1:N = −ν(K)0 =

N∑

j=1

w(K)j D

(K)j,1:Nν

(K)1:N . (5–87)

Next, let ν(τ) be the polynomial of degree N − 1 that satisfies ν(τi) = νi for 1 ≤ i ≤ N .

Using Eq. (5–78) together with the fact that the Legendre-Gauss-Radau quadrature is

exact for a polynomial of degree N − 2, then

−ν0 = −νN +N∑

i=1

wj ν(τi) = −νN +∫ +1

−1

ν(τ)dτ .

132

Because ν0 is the approximation to the state constraint multiplier at τ = −1, while νN

is the approximation at τ = +1, it is seen that ν0 in Eq. (5–74) is consistent with the

Fundamental Theorem of Calculus. Using the same logic as above, let 1 ∈ RN denote

the vector whose elements are all unity. Because the components of the vector D†1 are

the derivatives of the constant polynomial q(τ) = 1 at the LGR points, D†1 = 0, which

implies:N∑

j=1

D†j = 0. (5–88)

Because D is identical to the differentiation matrix D† introduced in Eq. (5–76) except for

the (N,N) element, substituting the values of D into Eq. (5–88) results in

[D1]i =

0 for (i = 1, ... ,N − 1),

− 1wN

for (i = N).(5–89)

Thus, substituting the values of ν(k)1:N obtained from Eq. (5–85) into the right-hand side of

Eq. (5–87) results in the relationship

D(k)0

⊤W(k)ν(k)1:N = −ν(k)0 = −ν(k+1)0 +

N∑

j=1

w(k)j [W

−1Γ(k)1:N]j , (k = 1, ... ,K − 1). (5–90)

Furthermore, substituting the expressions given by Eq. (5–82) into Eq. (5–90) results in

the expression

− ν(k)0 = −ν(k+1)0 − η(k) +N∑

j=1

w(k)j µj , (k = 1, ... ,K − 1). (5–91)

which says that the value of the state constraint mutliplier at the first point of an interval

is given by a Radau quadrature which approximates the backward integral of the state

constraint dynamics across that interval summed with a terminal condition given by

ν(k)N = ν

(k+1)0 + η(k). Finally, in a similar fashion, it can be shown that Eq. (5–73) implies

133

the following relation:

D0Wp1:N = −p0 = −pN +N∑

j=1

wjD†j,1:Np1:N. (5–92)

Because D† is a differentiation matrix, it is seen that Eq. (5–73) is also consistent with

the Fundamental Theorem of Calculus.

Now, using the transformations described in Eqs. (5–71)–(5–74) along with

Eqs. (5–75) and (5–77), the KKT conditions of the NLP given by Eqs. (5–64)–(5–69)

can be written as

D(k)Y(k)0:Nk = f(Y(k)1:Nk,U(k)1:Nk ), φ(Y

(1)0 ) = 0, (k = 1, ... ,K), (5–93)

0 = ∇UH(Y(k)1:N,U(k)1:N, p(k)1:N,ν(k)1:N), (k = 1, ... ,K), (5–94)

D†(k)1:Nkp(k)1:Nk= −∇YH(Y(k)1:N,U(k)1:N , p(k)1:N,ν(k)1:N) (5–95)

+eNkwNk

[

p(k)Nk− p(k+1)0

]

, (k = 1, ... ,K − 1), (5–96)

D†(K)1:NKp(K)1:NK= −∇YH(Y(K)1:N ,U(K)1:N , p(K)1:N ,ν(K)1:N)

+eNKwNK

[

p(K)N −∇Y

(

Φ(Y(K)N )− 〈ν(K)N ,S(Y(K)N )〉

)]

(5–97)

p(1)0 = ∇Y

(

〈ψ,φ(Y(1)0 )〉+ 〈ν0,S(Y0)〉)

(5–98)

S(Y(k)1:N) ≤ 0, D(k)ν(k)1:N ≥ 0, 〈D(k)ν(k)1:N,S(Y(k)1:N)〉 = 0, (k = 1, ... ,K), (5–99)

where H is the continuous Hamiltonian defined in Eq. (5–13). Next, substituting

Eq. (5–97) into Eq. (5–92) yields

−p(K)0 = −∇Y(

Φ(Y(K)N )− 〈ν(K)N ,S(Y(K)N )〉

)

−NK∑

i=1

w(K)i ∇YH(Y(K)i ,U(K)i , p(K)i ,ν(K)i ). (5–100)

If p(K)i were the continuous costate evaluated at τi in interval K , then by the continuous

adjoint equation, the sum in Eq. (5–100) approximates the integral of p(K) between −1

134

and +1. Eq. (5–100) amounts to an approximation to the relation

p(K)N = ∇Y

(

Φ(Y(K)N ) + 〈ν(K)N ,S(Y(K)N )〉

)

, (5–101)

where Eq. (5–101) is a discrete form of the continuous optimality condition given in

Eq. (5–11). Consequently, it is expected that the eN term in Eq. (5–97) should be

small, while the remaining terms in Eq. (5–97) amount to a collocation scheme for the

continuous adjoint equation. Next, it is shown that the last term in the brackets on the

right-hand side of Eq. (5–96) will be small. Substituting Eq. (5–96) into Eq. (5–92) yields

p(k+1)0 = p

(k)0 −

Nk∑

i=1

w(k)i ∇YH(Y(k)i ,U(k)i , p(k)i ,ν(k)i ), (k = 1, ... ,K − 1). (5–102)

If p(k)i were the continuous costate evaluated at τi in interval k , then by the continuous

adjoint equation, the sum in Eq. (5–102) approximates the integral of p(K) between −1

and +1. Eq. (5–102) amounts to an approximation to the relation

p(k)Nk= p

(k+1)0 . (5–103)

This condition shows that the costate will be continuous across mesh interval boundaries.

Consequently, it is expected that the eN term in Eq. (5–96) should be small, while the

remaining terms in Eq. (5–96) amount to a collocation scheme for the continuous adjoint

equation.

The connection between the transformed optimality conditions and the original

continuous optimality conditions is quite subtle. For example, the nonnegativity

conditions for the derivative of the state multiplier ν(k) and the complementary slackness

conditions in Eq. (5–12) are embedded in a very unusual way in the discrete optimality

conditions. As pointed out in Eq. (5–78), if the discrete multiplier νk)1:N associated with

the state constraint is interpolated by a polynomial ν(τ) of degree N − 1, then the

nonnegativity conditions in Eq. (5–98) only ensure nonnegativity of the polynomial

135

derivative at τ1 through τN−1. At τN , the discrete positivity condition amounts to

ν(K)(τN)−ν(K)(τN)

w(K)N

≥ 0, (5–104)

ν(K)(τN) +−ν(k)(τN) + ν(k+1)(τ0)

w(k)N

≥ 0, (k = 1, ... ,K). (5–105)

To illustrate how these conditions work, suppose that the state constraint is inactive over

the entire final interval [−1,+1] for the discrete problem. That is, S(Y(K)i ) < 0 for all i . In

this case, complementary slackness implies that

ν(K)(τi) = 0 for 1 ≤ i ≤ N − 1. (5–106)

Because the derivative of a polynomial of degree N − 1 is N − 2, the N − 1 conditions

Eq. (5–106) imply that the derivative is identically zero. Hence, ν(K)(τN) = 0 in

Eq. (5–104), and it is concluded that ν(K)(τN) = ν(+1) ≤ 0. Finally, from the

complementary slackness condition,

0 = S(Y(K)N )

T(ν(K)(τN)− ν(K)(τN)/wN)

= −S(Y(K)N )Tν(K)(τN)/wN ,

which implies that ν(K)(τN) = ν(K)N = 0 when S(Y(K)N ) < 0. Hence, the continuous

optimality conditions ν(K)(+1) ≤ 0 and 〈ν(K)(+1),S(y(K)(+1))〉 = 0 are satisfied

in the discrete problem. Furthermore, if complementary slackness holds in interval

K , then ν(K) is a non-decreasing function in interval K , and ν(K)0 ≥ ν(K−1)N , thus the

second term in Eq. (5–105) will be greater than zero. A similar argument can be made

over each interval, such that the condition ν(k) ≥ 0 is satisfied for (k = 1, ... ,K).

Thus, it has been shown that the conditions of the transformed adjoint system given

by Eqs. (5–93)–(5–99) are a discrete form of the first-order optimality conditions of the

continuous-time optimal control problem given by Eqs. (5–7)–(5–12).

The transformed adjoint system for variable-order collocation at the LGR points is

complex. For instance, the differentiation matrix associated with the state dynamics, D,

136

is a N × (N + 1) full-rank differentiation matrix associated with the space of polynomials

of degree at most N . Conversely, the differentiation matrix associated with the costate

dynamics, D† is a rank-defficient N × N differentiation matrix associated with the space

of polynomials of degree at most N − 1. Finally, the matrix associated with the state

constraint multiplier, D, is a full-rank N × N matrix that differentiates and evaluates the

terminal condition simultaneously.

5.4 Discussion

In this chapter costate estimates were derived for estimating the costate of state

inequality path constrained optimal control problems using orthogonal collocation

at the Legendre-Gauss and the flipped Legengre-Gauss-Radau Points. These

conditions result in a continuous approximation to the costate even in the presence

of state inequality path constraints. Furthermore, the costate estimate derived here

reduces to the costate estimate given by Ref. [1], presented in Chapter 4, when no

state inequality path constraints are present in the optimal control problem. Finally,

It was shown that the costate estimate using the method of indirect adjoining with

continuous multipliers resulted in a transformed adjoint system that is a discrete form of

the first-order optimality conditions of the continuous-time problem. Fig. (5-1) illustrates

the equivalence between the transformed adjoint system derived from the NLP and the

first-order optimality conditions of the continuous-time problem derived from the calculus

of variations.

137

Figure 5-1. Relationship between the direct and indirect methods for solving an optimalcontrol problem. In the indirect method, the problem is first optimized through thecalculus of variations, leading to a set of conditions which can then be discretizedand solved. In the direct method, the problem is first discretized and transcribed toan NLP, then it is optimized by solving the KKT system. The two systems are shown tobe equivalent even in the presence of a discontinuous costate.

138

CHAPTER 6EXAMPLES

In this chapter, four examples are studied using the methods developed in Chapters

3 and 5. The first two examples demonstrate the effectiveness of the costate estimation

methods derived in Chapter 3 using the integral form of LG and LGR collocation.

The first example is a single state nonlinear Mayer optimal control problem while

the second example is a single state nonlinear Lagrange optimal control problem.

Next, two state inequality path constrained optimal control problems are solved using

variable-order LG and LGR collocation as described by Chapter 5. The first state

inequality constrained example contains a first-order state inequality path constraint,

while the second state inequality constrained example contains a second-order state

inequality path constraint. The LG and LGR costate estimates derived in Ref. [1]

are shown to produce inaccurate estimates of the dual variables for both examples,

while the LG and LGR costate estimates using the method of indirect adjoining with

continuous multipliers (as described in Chapter 5) are shown to produce accurate

approximations of the dual variables.

Three main observations are made from the examples solved in this chapter. First,

it is shown that variable-order collocation at the LG and LGR points produces accurate

approximations to state inequality path constrained optimal control problems. Because

collocation at the LG points does not provide an approximation to the optimal control at

any of the mesh points whereas collocation at the LGR points provides an approximation

of the optimal control at all interior mesh points, collocation at the LGR method is found

to be the preferred method of solution. Second, it is shown that for state inequality path

constraints of at most order two, it is not necessary to reformulate the optimal control

problem by reducing the index of the DAE in order to obtain an accurate approximation.

Index-reduction requires an analytic reformulation of the optimal control problem which

may be cumbersome, if not impossible, to implement when using mesh refinement.

139

Third, because the method of indirect adjoining with continuous multipliers produces

a costate estimate that is continuous even in the presence of state inequality path

constraint, highly accurate estimates of the costate can be obtained even when using

low-order polynomial approximations in the state (that is, a small number of collocation

points).

6.1 Example 1: Mayer Optimal Control Problem

The first example considered is a nonlinear one-dimensional Mayer optimal control

problem [64]. It is stated as follows:

Minimize J = −y(2) subject to

y = 52(−y + yu − u2),

y(0) = 1.

The optimal solution is given as

y ∗(t) =4

a(t), u∗(t) =

y ∗(t)

2,

p∗y(t) = −(15 exp(5t/2)− 1)(−3 exp(5t/2)− 1)(2b exp(5t/2))

, λ∗y(t) = −exp(2 ln(a(t))− 5t/2)

b,

where a(t) = 1 + 3 exp(5t/2), and b = exp(−5) + 6 + 9 exp(5). This example was solved

using the integral LG and LGR collocation methods using the NLP solver SNOPT [22],

where SNOPT was implemented using optimality and feasibility tolerances of 1 × 10−8

and 2 × 10−8, respectively. The initial guess used for the state and control was a linear

interpolation from the initial state value to zero. For collocation at either set of points,

the integral costate, py(τ), was estimated from the KKT multipliers of the NLP, and

the differential costate, λy(τ), was subsequently computed from the integral costate

approximation.

6.1.1 Solution Using Collocation at Legendre-Gauss Points

Example 1 was solved using integral collocation at LG points as described in

Chapter 3. Figure 6-1 shows the state and control approximation obtained using N = 20

140

00

0.1

0.2

0.2

0.3

0.4

0.4

0.5

0.6

0.6

0.7

0.8

0.8

0.9

1.2 1.4 1.6 1.8

1

1 2t

Sta

te

y∗(t)

y(t)

(A) State.

00

0.1

0.2

0.2

0.3

0.4

0.4

0.5

0.6 0.8 1.2 1.4 1.6 1.8

0.05

0.15

0.25

0.35

0.45

1 2t

Con

trol

u∗(t)

u(t)

(B) Control.

Figure 6-1. Primal solution for Example 1 obtained using integral collocation at LGpoints.

LG collocation points. It is seen that integral collocation at the LG points provides a

highly accurate approximation to the optimal solution.

Next, the integral costate, py (τ), was computed at the LG points using Eq. (3–62),

and the differential costate, λy(τ), was estimated at the LG points plus the noncollocated

endpoints τ0 and τN+1 using the results of Section 3.2.5. Figure 6-2 shows both the

141

Cos

tate

-0.5

0

0

0.5

0.5

1.5

1.5

2.5

-1

1

1

2

2t

p∗y

λ∗

y

py

λy

Figure 6-2. Integral and differential costate solutions for Example 1 obtained using LGcollocation.

integral and the differential costates obtained using N = 20 LG collocation points.

It is seen that the costate estimate is indistinguishable from the optimal costate.

Furthermore, Fig. 6-3 shows the base ten logarithm of the L∞-norm error for the

integral and differential costates when approximated using (N = 2, 4, 6, ... , 20) LG

collocation points. It is interesting to note that the differential costate estimate converges

exponentially as a function of N and reaches an accuracy of O(10−12) for N = 20,

whereas the integral costate estimate achieves an accuracy of approximately O(10−11)

for N = 20.

142

0

0

-8

-6

-4

-2

4 8 12-12 16 20

-10

N

log10

Infin

ity-N

orm

Err

or

∥

∥λy − λ∗y

∥

∥

∞

∥

∥py − p∗y∥

∥

∞

Figure 6-3. Integral and differential costate errors for Example 1 obtained using LGcollocation.

143

6.1.2 Solution Using Collocation at Legendre-Gauss-Radau Points

Next, Example 1 was solved using integral LGR collocation as described in Chapter

3. Figure 6-4 shows the state and control approximation obtained using N = 20

LGR collocation points. It is seen that, similar to collocation at the LG points, integral

collocation at the LGR points provides highly accurate approximation to the optimal

solution. However, unlike collocation at the LG points, collocation at the LGR points

provides an approximation of the control at the terminal boundary point, making

collocation at the LGR points more desirable than collocation at the LG points.

Next, the integral costate, py (τ), was computed at the LGR points using Eq. (3–116),

and the differential costate, λy(τ), was estimated at the LGR points plus the noncollocated

endpoint τN+1 using the results of Section 3.3.5. Note that the value py(τ1) (where

τ1 = −1 for the integral LGR collocation method) was found by extrapolating the

Lagrange interpolating polynomial as described by Eqs. (3–118) and (3–119). Figure

6-5 shows both the integral and the differential costates obtained using N = 20

LGR collocation points. It is seen that the costate estimate is indistinguishable from

the optimal solution. Furthermore, Fig. 6-6 shows the base ten logarithm of the

L∞-norm error for the integral and differential costates when approximated using

(N = 2, 4, 6, ... , 20) LGR collocation points. It is seen that the differential and integral

costate estimates converges exponentially as a function of N until the error reaches

approximately O(10−10) for N = 20.

144

00

0.1

0.2

0.2

0.3

0.4

0.4

0.5

0.6

0.6

0.7

0.8

0.8

0.9

1.2 1.4 1.6 1.8

1

1 2t

Sta

te

y∗(t)

y(t)

(A) State.

00

0.1

0.2

0.2

0.3

0.4

0.4

0.5

0.6 0.8 1.2 1.4 1.6 1.8

0.05

0.15

0.25

0.35

0.45

1 2t

Con

trol

u∗(t)

u(t)

(B) Control.

Figure 6-4. Primal solution for Example 1 obtained using integral collocation at LGRpoints.

145

Cos

tate

-0.5

0

0

0.5

0.5

1.5

1.5

2.5

-1

-1.5

1

1

2

2t

p∗y

λ∗

y

py

λy

Figure 6-5. Integral and differential costate solutions for Example 1 obtained using LGRcollocation.

0

0

-8

-6

-4

-2

4 8 12 16 20

-10

-12

N

log10

Infin

ity-N

orm

Err

or

∥

∥λy − λ∗y

∥

∥

∞

∥

∥py − p∗y∥

∥

∞

Figure 6-6. Integral and differential costate errors for Example 1 obtained using LGRcollocation.

146

6.2 Example 2: Lagrange Optimal Control Problem

This second example considered is a nonlinear one-dimensional Lagrange optimal

control problem given as follows.

Minimize J = 12

∫ tf

0

(log2 y + u2)dt subject to

y = y log y + yu,

y(0) = 5,

y(tf ) = 3.

The optimal solution to this example is given as

y ∗(t) = exp(x∗(t)),

λ∗y(t) = λ∗

x(t)/y∗(t),

p∗y (t) = − λ∗x(t)y

∗(t)− λ∗x(t)y

∗(t)

(y ∗(t))2,

(6–1)

where

x∗(t) = c1 exp(−t√2) + c2 exp(t

√2),

λ∗x(t) = c1(1 +

√2) exp(−t

√2) + c2(1−

√2) exp(t

√2),

(6–2)

and

c1

c2

=

1 1

exp(−tf√2) exp(tf

√2)

log y0

log yf

. (6–3)

The example was solved using the integral LG and LGR collocation methods using the

NLP solver SNOPT [22], where SNOPT was implemented using optimality and feasibility

tolerances of 1 × 10−8 and 2 × 10−8, respectively, with the exact state and control

evaluated at the discretization points as the initial guess. For collocation at either set of

points, the integral costate, py (t), was estimated from the KKT multipliers of the NLP,

and the differential costate, λy(t), was subsequently computed from the integral costate

approximation.

147

3.5

3.5

4.5

4.5

5.5

0 0.5

1.5

1.5

2.5

2.51 1

2

2

3

3

4

4

5

5t

Sta

te

y∗(t)

y(t)

(A) State.

-3.5

-2.5

-1.5

3.5 4.5

-0.5

0

0

0.5

0.5 1.5 2.5-4

-3

-2

-1

1 2 3 4 5t

Con

trol

u∗(t)

u(t)

(B) Control.

Figure 6-7. State and control for Example 2 obtained using integral LG collocation.


Example 2 was solved using integral collocation at LG points, as described in

Chapter 3. Figure 6-7 shows the primal solution (that is, the state and control) obtained

using N = 32 LG collocation points. It is seen that integral collocation at the LG points

provides highly accurate approximation to the optimal solution.

148

-0.8

-0.6

-0.4

-0.2

0

0

0.2

0.4

0.6

0.8

1

1 2 3 4 5t

Cos

tate

p∗y

λ∗

y

py

λy

Figure 6-8. Integral and differential costate for Example 2 obtained using LG collocation.

Next, the integral costate, py (τ), was estimated at the LG points using Eq. (3–62),

and the differential costate, λy(τ), was estimated at the LG points plus the noncollocated

endpoints τ0 and τN+1 using the results of Section 3.2.5. Figure 6-8 shows both the

integral and the differential costates obtained using N = 32 LG collocation points. It is

seen that both the differential and integral costate estimates are indistinguishable from

the optimal costates. Figure 6-9 shows the base ten logarithm of the L∞-norm error for

the integral and differential costates for (N = 4, 8, 12, ... , 32) LG collocation points. Both

the differential and integral costate estimates converges exponentially as a function of N

until the error reaches approximately O(10−12) for N = 32.

149

322824-12

0

0

-8

-6

-4

-2

4 8 12 16 20

-10

N

log10

Infin

ity-N

orm

Err

or

∥

∥λy − λ∗y

∥

∥

∞

∥

∥py − p∗y∥

∥

∞

Figure 6-9. Integral and differential costate errors for Example 2 obtained using integralLG collocation.

150

6.2.2 Solution Using Collocation at Legendre-Gauss-Radau Points

Next, Example 2 was solved using integral collocation at LGR points, as described

in Chapter 3. Figure 6-10 shows the state and control approximations obtained using

N = 32 LGR collocation points. It is seen that, similar to collocation at the LG points,

integral collocation at the LGR points provides highly accurate approximation to the

optimal solution. However, unlike collocation at the LG points, collocation at the LGR

points provides an approximation of the control at the terminal boundary point, making

collocation at the LGR points more desirable than collocation at the LG points.

The integral costate, py (τ), was computed at the LGR points using Eq. (3–116), and

the differential costate, λy(τ), was estimated at the LGR points plus the noncollocated

endpoint τN+1 using the results of Section 3.3.5. Note that the value py(τ1) (where

τ1 = −1 for the integral LGR collocation method) was found by extrapolating the

Lagrange interpolating polynomial as described by Eqs. (3–118) and (3–119). Figure

6-11 shows both the integral and the differential costates obtained using N = 32

LGR collocation points. It is seen that the costate estimate is indistinguishable from

the optimal solution. Furthermore, Fig. 6-12 shows the base ten logarithm of the

L∞-norm error for the integral and differential costates when approximated using

(N = 4, 8, 12, ... , 32) LGR collocation points. It is seen that the differential and integral

costate estimates converges exponentially as a function of N until the error reaches

approximately O(10−12) for N = 32.

151

3.5

3.5

4.5

4.5

5.5

0 0.5

1.5

1.5

2.5

2.51 1

2

2

3

3

4

4

5

5t

Sta

te

y∗(t)

y(t)

(A) State.

-3.5

-2.5

-1.5

3.5 4.5

-0.5

0

0

0.5

0.5 1.5 2.5-4

-3

-2

-1

1 2 3 4 5t

Con

trol

u∗(t)

u(t)

(B) Control.

Figure 6-10. State and control for Example 2 obtained using integral LGR.

152

-0.8

-0.6

-0.4

-0.2

0

0

0.2

0.4

0.6

0.8

1

1 2 3 4 5t

Cos

tate

p∗y

λ∗

y

py

λy

Figure 6-11. Integral and differential costate for Example 2 obtained using integral LGRcollocation with N = 32.

322824-12

0

0

-8

-6

-4

-2

4 8 12 16 20

-10

N

log10

Infin

ity-N

orm

Err

or

∥

∥λy − λ∗y

∥

∥

∞

∥

∥py − p∗y∥

∥

∞

Figure 6-12. Integral and differential costate errors for Example 2 obtained using integralLGR collocation.

153

6.3 Example 3: First-Order State Inequality Path Constraint Problem

Consider the following state inequality path constrained optimal control problem:

Minimize

∫ 3

0

e−tu dt subject to

y = u, y(0) = 0,

y − 1 + (t − 2)2 ≥ 0,

0 ≤ u ≤ 3.

(6–4)

The optimal state and control for this example are given as

y ∗ =

0 , t ∈ [0, 1),

1− (t − 2)2 , t ∈ [1, 2],

1 , t ∈ (2, 3],

u∗ =

0 , t ∈ [0, 1),

2(2− t) , t ∈ [1, 2],

0 , t ∈ (2, 3],

(6–5)

This example was solved using LG and flipped LGR collocation with the NLP solver

SNOPT, where SNOPT was implemented using default settings. The solution domain

was divided into three intervals with N collocation points in each interval. The boundaries

between the intervals were chosen to be the time instants where the state constraint

changes between active and inactive, namely, the interval boundaries were at t = 1 and

t = 2. Furthermore, a straight line initial guess between the initial state and unity was

used for the state and control.


Example 3 was solved using variable-order collocation at the Legendre-Gauss

points, as described in Chapter 2. Figure 6-13 shows the state and control approximations

obtained using N = 10 collocation points per interval. It is seen that variable-order LG

collocation provides highly accurate approximations to the optimal solution even though

index-reduction of the state inequality path constraint was not performed. Figure 6-14

154

0

0

0.2

0.4

0.5

0.6

0.8

1.2

1.5 2.5

1

1 2 3-0.2

t

Sta

te

y∗(t)

y(t)

(A) State.

00

0.5

0.5

1.5

1.5

2.5

2.5

1

1

2

2 3t

Con

trol

u∗(t)

u(t)

(B) Control.

Figure 6-13. Primal solution for Example 3 obtained using collocation at LG points.

shows the base ten logarithm of the L∞-norm error for the state and the control for

(N = 2, 4, 6, ... , 20) collocation points per interval. It is interesting to see that the LG

state and control approximations are highly accurate even using low-degree state

approximations (that is, using a small number of collocation points).

155

-8.5

-9.5

-10.5

-11

-11.5

12

-9

-8

2 4 6 8 10 12 14 16 18 20

-10

N

log10

Infin

ity-N

orm

Err

or

‖y − y∗‖∞

‖u − u∗‖∞

Figure 6-14. State and control errors for Example 3 obtained using LG collocation.

156

6.3.1.1 Previously derived costate estimate

The accuracy of the costate estimate of Ref. [1] is now compared to the costate

estimate derived in Chapter 5. The analytic optimal costate for Example 3 using

the method of direct adjoining can be found by applying the first-order optimality

conditions derived in Section 2.3.2. The costate and state constraint multipliers are

given, respectively, as

λ∗ =

−e−1 , t ∈ [0, 1),

−e−t , t ∈ [1, 2],

0 , t ∈ (2, 3],

µ∗ =

0 , t ∈ [0, 1),

−e−t , t ∈ [1, 2],

0 , t ∈ (2, 3].

(6–6)

Figure 6-15 shows the result of the costate approximation for N = 10 collocation

points per interval. It can be seen that the costate, λ(t), is not approximated correctly at

the interval boundaries where the discontinuities occur. Furthermore, the approximation

of the state constraint multiplier, µ, is quite poor. Figure 6-16 shows the base ten

logarithm of the L∞-norm error for the costate and the state constraint mutlipliers for

(N = 2, 4, 6, ... , 20) collocation points per interval. It is seen that the costate has large

errors near the known discontinuities in the optimal costate. Furthermore, the state

constraint multiplier estimate diverges.

157

0

0 0.5 1.5 2.5

0.05

1 2 3-0.4

-0.35

-0.3

-0.25

-0.15

-0.1

-0.05

-0.2

t

Cos

tate

λ∗(t)

λ(t)

(A) Costate.

-0.5

0

0 0.5 1.5 2.5-4

-3

-2

-1

1 2 3

-3.5

-2.5

-1.5

t

Sta

teC

onst

rain

tMul

tiplie

r

µ∗(t)

µ(t)

(B) State Constraint Multiplier.

Figure 6-15. Costate Estimate as derived by Ref. [1] for Example 3 obtained usingcollocation at LG points.

158

0

1.2

1

2 4 6 8 10 12 14 16 18 20

-0.2

-0.4

-0.6

0.2

0.4

0.6

0.8

N

log10

Infin

ity-N

orm

Err

or

‖λ− λ∗‖∞

‖µ− µ∗‖∞

Figure 6-16. Errors in costate estimate derived by Ref. [1] for Example 3 obtained usingLG collocation.

159

6.3.1.2 Costate estimate using method of indirect adjoining with continuousmultipliers

The costate estimate derived using the method of indirect adjoining with continuous

multipliers for collocation at the LG points, described in Chapter 5, is now analyzed. The

optimal costate for Example 3 using the method of indirect adjoining with continuous

multipliers can be found by applying the first-order optimality conditions derived in

Section 2.3.3. The costate and the state constraint multipliers are given, respectively, as

p∗ =

0 , t ∈ [0, 1),

0 , t ∈ [1, 2],

0 , t ∈ (2, 3],

ν∗ =

−e−1 , t ∈ [0, 1),

−e−t , t ∈ [1, 2],

0 , t ∈ (2, 3].

(6–7)

Figure 6-17 shows the costate approximation for N = 10. It can be seen that the

estimates presented in Chapter 5 provide an accurate costate approximation of the

continuous optimal control problem. Figure 6-18 shows the base ten logarithm of the

L∞-norm error for the costate and state constraint multiplier. It can be seen that the

error on the state inequality constraint multiplier decreases as the number of collocation

points is increased. Furthermore, the error on the costate approximation remains

approximately zero. Therefore, the costate estimate produces an accuracy of O(10−12)

even when inacuracies in the state constraint multiplier are present due to the use of

low-order polynomial approximations in the state.

160

0.5 1 1.5 2 2.5 3-0.4

-0.35

-0.3

-0.25

-0.2

-0.15

-0.1

-0.05

0

0

0.05

t

Dua

lVar

iabl

es

p∗(t)

ν∗(t)

p(t)

ν(t)

Figure 6-17. Dual variables for Example 3 obtained using collocation at LG points.

0

-2

-4

-6

-8

-10

-12

-142 4 6 8 10 12 14 16 18 20

N

log10

Abs

olut

eE

rror

ν(t)

p(t)

Figure 6-18. Costate errors for Example 3 obtained using collocation at LG points.

161

6.3.2 Solution Using Collocation at Flipped Legendre-Gauss-Radau Points

Example 3 is now solved using variable-order flipped LGR collocation as described

in Chapter 2. Figure 6-19 shows the state and control approximations obtained using

N = 10 collocation points per interval. It is seen that variable-order collocation at the

LG points provides highly accurate approximations to the optimal solution even though

index-reduction of the state inequality path constraint was not performed. Figure 6-20

shows the base ten logarithm of the L∞-norm error for the state and the control using

(N = 2, 4, 6, ... , 20) collocation points per interval. It is seen that the solution using LGR

collocation is less accurate than the solution obtained using collocation at the LG points.

This difference in accuracy is expected, as the LG points are known to have a higher

accuracy quadrature than the LGR points. Furthermore, it is seen that the state and

control reach accuracies of O(10−5) and O(10−8), respectively, for N = 20.


The accuracy of the costate estimate of Ref. [1] using flipped LGR collocation is

now compared against the accuracy of the costate estimate derived in this research.

The optimal costate for Example 3 using the method of direct adjoining can be found by

applying the first-order optimality conditions derived in Section 2.3.2. The costate and

state constraint multipliers are given, respectively, as

λ∗ =

−e−1 , t ∈ [0, 1),

−e−t , t ∈ [1, 2],

0 , t ∈ (2, 3],

µ∗ =

0 , t ∈ [0, 1),

−e−t , t ∈ [1, 2],

0 , t ∈ (2, 3].

(6–8)

Figure 6-21 shows the result of the costate approximation when N = 10 collocation

points per interval were used. It can be seen that although the costate is approximated

162

0

0

0.2

0.4

0.5

0.6

0.8

1.2

1.5 2.5

1

1 2 3-0.2

t

Sta

te

y∗(t)

y(t)

(A) State.

00

0.2

0.4

0.5

0.6

0.8

1.2

1.4

1.5

1.6

1.8

2.5

1

1

2

2 3t

Con

trol

u∗(t)

u(t)

(B) Control.

Figure 6-19. Primal solution for Example 3 obtained using variable-order collocation atLGR points.

163

−16

−14

12

-8

-6

-4

-2

2 4 6 8 10 12 14 16 18 20

-10

N

log10

Infin

ity-N

orm

Err

or

‖y − y∗‖∞

‖u − u∗‖∞

Figure 6-20. State and control errors for Example 3 obtained using variable-ordercollocation at LGR points.

accurately, the state constraint multiplier, µ, is approximated very poorly where the

costate is discontinuous. Figure 6-22 shows the base ten logarithm of the L∞-norm

error for the costate and the state constraint mutliplier when approximated using

(N = 2, 4, 6, ... , 20) collocation points per interval. It is seen that the costate estimate

has large errors near the known discontinuities in the optimal costate. Furthermore, it is

seen that the state constraint multiplier estimate diverges.

6.3.2.2 Costate estimation using method of indirect adjoining with continuousmultipliers


multipliers for collocation at the flipped LGR points,described in Chapter 5, is now

analyzed. The analytic optimal costate for Example 3 using the method of indirect

adjoining with continuous multipliers can be found by applying the first-order optimality

conditions derived in Section 2.3.3. The costate and the state constraint multipliers are

164

Dua

l Sol

utio

n

0

0 0.5 1.5 2.51 2 3-0.4

-0.35

-0.3

-0.25

-0.15

-0.1

-0.05

-0.2

t

λ∗(t)

λ(t)

(A) Costate.

−14

−12

−10

−8

−6

Dua

l Sol

utio

n

0

0 0.5 1.5 2.5

-4

-2

1 2 3t

µ∗(t)

µ(t)

(B) State Constraint Multiplier.

Figure 6-21. Costate Estimate as derived by Ref. [1] for Example 3 obtained using LGRcollocation.

165

−0.5

0.5

0

1.5

1

2

2 4 6 8 10 12 14 16 18 20N

log10

Infin

ity-N

orm

Err

or‖λ− λ∗‖

∞

‖µ− µ∗‖∞

Figure 6-22. Errors in costate estimate derived by Ref. [1] for Example 3 obtained usingLGR collocation.


p∗ =

0 , t ∈ [0, 1),

0 , t ∈ [1, 2],

0 , t ∈ (2, 3],

ν∗ =

−e−1 , t ∈ [0, 1),

−e−t , t ∈ [1, 2],

0 , t ∈ (2, 3].

(6–9)

Figure 6-23 shows the costate approximation for N = 10 obtained by using the

method described in Chapter 5 for collocation at LGR points. It can be seen that the

estimate presented in this research provides an accurate approximation of the costate.

Figure 6-24 shows the base ten logarithm of the L∞-norm error for the costate and state

constraint multiplier approximations. It can be seen that the error on the state inequality

constraint multiplier decreases exponentially as the number of collocation points is

increased. Furthermore, the error on the costate approximation remains approximately

zero. Therefore, the costate estimate produces an accuracy of O(10−12) even when

166

0.5 1 1.5 2 2.5 3-0.4

-0.35

-0.3

-0.25

-0.2

-0.15

-0.1

-0.05

0

0

0.05

t

Dua

lVar

iabl

es

p∗(t)

ν∗(t)

p(t)

ν(t)

Figure 6-23. Costate estimate for Example 3 obtained using collocation at LGR points.

0

-2

-4

-6

-8

-10

-12

-142 4 6 8 10 12 14 16 18 20

N

log10

Abs

olut

eE

rror

ν(t)

p(t)

Figure 6-24. Costate errors for Example 3 obtained using collocation at LGR points.

inacuracies in the state constraint multiplier are present due to the use of low-order

polynomial approximations in the state.

167

6.4 Example 4: Second-Order State Inequality Path Constraint Example

Consider the following second-order state inequality constrained optimal control

problem from Ref. [10]:

minimize 12

∫ 1

0

u2dt subject to

x = v ,

v = u,

x(0) = 0,

x(1) = 0,

v(0) = 1

v(1) = −1,

x(t) ≤ ℓ.

It is known for this example that the inequality path constraint is inactive for ℓ > 1/4,

is active at only a single point for 1/6 < ℓ ≤ 1/4, and is active along a nonzero duration

arc for 0 < ℓ ≤ 1/6. In the case where 0 < ℓ ≤ 1/6, the optimal state and control are

given as

x∗(t) =

ℓ[

1−(

1− t3ℓ

)3]

,

ℓ,

ℓ[

1−(

1− 1−t3ℓ)3]

,

t ∈ [0, 3ℓ],

t ∈ [3ℓ, 1− 3ℓ],

t ∈ [1− 3ℓ, 1],

v∗(t) =

(

1− t3ℓ

)2,

0,

−(

1− 1−t3ℓ)2,

t ∈ [0, 3ℓ],

t ∈ [3ℓ, 1− 3ℓ],

t ∈ [1− 3ℓ, 1],

u∗(t) =

− 23ℓ(

1− t3ℓ

)

,

0,

− 23ℓ(

1− 1−t3ℓ)

,

t ∈ [0, 3ℓ],

t ∈ [3ℓ, 1− 3ℓ],

t ∈ [1− 3ℓ, 1],

A value of ℓ = 1/10 was used in the analysis of this example. The solution domain was

divided into three intervals with N collocation points in each interval. The boundaries

between the intervals were chosen to be the time instants where the state constraint

changes between active and inactive, namely, t = 3/10 and t = 7/10. The solution was

168

approximated using N = 5 collocation points. All problems were solved using the NLP

solver SNOPT with default optimality and feasibility tolerances. [22]. The initial guess

used was the exact solution.


Example 4 was solved using variable-order collocation at the Legendre-Gauss

points, as described in Chapter 2. Figure 6-25 shows the state and control approximations

obtained using N = 5 collocation points per interval. It is seen that variable-order

collocation at the LG points provides highly accurate approximations to the optimal

solution even though index-reduction of the state inequality path constraint was not

performed. Figure 6-26 shows the base 10 logarithm of the L∞-norm error for the state

and the control when approximated using (N = 2, 3, ... , 10) collocation points per

interval. It is interesting to see that the errors in the primal solution for this example are

larger than the errors observed for the primal solution of Example 3. This difference

in accuracy can be attributted to the increase in the order of the state inequality path

constraint. Although the errors in this example are larger than for Example 3, it can be

seen that an accuracy of O(10−6) and O(10−5) for the state and control, respectively,

can be obtained using N = 3 collocation points per interval.

169

00

0.1

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

0.02

0.04

0.06

0.08

0.12

t

Sta

teC

ompo

nent

x∗(t)

x(t)

(A) x(t).

0

0 0.1

0.2

0.2 0.3

0.4

0.4 0.5

0.6

0.6 0.7

0.8

0.8 0.9

1

1

-0.2

-0.4

-0.6

-0.8

-1t

Sta

teC

ompo

nent

v∗(t)

v(t)

(B) v(t).

0

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9

-4

-3

-2

1

1

-5

-6

-7

-1

t

Con

trol

u∗(t)

u(t)

(C) u(t).

Figure 6-25. Primal solution for Example 4 obtained using LG collocation.

170

N

0

-8

-7

-6

-5

-4

-3

-2

-1

2 3 4 5 6 7 8 9 10

log10

Infin

ity-N

orm

Err

or

x(t)

v(t)

u(t)

Figure 6-26. State and control errors for Example 4 using collocation at LG points.

171


The accuracy of the costate estimate using LG collocation derived in Ref. [1] is

now compared against the accuracy of the costate estimate derived in this research.

The optimal costate for Example 4 using the method of direct adjoining can be found by


state constraint multiplier are given, respectively, as

λ∗x =

29ℓ2

, t ∈ [0, 3ℓ],

0 , t ∈ [3ℓ, 1− 3ℓ],

− 29ℓ2

, t ∈ [1− 3ℓ, 1],

λ∗v =

23ℓ

(

1− t3ℓ

)

, t ∈ [0, 3ℓ],

0 , t ∈ [3ℓ, 1− 3ℓ],23ℓ

(

1− 1−t3ℓ

)

, t ∈ [1− 3ℓ, 1],

µ∗ =

0 , t ∈ [0, 3ℓ],

0 , t ∈ [3ℓ, 1− 3ℓ],

0 , t ∈ [1− 3ℓ, 1].

(6–10)

Figure 6-27 shows the costate approximation for N = 5 collocation points per

interval. It can be seen that the costate, λ(t), is not approximated correctly at the

interval boundaries where the discontinuities occur. Furthermore, the state constraint

multiplier, µ, is approximated very poorly. Figure 6-28 shows the base ten logarithm of

the L∞-norm error for the costate and the state constraint mutliplier when approximated

using (N = 2, 3, ... , 10) collocation points per interval. It can be seen that large errors

around the costate discontinuities prevent the costate estimate from converging to its

optimal solution.

172

0

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

25

20

15

10

5

-5

-10

-15

-20

-25t

Cos

tate

Com

pone

nt

λ∗

x (t)

λx (t)

(A) Costate.

0

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9

6

7

-1

1

1

2

3

4

5

t

Cos

tate

Com

pone

nt

λ∗

v (t)

λv(t)

(B) Costate.

0

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

-50

-100

-150

-200

-250

-300

-350

-400t

µ∗(t)

µ(t)

Sta

teC

onst

rain

tMul

tiplie

r

(C) State Constraint Multiplier.

Figure 6-27. Costate Estimate as derived by Ref. [1] for Example 4 obtained usingcollocation at LG points.

173

1.5

2.5

1

2

2

3

3 4 5 6 7 8 9 10

3.5

0.5

N

log10

Infin

ity-N

orm

Err

or

‖λx − λ∗x ‖∞

‖λv − λ∗v ‖∞

‖µ− µ∗‖∞

Figure 6-28. Errors in costate estimate derived by Ref. [1] for Example 4 obtained usingLG collocation.

174

6.4.1.2 Costate Estimation using method of indirect adjoining with continuousmultipliers


multipliers for collocation at the LG points, described in Chapter 5, is now analyzed. The

optimal costate for Example 4 using the method of indirect adjoining with continuous

multipliers can be found by applying the first-order optimality conditions derived in

Section 2.3.3. The costate and the state constraint multiplier are given, respectively, as

p∗x(t) =

{

− 29ℓ2, t ∈ [0, 1],

p∗v (t) =

23ℓ

(

1− t3ℓ

)

,

0,

23ℓ

(

1− 1−t3ℓ

)

,

t ∈ [0, 3ℓ],

t ∈ [3ℓ, 1− 3ℓ],

t ∈ [1− 3ℓ, 1].

ν∗(t) =

− 49ℓ2,

− 29ℓ2,

0,

t ∈ [0, 3ℓ],

t ∈ [3ℓ, 1− 3ℓ],

t ∈ [1− 3ℓ, 1].

Figure 6-29 shows the dual variable approximations for collocation at the LG points.

It can be seen that the mapping presented in Chapter 5 provides an accurate estimate

for the dual variables of the continuous optimal control problem. Figure 6-30 shows the

base ten logarithm of the L∞-norm error for the costate and state constraint multiplier

approximations obtained using Eqs. (5–32)–(5–38) and Eq. (5–79) for N collocation

points in each of the three mesh intervals. It can be seen that the error on the dual

variables reach an accuracy of O(10−5) for low-order polynomial approximations in the

state (that is, a small number of collocation numbers).

175

-22.5

-22.4

-22.3

-22.2

-22.1

-22

-21.9

-21.8

-21.7

-21.6

-21.5

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1t

p∗x (t)

px (t)

Cos

tate

Com

pone

nt

(A) px(t).

0

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9

6

7

-1

1

1

2

3

4

5

t

p∗v (t)

pv (t)

Cos

tate

Com

pone

nt

(B) pv (t).

0

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

-5

-10

-15

-20

-25

-30

-35

-40

-45t

ν∗(t)ν(t)

Sta

teC

onst

rain

tMul

tiplie

r

(C) ν(t).

Figure 6-29. Costate estimate for Example 4 obtained using collocation at LG points.

176

0

-6

-5

-4

-3

-2

-1

1

2

2 3 4 5 6 7 8 9 10

N

log10

Abs

olut

eE

rror

ν(t)

px(t)

pv (t)

Figure 6-30. Costate errors for Example 4 obtained using collocation at LG points.

177

6.4.2 Solution Using Collocation at Flipped Legendre-Gauss-Radau Points

Next, Example 4 was solved using variable-order collocation at the flipped

Legendre-Gauss-Radau points, as described in Chapter 2. Figure 6-31 shows the

state and control approximations obtained using N = 5 collocation points per interval. It

is seen that variable-order collocation at the LG points provides accurate approximations

to the optimal solution even though index-reduction of the state inequality path constraint

was not performed. Figure 6-32 shows the base 10 logarithm of the L∞-norm error for

the state and the control when approximated using (N = 2, 3, ... , 10) collocation points

per interval. It is seen that the state reaches an accuracy of O(10−5) for N = 3, whereas

the control only reaches an accuracy of O(10−2) for N = 10.

178

0.01

0.03

0.07

0.09

00

0.1

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9

0.05

1

0.02

0.04

0.06

0.08

t

Sta

teC

ompo

nent

x∗(t)

x(t)

(A) x(t).

0

0 0.1

0.2

0.2 0.3

0.4

0.4 0.5

0.6

0.6 0.7

0.8

0.8 0.9

1

1

-0.2

-0.4

-0.6

-0.8

-1t

Sta

teC

ompo

nent

v∗(t)

v(t)

(B) v(t).

0

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9

-4

-3

-2

1

-5

-6

-7

-1

t

Con

trol

u∗(t)

u(t)

(C) u(t).

Figure 6-31. Primal solution for Example 4 obtained using LGR collocation.

179

N

0

-5

2 3 4

5

5 6 7 8 9 10

-10

-20

-25

-30

-35

-15

log10

Infin

ity-N

orm

Err

or

x(t)

v(t)

u(t)

Figure 6-32. State and control errors for Example 4 using collocation at LGR points.

180


he accuracy of the costate estimate using LGR collocation derived in Ref.[1] is now

compared against the accuracy of the costate estimate derived in this research. The

optimal costate for Example 4 using the method of direct adjoining can be found by


state constraint multipliers are given, respectively, as

λ∗x =

29ℓ2

, t ∈ [0, 3ℓ],

0 , t ∈ [3ℓ, 1− 3ℓ],

− 29ℓ2

, t ∈ [1− 3ℓ, 1],

λ∗v =

23ℓ

(

1− t3ℓ

)

, t ∈ [0, 3ℓ],

0 , t ∈ [3ℓ, 1− 3ℓ],23ℓ

(

1− 1−t3ℓ

)

, t ∈ [1− 3ℓ, 1],

µ∗ =

0 , t ∈ [0, 3ℓ],

0 , t ∈ [3ℓ, 1− 3ℓ],

0 , t ∈ [1− 3ℓ, 1].

(6–11)

Figure 6-33 shows the costate approximation for N = 5 collocation points per

interval.. It can be seen that although the costate is approximated accurately, the state

constraint multiplier, µ, is approximated very poorly. Figure 6-34 shows the base ten

logarithm of the L∞-norm error for the costate and the state constraint mutliplier when

approximated using (N = 2, 3, ... , 10) collocation points per interval. It can be seen

that large errors around the costate discontinuities prevent the costate estimate from

converging to its optimal solution.

181

0

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

25

20

15

10

5

-5

-10

-15

-20

-25t

Cos

tate

Com

pone

nt

λ∗

x (t)

λx (t)

(A) Costate.

0

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9

6

7

-1

1

1

2

3

4

5

t

Cos

tate

Com

pone

nt

λ∗

v (t)

λv(t)

(B) Costate.

−2000

−1500

−1000

−500

500

0

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1t

µ∗(t)

µ(t)

Sta

teC

onst

rain

tMul

tiplie

r

(C) State Constraint Multiplier.

Figure 6-33. Costate Estimate as derived by Ref. [1] for Example 4 obtained usingcollocation at LGR points.

182

0

-10

12

-8

-6

-4

-2

2

2 3 4 5 6 7 8 9 10N

log10

Infin

ity-N

orm

Err

or

‖λx − λ∗x ‖∞

‖λv − λ∗v ‖∞

‖µ− µ∗‖∞

Figure 6-34. Errors in costate estimate derived by Ref. [1] for Example 4 obtained usingLGR collocation.

183

6.4.2.2 Costate Estimation using method of indirect adjoining with continuousmultipliers


multipliers for collocation at the flipped LGR points, described in Chapter 5, is now

analyzed. The analytic optimal costate for Example 4 using the method of indirect

adjoining with continuous multipliers can be found by applying the first-order optimality

conditions derived in Section 2.3.3. The costate and the state constraint multiplier are


p∗x(t) =

{

− 29ℓ2, t ∈ [0, 1],

p∗v (t) =

23ℓ

(

1− t3ℓ

)

,

0,

23ℓ

(

1− 1−t3ℓ

)

,

t ∈ [0, 3ℓ],

t ∈ [3ℓ, 1− 3ℓ],

t ∈ [1− 3ℓ, 1].

ν∗(t) =

− 49ℓ2,

− 29ℓ2,

0,

t ∈ [0, 3ℓ],

t ∈ [3ℓ, 1− 3ℓ],

t ∈ [1− 3ℓ, 1].

Figure 6-35 shows the dual variable approximations obtained by using the method

described in this paper for collocation at LGR points. It can be seen that the mapping

presented in Chapter 5 provides an accurate estimate for the dual variables of the

continuous optimal control problem. Figure 6-36 shows the base 10 logarithm of the

L∞-norm error for the costate and state constraint multiplier approximations for N

collocation points in each of the three mesh intervals. It can be seen that the error

on the estimate of the state inequality constraint multiplier and the costate reach an

accuracy of O(10−5) for N larger than two.

184

-22.5

-22.4

-22.3

-22.2

-22.1

-22

-21.9

-21.8

-21.7

-21.6

-21.5

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1t

p∗x (t)

px (t)

Cos

tate

Com

pone

nt

(A) px(t).

0

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9

6

7

-1

1

1

2

3

4

5

t

p∗v (t)

pv (t)

Cos

tate

Com

pone

nt

(B) pv (t).

0

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

-5

-10

-15

-20

-25

-30

-35

-40

-45t

ν∗(t)ν(t)

Sta

teC

onst

rain

tMul

tiplie

r

(C) ν(t).

Figure 6-35. Costate estimate for Example 4 obtained using collocation at LGR points.

185

0

-6

-5

-4

-3

-2

-1

1

2

2 3 4 5 6 7 8 9 10

N

log10

Abs

olut

eE

rror

ν(t)

px(t)

pv (t)

Figure 6-36. Costate errors for Example 4 obtained using collocation at LGR points.

186

CHAPTER 7CONCLUSIONS

Solving an optimal control problem is not easy. For most engineering applications,

it is impossible to derive an analytic solution to an optimal control problem using the

first-order optimality conditions derived from the calculus of variations. Thus, numerical

methods must be used to approximate the solution to the continuous-time problem.

Applying the first-order optimality conditions of the continuous-time optimal control

problem result in a Hamiltonian boundary-value problem which must be solved.

Numerical methods that aproximate a solution to the Hamiltonian boundary-value

problem stemming from the first-order optimality conditions of the optimal control

problem are called indirect methods. Numerical methods that employ the indirect

method tend to result in a highly accurate solution because the first-order optimality

conditions of the optimal control problem are satisfied. Convergence using indirect

methods, however, can be very hard to achieve due to the unstable nature of the

Hamiltonian boundary-value problem. Thus, intuitive initial guesses are required to

achieve convergence using indirect methods.

Numerical methods for solving optimal control problems that do not formulate the

first-order optimality conditions of the continous-time problem are called direct methods.

Direct methods convert the infinite-dimensional continuous control problem into a

finite-dimensional discrete nonlinear programming problem (NLP). The resulting NLP

can then be solved by well-developed NLP algorithms. Direct methods are attractive

because the first-order optimality conditions need not be derived. Furthermore, because

the Hamiltonian boundary-value problem is not formulated, convergence using direct

methods is usually easier to obtain. In this research a direct orthogonal collocation

method using collocation at the Legendre-Gauss and Legendre-Gauss-Radau

points was analyzed. In particular, an estimate of the continuous-time costate of the

187

continuous-time optimal control problem was derived from the KKT multipliers of the

nonlinear programming problem of the discrete problem.

Costate estimation is an important step in the numerical solution of optimal control

problems. Mapping the dual variables of the numerical solution to the costate of the

continuous-time problem not only allows for a verification of the dual solution, but

also allows the first-order optimality conditions of the nonlinear programming problem

(NLP) to take a form that is equivalent to the first-order optimality conditions of the

continuous-time problem. Thus, having an accurate costate estimate shows that the

KKT conditions satisfied by the NLP are a discrete form of the first-order optimality

conditions of the continuous problem given by the calculus of variations, and will

converge to an optimal solution of the continuous-time problem if discretized correctly.

Costate estimates for direct methods using orthogonal collocation at the Legendre-Gauss

and Legendre-Gauss-Radau points have previously been derived for optimal control

problems with no active state inequality path constraints and when the dynamic

constraints are formulated in their differential form. In this research a gap of costate

estimation theory was closed by deriving a mapping for the costate estimate for the case

when the dynamic constraints are expressed in integral form and in the presence of

state inequality path constraints.

In the first part of this research a costate estimate was developed for problem stated

with integral constraints. While the differential and integral forms of the LG and LGR

methods are mathematically equivalent with regard to the primal variables (that is, the

state and control), the two formulations produce completely different dual variables. In

particular, the relationship between the Lagrange multipliers of the collocation conditions

of the dynamic constraints and the costate of the optimal control problem has been

well documented. On the other hand, the corresponding relationship between the

Lagrange multipliers associated with the integral forms of LG and LGR collocation and

the costate of the optimal control problem has not been established. When employing

188

the integral forms of LG and LGR collocation, however, it may be of interest to either

verify optimality or perform sensitivity analysis in a manner consistent with that which

would be performed when using variational methods. In such cases it is useful to

obtain a costate estimate when using the integral forms of the LG and LGR methods.

Thus, in this research, a costate estimate for collocation at Legendre-Gauss and

Legendre-Gauss-Radau points was derived for the case when the dynamic constraints

of the optimal control problem are formulated in integral form. It was demonstrated

that the costate mapping derived for collocation at the LG and LGR points leads to a

set of transformed optimality conditions of the NLP which were shown to be a discrete

representation of the necessary conditions for optimality of the continuous-time problem.

Finally, a relationship between the integral and the differential forms of the costate

estimate was given and it was shown that the two sets of optimality conditions are

equivalent.

The second part of this research focused on problems with active state inequality

path constraints. Although previous research has successfully derived a high-accuracy

estimate of the costate from the KKT multipliers of the NLP for the case of a problem

with no active state inequality path constraints, Ref. [1] subsequently showed that in the

case when the costate is discontinuous (as is the case in the presence of active state

inequality path constraints), this costate estimate leads to a set of first-order optimality

conditions of the NLP that are not equivalent to the discrete form of the variational

optimality conditions. This lack of equivalence leads to an inaccurate approximation

of the costate. Therefore, in this research costate estimates for collocation at LG and

LGR points were derived for problems with active state inequality path constraints.

The derived costate estimate was shown to lead to a transformed adjoint system of

the NLP which is a discrete approximation of the necessary conditions for optimality

of the continuous-time optimal control problem. This equivalence was not existent with

prior costate estimates using LG and LGR collocation. The costate estimates derived

189

in this dissertation were implemented in four problems to assess their accuracy. It was

shown that each discrete costate estimate led to an accurate approximation of the

continuous-time costate.

190

REFERENCES

[1] Darby, C. L., Garg, D., and Rao, A. V., “Costate Estimation Using Multiple-IntervalPseudospectral Methods,” Journal of Spacecraft and Rockets, Vol. 48, No. 5,September–October 2011, pp. 856–866.

[2] Betts, J., Practical Methods for Optimal Control and Estimation Using NonlinearProgramming, SIAM Press, Philadelphia, 2nd ed., 2009.

[3] Unger, J., Kroner, A., and Marquardt, W., “Structural analysis ofdifferential-algebraic equation systems - theory and applications,” Computersand Chemical Engineering, Vol. 19, No. 8, 1995, pp. 867 – 882.

[4] Feehery, W. F. and Barton, P. I., “Dynamic optimization with state variable pathconstraints,” Computers and Chemical Engineering, Vol. 22, No. 9, 1998, pp. 1241– 1256.

[5] Maurer, H. and Pesch, H. J., “Direct optimization methods for solving a complexstate-constrained optimal control problem in microeconomics,” Applied Mathematicsand Computation, Vol. 204, No. 2, 2008, pp. 568 – 579.

[6] Boskens, Christof, M. H., “SQP-methods for solving optimal control problems withcontrol and state constraints: adjoint variables, sensitivity analysis and real-timecontrol,” J. Comput. Appl. Math., Vol. 120, No. 1-2, Aug. 2000, pp. 85–108.

[7] Bryson, A. E. and Ho, Y.-C., Applied Optimal Control , Hemisphere Publishing, NewYork, 1975.

[8] Speyer, J. L., “Necessary Conditions for Optimality For Paths Lying on a Corner,”Management Science, Vol. 19, No. 11, July 1973.

[9] Kirk, D. E., Optimal Control Theory: An Introduction, Dover Publications, Mineola,New York, 2004.

[10] Bryson, A. E., Denham, W. F., and Dreyfus, S. E., “Optimal Programming Problemswith Inequality Constraints I: Necessary Conditions for Extremal Solutions,” AIAAJournal , Vol. 1, No. 11, 1962, pp. 2544–2550.

[11] Dreyfus, S. E., Variational Problems with State Variable Inequality Constraints,Ph.D. thesis, Harvard University, Cambridge, Massachussets, 1962.

[12] Jacobson, D. H., Lele, M. M., and Speyer, J. L., “New Necessary Conditions ofOptimality for Control Problems with State-Variable Inequality Constraints,” Tech.rep., Division of Engineering and Applied Physics - Harvard University, Cambridge,Massachussets, 1969.

[13] Speyer, J. L. and Bryson, A. E., “Optimal Programming Problems with a BoundedState Space,” AIAA Journal , Vol. 6, No. 8, August 1968, pp. 1488–1491.

191

[14] Denham, W. F. and Bryson, A. E., “Optimal Programming Problems with InequalityConstraints II: Solution by Steepest-Ascent,” AIAA Journal , Vol. 2, No. 1, January1964, pp. 25–34.

[15] Hartl, R. F., Sethi, S. P., and Vickson, R. G., “A Survey of the Maximum Principlesfor Optimal Control Problems with State Constraints,” SIAM Review , Vol. 37, No. 2,June 1995, pp. 181–218.

[16] Gollan, B., “On Optimal Control Problems with State Constraints,” Journal ofOptimization Theory and Applications, Vol. 32, 1980, pp. 75–80.

[17] Russak, I. B., “On Problems with Bounded State Variable,” Journal of OptimizationTheory and Applications, Vol. 5, 1970, pp. 114–157.

[18] Russak, I. B., “On General Problems with Bounded State Variable,” Journal ofOptimization Theory and Applications, Vol. 6, 1970, pp. 424–452.

[19] Vinter, R., Optimal Control (Systems & Control: Foundations and Applications),Birkhauser, Boston, 2000.

[20] Russel, R. and Shampine, L., “A Collocation Method for Boundary Value Problems,”Numerical Mathematics, Vol. 19, 1972, pp. 1–28.

[21] Rao, A. V. and Mease, K. D., “Dichotomic Basis Approach to solvingHyper-Sensitive Optimal Control Problems,” Automatica, Vol. 35, No. 4, April1999, pp. 633–642.

[22] Gill, P. E., Murray, W., and Saunders, M. A., “SNOPT: An SQP Algorithm forLarge-Scale Constrained Optimization,” SIAM Review , Vol. 47, No. 1, January2005, pp. 99–131.

[23] Biegler, L. T. and Zavala, V. M., “Large-Scale Nonlinear Programming Using IPOPT:An Integrating Framework for Enterprise-Wide Optimization,” Computers andChemical Engineering, Vol. 33, No. 3, March 2008, pp. 575–582.

[24] Wachter, A., An Interior Point Algorithm for Large-Scale Nonlinear Optimizationwith Applications in Process Engineering, Ph.D. thesis, Department of ChemicalEngineering, Carnegie-Mellon University, Pittsburgh, PA, 2002.

[25] Wachter, A. and Biegler, L. T., “On the Implementation of an Interior-Point FilterLine-Search Algorithm for Large-Scale Nonlinear Programming,” MathematicalProgramming, Vol. 106, No. 1, 2006, pp. 25–57.

[26] Fahroo, F. and Ross, I. M., “Costate Estimation by a Legendre PseudospectralMethod,” Journal of Guidance, Control, and Dynamics, Vol. 24, No. 2, March–April2001, pp. 270–277.

[27] Benson, D. A., Huntington, G. T., Thorvaldsen, T. P., and Rao, A. V., “DirectTrajectory Optimization and Costate Estimation via an Orthogonal Collocation

192

Method,” Journal of Guidance, Control, and Dynamics, Vol. 29, No. 6,November-December 2006, pp. 1435–1440.

[28] Bock, H. G. and Plitt, K. J., “A Multiple Shooting Algorithm for Direct Solution ofOptimal Control Problems,” IFAC 9th World Congress, Budapest, Hungary, 1984.

[29] Williamson, W. E., “Use of Polynomial Approximations to Calculate SuboptimalControls,” AIAA Journal , Vol. 9, No. 11, 1971, pp. 2271–2273.

[30] Kraft, D., On Converting Optimal Control Problems into Nonlinear ProgrammingCodes, Vol. F15 of NATO ASI Series, in Computational Mathematical Program-ming, Springer-Verlag, Berlin, 1985, pp. 261–280.

[31] Von-Stryk, O. and Bulirsch, R., “Direct and Indirect Methods for TrajectoryOptimization,” Annals of Operations Research, Vol. 37, 1992, pp. 357–373.

[32] Hargraves, C. R. and Paris, S. W., “Direct Trajectory Optimization Using NonlinearProgramming Techniques,” Journal of Guidance, Control, and Dynamics, Vol. 10,No. 4, July–August 1987, pp. 338–342.

[33] Enright, P. J. and Conway, B. A., “Discrete Approximations to Optimal TrajectoriesUsing Direct Transcription and Nonlinear Programming,” Journal of Guidance,Control, and Dynamics, Vol. 19, No. 4, July–August 1996, pp. 994–1002.

[34] Benson, D. A., A Gauss Pseudospectral Transcription for Optimal Control , Ph.D.thesis, Department of Aeronautics and Astronautics, Massachusetts Institute ofTechnology, Cambridge, Massachusetts, 2004.

[35] Elnagar, G., Kazemi, M., and Razzaghi, M., “The Pseudospectral Legendre Methodfor Discretizing Optimal Control Problems,” IEEE Transactions on AutomaticControl , Vol. 40, No. 10, 1995, pp. 1793–1796.

[36] Garg, D., Patterson, M. A., Darby, C. L., Francolin, C., Huntington, G. T., Hager,W. W., and Rao, A. V., “Direct Trajectory Optimization and Costate Estimationof Finite-Horizon and Infinite-Horizon Optimal Control Problems using a RadauPseudospectral Method,” Computational Optimization and Applications, Vol. 49,No. 2, June 2011, pp. 335–358.

[37] Biegler, L. T., Ghattas, O., Heinkenschloss, M., and van Bloemen Waanders,B., editors, Large-Scale PDE Constrained Optimization, Lecture Notes inComputational Science and Engineering, Vol. 30, Springer-Verlag, Berlin, 2003.

[38] Betts, J. T., Campbell, S. L., and Engelsone, A., “Direct Transcription Solution ofOptimal Control Problems with Higher Order State Constraints: Theory vs Practice,”Optimization and Engineering, Vol. 8, No. 1, 2007, pp. 1–19.

[39] Betts, J. T. and Huffman, W. P., “Sparse Optimal Control Software – SOCS,” Tech.Rep. MEA-LR-085, Boeing Information and Support Services, Seattle, Washington,July 1997.

193

[40] Ross, I. M. and Fahroo, F., User’s Manual for DIDO 2001 α: A MATLAB Applicationfor Solving Optimal Control Problems, Department of Aeronautics and Astronautics,Naval Postgraduate School, Technical Report AAS-01-03, Monterey, California,2001.

[41] Von-Stryk, O., User’s Guide for DIRCOL Version 2.1: A Direct Collocation Methodfor the Numerical Solution of Optimal Control Problems, Technische UniversitatDarmstadt, Darmstadt, Germany, 1999.

[42] Rao, A. V., Benson, D. A., Darby, C. L., Francolin, C., Patterson, M. A., Sanders,I., and Huntington, G. T., “Algorithm 902: GPOPS, A Matlab Software for SolvingMultiple-Phase Optimal Control Problems Using the Gauss PseudospectralMethod,” ACM Transactions on Mathematical Software, Vol. 37, No. 2, April–June2010, pp. 22:1–22:39.

[43] J.T.Betts, “Survey of Numerical Methods for Trajectory Optimization,” Journal ofGuidance, Control, and Dynamics, Vol. 21, No. 2, March–April 1998, pp. 193–207.

[44] Hager, W. W., “Runge-Kutta Methods in Optimal Control and the TransformedAdjoint System,” Numerische Mathematik , Vol. 87, 2000, pp. 247–282.

[45] Dontchev, A. L., Hager, W. W., and Veliov, V. M., “Second-Order Runge-KuttaApproximations In Constrained Optimal Control,” SIAM Journal on NumericalAnalysis, Vol. 38, 2000, pp. 202–226.

[46] Betts, J. T., “Sparse Jacobian Updates in the Collocation Method for OptimalControl Problems,” Journal of Guidance, Control, and Dynamics, Vol. 13, No. 3,May–June 1990, pp. 409–415.

[47] Dontchev, A. L., Hager, W. W., and Malanowski, K., “Error Bounds for the EulerApproximation and Control Constrained Optimal Control Problem,” NumericalFunctional Analysis and Applications, Vol. 21, 2000, pp. 653–682.

[48] Dontchev, A., Hager, W., and Veliov, V., “Uniform Convergence and Meshindependence of Newton’s method for Discretized Variational Problems,” SIAMJournal on Control and Optimization, Vol. 39, 2000, pp. 961–980.

[49] Kameswaran, S. and Biegler, L. T., “Convergence Rates for Direct Transcriptionof Optimal Control Problems Using Collocation at Radau Points,” ComputationalOptimization and Applications, Vol. 41, No. 1, 2008, pp. 81–126.

[50] Darby, C. L., Hager, W. W., and Rao, A. V., “An hp-Adaptive Pseudospectral Methodfor Solving Optimal Control Problems,” Optimal Control Applications and Methods,Vol. 32, No. 4, July–August 2011, pp. 476–502.

[51] Darby, C. L., Hager, W. W., and Rao, A. V., “Direct Trajectory Optimization Using aVariable Low-Order Adaptive Pseudospectral Method,” Journal of Spacecraft andRockets, Vol. 48, No. 3, May–June 2011, pp. 433–445.

194

[52] Vlassenbroeck, J. and Dooren, R. V., “A Chebyshev Technique for SolvingNonlinear Optimal Control Problems,” IEEE Transactions on Automatic Control ,Vol. 33, No. 4, 1988, pp. 333–340.

[53] Vlassenbroeck, J., “A Chebyshev Polynomial Method for Optimal Control with StateConstraints,” Automatica, Vol. 24, No. 4, 1988, pp. 499–506.

[54] Elnagar, G. and Razzaghi, M., “A Collocation-Type Method for Linear QuadraticOptimal Control Problems,” Optimal Control Applications and Methods, Vol. 18,No. 3, 1998, pp. 227–235.

[55] Elnagar, G. and Kazemi, M., “Pseudospectral Chebyshev Optimal Control ofConstrained Nonlinear Dynamical Systems,” Computational Optimization andApplications, Vol. 11, No. 2, 1998, pp. 195–217.

[56] Elnagar, G. N. and Razzaghi, M. A., “Pseudospectral Legendre-Based OptimalComputation of Nonlinear Constrained Variational Problems,” Journal of Computa-tional and Applied Mathematics, Vol. 88, No. 2, March 1998, pp. 363–375.

[57] Williams, P., “Jacobi Pseudospectral Method for Solving Optimal Control Problems,”Journal of Guidance, Control, and Dynamics, Vol. 27, No. 2, March–April 2004,pp. 293–297.

[58] Williams, P., “Application of Pseudospectral methods for Receding Horizon Control,”Journal of Guidance, Control, and Dynamics, Vol. 27, No. 2, 2004, pp. 310–314.

[59] Williams, P., “Hermite-Legendre-Gauss-Lobatto Direct Transcription Methods inTrajectory Optimization,” Americal Astronautical Society, Spaceflight MechanicsMeeting, August 2005.

[60] Huntington, G. T., Advancement and Analysis of a Gauss Pseudospectral Tran-scription for Optimal Control , Ph.D. thesis, Massachusetts Institute of Technology,Cambridge, Massachusetts, 2007.

[61] Huntington, G. T., Benson, D. A., and Rao, A. V., “Optimal Configuration ofTetrahedral Spacecraft Formations,” The Journal of the Astronautical Sciences,Vol. 55, No. 2, April-June 2007, pp. 141–169.

[62] Huntington, G. T. and Rao, A. V., “Optimal Reconfiguration of SpacecraftFormations Using the Gauss Pseudospectral Method,” Journal of Guidance,Control, and Dynamics, Vol. 31, No. 3, May-June 2008, pp. 689–698.

[63] Fahroo, F. and Ross, I. M., “Direct Trajectory Optimization by a ChebyshevPseudospectral Method,” Journal of Guidance, Control, and Dynamics, Vol. 25,No. 1, 2002, pp. 160–166.

[64] Garg, D., Patterson, M. A., Hager, W. W., Rao, A. V., Benson, D. A., and Huntington,G. T., “A Unified Framework for the Numerical Solution of Optimal Control Problems

195

Using Pseudospectral Methods,” Automatica, Vol. 46, No. 11, December 2010,pp. 1843–1851.

[65] Garg, D., Hager, W. W., and Rao, A. V., “Pseudospectral Methods for SolvingInfinite-Horizon Optimal Control Problems,” Automatica, Vol. 47, No. 4, April 2011,pp. 829–837.

[66] Dontchev, A. L. and Hager, W. W., “The Euler Approximation in State ConstrainedOptimal Control,” Mathematics of Computation, Vol. 70, 2001, pp. 173–203.

[67] Maurer, H. and Gillessen, W., “Application of multiple shooting to the numericalsolution of optimal control problems with bounded state variables,” Computing,Vol. 15, 1975, pp. 105–126.

[68] Garg, D., Advances In Global Pseudospectral Methods For Optimal Control , Ph.D.thesis, University of Florida, Gainesville, Florida, 2011.

[69] Patterson, M. A. and Rao, A. V., “GPOPS-II: A MATLAB Software for SolvingMultiple-Phase Optimal Control Problems Using hp-Adaptive Gaussian QuadratureCollocation Methods and Sparse Nonlinear Programming,” ACM Transactions onMathematical Software, February 2013, Submitted.

[70] Patterson, M. A., Hager, W. W., and Rao, A. V., “A ph-Collocation Scheme forOptimal Control,” Automatica, January 2013, Submitted.

[71] Vlases, W. G., Paris, S. W., Lajoie, R. M., Martens, M. J., and Hargraves, C. R.,“Optimal Trajectories by Implicit Simulation,” Tech. Rep. WRDC-TR-90-3056,Boeing Aerospace and Electronics, Wright-Patterson Air Force Base, Ohio, 1990.

[72] Falugi, P., Kerrigan, E., and van Wyk, E., Imperial College London Optimal ControlSoftware User Guide (ICLOCS), Department of Electrical Engineering, ImperialCollege London, London, UK, May 2010.

[73] Houska, B., Ferreau, H. J., and Diehl, M., “ACADO Toolkit–An Open-SourceFramework for Automatic Control and Dynamic Optimization,” Optimal ControlApplications and Methods, Vol. 32, No. 3, 2011, pp. 298–312.

[74] Darby, C. L., hp-Pseudospectral Method For Solving Continuous-Time NonlinearOptimal Control Problems, Ph.D. thesis, University of Florida, Gainesville, Florida,2011.

196

BIOGRAPHICAL SKETCH

Camila Francolin was born in Rio de Janeiro, Brazil. She received her dual Bachelor

of Science degrees in aerospace and mechanical engineering in December 2007

from the University of Florida. She then received her Master of Science degree in

aerospace engineering in May 2010, and her Doctor of Philosophy in aerospace

engineering in August 2013 from the University of Florida. Her research interests include

numerical approximations to differential equations, optimal control theory, and numerical

approximations to the solution of optimal control problems.

197

COSTATE ESTIMATION FOR OPTIMAL CONTROL PROBLEMS...

Documents

Transcript of COSTATE ESTIMATION FOR OPTIMAL CONTROL PROBLEMS...