Linear Optimal control - uniroma1.itiacoviel/Materiale/OptimalControl-Lecture1.pdfD. Liberzon,...
Transcript of Linear Optimal control - uniroma1.itiacoviel/Materiale/OptimalControl-Lecture1.pdfD. Liberzon,...
Optimal Control
Lecture
Prof. Daniela Iacoviello
Department of Computer, Control, and Management Engineering Antonio Ruberti
Sapienza University of Rome
13/10/2015 Controllo nei sistemi biologici
Lecture 1
Pagina 2
Prof. Daniela Iacoviello
Department of computer, control and management
Engineering Antonio Ruberti
Office: A219 Via Ariosto 25
http://www.dis.uniroma1.it/~iacoviel
Prof.Daniela Iacoviello- Optimal Control
Grading
Project + oral exam
The exam must be concluded before the second
part of Identification that will be held by Prof. Battilotti
Grading
Project+ oral exam
Example of project:
- Read a paper on an optimal control problem
- Study: background, motivations, model, optimal control, solution,
results
- Simulations
You must give me, before the date of the exam:
- A .doc document
- A power point presentation
- Matlab simulation files
The exam must be concluded before the second
part of Identification that will be held by Prof. Battilotti
Some projects studied in 2014-15
Application of Optimal Control to malaria: strategies and simulations
Performance compare between LQR and PID control of DC Motor
Optimal Low-Thrust LEO (low-Earth orbit) to GEO (geosynchronous-Earth orbit)
Circular Orbit Transfer
Controllo ottimo di una turbina eolica a velocità variabile attraverso il metodo
dell'inseguimento ottimo a regime permanente
Optimal Control in Dielectrophoresis
On the Design of P.I.D. Controllers Using Optimal Linear Regulator Theory
Rocket Railroad Car
………
THESE SLIDES ARE NOT SUFFICIENT
FOR THE EXAM: YOU MUST STUDY ON THE BOOKS
Prof.Daniela Iacoviello- Optimal Control
Part of the slides has been taken from the References indicated below
B.D.O.Anderson, J.B.Moore, Optimal control, Prentice Hall, 1989 C.Bruni, G. Di Pillo, Metodi Variazionali per il controllo ottimo, Masson , 1993 L. Evans, An introduction to mathematical optimal control theory, 1983 H.Kwakernaak , R.Sivan, Linear Optimal Control Systems, Wiley Interscience, 1972 D. E. Kirk, "Optimal Control Theory: An Introduction, New York, NY: Dover, 2004 D. Liberzon, "Calculus of Variations and Optimal Control Theory: A Concise Introduction", Princeton University Press, 2011 How, Jonathan, Principles of optimal control, Spring 2008. (MIT OpenCourseWare: Massachusetts Institute of Technology). License: Creative Commons BY-NC-SA.
References
Prof.Daniela Iacoviello- Optimal Control
Course outline
• Introduction to optimal control
• Nonlinear optimization
• Dynamic programming
• Calculus of variations
• Calculus of variations and optimal control
Prof.Daniela Iacoviello- Optimal Control
Course outline
• Introduction to optimal control
• Nonlinear optimization
• Dynamic programming
• Calculus of variations
• Calculus of variations and optimal control
Prof.Daniela Iacoviello- Optimal Control
First course on linear systems (free evolution, transition matrix, gramian matrix,…)
Background
Prof.Daniela Iacoviello- Optimal Control
Notations
nRtx )( State variable
pRtu )( Control variable
RRRRf pn ::
Function (function with second derivative continuous a.e.)
2C
Prof.Daniela Iacoviello- Optimal Control
Introduction
Optimal control is one particular branch of modern control that sets out to provide analytical designs of a special appealing type.
The system that is the end result of an optimal design is supposed to be the best possible system of a particular type
A cost index is introduced Prof.Daniela Iacoviello- Optimal Control
Introduction
Linear optimal control is a special sort of optimal control:
the plant that is controlled is assumed linear
the controller is constrained to be linear
Linear controllers are achieved by working with quadratic cost indices
Prof.Daniela Iacoviello- Optimal Control
Introduction
Advantages of linear optimal control
Linear optimal control may be applied to nonlinear systems
Nearly all linear optimal control problems have computational solutions
The computational procedures required for linear optimal design may often be carried over to nonlinear optimal problems
Prof.Daniela Iacoviello- Optimal Control
History
Prof.Daniela Iacoviello- Optimal Control
History
In 1696 Bernoulli posed the
Brachystochrone problem to his
contemporaries: “it seems to be the
First problem which explicitely dealt
with optimally controlling the path or or the behaviour of a dynamical
system”.
Prof.Daniela Iacoviello- Optimal Control
Motivations
Example 1 (Evans 1983)
Reproductive strategies in social insects
Let us consider the model describing how social insects
(for example bees) interact:
represents the number of workers at time t
represents the number of queens
represents the fraction of colony effort
devoted to increasing work force
lenght of the season
)(tw
)(tq
)(tu
TProf.Daniela Iacoviello- Optimal Control
0)0(
)()()()()(
ww
twtutsbtwtw
Evolution of the worker population
Death rate Known rate at which each worker contributes to the bee economy
0)0(
)()()(1)()(
twtstuctqtq
Evolution of the Population of queens
Constraint for the control: 1)(0 tuProf.Daniela Iacoviello- Optimal Control
The bees goal is to find the control that maximizes the number of queens at time T: The solution is a bang- bang control
)()( TqtuJ
Prof.Daniela Iacoviello- Optimal Control
Motivations Example 2 (Evans 1983…and everywhere!)
A moon lander
Aim: bring a spacecraft to a soft landing on the lunar surface, using the least amount of fuel
represents the height at time t
represents the velocity =
represents the mass of spacecraft
represents thrust at time t
We assume
)(th
)(tv
)(tm
Prof.Daniela Iacoviello- Optimal Control
)(th
)(tu
1)(0 tu
Considere the Newton’s law:
We want to minimize the amount of fuel
that is maximize the amount remaining once we have
landed
where
is the first time
Prof.Daniela Iacoviello- Optimal Control
ugmthm )(
0)(0)()()(
)()(
)(
)()(
tmthtkutm
tvth
tm
tugtv
)())(( muJ
0)(0)( vh
Analysis of linear control systems
Essential components of a control system
The plant
One or more sensors
The controller
Controller Plant
Sensor
Prof.Daniela Iacoviello- Optimal Control
Analysis of linear control systems
Feedback: the actual operation of the control system is compared to the desired operation and the input to the plant is adjusted on the basis of this comparison.
Feedback control systems are able to operate satisfactorily despite adverse conditions, such as disturbances and variations in plant properties
Prof.Daniela Iacoviello- Optimal Control
Course outline
• Introduction to optimal control
• Nonlinear optimization
• Dynamic programming
• Calculus of variations
• Calculus of variations and optimal control
Prof.Daniela Iacoviello- Optimal Control
Definitions
Consider a function
And
denotes the Eucledian norm
A point is a local minimum of over
If
Prof.Daniela Iacoviello- Optimal Control
RRf n ::
nRD
Dx *f
nRD
)()(
0
*
*
xfxf
xxsatisfyingDxallforthatsuch
Definitions
Consider a function
And
denotes the Eucledian norm
A point is a strict local minimum of over
If
Prof.Daniela Iacoviello- Optimal Control
RRf n ::
nRD
Dx * fnRD
**
*
),()(
0
xxxfxf
xxsatisfyingDxallforthatsuch
Definitions
Consider a function
And
denotes the Eucledian norm
A point is a global minimum of over
If
Prof.Daniela Iacoviello- Optimal Control
RRf n ::
nRD
Dx *f
nRD
)()( * xfxf
Dxallfor
Definitions
The notions of a local/strict/global maximum are defined
similarly
If a point is either a maximum or a minimum is called an
extremum
Prof.Daniela Iacoviello- Optimal Control
Unconstrained optimization - first order necessary conditions
All points sufficiently near in are in
Assume and its local minimum. Let an arbitrary vector.
Being in the unconstrained case:
Let’s consider:
0 is a minimum of g
Prof.Daniela Iacoviello- Optimal Control
x
nRd
DnR
1Cf *x
)(:)( * dxfg
0* toenoughcloseRDdx
*x
First order Taylor expansion of g around
Proof: assume
For these values of
Prof.Daniela Iacoviello- Optimal Control
0
0)(
lim),()0(')0()(0
ooggg
0)0(' g
0)0(' g
)0(')(
0
gofor
thatsoenoughsmall
)0(')0(')0()( gggg
Unconstrained optimization - first order necessary conditions
If we restrict to have the opposite sign to
contradiction.
d was arbitrary
First order necessary condition for
optimality
Prof.Daniela Iacoviello- Optimal Control
)0(')0(')0()( gggg
)0('g
0)0()( gg
fofgradienttheisfffwhereddxfgT
xx n
1:)()(' *
0)()0(' * dxfg
0)( * xf
Unconstrained optimization - first order necessary conditions
A point satisfying this condition is a stationary point
Prof.Daniela Iacoviello- Optimal Control
*x
Unconstrained optimization - first order necessary conditions
Unconstrained optimization- second order conditions
Assume and its local minimum. Let an
arbitrary vector.
Second order Taylor expansion of g around
Since
Proof: suppose
Prof.Daniela Iacoviello- Optimal Control
nRd2Cf *x
0
0)(
lim),()0(''2
1)0(')0()(
2
2
0
2
oogggg
0)0(' g
0)0('' g
0)0('' g
22 )0(''2
1)(
0
gofor
thatsoenoughsmall
For these values of
contradiction
By differentiating both sides with respect to
Unconstrained optimization- second order conditions
Prof.Daniela Iacoviello- Optimal Control
0)0()( gg
n
i
ix ddxfgi
1
* )()('
dxfdddxfg
dddxfg
T
j
n
ji
ixx
j
n
ji
ixx
ji
ji
)()()0(''
)()(''
*2
1,
*
1,
*
nnn
n
xxxx
xxxx
ff
ff
f
1
111
2
Hessian matrix
Unconstrained optimization- second order conditions
Second order necessary condition for
optimality
Remark:
The second order condition distinguishes minima from
maxima:
At a local maximum the Hessian must be negative
semidefinite
At a local minimum the Hessian must be positive
semidefinite
Prof.Daniela Iacoviello- Optimal Control
0)( *2 xf
Unconstrained optimization- second order conditions
Let and
is a strict local minimum of f
Prof.Daniela Iacoviello- Optimal Control
0)( * xf2Cf 0)( *2 xf
*x
Definitions- Remarks
A vector is a feasible direction at if
If D in not the entire then D is the constraint set over
which f is being minimized
Prof.Daniela Iacoviello- Optimal Control
*xnRd
0* enoughsmallforDdx
nR
Global minimum
Weierstrass Theorem
Let f be a continuous function and D a compact set
there exist a global minimum of f over D
Prof.Daniela Iacoviello- Optimal Control
Let
Equality constraints
Inequality constraints
Regularity condition:
Lagrangian function
If the stationary point is called normal and we can assume .
Constrained optimization
Prof.Daniela Iacoviello- Optimal Control
1, CfRD n 1,:,0)( ChRRhxh p
1,:,0)( CgRRgxg q
a
a
a
x
a
qg
ofconstrainactivethearegwhere
qpx
ghrank
dimensionwith
,
*
)()()(,,, 00 xgxhxfxL TT
*x00
10
From now on and therefore the Lagrangian is
If there are only equality constraints the are called
Lagrange multipliers
The inequality multipliers are called Kuhn – Tucker
multipliers
Prof.Daniela Iacoviello- Optimal Control
Constrained optimization
i
10
)()()(,, xgxhxfxL TT
First order necessary conditions for constrained optimality:
Let and
The necessary conditions for to be a constrained local
minimum are
If the functions f and g are convex and the functions h are
linear these conditions are necessary and sufficient!!!
Constrained optimization
Prof.Daniela Iacoviello- Optimal Control
i
ixg
x
L
i
ii
T
T
x
0
,0)(
0
*
*
Dx * 1,, Cghf
*x
Constrained optimization
Second order sufficient conditions for constrained optimality:
Let and and assume the conditions
is a strict constrained local minimum if
Prof.Daniela Iacoviello- Optimal Control
ixgx
Liii
T
T
x
0,0)(0 *
*
Dx * 2,, Cghf
*x
piddx
xdhthatsuchdd
x
Ld
x
i
x
T ,...,1,0)(
0**
2
2
Function spaces
Functional
Vector space V,
is a local minimum of J over A if there exists an
Prof.Daniela Iacoviello- Optimal Control
RVJ :
VA
Az *
)()(
0
*
*
zJzJ
zzsatisfyingAzallforthatsuch
Consider function in V of the form
The first variation of J at z is the linear function
such that
First order necessary condition for optimality:
For all admissible perturbation we must have:
Prof.Daniela Iacoviello- Optimal Control
RVz ,,
RVJz
:
and
)()()()( oJzJzJ z
0)(* z
J
Function spaces
A quadratic form is the second variation
of J at z if
we have:
second order necessary condition for optimality:
If is a local minimum of J over
for all admissible perturbation we must have:
Prof.Daniela Iacoviello- Optimal Control
RVJz
:2
and
)()()()()( 222 oJJzJzJ zz
0)(*
2 z
J
Az * VA
Function spaces
The Weierstrass Theorem is still valid
If J is a convex functional and is a convex set
A local minimum is automatically a global one and the first
order condition are necessary and sufficient condition for
a minimum
Prof.Daniela Iacoviello- Optimal Control
VA
Function spaces