Presentazione di PowerPoint - uniroma1.itiacoviel/Materiale/Lecture_5_2019_SITO.pdfPart of the...
Transcript of Presentazione di PowerPoint - uniroma1.itiacoviel/Materiale/Lecture_5_2019_SITO.pdfPart of the...
Optimal Control
Prof. Daniela Iacoviello
Lecture 5
24/10/2019 Pagina 2
THESE SLIDES ARE NOT SUFFICIENT
FOR THE EXAM:YOU MUST STUDY ON THE BOOKS
Part of the slides has been taken from the References indicated below
Pagina 224/10/2019Prof. D.Iacoviello - Optimal Control
24/10/2019 Pagina 3
References
B.D.O.Anderson, J.B.Moore, Optimal control, Prentice Hall, 1989
E.R.Pinch, Optimal control and the calculus of variations, Oxford science publications,
1993
C.Bruni, G. Di Pillo, Metodi Variazionali per il controllo ottimo, Masson , 1993
A.Calogero, Notes on optimal control theory, 2017
L. Evans, An introduction to mathematical optimal control theory, 1983
H.Kwakernaak , R.Sivan, Linear Optimal Control Systems, Wiley Interscience, 1972
D. E. Kirk, "Optimal Control Theory: An Introduction, New York, NY: Dover, 2004
D. Liberzon, "Calculus of Variations and Optimal Control Theory: A Concise
Introduction", Princeton University Press, 2011
How, Jonathan, Principles of optimal control, Spring 2008. (MIT OpenCourseWare:
Massachusetts Institute of Technology). License: Creative Commons BY-NC-SA.
……………………………………..
Pagina 324/10/2019Prof.Daniela Iacoviello- Optimal Control
24/10/2019 Pagina 4
Course outline
• Introduction to optimal control
• Calculus of variations
• Calculus of variations and optimal control
• The maximum principle
• The Hamilton Jacobi Bellman equation
• The linear quadratic regulator
• The minimum time problem
• The linear quadratic gaussian problem
Pagina 424/10/2019Prof.Daniela Iacoviello- Optimal Control
• Hamilton-Jacobi- Bellman equation
Prof. D.Iacoviello - Optimal Control 524/10/2019
William Rowan Hamilton
(4 August 1805 – 2 September 1865)
He was an Irish physicist, astronomer, and mathematician, who
made important contributions to classical mechanics,
optics and algebra.
His studies of mechanical and optical systems led
him to discover new mathematical concepts and techniques.
His greatest contribution is perhaps the reformulation of
Newtonian mechanics, now called Hamiltonian mechanics.
This work has proven central to the modern study of classical
field theories such as electromagnetism and to the development
of quantum mechanics. In mathematics, he is perhaps best
known as the inventor of quaternions.
.
Prof. D.Iacoviello - Optimal Control 624/10/2019
Jacobi (Posdam 1804- Berlin 1851)
He was still only 12 years old yet he had reached the necessary
standard to enter University in 1817
Around 1825 Jacobi changed from the Jewish faith to
become a Christian which now made university
teaching possible for him.
By the academic year 1825-26 he was teaching at the University
of Berlin.
Jacobi carried out important research in partial differential
equations of the first order and applied them to the differential
equations of dynamics.
He also worked on determinants and studied the functional
determinant now called the Jacobian.
Prof. D.Iacoviello - Optimal Control 724/10/2019
Richard Bellman (1920- 1984, USA)
He was interested in dynamic programming, with
applications in biology and medicine.
Prof. D.Iacoviello - Optimal Control 824/10/2019
Principle of optimality
Prof. D.Iacoviello - Optimal Control 924/10/2019
Principle of optimality
Prof. D.Iacoviello - Optimal Control 1024/10/2019
Principle of optimality
Prof. D.Iacoviello - Optimal Control 1124/10/2019
Principle of optimality
Prof. D.Iacoviello - Optimal Control 1224/10/2019
Principle of optimality
Prof. D.Iacoviello - Optimal Control 1324/10/2019
The Hamilton-Jacobi equation
It is an approach in some sense alternative to the Minimum
Principle of Pontryagin, combined with the Euler Lagrange
equation.
Actually the Hamilton – Jacobi equation has so far rarely
proved useful except for linear regulator problems, to which
it seems particularly well suited.
Prof. D.Iacoviello - Optimal Control 1424/10/2019
The Hamilton-Jacobi equation
The Hamilton-Jacobi (H-J) equation is satisfied by the optimal
performance index under suitable differentiability and
continuity assumptions.
If a solution to the H-J equation has certain differentiability
properties, then this solution is the desired performance index.
Prof. D.Iacoviello - Optimal Control 1524/10/2019
The Hamilton-Jacobi equation
- Such a solution need not exist
- Not every optimal performance index satisfies the Hamilton –
Jacobi equation
The HJ equation represents only a sufficient condition on the
optimal performance index
Prof. D.Iacoviello - Optimal Control 1624/10/2019
Principle of optimality
Minimizing over [t, tf] is equivalent to minimizing over [t, t1]
and [t1, tf]
t t1 tf
x(t)
xx1(t1)
x2(t1)
x3(t1)
x1(tf)
x2(tf)
x3(tf)
Prof. D.Iacoviello - Optimal Control 1724/10/2019
The Hamilton-Jacobi equation
Principle of optimality:
OR:
( )
( )
( )
++
= )((),(),(min),(),(min),(
1
1
1
1 ,,
*f
t
tttu
t
tttu
txGduxLduxLttxJ
f
f
( )
( ) ( )
+= 11
*
,
* ),(),(),(min),(
1
1
ttxJduxLttxJ
t
tttu
Prof. D.Iacoviello - Optimal Control 1824/10/2019
The Hamilton-Jacobi equation
Principle of optimality:
The optimal cost for trajectories commencing at t and
finishing at tf is incurred by minimizing the sum of the
cost in transiting to x1(t1) and the optimal cost from there
onwards that is:
and
( )
( ) ( )
+= 11
*
,
* ),(),(),(min),(
1
1
ttxJduxLttxJ
t
tttu
( )1
),(),(
t
t
duxL ( )ttxJ ),(*
Prof. D.Iacoviello - Optimal Control 1924/10/2019
Principle of optimality
A control policy optimal over the interval [t, tf] is
optimal over all subintervals [t1, tf]
t t1 tf
x(t)
xx1(t1)
x2(t1)
x3(t1)
x1(tf)
x2(tf)
x3(tf)
Prof. D.Iacoviello - Optimal Control 2024/10/2019
The Hamilton-Jacobi equation has so far rarely proved
useful except for linear regulator problems
The Hamilton-Jacobi equation
Prof. D.Iacoviello - Optimal Control 2124/10/2019
The Hamilton-Jacobi equationAssume the Hamiltonian regular: there exists a unique minimum
with respect to u(t).
The Hamilton-Jacobi equation:
Prof. D.Iacoviello - Optimal Control 2224/10/2019
0),(
,,),(
),(),(),(
=
+
t
x
txVt
x
txVtxutxH
t
txVTT
0,,),(
),(),(),(
,),(
,,),(
),(),(),(
=
+
+
ttx
txVtxutxf
x
txV
tx
txVt
x
txVtxutxL
t
txV
TT
TT
The Hamilton-Jacobi equation
Problem:
Consider the system:
Find the optimal control
that minimizes the cost:
( ) giventxtuxfx )(,,, 0=
fttttu ,),( 0*
( ) ( ) ( ))(),(),(),(),(
0
f
t
t
txGduxLtutxJ
f
+=
Prof. D.Iacoviello - Optimal Control 2324/10/2019
Define the Hamiltonian:
Assume:
➢ f, L and G are continuously differentiable in all
arguments
➢ The Hamiltonian regular: there exists a unique minimum
with respect to u(t),
Prof. D.Iacoviello - Optimal Control 24
( ) ( ) ( )tuxftuxLtuxH T ,,,,,,, +=
24/10/2019
Derivation of the Hamilton-Jacobi equation
Let’s consider the system:
with the cost index:
Notation:
Initial instant
initial state
freeorfixedttxfixedxuxfx ff ),(,),,( 0=
+= ft
tftxGdtttutxLJ
0
))(()),(),((minmin
+=
f
f
t
tf
ttttu
txGdtttutxLxtJ0
0
))(()),(),((min),(
,)(
00
Prof. D.Iacoviello - Optimal Control 2524/10/2019
++=
++=
+=
f
f
f
f
f
f
t
tf
t
tttt
tu
t
tf
t
tttt
tu
t
tf
ttttu
txGdtttutxLdtttutxL
txGdtttutxLdtttutxL
txGdtttutxLxtJ
1
1
0
0
1
1
0
0
0
0
))(()),(),(()),(),((min
))(()),(),(()),(),((min
))(()),(),((min),(
,)(
,)(
,)(
00
Prof. D.Iacoviello - Optimal Control 2624/10/2019
( ) ),,,(,,,),()),(),((min
))(()),(),(()),(),((min
],[00110111*
,)(
,)(
10
1
0
0
1
1
0
0
ttf
t
tttt
tu
t
tf
t
tttt
tu
uxttxxtttxtJdtttutxL
txGdtttutxLdtttutxL
f
f
f
=
+=
++=
Principle of optimality
Idea: dttt += 01
,)(,()),(),((min
))(()),(),((min),(
*
],[
,)(
*
+++=
++=
+
+dttxdttJdtttutxL
txGdtttutxLxtJ
dtt
tdttt
f
t
ttttu
f
f
Prof. D.Iacoviello - Optimal Control 2724/10/2019
smalldt
],[)()( dttttuu +
dtttutxLduxLdtt
t)),(),(()),(),((
+
dtd
Jdx
x
JtxtJdttxdttJ
+
+++
**** ))(,())(,(
+
++=
+++= +
+
dtt
xtJdx
x
xtJxtJdtttutxL
dttxdttJdtttutxLxtJdtt
tdttt
),(),(),()),(),((min
)(,()),(),((min),(
***
*
],[
*
Prof. D.Iacoviello - Optimal Control 2824/10/2019
+=
− dx
x
xtJdtttutxLdt
t
xtJ
tu
),()),(),((min
),( *
)(
*
Divide for dt and consider 0
lim→t
+=
− ),(
),()),(),((min
),( **
uxfx
xtJttutxL
t
xtJ
Prof. D.Iacoviello - Optimal Control 2924/10/2019
J*(x(t),t) is a solution of the Hamilton-Jacobi (H-J) equation:
with boundary conditions:
-J*(x(t),t) continuous with respect to its arguments
-the first derivatives of J*(x(t),t) continuous with respect to its
arguments
-
-
−
−=
tttx
Jtxutxf
x
J
ttx
JtxutxL
t
J
,,,),(),('
,,),(),(
**
**
( ) ( ))(),(*fff txGttxJ =
Prof. D.Iacoviello - Optimal Control 30
( ) ( )
= tttx
x
Jtxuttxu
T
o ,),(),(),(*
24/10/2019
is the optimal performance index for
( )( )
( )tutxJttxJfttu
),(),(min),(,
* =
( ) ( ) ( ))(),(),(),(),( f
t
t
txGduxLtutxJ
f
+=
Prof. D.Iacoviello - Optimal Control 31
and the control:
Is the optimal control at time t for the class problems with cost index
( ) ( )
= tttx
x
Jtxuttxuo ,),(),(),(
*
( ) ( ) ( ))(),(),(),(),( f
t
txGduxLuxJ
f
+=
24/10/2019
The Hamilton-Jacobi equation
Example 2 (from Anderson)
Consider the system:
With performance index:
Applying the H-J equation we have:
ux =
( ) 0,2
1)(),0( 0
422
0
=
++= twithdtxxuuxJ
ft
t
Prof. D.Iacoviello - Optimal Control 3224/10/2019
The minimizing control is:
And we have:
With boundary condition:
+++−=
u
x
Jxxu
t
J
tu
*422
)(
*
2
1min
x
Ju
−=
*
2
1
42
2**
2
1
4
1xx
x
J
t
J−−
=
( ) 0),( =ff ttxJ
In actual fact, it is rarely possible to solve a Hamilton-
Jacobi equation
Prof. D.Iacoviello - Optimal Control 3324/10/2019
The Hamilton-Jacobi equation
Main results
The Hamilton-Jacobi (H-J) equation is satisfied by the
optimal performance index under suitable differentiability
and continuity assumptions.
If a solution to the H-J equation has certain differentiability
properties, then this solution is the desired performance
index.
Note that the H-J equation represents only a sufficient
condition on the optimal performance index
Prof. D.Iacoviello - Optimal Control 3424/10/2019