Optimal Control: Variational Approach to Optimal Control...
Transcript of Optimal Control: Variational Approach to Optimal Control...
Optimal Control: Variational Approach Dr. Timm Faulwasser, Laboratoire d‘Automatique, EPFL
28.05.2013
Optimal Control: Indirect Solution Methods and Research Outlook
Timm Faulwasser <[email protected]>
Laboratoire d’Automatique
Overview
Indirect Solution Methods for OCPs
Research Outlook – Model Predictive Control
– Real Time Optimization
Optimal Control: Variational Approach Dr. Timm Faulwasser, Laboratoire d‘Automatique, EPFL
28.05.2013
Overview of Numerical Methods for Optimal Control
Continuous Time Optimal Control
Hamilton-Jacobi-Bellman-Equation
• tabulation in state space • closed-loop opt. control
Indirect Methods
• Pontryagin’s Maximum Principle (NCO of OCP)
Direct Solution Methods
• convert OCP into NLP
Direct Single Shooting
• only discretized controls in NLP • sequential approach
Direct Collocation
• discretized states and controls in NLP (collocation) • simultaneous approach
Direct Multiple Shooting
• discretized controls and node start values in NLP • simultaneous approach
First optimize then discretize!
Indirect and Direct Solution Methods
Indirect Methods
• based on Pontryagin’s Maximum Principle or variational principles (NCO of OCP) • no discretization of control profile needed • guess for optimal solution structure needed
Direct Solution Methods
• convert OCP into NLP • no knowledge about solution structure required • approximate solutions (control parametrization)
First discretize then optimize!
Lecture II on Optimal Control this lecture
Optimal Control: Variational Approach Dr. Timm Faulwasser, Laboratoire d‘Automatique, EPFL
28.05.2013
Optimal Control Problem with Terminal Constraints
Necessary Conditions of Optimality
Indirect Solution Approaches
General Idea
• solve NCO directly
• split NCO into two parts
– NCOs enforced at each iteration
– NCOs which are modified at each iteration
Basic Indirect Shooting
1. Guess
2. Integrate from
3. Compute defect of transversality and terminal conditions
4. Update to enforce that
Optimal Control: Variational Approach Dr. Timm Faulwasser, Laboratoire d‘Automatique, EPFL
28.05.2013
Indirect Solution Approaches
Basic Algorithm for Indirect Shooting
1. Guess , choice tolerance , set
2. Integrate from
3. Compute defect of transversality conditions
If STOP
4. Compute gradient
and solve
5. Update
6. Set . Goto 1.
Example: Pontryagin‘s Maximum Principle
• Formulate the NCO. • Formulate the defect .
Optimal Control: Variational Approach Dr. Timm Faulwasser, Laboratoire d‘Automatique, EPFL
28.05.2013
Indirect Gradient Projection Method
Input constraints? e.g. via gradient projection methods
Necessary Conditions of Optimality
Indirect Gradient Projection Method
0. Guess input trajectory , choose tolerance , set
1. Integrate forward
2. Integrate backward
3. Compute search direction
4. Compute step size (line search)
5. Compute control trajectory via projection onto input constraints
6. Integrate forward
7. IF STOP
ELSE set GOTO 2.
Dunn, J. (1996). On l2 conditions and the gradient projection method for optimal control problems, SIAM Journal on Control and Optimization 34(4): 1270–1290. K. Graichen and B. Käpernick, A real-time gradient method for nonlinear model predictive control, in Frontiers of Model Predictive Control, T. Zheng, Ed. Rijeka, Croatia: InTech, Feb. 2012, pp. 9–28.
Optimal Control: Variational Approach Dr. Timm Faulwasser, Laboratoire d‘Automatique, EPFL
28.05.2013
First optimize then discretize!
Numerical Methods for Optimal Control
Indirect Methods
• based on Pontryagin’s Maximum Principle or variational principles (NCO of OCP) • no discretization of control profile needed • guess for optimal solution structure needed
Direct Solution Methods
• convert OCP into NLP • no knowledge about solution structure required • approximate solutions (control vector parametrization)
First discretize then optimize!
Which method should be used? • no conclusive answer depends on considered OCP, preferences, … • accurate solution to OCP first approximate solution by direct method
refine solution with indirect approach • fast online solutions (e.g. NMPC):
• without state constraints direct and indirect methods suitable • with state constraints direct methods + safety margin on constraints
Overview
Indirect Solution Methods for OCPs
Research Outlook – Model Predictive Control
– Real Time Optimization
Optimal Control: Variational Approach Dr. Timm Faulwasser, Laboratoire d‘Automatique, EPFL
28.05.2013
Model Predictive Control: Basic Principle
1. Obtain state
2. Predict system and optimize input
3. Apply “optimal” input signal
model predictive control = control based on repeated prediction/optimization
Model Predictive Control
1. State measurement at
2. Solve
3. Apply for
Main Idea of NMPC
Stability, robustness, achievable sampling rates, …?
Optimal Control: Variational Approach Dr. Timm Faulwasser, Laboratoire d‘Automatique, EPFL
28.05.2013
Detour: Does the application of optimal controls lead to stability?
• optimal control often computed over finite horizon (e.g. MPC)
• stability deals with long term behavior of trajectories
• even for optimal feedbacks stability conditions need to be checked
• typical example – infinite horizon LQR control
optimal feedback for infinite horizon via (static) Riccati equation
sufficient stability condition: observable and controllable
Kalman, R. Contributions to the theory of optimal control. Bol. Soc. Mat. Mexicana, 1960, 5, 102-119
Optimality does not imply stability!
Stability of MPC?
Model Predictive Control
1. State measurement at
2. Solve
3. Apply for
• stability via design of and/or via controllability assumptions • additional restrictions on stage cost • guaranteed satisfaction of constraints stability can be rigorously shown (details beyond the scope of this course) • MPC can be extended to various problems beyond stabilization: tracking, path following, …
Optimal Control: Variational Approach Dr. Timm Faulwasser, Laboratoire d‘Automatique, EPFL
28.05.2013
Output Path-following Problem
Control Task
• convergence to path
• convergence on path
• satisfaction of constraints
Questions
• problem structure suitable formulation? • constraints controller design?
• dynamic system
• output path regular, 1d curve
Analysis of Path-following Problems
state space output space
• = zero-path-error manifold
Path Followability Given a system and a path , under which conditions is it possible to follow exactly? I.e., does an input exist, such that while ?
Augmented System Description
Augmented system:
Optimal Control: Variational Approach Dr. Timm Faulwasser, Laboratoire d‘Automatique, EPFL
28.05.2013
Principle of Predictive Path Following
Idea • prediction based on augmented dynamics • costs penalize path-following error
Optimal Control Problem
Path convergence, stability? Faulwasser, T. Optimization-based solutions to constrained trajectory-tracking and path-following problems. Shaker, Aachen, Germany, 2013
Outlook on Applications – Implementation on 2-DoF Robot
KUKA LWR Robot
Unscented Kalman Filter
Predictive Path-following Controller
• two actuated axes • continuous-time nonlinear model • input constraints
Performance?
Optimal Control: Variational Approach Dr. Timm Faulwasser, Laboratoire d‘Automatique, EPFL
28.05.2013
Outlook on Applications – Implementation on 2-DoF Robot
Circular Path States and Inputs Cartesian Workspace
• Coriolis and friction terms neglected • prediction horizon 200ms • sampling time 5ms • constraint satisfaction • disturbance attenuation
Predictive Path-following: Implementation on 2-DoF Robot
Circular Path Path Deviation Cartesian Workspace
Benefits?
• Coriolis and friction terms neglected • prediction horizon 200ms • sampling time 5ms • constraint satisfaction • disturbance attenuation
Optimal Control: Variational Approach Dr. Timm Faulwasser, Laboratoire d‘Automatique, EPFL
28.05.2013
Example: 2-DoF Robot
Circular Path
Trajectory Tracking (KUKA controller) Path Following
improved performance
J. Matschek. Echtzeitfähige Implementierung einer optimierungsbasierten prädiktiven Pfadverfolgung. Master thesis, OvG University Magdeburg, Germany, 2013.
Model Predictive Control
Remarks:
• achievable sampling rate depends on problem size, problem formulation and computational platform
• special case linear time-discrete systems convex quadratic programs
• down to submillisecond sampling rates for small or linear systems
• nonlinear systems (< 10 states) down to millisecond range
• large progress in theory and computation (Moore‘s Law & improved algorithms) over last two decades
• highly active research field & many open issues
• extensions to trajectory tracking, set-point tracking, path-following, economic MPC, …
Further reading: • Mayne, D.; Rawlings, J.; Rao, C. & Scokaert, P. Constrained model predictive control: Stability and optimality.
Automatica, 2000.
• Rawlings, J. & Mayne, D. Model Predictive Control: Theory & Design, Nob Hill Publishing, Madison, WI, 2009
• Grüne, L. & Pannek, J. Nonlinear Model Predictive Control: Theory and Algorithms, Springer Verlag, 2011
model predictive control = optimization-based control
Feedback-based optimization?
Optimal Control: Variational Approach Dr. Timm Faulwasser, Laboratoire d‘Automatique, EPFL
28.05.2013
Static Real-time Optimization
subject to subject to
Consider: • repeated static optimization of real plant • uncertain plant data /plant-model mismatch
Inputs which are optimal for the model are usually not be optimal for the plant!
Plant RTO
Static Real-time Optimization
Remedies • quantify uncertainty and compute min-max solution conservative • update plant model during run time parameter estimation • update optimization problem based on measurements, e.g. modifier adaptation
Inputs which are optimal for the model might not be optimal for the plant!
RTO Plant
subject to
Main idea
Modifier Adaptation
• use linear update terms in optimization problem • needs information about plant gradients estimation problem
-
Optimal Control: Variational Approach Dr. Timm Faulwasser, Laboratoire d‘Automatique, EPFL
28.05.2013
Modifier Adaptation with Noise Filtering
RTO Plant
Filter
subject to
Main idea
Filter update
Further reading on modifier adaptation: A. Marchetti, B. Chachaut and D. Bonvin. Modifier-Adaptation Methodology for Real-Time Optimization, in Industrial & Engineering
Chemistry Research, vol. 48, num. 13, p. 6022-6033, 2009.
Upon convergence a KKT point of the plant is obtained!
Modifier Adaptation - Example
QP with uncertain parameters
Example taken from: A. G. Marchetti. Modifier-adaptation methodology for real-time optimization. Thèse EPFL, n° 4449
Filter
Parameters:
Optimal Control: Variational Approach Dr. Timm Faulwasser, Laboratoire d‘Automatique, EPFL
28.05.2013
Real-time Optimization
Main idea Use feedback to achieve optimal process operation in the presence of uncertainties. Remarks • static and dynamic optimization problem can be considered • different methods available: G. François and D. Bonvin. Measurement-based Real-Time Optimization of Chemical Processes, in Advances in Chemical Engineering, p. 1-50, Identification, Control and Optimisation of Process Systems 43, 2013
• NCO tracking Bonvin, D. & Srinivasan, B. On the role of the necessary conditions of optimality in structuring dynamic real-time optimization schemes. Computers & Chemical Engineering, Elsevier, 2012
• self-optimizing control S. Skogestad, ``Near-optimal operation by self-optimizing control: From process control to marathon running and business systems'', Computers and Chemical Engineering, 29 (1), 127-137 (2004).
• static RTO, e.g. modifier adaptation A. Marchetti, B. Chachaut and D. Bonvin. Modifier-Adaptation Methodology for Real-Time Optimization, in Industrial & Engineering Chemistry Research, vol. 48, num. 13, p. 6022-6033, 2009.
How to Earn Credits for this Course?
• project-based evaluation (max 2 persons on 1 project)
– apply optimal control/calculus of variations to one of your research problems
– simplify models if necessary
– numerical implementation and theoretical discussion are both possible
– in both cases you should be able to explain what you have been doing ;-)
• written report due on June 28th (hard deadline)
– approximately 10 pages
– earlier submission of reports is welcome
• presentation to teachers and class on July 4th
– 15 minutes plus 10 minutes of discussion
– room ME C2 405