@let@token OPTIMAL CONTROL - ELEC732001ncbs.knu.ac.kr/Teaching/OC_Files/OpC_Lecture1.pdf ·...
Transcript of @let@token OPTIMAL CONTROL - ELEC732001ncbs.knu.ac.kr/Teaching/OC_Files/OpC_Lecture1.pdf ·...
OPTIMAL CONTROLELEC732001
Kalyana C. Veluvolu#IT1− 817
Tel: 053− 950− 7232
Website: http://ncbs.knu.ac.kr
Email: [email protected] of IT Engineering
Kyungpook National University
Kalyana C. Veluvolu Optimal Control #1
Text Books
Grewal 2006 Simon 2008 Anderson 2007 Lewis 2012
Kalman filtering: Theory and Practice using MatLab by Mohinder S.Grewal, Wiley, Third Edition, 2006
Optimal state estimation- Kalman, H∞, and Nonlinear approachesby Dan Simon, Wiley, Fourth Edition, 2008
Optimal Control-Linear Quadratic methods by Anderson,Prentice-Hall, First Edition, 2007
Optimal Control by Frank L. Lewis et al., Wiley, Third Edition, 2012
Kalyana C. Veluvolu Optimal Control #2
Grading Policy
The grading for the students will be done on following basis:
Assignments: 45%
Final Project: 50% (might be a group project)
Attendance: 5%
Remark
For assignments and projects there will zero tolerance for malpractice.
Kalyana C. Veluvolu Optimal Control #3
Course Description and Prerequisites
Course Description
This course provides a comprehensive introduction to optimalcontrol theory, emphasizing application of its basic concepts to realproblems such as tracking, estimation, and navigation.
The theoretical foundations of optimal state estimation - Kalmanfiltering and other nonlinear filtering techniques - and optimal control– linear quadratic, H2 and H∞ methods – are discussed in detail.
This course will make use of MATLAB to emphasize the practicalaspects of the concepts discussed
Pre-requisites
Solid knowledge of linear algebra, differential equations, signal andsystem analysis is required.
Knowledge of simulation on MATALB would be helpful as studentswill often be required to perform simulations for verification oftheoretical results.
Kalyana C. Veluvolu Optimal Control #4
Syllabus
Week #1: Introduction to the Course
Week #2: Review of Linear Algebra, Linear and Nonlinear Systems
Week #3: Basics of Probability
Week #4: Stochastic Systems Assignment #1
Week #5 and #6: Kalman Filtering
Week #7: Nonlinear Filtering
Week #8: Applications Assignment #2
Week #9: Midterm Final Project Objectives
Week #10: Basics of Optimal Control, Linear Quadratic Regulators
Week #11: LQ Tracking Controllers and Other Extensions
Week #12: Systems with Disturbances: H2 and H∞ Methods
Week #13: Model Predictive Control Assignment #3
Week #14: Final Project
Week #15: Final Exam
Kalyana C. Veluvolu Optimal Control #5
Introduction
Optimal control essentially deals with the operation of a dynamicalsystem in the desired best possible way.
Let us consider some examples:
A factoryManufacture products at minimal financial costs.
A rocket launchReach the target as quickly as possible.
A passenger busReach the destination with minimum fuel.
A disease treatment protocolRecover as quickly as possible without any side effects.
The ’best possible way’ can be subjective but is usually based on somewell-defined performance criterion.
Kalyana C. Veluvolu Optimal Control #6
Dynamical Systems
Dynamical system : A system whose behaviour or state evolves overtime. e.g., a pendulum, a heavenly body, a development project, thestock market
State x represents the current condition of the system. It may notbe fully known or observed.
System behaviour is observed though the output, y .
Control input u and disturbance w can both change the behaviourof the system.
Kalyana C. Veluvolu Optimal Control #7
Models of Dynamical Systems
Dynamical systems are everywhere. To study a system’s behaviour, weneed its mathematical model.
Different systems are described with different models:
Differential or difference equations, e.g., pendulum motion
Partial differential or difference equations, e.g., tubular reactor
Finite state machines, e.g., elevator
Markov chains, e.g., network congestion
These models are in time domain.
Differential or difference equation based models can be expressed infrequency domain models:
Differential eqn.Laplace/inv.Laplace transform←−−−−−−−−−−−−−−−−−−→ Conti.-time transfer fn.
Difference eqn.Z/inv.Z transform←−−−−−−−−−−→ Discrete-time transfer fn.
Kalyana C. Veluvolu Optimal Control #8
State-Space Description
We focus on systems modeled with differential or difference equationsand use the state-space description:
Differential equations: Continuous-time Systems
dx
dt= f(x(t), u(t),w(t), t
), y(t) = g
(x(t), u(t), v(t), t
)Difference equations: Discrete-time systems
x(t + 1) = f(x(t), u(t),w(t), t
), y(t) = g
(x(t), u(t), v(t), t
)In discrete-time, t represents discrete time steps.
For linear time-invariant (LTI) systems:
f(x(t), u(t),w(t), t
)= Ax(t) + Bu(t) + D1w(t)
g(x(t), u(t), v(t), t
)= Cx(t) + Du(t) + D2w(k)
Usually, a performance output z(t) = h(x(t), u(t)) = Cz x(t) + Dzu(t) isalso considered.
Kalyana C. Veluvolu Optimal Control #9
What is State Estimation?
The observed output y is usually a noisy version of (a part of) the state.
State estimation or filtering deals with estimating the state x through theobserved output y using the model of the system. Goals:
Reconstruct the full state from the measured part of the state
Minimize effects of noise in the knowledge of the state
Examples:
GPS: GPS sensor signals are filtered to get an estimate of position.
Satellite navigation: Orbital position and velocity of a satellite areestimated from position sensor measurements.
Wireless communication: Signals received from a channel are filteredto estimate the transmitted signal.
To assess and manage the behaviour of a dynamical system, a reliableestimate of its state is necessary.
Kalyana C. Veluvolu Optimal Control #10
What is Control?
Control deals with suitably applying the control input u to a system so asto achieve a desired behaviour or operation.
Control is aimed at
Ensuring system stability.
Achieving or improving system performance as per specifications.
Traditionally, control designers considered various factors and tools forsimple, SISO, LTI systems:
Factors: Transient response, disturbance rejection, bandwidth,steady-state error, robustness
Tools: Laplace/Z-transform, Routh test, Nyquist plots, Nicholaschart, empirical knowledge
These classical tools are insufficient to handle complex multivariablesystems.
Kalyana C. Veluvolu Optimal Control #11
Towards Optimal State Estimation and Control
Optimal estimation and optimal control techniques systematically handlemultivariable system operations with specific performance considerations.
These techniques are based on the optimization of a mathematicallydefined performance index or cost function.
In optimal state estimation, the estimator or observer(which is a
dynamical system with state x(t))
is optimized so that z(t) is steered‘closest’ to z(t) in some sense.
In optimal control, the controller (or control input) is optimized so thatstate z(t) is steered to a reference signal zd(t) in some optimal way.
Kalyana C. Veluvolu Optimal Control #12
Optimal Estimation Performance Functions
The cost function is expressed in terms of an error signal:
e(t) = z(t)− z(t) = Cz
(x(t)− x(t)
)Estimate x(t) is obtained by minimizing the squared weighted 2-norm ofe(t). Cost function J(.) depends on the nature of disturbances:
Stochastic disturbances (disturbance spectral properties known)
minimize J = E∫ tft=0
eT (t)W (t)e(t)dt
For a linear system, the solution results in a linear estimator and iscalled Kalman filter.Steady-state Kalman filter (tf → ∞) is the H2 optimal estimator.
Non-stochastic disturbances (only disturbance bounds are known)
minimize J = maxw∈W,v∈V∫ tft=0
eT (t)W (t)e(t)dt
This is called minimax filter. It is related to an H∞ filter whichminimizes a similar worst-case cost over infinite horizon.
Kalyana C. Veluvolu Optimal Control #13
Optimal Control Performance Functions
Control performance criteria may vary over various applications and thenature of disturbances.
A few approaches for deterministic systems (no disturbances):
Minimum timeminimize J = tf such that x(tf ) = xd , u(t) ∈ U .
Minimum fuel or financial costminimize J =
∫ tft=0‖u(t)‖, such that x(tf ) = xd . ‖.‖: suitable norm
Minimum energyminimizeJ =
∫ tft=0
{zT (t)Q(t) z(t) + uT (t)R(t) u(t)
}+ zT (tf )P z(tf )
The minimum energy control uses a quadratic cost function and ischosen to obtain a trade-off between performance and fuel usage.
For a linear system, the solution to the minimum energy problem is calledthe linear quadratic regulator (LQR).
Kalyana C. Veluvolu Optimal Control #14
Optimal Control Performance Functions
When disturbances are present, different variants of cost functions areconsidered:
Stochastic disturbancesminimize
J = E{∫ tf
t=0
{zT (t)Q(t) z(t) + uT (t)R(t) u(t)
}+ zT (tf )P z(tf )
}For a linear system with Gaussian disturbances, the optimalcontroller is called LQG.Steady state LQG controller is H2 optimal.
Non-stochastic norm bounded disturbancesminimize
J = maxw∈W,v∈V
{∫ tf
t=0
{zT (t)Q(t) z(t) + uT (t)R(t) u(t)
}+zT (tf )P z(tf )
}This is minimax cost function. An H∞ controller minimizes asimilar but infinite-horizon cost.
Kalyana C. Veluvolu Optimal Control #15
Open-loop and Closed-loop Approaches
Open-loop Solution: Solution is in the form of actual values (e.g., inputtrajectory) specific to the particular system state and time.
Closed-loop Solution: Solution to the optimal estimation/constrolproblem is in the form of a function of the measured output or state.
The open-loop problem may be easier to solve but the solution may notbe give the desired performance when the system model is imperfect.
The closed-loop solution may be more robust to the presence ofdisturbances and minor system/model mismatches but is more difficult tocompute for general problems.
An alternative is to ’achieve’ feedback by repeating the open-loopsolution at frequent times as the new measurements become available.This approach is called receding horizon approach. e.g., recedinghorizon estimation, model predictive control
Kalyana C. Veluvolu Optimal Control #16
How Optimal Estimation and Optimal Control are Related?
Optimal state estimation and optimal control problems are considered tobe dual to each other.
For linear time invariant (LTI) systems, the Riccati equations that definethe solution of the two problems are identical in form.
For LTI systems with Gaussian disturbances and measurement errors, thesolution to the stochastic optimal control problem is the combination ofthe optimal estimator (Kalman filter) and the optimal controller (LQR).
This implies that the stochastic problem can be decoupled intoestimation and control problems that can be solved separately. This isreferred to as the Separation Principle.
The Separation Principle is often applied to other situations to simplifythe design process.
In the subsequent classes, we discuss the two problems mostlyindependently.
Kalyana C. Veluvolu Optimal Control #17
In the next class...
Review of Linear Algebra and other background topics ...
Kalyana C. Veluvolu Optimal Control #18
MATLAB- Functions
The various MATALAB functions that are useful for understanding theconcepts of this lecture are
Functions
laplace, invlaplace Returns the Laplace/ inverse Laplace transform
tf Creates the transfer function of a system.
step, impulse Gives step/impulse response of a system.
lyap, dlyap Solves the continuous/discrete-time Lyapunov equation
lqr, dlqr Solves the continuous/discrete-time LQR problem
care, dare Solves the continuous/discrete-time Riccati equation.
Use‘‘Help’’ in MATLAB for more details on how to use thesefunctions.
Kalyana C. Veluvolu Optimal Control #19