con humo - MIT · 2019. 6. 4. · con humo Technische Univ ersitä t M ünchen Chair of I nf or ma...
1
humo con Technische Universität München Chair of Information-oriented Control Identification and Control with Gaussian Processes with formal guarantees Thomas Beckers [email protected] Sandra Hirche [email protected] Identification and Control with Gaussian Processes with formal guarantees Thomas Beckers [email protected] Sandra Hirche [email protected] Motivation Precise and safe control for real-world systems is challenging due to uncertain system models and external disturbances [Soft Robotics Inc] [KUKA] [DARPA] [Schaeffler] Idea: Gaussian process regression for identification and control Challenge: Providing formal guarantees for stable and safe operation Research Statement Safe, efficient and rational use of machine learning in control Recent research Stability of data-driven systems Model error estimation for efficient learning Integration of prior knowledge for hybrid learning approaches Future research Holistic learning including task, system and controller Exploiting the properties of data-driven models for smarter control Gaussian Process Models Flexible nonparametric regression Based on Bayesian probability Noisy training data Explicit uncertainty description Input space Output space Training points Mean Variance µ(y ∗ |x ∗ , D )= k ⊤ φ (x ∗ , X )(K φ (X , X )+ Iσ 2 n ) −1 Y Σ(y ∗ |x ∗ , D )= k φ (x ∗ , x ∗ ) − k ⊤ φ (x ∗ , X )(K φ (X , X )+ Iσ 2 n ) −1 k φ (x ∗ , X ) Identification Rational kernel selection Kernel constrains the capabilities of the model Active injection of properties System Kernel GP model Desired model behavior Property transfer Squared exponential kernel generates always bounded systems Time State System GP model Model error Misspecified GPs underestimate the model error ⇒ Derivation of an upper bound for prediction error Actual functions GP model Input Output Hyperparameter optimization Including the task into the optimization process Controller System Kernel-based model Bayesian optimization Cost function Reference Classical approaches focus on data-based optimization only Closed-loop performance gives valuable information for the optimization ⇒ More comprehensive interpretation of the data ⇒ Stability can be preserved Task-based hyperparameter tuning using Bayesian optimization improves the closed-loop performance 10 20 30 40 50 100 200 300 400 Task evaluations Min. cost C Control Gaussian process based computed torque control PD controller GP model Prior knowledge System ¨ x d , ˙ x d , x d Variance Σ d , Σ p Mean µ τ States Adaptive feedback gains based on the model uncertainty τ = Prior knowledge + µ( ˜ τ | ¨ x, ˙ x, x, D ) | {z } Feedforward −K d (Σ d ( ˜ τ | ˙ x, x, D )) ˙ e − K p (Σ p ( ˜ τ |x, D ))e | {z } Feedback Stability analysis The tracking error of the closed loop is uniformly ultimately bounded Tracking error r ∼ GP model error ∥Feedback gain functions∥ e ˙ e γ r Improved performance inside and outside the training data set Safe interaction due to low feedback gains Selected Publications T. Beckers, D. Kulic, and S. Hirche. Stable Gaussian process based tracking control of Euler-Lagrange systems. Automatica, 103:390–397, 2019. T. Beckers, J. Umlauft, and S. Hirche. Mean square prediction error of misspecified Gaussian process state space models. In Proc. of the Conference on Decision and Control, 2018. T. Beckers and S. Hirche. Passive rendering of a class of nonlinear systems with Gaussian process models. In Proc. of the European Control Conference, 2018. T. Beckers, J. Umlauft, D. Kulić, and S. Hirche. Stable Gaussian process based tracking control of Lagrangian systems. In Proc. of the Conference on Decision and Control, 2017. T. Beckers, J. Umlauft, and S. Hirche. Stable model-based control with Gaussian process regression for robot manipulators. In Proc. of the 20th IFAC World Congress, 2017. T. Beckers and S. Hirche. Stability of Gaussian process state space models. In Proc. of the European Control Conference, 2016. T. Beckers and S. Hirche. Equilibrium distributions and stability analysis of Gaussian process state space models. In Proc. of the Conference on Decision and Control, 2016. Evaluation Simulation of stable learning Property injection analysis Learning of robot dynamics From simulation to application in real-world scenarios
Transcript of con humo - MIT · 2019. 6. 4. · con humo Technische Univ ersitä t M ünchen Chair of I nf or ma...
humocon
Technische Universität MünchenChair of Information-oriented Control
Identification and Control with Gaussian Processes with formal guaranteesThomas [email protected]
Sandra [email protected]
Identification and Control with Gaussian Processes with formal guaranteesThomas [email protected]
Sandra [email protected]
MotivationPrecise and safe control for real-world systems is challenging due touncertain system models and external disturbances
[Soft Robotics Inc] [KUKA] [DARPA] [Schaeffler]
Idea: Gaussian process regression for identification and control
Challenge: Providing formal guarantees for stable and safe operation
Research Statement
Safe, efficient and rational use of machine learning in control
Recent research
Stability of data-driven systems
Model error estimation for efficient learning
Integration of prior knowledge for hybrid learning approaches
Future research
Holistic learning including task, system and controller
Exploiting the properties of data-driven models for smarter control
Gaussian Process Models
Flexible nonparametric regression
Based on Bayesian probability
Noisy training data
Explicit uncertainty description
Input space
Outpu
tspa
ce
Training pointsMean
Variance
µ(y∗|x∗,D) = k⊤φ (x
∗, X)(Kφ(X,X) + Iσ2n)
−1Y
Σ(y∗|x∗,D) = kφ(x∗,x∗)− k⊤
φ (x∗, X)(Kφ(X,X) + Iσ2
n)−1kφ(x
∗, X)
IdentificationRational kernel selection
Kernel constrains the capabilities of the model
Active injection of properties
System Kernel GP model
Desired modelbehavior
Property transfer
Squared exponential kernel generatesalways bounded systems
Time
State
SystemGP model
Model error
Misspecified GPs underestimate themodel error
⇒Derivation of an upper bound forprediction error
Actual functions
GP model
Input
Outpu
t
Hyperparameter optimization
Including the task into the optimization process
Controller System
Kernel-basedmodel
Bayesianoptimization
Costfunction
Reference
Classical approaches focus on data-based optimization only
Closed-loop performance gives valuable information for the optimization
⇒ More comprehensive interpretation of the data
⇒ Stability can be preserved
Task-based hyperparameter tuningusing Bayesian optimization improvesthe closed-loop performance
10 20 30 40 50
100
200
300
400
Task evaluations
Min.co
stC
ControlGaussian process based computed torque control
PD controller
GP model
Prior knowledge
System
xd, xd,xd
Variance Σd,Σp
Mean µ τ
States
Adaptive feedback gains based on the model uncertainty
τ =Prior knowledge+µ(τ |x, x,x,D)︸ ︷︷ ︸Feedforward
−Kd(Σd(τ |x,x,D))e−Kp(Σp(τ |x,D))e︸ ︷︷ ︸Feedback
Stability analysisThe tracking error of the closed loop is uniformly ultimately bounded
Tracking error r ∼ GP model error
∥Feedback gain functions∥ e
e
γ r
Improved performance inside and outside the training data set
Safe interaction due to low feedback gains
Selected PublicationsT. Beckers, D. Kulic, and S. Hirche. Stable Gaussian process based tracking control of Euler-Lagrange systems. Automatica, 103:390–397, 2019.
T. Beckers, J. Umlauft, and S. Hirche. Mean square prediction error of misspecified Gaussian process state space models. In Proc. of the Conference on Decision and Control, 2018.
T. Beckers and S. Hirche. Passive rendering of a class of nonlinear systems with Gaussian process models. In Proc. of the European Control Conference, 2018.
T. Beckers, J. Umlauft, D. Kulić, and S. Hirche. Stable Gaussian process based tracking control of Lagrangian systems. In Proc. of the Conference on Decision and Control, 2017.
T. Beckers, J. Umlauft, and S. Hirche. Stable model-based control with Gaussian process regression for robot manipulators. In Proc. of the 20th IFAC World Congress, 2017.
T. Beckers and S. Hirche. Stability of Gaussian process state space models. In Proc. of the European Control Conference, 2016.
T. Beckers and S. Hirche. Equilibrium distributions and stability analysis of Gaussian process state space models. In Proc. of the Conference on Decision and Control, 2016.
Evaluation
Simulation of stable learning
Property injection analysis
Learning of robot dynamics
From simulation to application in real-world scenarios
