March 1956! - JuliaOpt · 2020. 11. 8. · code generation using the OSQP solver. IEEE Conference...
Transcript of March 1956! - JuliaOpt · 2020. 11. 8. · code generation using the OSQP solver. IEEE Conference...
JuMP Developers Meetup, 13 Jun 2017
Bartolomeo Stellato
OSQP.jl A Julia wrapper for the Operator Splitting QP solver
joint work with Goran Banjac,
Nicholas Moehle, Paul Goulart, Alberto Bemporad, Stephen Boyd
Why quadratic programming?
3
March 1956!
5
First-order methods
Handle large-scale problems
Warm starting
Pros Cons
Embeddable
Low accuracy solutions
Don’t detect infeasibility
Problem data dependent
6
General Purpose QP Solver
Detects InfeasibilityRobust Accurate
Based on first-order methods
The OSQP Solver
8
The problem
1
2xT Px + qT x
minimize 12xT Px + qT x
subject to l � Ax � u
Quadratic Program
l � Ax � u
9
ADMM
minimize f(x̃) + g(x)subject to x̃ = x
minimize f(x) + g(x)
9
ADMM
minimize f(x̃) + g(x)subject to x̃ = x
x̃k+1 � argminx̃
�f(x̃) +
�
2
����x̃ � xk +yk
�
����2�
xk+1 � argminx
�g(x) +
�
2
����x � x̃k+1 � yk
�
����2�
yk+1 � yk + ��x̃k+1 � xk+1
�
minimize f(x) + g(x)
1
2
3
10
How to split the QP?
minimize 12 x̃T Px̃ + qT x̃ + IAx=z(x̃, z̃) + I[l,u](z)
subject to (x̃, z̃) = (x, z)
minimize 12xT Px + qT x
subject to Ax = zl � z � u
10
How to split the QP?
f
f
minimize 12 x̃T Px̃ + qT x̃ + IAx=z(x̃, z̃) + I[l,u](z)
subject to (x̃, z̃) = (x, z)
minimize 12xT Px + qT x
subject to Ax = zl � z � u
10
How to split the QP?
f
f
g
g
minimize 12 x̃T Px̃ + qT x̃ + IAx=z(x̃, z̃) + I[l,u](z)
subject to (x̃, z̃) = (x, z)
minimize 12xT Px + qT x
subject to Ax = zl � z � u
11
OSQP
(xk+1, �k+1) � solve�P + �I AT
A � 1�I
� �xk+1
�k+1
�=
��xk � qzk � 1
�yk
�
z̃k+1 � zk + 1� (�k+1 � yk)
zk+1 � ��z̃k+1 + 1
�yk�
yk+1 � yk + ��z̃k+1 � zk+1
�
minimize 12xT Px + qT x
subject to l � Ax � u
Problem
Algorithm
1
23
{{{
11
OSQP
(xk+1, �k+1) � solve�P + �I AT
A � 1�I
� �xk+1
�k+1
�=
��xk � qzk � 1
�yk
�
z̃k+1 � zk + 1� (�k+1 � yk)
zk+1 � ��z̃k+1 + 1
�yk�
yk+1 � yk + ��z̃k+1 � zk+1
�
Linear systemsolve
minimize 12xT Px + qT x
subject to l � Ax � u
Problem
Algorithm
11
OSQP
(xk+1, �k+1) � solve�P + �I AT
A � 1�I
� �xk+1
�k+1
�=
��xk � qzk � 1
�yk
�
z̃k+1 � zk + 1� (�k+1 � yk)
zk+1 � ��z̃k+1 + 1
�yk�
yk+1 � yk + ��z̃k+1 � zk+1
�
Linear systemsolve
Easyoperations
minimize 12xT Px + qT x
subject to l � Ax � u
Problem
Algorithm
11
OSQP
(xk+1, �k+1) � solve�P + �I AT
A � 1�I
� �xk+1
�k+1
�=
��xk � qzk � 1
�yk
�
z̃k+1 � zk + 1� (�k+1 � yk)
zk+1 � ��z̃k+1 + 1
�yk�
yk+1 � yk + ��z̃k+1 � zk+1
�
Linear systemsolve
Easyoperations
Factorization caching
minimize 12xT Px + qT x
subject to l � Ax � u
Problem
Algorithm
11
OSQP
(xk+1, �k+1) � solve�P + �I AT
A � 1�I
� �xk+1
�k+1
�=
��xk � qzk � 1
�yk
�
z̃k+1 � zk + 1� (�k+1 � yk)
zk+1 � ��z̃k+1 + 1
�yk�
yk+1 � yk + ��z̃k+1 � zk+1
�
Linear systemsolve
Easyoperations
Warm starting
Factorization caching
minimize 12xT Px + qT x
subject to l � Ax � u
Problem
Algorithm
11
OSQP
(xk+1, �k+1) � solve�P + �I AT
A � 1�I
� �xk+1
�k+1
�=
��xk � qzk � 1
�yk
�
z̃k+1 � zk + 1� (�k+1 � yk)
zk+1 � ��z̃k+1 + 1
�yk�
yk+1 � yk + ��z̃k+1 � zk+1
�
Linear systemsolve
Easyoperations
Warm starting
Factorization caching
minimize 12xT Px + qT x
subject to l � Ax � u
Problem
Algorithm
Solution polishing
13
Compiled code size ~80kb
300x Reduction!
CPLEX
GUROBI
OSQP
14
InterfacesLanguages Parsers
JuMP
CVXPY
YALMIP
15
OSQP interface
% Create OSQP objec tm = osqp ( ) ;
% I n i t i a l i z e so l ve rm. setup (P , q , A, l , u ,
se t t i ngs ) ;
% Solver e s u l t s = m. so lve ( ) ;
% Update cost w i th q_newm. update ( ’q ’ , q_new ) ;
% Solve againresu l ts_new = m. so lve ( ) ;
# Create OSQP objec tm = osqp .OSQP( )
# I n i t i a l i z e so l ve rm. setup (P , q , A, l , u ,
se t t i ngs )
# Solver e s u l t s = m. so lve ( )
# Update cost w i th q_newm. update (q=q_new )
# Solve againresu l ts_new = m. so lve ( )
16
Code generation
# Create OSQP objec tm = osqp .OSQP( )
# I n i t i a l i z e so l ve rm. setup (P , q , A, l , u ,
se t t i ngs )
# Generate C codem. codegen ( ’ folder_name ’ )
Optimized C code
Embedded hardware
Numerical Example
18
Lasso
minimize ∥Ax− b∥22 + λ∥x∥1
Features Data pointsn m = 100n
18
Lasso
minimize ∥Ax− b∥22 + λ∥x∥1
Weighting parameter
Features Data pointsn m = 100n
19
Lasso timingsTi
me
[s]
0.001
0.01
0.1
1
10
100
1000
Data points
1,000 5,000 10,000 50,000 100,000
OSQPGUROBIMOSEK
10x
6x
OSQP in
“meta-algorithms”
21
MIQP
minimize 12xT Px + qT x
subject to l � Ax � uxi � Z �i � I
21
MIQP
Integer constraints
minimize 12xT Px + qT x
subject to l � Ax � uxi � Z �i � I
21
MIQP
Integer constraints
minimize 12xT Px + qT x
subject to l � Ax � uxi � Z �i � I
QP(��, +�)
QP(3, +�)QP(��, 3)
QP(2, 3)QP(��, 2)
22
Inner QPs
minimize 12xT Px + qT x
subject to l � Ax � uxi � xi � xi �i � I
QP(x, x)
22
Inner QPs
minimize 12xT Px + qT x
subject to l � Ax � uxi � xi � xi �i � I Changing
bounds
Reusing factorization
QP(x, x)
23
Saving computations
Factorization caching
Warm starting
ADMM
+
QP(x, x) Repeated MIQPs
24
MIQP TimingsTi
me
[s]
0.001
0.01
1 2 3 4 5
miOSQP GUROBI
4x
2.8xSimple Python
Implementation!
Hybrid Model Predictive Control
NHorizon Length
Conclusions
26
Acknowledgements
Goran Banjac
NicholasMoehle
PaulGoulart
AlbertoBemporad
StephenBoyd
Oxford Stanford Oxford IMT Lucca Stanford
27
Final remarksOSQP
Robust Embeddable
New high-level algorithms
Julia interface
C code generation
Exploit Initialization
Simple
28
References
G. Banjac, P. Goulart, B. Stellato, and S. Boyd. Infeasibility detection in the alternating direction method of multipliers for convex optimization. optimization-online.org, 2017
B. Stellato, V. Naik, A. Bemporad, P. Goulart, and S. Boyd. Embedded mixed-integer quadratic optimization using the OSQP solver. IEEE Conference on Decision and Control (CDC) (submitted), 2017
G. Banjac, B.Stellato, N. Moehle, P. Goulart, A. Bemporad and S. Boyd. Embedded code generation using the OSQP solver. IEEE Conference on Decision and Control (CDC) (submitted), 2017
B. Stellato, G. Banjac, P. Goulart, A. Bemporad and S. Boyd. OSQP: An Operator Splitting Solver for Quadratic Programs. (Coming soon!)