Conic Optimisation:
Why, When and How Should it be Used.
A Short Trip Along a Basic yet Powerful Optimisation Tool
OSE Workshop 2015 – 17 September 2015 Glasgow (UK)
Andrea Cassioli, [email protected]
www.mosek.com
Conic Optimization I
Standard form
min∑k
cTk xk +∑k
〈Ck , Xk〉
s.t. ∑k
aTikxk +∑k
〈Aik , Xik〉 = bi i = 1, . . .
xk ∈ Kk k = 1, . . .Xk ∈ Kk k = 1, . . .
where Kk , Kk are proper cones.
1 / 24
Conic Optimization II
We only consider
Positive Orth. R+ = x ∈ Rn|x ≥ 0
Lorentz Q = (t, x) ∈ R× Rn|t ≥ ‖x‖2
Rotated Qr =
(z , y , x) ∈ R× R× Rn|2zy ≥ ‖x‖22, z ,w ≥ 0
PSD Matrices S+ =X ∈ Rn×n|X is PSD
Other more exotic cones can be defined, still an active researchfield.
2 / 24
Conic Optimization III
Properties
• convexity guarantee,
• rich duality theory,
• (almost) avoid ill-posedness,
• polynomial time complexity,
• deeply investigated, but still active research field,
• many good implementations!
3 / 24
Convexity guarantee
If you can cast your problem in conic form, then it is convex!No need for long and tedious convexity proof!
Plenty of rules and examples:
Boyd, S. and Vandenberghe, L., Convex optimization,Cambridge University Press, 2004.
Ben-Tal, A. and Nemirovski, A, Lectures on Modern ConvexOptimization: Analysis, Algorithms, and EngineeringApplications, SIAM, 2001.
Alizadeh, F. and Goldfarb, D. Second-order coneprogramming, Math. Programming 95-1, 2003.
4 / 24
Conic Modeling I
What can we represent with CO?
Trivially:
• LP
• QP
• QCQP
• SOCP
• SDP
5 / 24
Conic Modeling II
• Geometric mean:(x , t) ∈ Rn+1|(x1x2 . . . xn)1/n ≥ t, x ≥ 0
• Harmonic mean:
(x , t) ∈ Rn+1|(
1
2
∑x−1i
)−1≥ t, x ≥ 0
6 / 24
Conic Modeling III
• Increasing powers:(x , t)|xp/q ≤ t, x ≥ 0
, p/q ≥ 1, p, q ∈ Z+
• Decrease powers:(x , t)|x−p/q ≤ t, x ≥ 0
, p, q ∈ Z+
• Power cones:(x , y) ∈ Rn
+ × R| |y | ≤∏
xαj
j
, α > 0,
∑αj = 0
7 / 24
Conic Modeling IV
Eigenvalue related problems:
• largest/smallest eigenvalue
• eigenvalue spread
• spectral radius
• minimize condition number of a PSD matrix
• singular values.
8 / 24
Conic Modeling - Example
(Linear) Regressions/Regularization
min φ(δ)δ = Ax − yx ∈ Ω
φ(·) is typically
• ‖ · ‖1, ‖ · ‖2, ‖ · ‖∞• 3/2−regression
• k−largest penalty
• Huber penalty
• Dead-zone linear penalty
9 / 24
Conic Modeling - Example
Nearest correlation matrix
min ‖X − Y ‖FXii = 1 i = 1, . . . , nX ∈ Sn+
where ‖A‖F =√
tr(ATA) =
√∑i ,j
A2ij , and hence
min t(t, vec(X − Y )) ∈ QXii = 1 i = 1, . . . , nX ∈ Sn+
10 / 24
Conic Modeling - Examples
For a polyhedron, find an ellipsoid E defined by a matrix C :
Maximal inscribed ellipsoid: Vol(E) ≈ det(C )1/n
Minimal enclosing ellipsoid: Vol(E) ≈ det(C )−1/n
Much interest in the robotic community.11 / 24
Conic Modeling - Examples
Polynomial curve-fitting
For a given degree, condition on
• interpolation
• non-negativity
• monotonicity
• convexity/concavity
f2(t)
f4(t)
f8(t)12
1
32
−2 −1 0 1 2 t
12 / 24
Conic Modeling
There is plenty of material out there.
Advice
Always try to cast your problem in conic form!
13 / 24
CO polynomial complexity
The problem size is not just the combination of the number ofvariables and constraints, but also the number of bits to representthe problem!
Advise
Do not be afraid of transformations that introduce additionalvariables/constraints.
A larger but sparser problem can have the same complexity that asmaller but denser.
14 / 24
SOCP vs. QCQP
SOCP contains QCQP, which one should one use?
• in general no significant differences in complexity
• SOCP more general, ready for extensions
• some specialized algorithms for structured QP/QCQP exist
• our experience indicates SOCP more robust.
15 / 24
From QCQP to SOCP I
To convert QCQP to SOCP you need to factorize the Q matrix...
min xTQxx ∈ X
↓
min t
t ≥ xTQx = xTFFT x = ‖Fx‖2x ∈ X
↓
min t(1/2, t,Fx) ∈ Qr
x ∈ X16 / 24
From QCQP to SOCP II
Forming F may sound tricky, but it can pay:
• You can exploit the structure better than the solver;
• Sometimes F already given;
• You may need to factorize to check Q ∈ S+;
• Recover from errors.
17 / 24
From QCQP to SOCP III
minimize −x1 − x2subject to (x1 − x2)2 ≤ 0,
0 ≤ x1, x2 ≤ 1
What if the quadratic constraints is affected by a small error ε, i.e.
xT[
1 −1−1 1 + ε
]x ≤ 0
Observe:
• ε < 0 : The problem is not convex.
• ε = 0 : x∗1 = x∗2 = 1.
• ε > 0 : x∗1 = x∗2 = 0.
18 / 24
From QCQP to SOCP III
Advise
Do try to cast a QCQP as a SOCP yourself!
19 / 24
Exploit duality I
In CO duality allows to
1 Automatic dual formulation (pretty much like LPs)
2 Numeric infeasibility certificate (approximate Farkas’ Lemma)
For primal-dual algorithms there is no theoretical difference inusing primal or dual formulation, but...
20 / 24
Exploit duality II
in practice:
• presolver may be affected
• caching issue from different memory setup
• linear algebra routines may be tuned better for (dual) primalformulation.
21 / 24
Exploit duality
In our experience some times up to 10x speed up!
Advise
Give the dual formulation a chance!
Some framework like CVX will do it automagically for you...
22 / 24
Software
Great tools/solvers for CO available:
• CVX, YALMIP, Julia Convex, PICOS, ...
• MOSEK, CPLEX, Gurobi,...
• Sedumi, SDPA, SDPT3, CVXOPT, ...
Advice
Do not re-invent the wheel!
23 / 24
Conclusion
My suggestions:
1 use CO whenever possible
2 rely on proven technology even for cutting edge research
3 give the dual problem a chance
4 move to SOCP to QP/QCQP unless a special case is at hand
5 don’t be afraid of larger but sparser formulations
6 share your experiences and issues with the developers!
24 / 24
Top Related