David Gao Alex Rubinov Prof. of Mathematics, Federation University
description
Transcript of David Gao Alex Rubinov Prof. of Mathematics, Federation University
Canonical Duality Theory for Solving General Canonical Duality Theory for Solving General Mixed Integer Nonlinear Programming Problems Mixed Integer Nonlinear Programming Problems
with Applications with Applications
David Gao
Alex Rubinov Prof. of Mathematics, Federation University
Research Prof. of Eng. Science, Australian National University
Supported by US Air Force AFOSR grants
Since 2008
1. Duality Gap between Math and Physics conceptual problems
3. Challenges Breakthrough
2. Canonical Duality-Triality:
Unified Modeling Unified Solutions
MINLP 2014, Carnegie Mellon, June 2-5, 2014
Gap between Math and MechanicsGap between Math and MechanicsNonlinear/Global Optimization Problem: min f (x) s.t. g(x) ≤ 0
f(x) is an “objective” functiong(x) is a general constraint.
(naive) questions: What is the objective function? target and cost? what is Lagrangian? …
“Mathematics is a part of physics. …In the middle of the twentieth century it was attempted to divide physics and mathematics. The consequences turned out to be catastrophic." — V.I. Arnold (1997)
Mathematics needs to remarry physics – A. JaffeGao-Ogden-Ratiu, Springer
Duality in mathematics is not a theorem, but a “principle” – Sir M.F. Atiyah
Duality gap is not allowed in mathematical physics!
Canonical Duality-Triality TheoryCanonical Duality-Triality Theory
A methodological theory comprises mainly
1. Canonical dual transformation
2. Complementary-Dual Principle
3. Triality Theory
Unified Modeling
Unified Solution
Identify both global and local extrema Design powerful algorithms
Unified understanding complexities
Gao-Strang, 1989 MIT and Gao, 1991 Harvard
x
PPd
min = max
max = max
min = min
Nothing is too wonderful to be true, if it be consistent with the laws of nature Michael Farady (1860 AC)
Philosophical FoundationPhilosophical Foundation I-Ching ( 2800 BC-2737 BC): The fundamental Law of Nature is the Dao : the complementarity of one yin (Ying) and one Yang
Canonical System = { (Ying, Yang) | H-Chi } = { ( X , X* ) | A }
Laozi: All things have the receptivity of the yin and the activity of the yang. Through union with the life-giving force (chi) they blend in harmony
Everything = {( Yin, Yang) ; Chi }
= { (subj. , obj.) ; verb }
Convex Canonical System: Unified ModelingConvex Canonical System: Unified Modeling
(P): min P(x) = W(Dx) - F( x)
s.t. x Xc= { xXa | Dx Ya}
W( y ) : Objective function (Gao, 2000): W( Q y ) = W( y ) QT = Q -1, det Q = 1
Exam: W(y) = ½ | y |2 , |Q y|2 = yTQTQ y = | y|2 The 2nd duality: y* = ∂ W(y) Constitutive law
F(x) = f T x Subjective function The 1st duality: x*=∂F( x) = f , action-reaction
Convex System
input
f out put
x
Xa x x* = f Xa* ∂F( x)
y* Ya*Ya yD= D D
∂W(y)
Legendre transf. W*( y*) = y y - W(y)
(Pd ): max Pd(y*) = - W*( y*) s.t. Dy* = f
Lagrangian: L(x, y*) = (Dx) y* - W*( y*) - f T x = xDy* - f ) - W*( y*)
xP
Pd
min P(x) = max Pd(y*)
frame-indifference
Objectivity, Gao 2000Objectivity, Gao 2000Objectivity is not a hypothesis, but a principle. Objectivity is not a hypothesis, but a principle. P.G. Ciarlet, P.G. Ciarlet, Nonlinear Functional AnalysNonlinear Functional Analysis, 2013, SIAMis, 2013, SIAM
Manufacturing Company System Manufacturing Company System
Workers y
CompanyProducts x
D D
Price x*
Salary y*
(P): min P(x) = W(Dx) – F( x )
cost incomeTarget(Lose)
Xa
Ya
Xa*
Ya*
F( x ) = xT x*
Unified Understanding Constraints (Unified Understanding Constraints (Gao, 1997Gao, 1997))(P): min P(x) = W(Dx) – U( x) W(y) : Ya = { y Y | g(y) ≥ 0 } physically feasible U(x) : Xa = { x X | Bx ≤ 0 } geometrically feasible
Boundary (external) constraints in Xa
external KKT conditions
0 ≥ Bx = u ┴ u* = B*x* ≥ 0 Constitutive (objective) constraints in Ya
internal KKT conditions
0 ≤ g(y) = ┴ * = g*(y*) ≤ 0
Xa
Ya*
Xa*
Ya
D=D
Dmn
( y; y*
(x, x*
0 ≤ * ≤ 0
g(y)
┴ *
g*(y*)
0 ≥ u u* ≥ 0
uBx
u ┴ u*
uB*x*
W (y) = {
∂W constitutive law and KKT conditions
W(y) if g(y) ≥ 0 ∞ otherwise
Indicator ( J-J Moreau, 1963)
(P): min P(x) = W(Dx) – U( x) , x X = Obj. – Subj.
Math = { ( X, X* ) ; A}
Canonical Duality - Triality TheoryCanonical Duality - Triality Theory(P): min P(x) = W(Dx) – x T f
Legendre Trans: V*( T–V
Canonical Dual: Pd() x - ½ f T G -1f - V*( 2. Complemenary-Dual Principle:
3. Triality Theory:
If c S - , then either P(xc ) = max P(x) = max Pd() = Pd(c ) (Gao, 1996) or P(xc ) = min P(x) = min Pd() = Pd(c )
Total complementary function (Gao-Strang, 1989) (x, ) = x) T - V*() – x T f
Gap function
1. Canonical transf.
Let S+ = {G 0 }
If c is a critical point of Pd(), then xc = G c -1f is a critical solution of (P) and P(xc ) =Pdc
G-Strang (1989) If c S+, then P(xc ) = min P(x) = max Pd() = Pd(c )
choose an objective measure =x) W(D x) = V((x)) convex in canonical dual eqn (one-to-one): ∂ V ( )
(Quadratic ) = ½ x T G()x - V*() – x T f ∂xAnalytic solution: x = G -1f
S- = {G < 0 }
D
y*y
D*
x x*
* = ∂V
y
y∂W
*
t*
Nonconvex W(y)
Example: Nonconvex in RExample: Nonconvex in Rn Convex in Convex in RR1
P(x) = W(Dx) – F(x) = ½( ½ |x|2 - 1 )2 – x T f
Complementary-Dual Principle: Analytic solutions: xk = (k) -1 f P(xk ) = Pd(k ) k =1,2,3
x
PPd =½ |x|2 V() = ½ ( – 1 )2
Pd() = -½ | f |½ | f | 2
-1- ½ 2 -
∂Pd() = 0 2 (+ 1) = ½ | f |2
3 ≤ 2 ≤ 0 ≤ 1
f
n=2: Mexican hat
y
W(y) = ½ ( ½ y 2 - 1)2
Triality Theory: P(x1 ) = Pd(1 ) P(x2 ) = Pd(2 ) P(x3 ) = Pd(3) Open Problem (2003): If dim x ≠ dim P(x2 ) = min P(x) ≠ min Pd(= Pd(2 )
n=1: double-well = ∂ V() = - 1
Solved in 2012 f = 0 Multiple solution x
P
4
Pd
Perturbation: f ≠ 0 Unique solution
Buridan’s donkey
Quadratic Boolean ProgrammingQuadratic Boolean Programming(P): min P(x) = ½ xTAx – f T x s.t. x {-1,1}n
(Pd): max Pd() = - ½ f T [G( ) ]-1 f – i s.t. S + = { Rn | ≥ 0, G( ) 0 }
Canonical transformation: i = x i 2 – 1 ≤ 0
(x, P(x) + ixi 2 - 1 ) = ½ x TG() x - i-
f T x G ()A+2 Diag (
i ≠ 0 xi2 =1 integer!
KKT: i ≥ 0 , i = xi2 - 1 ≤ 0, ( xi
2 - 1 ) i = 0
Thm (Gao,2007): For each critical point c ≠ 0 ,
the vector xc = G -1(c) f {-1,1}n is a KKT point of P(x) and P(xc ) = Pd(c )
if G(c) 0 P(xc )= min P(x ) = max Pd ( ) =Pd (c ) if G(c) 0 P(xc )= min P(x ) = min Pd ( ) =Pd (c )
x(x, ) = 0 x = G() -1 f
P(x)
Pd()
(P) Could be NP-Hard if Pd ( ) has no critical point in S +
minP(x)= max
min = min
Results for Max-Cut Problem (NP-Complete) Wang-Fang-Gao-Xing (2012) J. Global Optimization
Comparison of the running time produced by the canonical dual approach and GW’s approach (Goemans and Williamson)
max P(x) = ½ xTAx s.t x {0,1}n
– f T x linear perturbation
(Pd): max Pd() = - ½ f T [G( ) ]-1 f – i s.t. G( ) ≥ 0
Max -Cut Problem (contin.)
■ Randomly produce 50 instances on graphs of sizes 20,50, 100, 150,200 and 500. The weight of each edge is uniformly from [0,10]
■ Ave ratio is the average approximate ratio, the ratio is close to 1 when the dimension increases
The 2The 2ndnd Canonical Dual for Integer Programming Canonical Dual for Integer Programming(P): min P(x) = ½ xTAx – f T x s.t. x {-1,1}n
The second canonical dual (Gao, 2009)(Pg): min Pg() = - ½ T A-1– fi - i | s.t Rn
Thm: If cis a solution of ( Pg ) , then
xc i = {
is a feasible solution of (P) and P(xc ) = Pg(c ) .
1 if fi > c i -1 if fi < c i
Nonconvex/nonsmooth minimization DIRECT method (Deterministic )
If A 0, P(xc )= minP(x)= maxPg( )= Pg(c )
If A 0, P(xc )= min P(x)= min Pg( ) = Pg(c )
P(x)
Pg( )
P(x)
Pg( )
If A = - B T B , B Rm n , Pg() = ½ T– fi - Bjij | m < n
n.m
General MINLP ProblemsGeneral MINLP Problems
(P): min P(x,y ) = W(x,y) + aT x – bT y , x Xa , y Ya
s.t. C1 x + C2 y ≤ c , D1 x + D2 y = d , Xa = {x Rn | 0 x u }, Ya = { y Zm | 0 y v } Let z = (x, y) , assume W(z ) is objective such that an objective measure =z ) and a convex V() W(z ) = V((z )) Canonical form: min P(z ) = V((z )) – f T z s.t. z Za
Mixed Integer (fixed Cost) ProblemMixed Integer (fixed Cost) Problem(with H.D. Sherali and N. Ruan)(with H.D. Sherali and N. Ruan)
(P): min P(x,y) = ½ xTA x + cT x – f T y s.t. -y ≤ x ≤ y, y { 0 , 1 }n
(Pd): maxPd() = - ½ cTG()-1c - ½ i fi )+ s.t. ≥ 0 , G() = A +2 Diag ( ) p.d.
Thm: If cis a solution of (Pd ) , then
xc = - G (c)-1 c ,
yci = {
is a global solution of (P) and P(xc , yc ) = Pd(c )
1 if fi < c i 0 if fi > c i
Applications to scheduling and decision science x Rd x n
Problems that can be solvedProblems that can be solved
Benchmark Problems:
1. Rosenbrock function
2. Lennard-Jones potential minimization
3. Three Hump Camel Back Problem
4. Goldstein-Price Problem
5. 2n order polynomials minimizations
6. Canonical functions … New math– Nonlinear space
Nonconvex constrained problemsNonconvex constrained problems
(P): min P(x) = || y – z || 2 s.t. h(y) = ½ y A y – r ellipsoid g (z) = ½ ( || z – c || 2 - b )2 – d t ( z - c)
Thm: If G ( ) 0 , (Pd) has at least one critical solution which gives to a global optimal solution to (P).
Lagrangian: x = ( y, z ) R2n L(x,) = || y – z || 2 + h(y) + g(z)
Let = z ) = || z – c || 2 , V() = ½ (b 2
= ∂ V() = b , V*(V() =½ 2
bTotal complementary function (x,) = || y – z || 2 + h(y) +(z ) - V() – d t ( z - c) ]
(Pd): Pd() = minx(x,) = - ½ F T G () -1 F - V()
0
yz
G () =
Challenges Challenges Super-Duality Super-DualitySince 2010, Zalinescu (+ 2) has wrote 11 papers + 1 letter challenging
the Canonical Duality Theory, which can be grouped in three categories: 1. Conceptual Duality (4 papers, two published and two rejected)
• min P(x) = V((x)) – F(x)
F (x) external energy (must be linear function)∂F(x) = x* = f
V() internal (stored) energy (must be objective ) ∂V() =
2. Moral Duality (6 papers) all on the same open problem left in 2003:
If dim P ≠ dim Pd min P(x) ≠ minPd( S- 3. Multi-scale duality (1 paper): Locally correct but globally wrong
Certain condition in S+ is missing Total complementary function (x,) , x = ( y , z ) R2n
0
yz
““Counter-Example” Counter-Example” Hidden truthHidden truth
Conclusion: The consideration of the Gao-Strang function (x,) is useless, at least for the problem studied in [3]. Morales-Gao (2012): linear perturbation (x,) – k -1 xT f
Mixed Integer Optim. Supply Chain Process
Nonconvex/nonsmoothVariational/V.I. Analysis
Graph, lattice, fuzzy max-plus algebra
FEM, FDM, FVM, SDP Meshless, Wavelet, SIP
Discrete optimization
Continuous Optimization
Unified Global Optimization
Combinatorial Algebra
Numerical Analysis
Combinatorial Optim. Integer Programming
Canonical Duality-Triality
Theory
Duality in Nonconvex Systems:Duality in Nonconvex Systems:Theory, Methods and ApplicationTheory, Methods and Application
David Yang GaoDavid Yang GaoKluwer Academic Publishers, 2000, 454pp
Part I Symmetry in Convex Systems1. Mono-duality in static systems2. Bi-duality in dynamical systems
Part II Symmetry Breaking: Triality Theory in Nonconvex Systems
3. Tri-duality in nonconvex systems4. Multi-duality and classifications of general systems
Part III Duality in Canonical Systems5. Duality in geometrically linear systems6. Duality in finite deformation systems7. Applications, open problems and concluding remarks duality in fluid mechanics ?
All happy families are alike,
Every unhappy family is unhappy in its own way
Anna Karenina --- Leo N Tolstoy
Reason: canonical duality
Reason: different duality gaps
Philosophy = Love of Canonical Duality
Proof: 1. By Greeks: Philosophy = Love of Wisdom
2. By Confucius: The highest Wisdom = Dao
3. By I-Ching (4000BC): Dao = one Ying + one Yang = Canonical Duality ---
Open Problem:
How to correctly understand the Triality
Canonical Duality –Triality Theory:
1. Non-convex concave
2. Discrete continuous
5. Diff. eqn Algebraic eqn.
4. Rescaling: Rn Rm Rr
n > m > r
7. Challenges Breakthrough
6. Non-deterministic deterministic
Rn
Rm
Rn
Rm
Rm n
x x*
yy*
yf
Rr Rr
oxxo
Rmr
x= 0
Open Problems: (P) is NP-Hard if (Pd) has no solution in Sa
+ ?
u
3. Non-smooth smooth
The The 44thth World Congress on Global Optimization World Congress on Global Optimization Gainesville, Florida - USA, Feb 22-25, 2015Gainesville, Florida - USA, Feb 22-25, 2015
Thanks! Thanks!
Some referencesSome references
[1] Gao, D.Y. and Sherali, H.D. (2008).
Canonical duality: Connection between
nonconvex mechanics and global optimization,
in Advances in Appl. Mathematics and Global Optimization, 249-316, Springer, 2008
[2] Gao, D.Y. (2009). Canonical duality theory: Unified understanding and generalized solution for global optimization problems,
Computers & Chemical Engineering, 33:1964–1972
[3] Daniel Morales-Silva, David Gao On the minimal distance between two surfaces, http://arxiv.org/abs/1210.1618
[4] Gao, DY and Wu, C, On the Triality Theory in Global Optimization
http://arxiv.org/abs/1104.2970
The The 44thth World Congress on Global Optimization World Congress on Global Optimization Gainesville, Florida - USA, Feb 22-25, 2015Gainesville, Florida - USA, Feb 22-25, 2015
Thanks! Thanks!
International Society of Global OptimizationInternational Society of Global Optimization (www.iSoGOp.org)(www.iSoGOp.org)
WCGO 2015WCGO 2015