The moment-LP and moment-SOS approaches in optimization€¦ · II. Let K ˆRn and S ˆK be given,...

The moment-LP and moment-SOSapproaches in optimization

Jean B. Lasserre

LAAS-CNRS and Institute of Mathematics, Toulouse, France

Master 2 Optimization: Paris-Orsay, 2016

Jean B. Lasserre semidefinite characterization

SOS-based certificate

K = {x : gj(x) ≥ 0, j = 1, . . . ,m }

Theorem (Putinar’s Positivstellensatz)If K is compact (+ a technical Archimedean assumption) andf > 0 on K then:

† f (x) = σ0(x) +m∑

σj(x) gj(x), ∀x ∈ Rn,

for some SOS polynomials (σj) ⊂ R[x].

LP-based certificate

K = {x : gj(x) ≥ 0; (1− gj(x)) ≥ 0, j = 1, . . . ,m}

Theorem (Krivine-Vasilescu-Handelman’s Positivstellensatz)

Let K be compact and the family {1,gj} generate R[x]. If f > 0on K then:

(?) f (x) =∑α,β

cαβm∏

gj(x)αj (1− gj(x))βj , , ∀x ∈ Rn,

for some NONNEGATIVE scalars (cαβ).

Dual side of Putinar’s theorem: The K -momentproblem

Given a real sequence y = (yα), α ∈ Nn, determine wherethere exists some finite Borel measure µ on K such that

† yα =

xα dµ, ∀α ∈ Nn.

TheoremIf K = {x : gj(x) ≥ 0, j = 1, . . . ,m} is compact (+Archimedeanassumption) then † holds if and only if

(?) Ly

(h2 gj)

)≥ 0, ∀h ∈ R[x]d .

The condition (?) is equivalent to countably many LINEARMATRIX INEQUALITIES on the yα’s

† yα =

(?) Ly

(h2 gj)

)≥ 0, ∀h ∈ R[x]d .

† yα =

(?) Ly

(h2 gj)

)≥ 0, ∀h ∈ R[x]d .

Dual side of Krivine’s theorem: The K -momentproblem

TheoremIf K = {x : gj(x) ≥ 0, j = 1, . . . ,m} is compact, 0 ≤ gj ≤ 1 on K,and {1,gj} generates R[x], then † holds if and only if

(??) Ly

m∏j=1

gjαj (1− gj

≥ 0, ∀α, β ∈ Nm.

The condition (??) is equivalent to countably many LINEARINEQUALITIES on the yα’s

Dual side of Krivine’s theorem: The K -momentproblem

TheoremIf K = {x : gj(x) ≥ 0, j = 1, . . . ,m} is compact, 0 ≤ gj ≤ 1 on K,and {1,gj} generates R[x], then † holds if and only if

(??) Ly

m∏j=1

gjαj (1− gj

≥ 0, ∀α, β ∈ Nm.

The condition (??) is equivalent to countably many LINEARINEQUALITIES on the yα’s

The Generalized problem of Moments

• Polynomials NONNEGATIVE ON A SET K ⊂ Rn areubiquitous. They also appear in many important applications(outside optimization),

. . . modeled asparticular instances of the so called

Generalized Moment Problem, among which:Probability, Optimal and Robust Control, Game theory, Signal

processing, multivariate integration, etc.

The Generalized Moment Problem (GMP) is theinfinite-dimensional linear program

(GMP) : infµi∈M(Ki )

fi dµi :s∑

hij dµi≥= bj , j ∈ J}

with M(Ki) space of Borel measures on Ki ⊂ Rni , i = 1, . . . , s.

The DUAL of the GMP is the linear program GMP∗:

supλj

λj bj : fi −∑j∈J

λj hij ≥ 0 on Ki , i = 1, . . . , s }

And one can see that ...the constraints of GMP∗ state that

some functions fi −∑

j∈J λj hij

must be nonnegative on a certain set Ki , i = 1, . . . , s.

� The GPM has great modelling power, in various fields.Global Optimization (continuous, discrete), Control (Robust andoptimal control), Nonlinear Equations, Probability and Statistics,Performance Evaluation (in e.g. Mathematical finance, Markovchains), Inverse Problems (cristallography, tomography),Numerical multivariate Integration, etc ...

A couple of examples

I: Global OPTIM → f ∗ = infx{ f (x) : x ∈ K }

is the SIMPLEST example of the GMP

because ...

f ∗ = infµ∈M(K)

Kf dµ :

1 dµ = 1}

• Indeed if f (x) ≥ f ∗ for all x ∈ K and µ is a probability measureon K, then

∫K f dµ ≥

∫f ∗ dµ = f ∗.

• On the other hand, for every x ∈ K the probability measureµ := δx is such that

∫f dµ = f (x).

because ...

Kf dµ :

1 dµ = 1}

∫K f dµ ≥

∫f ∗ dµ = f ∗.

∫f dµ = f (x).

because ...

Kf dµ :

1 dµ = 1}

∫K f dµ ≥

∫f ∗ dµ = f ∗.

∫f dµ = f (x).

because ...

Kf dµ :

1 dµ = 1}

∫K f dµ ≥

∫f ∗ dµ = f ∗.

∫f dµ = f (x).

II. Let K ⊂ Rn and S ⊂ K be given, and let Γ ⊂ Nn be alsogiven.

BOUNDS on measures with moment conditions

maxµ∈M(K)

{ 〈1S, µ〉 :

xα dµ = mα, α ∈ Γ }

to compute an upper bound on µ(S) over all distributionsµ ∈ M(K) with a certain fixed number of moments mα.

• If Γ = Nn then one may use this to compute the Lebesguevolume of a compact basic semi-algebraic setS ⊂ K := [−1,1]n.

Take mα :=

∫[−1,1]n

xα dx, α ∈ Nn.

II. Let K ⊂ Rn and S ⊂ K be given, and let Γ ⊂ Nn be alsogiven.

BOUNDS on measures with moment conditions

maxµ∈M(K)

{ 〈1S, µ〉 :

xα dµ = mα, α ∈ Γ }

to compute an upper bound on µ(S) over all distributionsµ ∈ M(K) with a certain fixed number of moments mα.

• If Γ = Nn then one may use this to compute the Lebesguevolume of a compact basic semi-algebraic setS ⊂ K := [−1,1]n.

Take mα :=

∫[−1,1]n

xα dx, α ∈ Nn.

III. For instance, one may also want:• To approximate sets defined with QUANTIFIERS, like .e.g.,

Rf := {x ∈ B : f (x , y) ≤ 0 for all y such that (x , y) ∈ K}

Df := {x ∈ B : f (x , y) ≤ 0 for some y such that (x , y) ∈ K}

where f ∈ R[x , y ], B is a simple set (box, ellipsoid).

• To compute convex polynomial underestimators p ≤ f of apolynomial f on a box B ⊂ Rn. (Very useful in MINLP.)

III. For instance, one may also want:• To approximate sets defined with QUANTIFIERS, like .e.g.,

Rf := {x ∈ B : f (x , y) ≤ 0 for all y such that (x , y) ∈ K}

Df := {x ∈ B : f (x , y) ≤ 0 for some y such that (x , y) ∈ K}

where f ∈ R[x , y ], B is a simple set (box, ellipsoid).

• To compute convex polynomial underestimators p ≤ f of apolynomial f on a box B ⊂ Rn. (Very useful in MINLP.)

The moment-LP and moment-SOS approachesconsist of using a certain type of positivity certificate(Krivine-Vasilescu-Handelman’s or Putinar’s certificate) inpotentially any application where such a characterization isneeded. (Global optimization is only one example.)

In many situations this amounts tosolving a HIERARCHY of :

LINEAR PROGRAMS, orSEMIDEFINITE PROGRAMS

... of increasing size!.

LP- and SDP-hierarchies for optimization

Replace f ∗ = supλ {λ : f (x)− λ ≥ 0 ∀x ∈ K} with:

The SDP-hierarchy indexed by d ∈ N:

f ∗d = sup {λ : f − λ = σ0︸︷︷︸SOS

σj︸︷︷︸SOS

gj ; deg (σj gj) ≤ 2d }

or, the LP-hierarchy indexed by d ∈ N:

θd = sup {λ : f −λ =∑α,β

cαβ︸︷︷︸≥0

m∏j=1

gjαj (1−gj)

βj ; |α+β| ≤ 2d}

LP- and SDP-hierarchies for optimization

Replace f ∗ = supλ {λ : f (x)− λ ≥ 0 ∀x ∈ K} with:

The SDP-hierarchy indexed by d ∈ N:

f ∗d = sup {λ : f − λ = σ0︸︷︷︸SOS

σj︸︷︷︸SOS

gj ; deg (σj gj) ≤ 2d }

or, the LP-hierarchy indexed by d ∈ N:

θd = sup {λ : f −λ =∑α,β

cαβ︸︷︷︸≥0

m∏j=1

gjαj (1−gj)

βj ; |α+β| ≤ 2d}

TheoremBoth sequence (f ∗d ), and (θd ), d ∈ N, are MONOTONE NONDECREASING and when K is compact (and satisfies atechnical Archimedean assumption) then:

f ∗ = limd→∞

f ∗d = limd→∞

•What makes this approach exciting is that it is at thecrossroads of several disciplines/applications:

Commutative, Non-commutative, and Non-linearALGEBRAReal algebraic geometry, and Functional AnalysisOptimization, Convex AnalysisComputational Complexity in Computer Science,

which BENEFIT from interactions!

• As mentioned ... potential applications are ENDLESS!

•What makes this approach exciting is that it is at thecrossroads of several disciplines/applications:

Commutative, Non-commutative, and Non-linearALGEBRAReal algebraic geometry, and Functional AnalysisOptimization, Convex AnalysisComputational Complexity in Computer Science,

which BENEFIT from interactions!

• As mentioned ... potential applications are ENDLESS!

• Has already been proved useful and successful inapplications with modest problem size, notably in optimization,control, robust control, optimal control, estimation, computervision, etc. (If sparsity then problems of larger size can beaddressed)

• HAS initiated and stimulated new research issues:in Convex Algebraic Geometry (e.g. semidefiniterepresentation of convex sets, algebraic degree ofsemidefinite programming and polynomial optimization)in Computational algebra (e.g., for solving polynomialequations via SDP and Border bases)Computational Complexity where LP- andSDP-HIERARCHIES have become an important tool toanalyze Hardness of Approximation for 0/1 combinatorialproblems (→ links with quantum computing)

• Has already been proved useful and successful inapplications with modest problem size, notably in optimization,control, robust control, optimal control, estimation, computervision, etc. (If sparsity then problems of larger size can beaddressed)

• HAS initiated and stimulated new research issues:in Convex Algebraic Geometry (e.g. semidefiniterepresentation of convex sets, algebraic degree ofsemidefinite programming and polynomial optimization)in Computational algebra (e.g., for solving polynomialequations via SDP and Border bases)Computational Complexity where LP- andSDP-HIERARCHIES have become an important tool toanalyze Hardness of Approximation for 0/1 combinatorialproblems (→ links with quantum computing)

There has been also recents attempts to use other types ofalgebraic certificates of positivity that try to avoid the size

explosion due to the semidefinite matrices associated with theSOS weights in Putinar’s positivity certificate

Recent work by Ahmadi et al. and Lasserre, Toh and Zhang

The moment-LP and moment-SOS approaches in optimization€¦ · II. Let K ˆRn and S ˆK be given,...

Documents

Transcript of The moment-LP and moment-SOS approaches in optimization€¦ · II. Let K ˆRn and S ˆK be given,...