Linear Programming 2010 1 (Convex) Cones Def: closed under nonnegative linear combinations, i.e. K...

Linear Programming 2010 1

(Convex) Cones

Def: closed under nonnegative linear combinations, i.e. K is a cone provided

a1, …, ap K Rn, 1, …, p 0 i = 1p i ai K

( Note: Usually cones are defined as closed only under nonnegative scalar multiplication)

Observations (Characteristics)Subspaces are cones

For any family of cones { Ki : i I }, i I Ki is a cone also.

Any nonempty cone contains 0

K1, K2 are cones, then so is K1+K2 = { (x+y): x K1, y K2 }

Halfspaces are cones: H = { x Rn : a’x 0 }


Description of cones:

Any subset A Rn generates a cone K(A) (define K() = {0} )

K(A) = { 1a1 + … + pap : p 1, i R+ , ai A},

called “conical span of A”

“conical hull of A” = Ki A, Ki is cone Ki (outside description)

They are the same.

Finite basis result is false for cones (e.g. ice cream cones).

Hence we will restrict our attention to the cones with finite conical basis.


Conical dual:

For any A Rn, define the (conical) dual of A to be

A+ = { x Rn : Ax 0 }, where Ax 0 means a’x 0 for all a A.

( some people use Ax 0 )

It is called a constrained cone since it is the solution set of some homogeneous inequalities.

When A is a cone, A+ is defined as dual cone (or polar cone) of A.

Note that A+ is always a cone – a constrained cone.

For A: m n, A+ (with rows of A regarded as the vectors in the set A) is finitely constrained (polyhedron).


(conical) dual of ARn

a1

a2

a3

A={a1, a2, a3}

{x: a1’x 0}

{x: a3’x 0}A+={ x: Ax0}


Prop: Suppose A, B Rn. Then

(1) B A A+ B+

(2) A A++

(3) A+ = A+++

(4) A = A++ A is a constrained cone

(5) If B A and B generates A conically, then A+ = B+.

Pf) parallels the cases for subspaces.


A++

A A++

a1

a2

a3

A={a1, a2, a3}

A+={ x: Ax0}


A++

A+ = A+++

a1

a2

a3

A={a1, a2, a3}

A+={ x: Ax0}A+++={ x: A++ x 0}


A

A = A++ A is a constrained cone

A = constrained cone

A+={ x: Ax0}

A++ = A


Thm (Weyl): Any nonempty finitely generated cone is polyhedral (finitely constrained).

Pf) Use Fourier-Motzkin elimination (later).

Cor 1: Among all subsets A Rn with finite conical basis

A = A++ A is a nonempty cone.

Cor 2: Given A: mn, consider K = { y’A: y 0}, L = { x: Ax 0}.

Then K+ = L, L+ = K.


A++

K+ = L, L+ = K

a1

a2

a3

A={a1, a2, a3}

A+={ x: Ax0}


Cor 3 (Farkas’ lemma):

Given A: mn, c Rn, exactly one of the two holds

(I) there exists y R+m s.t. y’A = c’

(II) there exists x Rn s.t. Ax 0, c’x > 0.

Pf) Show ~ (I) (II)

~ (I) c K { y’A : y 0}

c K++ (by Cor 1)

x K+ (Ax 0) s.t. cx > 0

(II) holds


K

Farkas’ Lemma

a1

a2

a3

K+={ x: Ax0}

A : m n, cRn

Case (1): c K ( y0 such that y’A=c’) c


K

Farkas’ Lemma

a1

a2

a3

K+={ x: Ax0}

Case (2): c K ( x such that Ax0, c’x>0)

c

c


Farkas’ lemma is core of LP duality theory (details later). There are many other forms of theorems of the alternatives and they are important and powerful tools in optimization theory.

ex)verifying that c* (c’x* = c* for some x*) is an optimal value of a LP (in

minimization form) is the same as to verify that the following system has no solution.

-c’x + c* > 0

Ax – b 0

Truth of claim can be verified by giving a solution to the alternative system.

question: similar result possible for integer form?Finding projection of a polyhedron to a lower dimensional space (later)absence of arbitrage condition in finance theory. KKT optimality condition for nonlinear program...


Absence of Arbitrage text Chapter 4, p167-169

Text use the form

(I) there exists some x 0 such that Ax = b

(II) there exists some vector p such that p’A 0’ and p’b < 0

(here, columns of A are generators of a cone)

Compare with

(I) there exists y R+m s.t. y’A = c’

(II) there exists x Rm s.t. Ax 0, c’x > 0.


n different assets are traded in a market (single period)

m possible states after the end of the period

rsi : return on investment of 1 dollar on asset i and the state is s at the

end of the period

payoff matrix R: mn

mnm

n

rr

rr

R

1

111

xi : amount held of asset i

xi can be negative

xi > 0: has bought xi units of asset i, receive rsixi if state s occurs

xi < 0: “short position”, selling |xi| units of asset i at the beginning,

with the promise to buy them back at the end.

(seller’s position in futures contract, payout rsi|xi|, i.e.

receiving a payoff of rsixi if state s occurs)


Given a portfolio x, the resulting wealth when state s occurs is,

ws = i = 1n rsixi w = Rx

Let pi be the price of asset i in the beginning, then cost of acquiring

portfolio x is p’x. What are the fair prices for the assets?

Absence of arbitrage condition: asset prices should always be such that no investor can get a guaranteed nonnegative payoff out of a negative investment (no free lunch)

Hence, if Rx 0, then we must have p’x 0, i.e. there exists no vector x such that x’R’ 0’, x’p < 0.

So, by Farkas’ lemma, there exists q 0 such that R’q = p, i.e.

pi = s = 1m qsrsi

( Here, R’ = A, p = b in Farkas’ lemma )


Fourier-Motzkin Elimination Solving system of inequalities (refer text section 2.8)

Idea similar to Gaussian elimination.

Eliminate one variable at a time with some mechanism reserved to recover the feasible values later.

Given a system of inequalities and equations:

n

j ijij mibxa1 ,...,1, (I)

eliminate xn in (I) and obtain system (II) which consists of linear

equations and inequalities now in variables x1, x2, … , xn-1 .

And we want to have (x1, x2, … , xn ) satisfies (I) for some xn (x1, x2, … ,

xn-1 ) satisfies (II). (i.e. we want (II) which does not miss any feasible

solution to (I) and does not include any vector not satisfying (I) )

Then (I) consistent (II) consistent (have a solution).


Related concept: projection of vectors to a lower dimensional space

Def: If x = ( x1, x2, … , xn) Rn and k n, the projection mapping k :

Rn Rk is defined as k(x) = k(x1, x2, … , xn) = ( x1, … , xk)

For S Rn, k(S) = { k(x) : x S}

Equivalently, k(S) = {(x1,…, xk): xk+1, …, xn s.t. (x1, … , xn)S}

To determine whether a polyhedron P is nonempty, find n-1(P) …

1(P) and determine whether the one dimensional polyhedron is

nonempty. (But it is inefficient)


Elimination algorithm:

(0) If all coefficients of xn in (I) are 0, then take (II) same as (I).

(1) some relation, say i-th, with ain 0, and this relation is ‘=‘. Then

derive (II) from (I) by Gauss-Jordan elimination.

( ai1x1 + … + ainxn = bi

xn = 1/ain( bi - ai1x1 - … - ainxn), substitute into (I).

Clearly ( x1, … , xn) solves (I) ( x1, … , xn-1) solves (II). )

(continued)


(continued)

(2) Rewrite each constraint j=1n aijxj bi as

ainxn - j=1n-1 aijxj +bi, i = 1, … , m

If ain0, divide both sides by ain.

By letting x = (x1, …, xn-1), we obtain

xn di + fi’x , if ain > 0

dj + fj’x xn , if ajn < 0

0 dk + fk’x , if akn = 0

, where di, dj, dk R and fi, fj, fk Rn-1

Let (II) be the system defined by

dj + fj’x di + fi’x , if ain > 0 and ajn < 0

0 dk + fk’x , if akn = 0

( and remaining equations )


Ex) x1 + x2 1

x1 + x2 + 2x3 2

2x1 + 3x3 3

x1 - 4x3 4

-2x1 + x2 - x3 5

0 1 - x1 - x2

x3 1 – (x1/2) – (x2/2)

x3 1 – (2x1/3)

-1 + (x1/4) x3

-5 – 2x1 + x2 x3

0 1 - x1 - x2

-1 + (x1/4) 1 – (x1/2) – (x2/2)

-1 + (x1/4) 1 – (2x1/3)

-5 – 2x1 + x2 1 – (x1/2) – (x2/2)

-5 – 2x1 + x2 1 – (2x1/3)


Thm 2.10: The polyhedron Q (defined by system (II)) constructed by the elimination algorithm is equal to n-1(P) of P.

Pf) If x n-1(P), xn such that (x, xn) P. In particular, x= (x, xn)

satisfies system (I), hence also satisfies system (II). It shows that n-

1(P) Q.

Let x Q. Then x satisfies

min{ j : ajn < 0} (dj + fj’x) max{ i : ain > 0} (di + fi’x).

Let xn be a number between the two sides of the above inequality.

Then (x, xn) P, which shows Q n-1(P).

Observe that for x = (x1, …, xn), we have n-2(n-1(x)) = n-2(x).

Also n-2(n-1(P)) = n-2(P). Hence obtain 1(P) recursively to

determine if P is empty or to find a solution.

A solution in P can be recovered recursively starting from 1(P) and

finding xi that lies in [ min{ j : ajn < 0} (dj + fj’x), max{ i : ain > 0} (di + fi’x) ].


Cor 2.4: Let P Rn+k be a polyhedron. Then, the set { x Rn : there exists y Rk such that (x, y) P} is also a polyhedron.

( Will be used to prove the Weyl’s Theorem. Other proof technique is not apparent.)

Cor 2.5: Let P Rn be a polyhedron and A be an mn matrix. Then the set Q = { Ax: x P} is also a polyhedron.

Pf) Q = { y Rm : y = Ax, x P}. Hence Q is the projection of the polyhedron {(x, y) Rn+m : y = Ax, x P} onto the y coordinates.

Cor 2.6: The convex hull of a finite number of vectors (called polytope) is a polyhedron.

Pf) The convex hull { i=1k ixi : i i =1, i 0} is the image of the

polyhedron { ( 1, … , k) : i i = 1, i 0} under the mapping that maps

( 1, …, k) to i ixi. (The mapping can be expressed as A, where the

columns of the matrix A are xi vectors. We will see a different proof later.)


Remarks

FM elimination not efficient as an algorithm. Number of inequalities grows exponentially as we eliminate variables.

Can also handle strict inequalities.

Can solve LP problem max{ c’x : Ax b} Consider Ax b, z = c’x and eliminate x and find z as large as possible in the one dimensional polyhedron. Solution can be recovered by backtracking.

FM gives an algorithm to find the projection of P = { (x, y) Rn+p : Ax + Gy b} onto the x space Prx(P) = { x Rn : (x, y) P for some y

Rp}.

But how can we characterize Prx(P) for arbitrary P?


Concept of projection becomes important in recent optimization theory (especially in integer programming) as new techniques using projections have been developed.

(e.g. RLT (reformulation and linearization technique))

Formulation in a higher dimensional space and use the projection to lower dimensional space may give stronger formulation in integer programming (in terms of strength of LP relaxation).

(e.g. Node + edge variable formulation stronger than edge formulation for weighted maximal b-clique problem)

: Given a complete undirected graph G = (V, E), weight ce, e E.

Find a clique (complete subgraph) of size (number of nodes in the clique) at most b and sum of edge weights in the clique is maximum.


Weyl’s Theorem:

Any nonempty finitely generated cone is polyhedral (i.e. finitely constrained).

Pf) K = { y’A: y 0} for A: m n

= { x : x – y’A = 0, y 0 is a consistent system in (x, y)}

Use FM elimination to get rid of y’s

= ( x : some linear homog. system in x is consistent}

Write these relation as Bx 0

Then K = { x: Bx 0} -- polyhedral

Note that we get homogeneous system Bx 0 if we apply FM.


Minkowski’s Theorem:

Any polyhedral cone is nonempty and finitely generated.

Pf) Let L be a polyhedral cone. Clearly L .

We know that L = L++ from earlier Prop.

Part 2 of Cor. 2 says L+ is finitely generated ( L+ = K )

By Weyl’s Thm, L+ itself is polyhedral.

By part 2 of Cor. 2 (L+)+ is finitely generated.

L++ = L from above L is finitely generated.

FM elimination leads to Weyl-Minkowski cone representation.

(nonempty finitely generated cone finitely constrained

(polyhedral)) Extend this result to affine version Def: The set of all convex combinations of a finite point set is called

a polytope.


Affine Weyl Theorem: (i.e. finitely generated are finitely constrained)

Suppose P = { x Rn: x’ = y’B + z’C, y 0, z 0, i zi = 1 },

B: p n, C: q n.

Then matrix A: m n and b Rm s.t. P = { x Rn: Ax b}.

(special case where B is vacuous is that polytope is polyhed.)

Pf) (use technique called homogenization)

If P = (i.e. B, C vacuous, i.e. p = q = 0), take A =[0,…,0], b = -1.

If P , consider P’ Rn+1 defined as

}0,,

1

1

0

0

)','(),(:),{(' 11

1

zy

C

B

zyxxRxxP nn

n

Observe that xP (x, 1)P’


and P’ is finitely generated nonempty cone in Rn+1.

Apply Weyl’s Thm to P’ in Rn+1 to get

P’ = { (x, xn+1) : A’(x, xn+1)’ 0} for some A’: m (n+1)

i.e. A’ = [ A: d] with d = last column of A’

Define b = -d, then A’ = [ A: -b]

Observe that x P (x, 1) P’ A’(x, 1)’ 0

(A: -b)(x, 1)’ 0

Ax b

Note that we changed the problem as the problem involving a cone in Rn+1, for which we know more properties, and used the results for cones to prove the theorem.


Affine Minkowski Theorem: (i.e. finitely constrained are finitely generated)

Suppose P = { x Rn : Ax b}, A: m n, b Rm.

Then matrices, B: p n, C: q n such that

P = { x Rn: x’ = y’B + z’C, y, z 0, i zi = 1}

Pf) For P = , take p = q = 0, i.e. B, C vacuous.

Otherwise, again homogenize and consider

100

'},0':),{('1

1 bA

Awherex

xAxxP

nn

Then x P (x, 1) P’

P’ is a polyhedral cone, so Minkowski’s Thm applies.

Hence matrices, B’: l (n+1) such that

P’ = { (x, xn+1) Rn+1 : (x, xn+1)’ = y’B’ , y R+l }


(continued)

Break B’ into 2 parts so that all rows of B’ with 0 last component come as top rows and rows with nonzero last component come as bottom rows. Note that all nonzero values in the last column of B’ must be > 0.

.)0'0

0'( 1

1

nn

xx

xA

C

BB implies 0 P'. in be must ofrow each

It doesn’t change P’. Then we have x P (x, 1) P’

.0

0'

C

BB Hence .

1

0'

C

BB get to B' of rows Scale

}0,,1

0)','(),'(:),{(' 1

11

zy

C

BzyxxRxxP n

nn

i.e. x P x = y’B + z’C, y 0, z 0, i zi = 1


Geometric view of homogenization in Affine Minkowski Thm

0

1

Rn

R

P={x: Axb}

P’Rn+1


Think about similar picture for affine Weyl.

Affine Weyl, Minkowski Thm together provides “Double Description Thm”

We can describe polyhedron as(finite) intersection of halfspaces

(finite) conical combination of points + convex combination of points ( i.e. P = C + Q, where C is a cone and Q is a polytope).

Existence of different representations has been shown. Next question is how to identify the representation.

Linear Programming 2010 1 (Convex) Cones Def: closed under nonnegative linear combinations, i.e. K...

Documents

Transcript of Linear Programming 2010 1 (Convex) Cones Def: closed under nonnegative linear combinations, i.e. K...