Course:

Advanced Algorithms CSG713, Fall 2008CCIS Department, Northeastern University

Dimitrios Kanoulas

A Randomized Polynomial-Time

Simplex Algorithmfor Linear Programming

by J. Kelner & D. Spielman

Linear Programming

Maximize cxSubject to: Ax ≤ b

Objective Function (c vector)Constraints

where: x and c are two d-size vectors

A is a nxd matrix

b is a n-size right-hand

vectorConvex Polyhedron: Set of feasible points P:={x|Ax ≤ b}where:the vertices can be

defined by the d constraints

Polytope: A bounded Convex Polyhedron

The form of the LP we will analyze:

Simplex Method(Intuition)

Picture from: commons.wikimedia.org

Starting from a vertex

Try to find that vertexthat optimizes the objective function

TIME:(given random input constraints)

• Worst-Case: exponential

Thus:An algorithm that uses the Simplex methodin worst-case polynomial time is needed.(open problem for 50 years)

Solution: Randomization

Opt.

Start

Main Result

Given a Linear Program, we can randomly perturb it, in such a way that:

• We can use (Shadow) Simplex Method to solve this new nearby but different Linear Program

and

• The solution of this new Linear Program to be the solution of the starting one.

and

• It is polynomial in time w.h.p.

A Randomized Polynomial-Time LP Simplex Method

Overview of the Algorithm1. Reduce the starting LP to a problem of certifying

boundedness

Boundedness does not depend on RHS vector b, of the constraints

Loop:2. Perturb randomly b vector of constraints and

run shadow-vertex Simplex Method on perturbed polytope to generate certificate of boundedness

3. If it fails in constant number of steps, change the distribution on perturbations and run the algorithm again.

TIME:The number of iterations of this loop is polynomial on the bit-length of the input, with high probability (w.h.p.)

Overview of the Algorithm1. Reduce the starting LP to a problem of certifying

boundedness

Boundedness does not depend on RHS vector b, of the constraints

The problem of determining whether a set of linear constraints defines an unbounded polyhedron or not.

A polyhedron is said to be boundedIf the solution set is bounded

Maximize cxSubject to: Ax ≤ b

Overview of the Algorithm1. Reduce the starting LP to a problem of certifying

boundedness

Boundedness does not depend on RHS vector b, of the constraints

Loop:2. Perturb randomly b vector of constraints and

run shadow-vertex Simplex Method on perturbed polytope to generate certificate of boundedness

3. If it fails in constant number of steps, change the distribution on perturbations and run the algorithm again.

TIME:The number of iterations of this loop is polynomial on the bit-length of the input, with high probability (w.h.p.)

Shadow of polytopeThe shadow of a polytope P onto a 2-dimensions plane Sis just the projection of P onto S.

The shadow is a polygon

Create shadow:Every vertex/edge of the shadow’s polygon

is the image of

a vertex/edge of the polytope P

Shadow of polytopeThe shadow of a polytope P onto a 2-dimensions plane Sis just the projection of P onto S.

The shadow is a polygon

Idea:Find S such that the set of vertices of P that projects onto the boundary of the shadow polygon,are exactly the vertices of P that optimizethe objective function in S.

Shadow-Vertex-Simplex Method

1. Input: a starting vertex u0 of starting polytope P. 2. Choose some random objective function f optimized at u0.3. Set S = span(c,f).

S is the 2-dimentional subspace which has c and f as bases.4. Find the shadow of P onto S.

Selection of 2-dimensions plane S

S = span(f,c)f

(opt. by u0)

objective function c

Image: BU`s prof. Teng presentation

Shadow-Vertex-Simplex Method

5. Each vertex y on P that projects onto the boundary of the shadow has a unique neighbor on P that projects onto the next vertex of the shadow in clockwise order.

6. Looking all the vertices on P that map to the boundary of the shadow we can move from vertex that optimize f to the vertex that optimize c.

u0

opt. f

uopt. c or -c

Disadvantage of using the Starting LP

Problem:By using the starting LP and the plane S we proposed,we cannot guarantee that:

the number of edges of the shadow will be polynomial in size.[There is the possibility to be exponentially large]

Solution:Guarantee that the number of edges of the shadow’s boundary will be polynomially large by perturbing the b-vector

Shadow Size(Number of shadow's edges)

The real problem is to bound the number of shadow’s edges

2 cases:• Polytope in k-near-isotropic position• Polytope in non k-near-isotropic position

k-near-isotropic position

€

B(0,1) ⊂ P ⊂ B(0,k)

P is the polytope: Ax≤b

k

1

k-near-isotropic position

• Upper bound of total shadow length (Shadow Size).• Lower bound expected length of each edge.

Perturb polytope P:

P := {x | for each i: aix ≤ bi} Q := {x | for each i: aix ≤ bi + ri}

ri: independent exponentially distributed random variable with

expectation λ( Pr[ri≥t] = e-t/l, t≥0 )

Proof of Poly Size of the shadow

Number of edges of the shadow is poly large w.h.p.

Until Now

1. Starting LP:

2. Perturb b vector: guarantee that the number of edges of the shadow of the new polytope will be polynomially large w.h.p.

3. From the shadow find the vertex that optimizes the objective function c in polynomial number of steps, w.h.p.

Maximize cxSubject to: Ax ≤ b

Boundedness Certification ProblemP is the polytope: Ax≤b

Starting LP

Maximize c xSubject to: Ax ≤ b

Boundedness Certification

Problem

B w ≤ b’

B -> A, c, b

Reduction

Dual LP

Minimize b ySubject to: ATy = c

y≥0

CombinePrimal & Dual

Need feasible x and y:Ax ≤ bATy = cy≥0c x = b y

Dual

Boundedness independent of b-vector

Boundedness Certification Problem

Boundedness Certification

Problem

B w ≤ b

Find certificate of boundednessby minimizing and maximizing the objective function (c)

Summary1. Reduce the starting LP to a problem of certifying

boundednessBoundedness does not depend on RHS vector b, of the constraints

2. We choose the correct 2-dimensions plane S: care only about the boundary of the shadow

3. Perturb randomly b vector: guarantee that the number of edges of the boundary of the shadow is polynomially large, w.h.p.

4. We can move from a starting vertex of the shadow to this that optimizes c or -c objective function in polynomial number of steps, by doing fixed number of steps.

5. If it fails in constant number of steps, change the distribution on perturbations and run the algorithm again.

Thank You

(From the movie: π)

I have uploaded the presentation on my web site:http://www.ccs.neu.edu/home/dkanou/

11:15, restate my assumptions:1. Mathematics is the

language of nature.2. Everything around us

can be represented and understood through numbers.

3. If you graph these numbers, patterns emerge.

Therefore:There are patterns everywhere in nature.