Combinatorial OptimizationCombinatorial Optimization Top Ten List Top Ten List Discrete Mathematics 2000

as selected by

William R. Pulleyblank

T. J. Watson Research CenterT. J. Watson Research Center IBM Corporation IBM Corporation Yorktown Heights, NY Yorktown Heights, NY

Euler's Theorem 1736 Theorem: A graph has a Euler tour if and only if there are zero or two nodes of odd degree

" As far as the problem of the seven bridges of Koenigsberg is concerned, it can be solved by making an exhaustive list of all possible routes, and then finding whether or not any route satisfies the conditions of the problem. Because of the number of possibilities, this method of solution would be too difficult and laborious, and in other problems with more bridges, it would be impossible..."

L. Euler, Solutio Problematis ad Geometriam Situs Pertinentis, Commentarii Academiae Scientiarum Imperalis Petropolitanae 8 (1736) 128-140.

" So whatever arrangement may be proposed, one can easily determine whether or not a journey can be made, crossing each bridge once, by the following rules:

If there are more than two areas to which an odd number of bridges lead, then such a journey is impossible.

If, however, the number of bridges is odd for exactly two areas, then the journey is possible if it starts in either of these areas.

If, finally, there are no areas to which an odd number of bridges leads, then the required journey can be accomplished starting from any area. "

Algorithm first published 137 years later

C. Hierholzer, Ueber die Moeglichkeit, einen Linienzug ohne Wiederholung und ohne Unterbrechnung zu umfahren, Mathematische Annalen 6 (1873), 30-32.

- start at an odd degree node; - walk until you are stuck * what's left has no odd degree nodes

- solve the remaining problem and splice in the answer.

seeN.L. Biggs, E.K.Lloyd, R.J.Wilson, Graph Theory 1736-1936, Oxford University Press, 1976.

Chinese Postman problem

The streets were given lengths, and any (even) number of odd degree intersections was permitted. The problem now became:

Find the shortest tour traversing each street at least once.

J. Edmonds and E.L. Johnson, Matching, Euler Tours and the Chinese Postman, Mathematical Programming 5 (1973) 88-124.

Guan Meigu, Graphic Programming using odd and even points, Chinese Math. 1 (1962) 273-277

Solved polynomially in

P.D. Seymour, The matroids with the max-flow min-cut property, J. Combin. Theory Ser. B 23 (1977) 189-222.

Many extensions - one in particular -

Herbert J. Ryser, Combinatorial Mathematics, The Mathematical Association of America, 1963.

" Combinatorial mathematics cuts across the many subdivisions of mathematics, and this makes a formal definition difficult. ... Two general types of problems appear throughout the literature. In the first the existence of the prescribed configuration is in doubt, and the study attempts to settle this issue. These we call existence problems. In the second the existence of the configuration is known, and the study attempts to determine the number of configurations or the classification of these configurations according to types. These we call enumeration problems. "

March 1971 -- Second Louisiana Conference on Combinatorics, Graph Theory and Computing, LSU, Baton Rouge, Ryser gave an invited address and included a third type of problem:

How do you efficiently find or construct the configurations?

Max-flow Min-cut Theorem 1956

Given a directed graph G = (V, E),with finite arc capacities ( uj : j

�

��

�

���� E), and designated source and sink nodes s and z,the maximum value of an s - z flow is the minimum capacity of an s - z cut.

L.R. Ford Jr. and D.R. Fulkerson, Maximal flow through a network, Canadian J. of Math. 8 (1956) 399-404.

A. Kotzig, "Suvislost' a Pravidelina Suvislost Konecnych Grafor", Bratislava: Vysoka Skola Economicka (1956).

P. Elias, A. Feinstein and C.E. Shannon, A note on the maximum flow through a network, IRE Transactions on Information Theory IT2 (1956) 117-119.

Equivalent Theorems:

Menger's Theorem (1927) (edge or node disjoint paths)Dilworth's Theorem (1950) (antichains in partially ordered sets)P. Hall's Theorem (1935) (Systems of Discrete Representatives)Koenig's Theorem (1931) (maximum matching and minimum cover in a bipartite graph)Hoffman's Circulation Theorem (1960)Gale's Supply Demand Flow (1957)...

These were all linked by total unimodularity - Hoffman and Kruskal (1956)

Write down a "natural" linear programming formulation, then there will be an integral optimal solution ...

Flow augmenting pathsDinit (1970) and Edmonds-Karp (1972) - shortest flow augmenting paths give polynomial performancePreflow-push algorithmsGeneralized flowsMulticommodity flows ...

and some extensions ...

Seymour's Theorem (1980): [A good characterization of total unimodularity] - a matrix is totally unimodular if and only if it can be composed from network matrices, their transposes, and two exceptions.

P.D. Seymour, Decomposition of regular matroids, J. of Combin. Theory Ser. B 28 (1980) 305-359.

Nonbipartite matching algorithm and polyhedral characterization 1965.

Jack Edmonds, Paths, trees and flowers, Canadian J. of Maths. 17 (1965) 449-467.

There exists a polynomially bounded algorithm which finds a maximum matching in a nonbipartite graph.

Generalized the Hungarian method of Kuhn 1955 and Munkres 1957 for bipartite graphs

Alternating paths not enough

You shrink the odd sets!

Paths, trees and flowers (Edmonds 1965) -

2. Digression

... I am claiming, as a mathematical result, the existence of a good algorithm for finding a maximum cardinality matching in a graph.... an algorithm whose difficulty grows only algebraically with the size of the graph...

...For practical purposes the difference between algebraic and exponential order is often more crucial than the difference between finite and non-finite.

also, Cobham (1965)

Foundation of Polyhedral Combinatorics

The convex hull of the incidence vectors x ofthe matchings of a nonbipartite graph is defined by

xj � 0 for all j � E,

� (xj : j ���(v)) ���� 1 for all v �V,� (xj : j ���E(S)) ���� (|S| -1)/2 for all odd

cardinality S ��V.

J. Edmonds, Maximum matching and a polyhedron with 0-1 vertices, J. Res. Nat. Bureau of Standards B 69 (1965) 125-130.

Extensions:

bidirected capacitated weighted matchings - includes network flows, edge covers and natchings

parity constraints - includes T-joins, cuts and postman problems

stable sets in claw free graphs

path systems

Matroid Intersection Theorem 1970

Matroids had been introduced by Hassler Whitney in 1935 as an abstraction of linear independence.

A family FFFF of (so called independent) subsets of a set E forms a matroid if

1. all subsets of an independent set are independent;2. for any A �������� E, all maximal independent subsets of A have the

same cardinality - denoted by r(A).

(If 1. is satisfied, then FFFF is called an independence system.)

Examples - edge sets of forests in a graph, linearly independent sets of columns of a matrix, systems of distinct representatives of a set family, ...

Matroid Optimization Problem: Let each elemente � E have a weight we. Find an independent set I for which ��(we: e� I) is maximized.

Theorem (Rado 1957, Edmonds 1970): An independence system is a matroid if and only if the greedy algorithm solves the Matroid Optimization Problem, for any vector w of weights.

Theorem (Edmonds 1970): The incidence vectors x of the independent sets of a matroid are the vertices of the polyhedron defined by xe � 0 for all e� E,

����(xe: e� A) � r(A) for all A � E.

Matroid Intersection Theorem (Edmonds(1970): Let M1 = (E, F1) and M2 = (E, F2) be matroids on a set E. Then maximum |I| = minimum {r1(S)+r2(E\S)}. I�F1�F2 S � E

Weighted Intersection Theorem (Edmonds 1970): Let M1 = (E, F1) and M2 = (E, F2) be matroids on a set E. Let each element e � E have weight we. Then x is the incidence vector of a maximum weight set independent in both matroids if and only if x is a basic optimal solution to the linear program

maximize wx subject to xe � 0 for all e� E,

����(xe: e� A) � r1(A) for all A � E,����(xe: e� A) � r2(A) for all A � E.

Consequences:Koenig's Theorem for maximum matchings in bipartite graphs;

Edmonds' Matroid Partition Theorem

Perfect's Theorem on Systems of Common Representatives

Extensions: Polymatroids and polymatroid intersection

Submodular and supermodular functions

Unification with nonbipartite matching: Matroid parity problem, or matroid matching - cardinality case

for linear matroids (Lovasz 1980)

Cook's Theorem 1971

If there exists a polynomially bounded algorithm for satisfiability, then if a class of decision problems has a polynomial length certificate for a "YES" answer, then there is a polynomial time algorithm to solve any decision problem in the class.

S.A. Cook, The complexity of theorem-proving procedures, Proc. 3rd Ann. ACM Symp. on Theory of Computing, Association for Computing Machinery, New York (1971).

NP NP NP NP - problems having short "YES" certificates - polynomially solvable on nondeterministic Turing machinesP P P P - problems solvable in polynomial timeCook's Theorem: SATISFIABILITY is NP-NP-NP-NP-complete - a polynomial algorithm for SATISFIABILITY implies PPPP = NPNPNPNP is PPPP = NPNPNPNP ?

R. M. Karp 1972 Reducibility among combinatorial problems in R.E. Miller and J.W. Thatcher (eds.), Complexity of Computer Computations, Plenum Press, New York, 85-103.

M.R. Garey and D. S. Johnson, Computers and Intractability: A Guide to the Theory of NPNPNPNP-Completeness, Freeman, San Francisco, 1979

* More than 300 NPNPNPNP-complete problems

Showed that many well known problems were NPNPNPNP-complete - Hamiltonian circuit; large independent set of nodes; scheduling problems; 3-d matching ...

Conventional Wisdom: P P P P ��������NPNPNPNP.

Garey and Johnson - 12 open problems:

1. Graph Isomorphism open2. Subgraph homeomorphism for a fixed graph PPPP 3. Graph genus NPNPNPNP-complete4. Chordal graph completion NPNPNPNP-complete5. Chromatic Index NPNPNPNP-complete6. Spanning tree parity PPPP 7. Partial Order Dimension NPNPNPNP-complete8. Precedence constrained 3-processor sched. open9. Linear Programming PPPP 10. Total Unimodularity PPPP 11. Composite number open12. Minimum length triangulation open

Dantzig, Fulkerson and Johnson - solution of the 49 city problem - 1954

G. Dantzig, R. Fulkerson and S. Johnson, "Solution of a large-scale traveling salesman problem", Operations Research 2, (1954), 393-410.

Optimal solution of a 49 city traveling salesman problem using cutting planes

... and the race was on

Size Researchers Year49 Nodes Dantzig, Fulkerson, and Johnson 195460 Nodes Held and Karp 197080 Nodes Helbig, Hansen, and Krarup 1974100 Nodes Camerini, Fratta, and Maffiol 1975120 Nodes Groetschel 1977120 Nodes Padberg and Hong 1980318 Nodes Crowder and Padberg 1980532 Nodes Padberg and Rinaldi 1987666 Nodes Groetschel and Holland 19881002 Nodes Padberg and Rinaldi 19882392 Nodes Padberg and Rinaldi 19883038 Nodes Applegate, Bixby, Chvatal, and Cook 19924461 Nodes Applegate, Bixby, Chvatal, and Cook 19937397 Nodes Applegate, Bixby, Chvatal, and Cook 199413509 Nodes Applegate, Bixby, Chvatal, and Cook 1998

Largest solved TSP problems

1950 1960 1970 1980 1990 200010

100

1000

10000

100000number of cities

Size of largest TSP solvedto provable optimality

Held and Karp's Subgradient relaxation of the TSP - 1970, 71

Start with a minimal integer programming formulation of the TSP:

minimize � dij xij

subject to xij {0,1} for all i, j����

� xij = 2 for all ji

� xij |S| - 1i, j �s

for all ����S ��V.�

minimize � dij xij

subject to xij {0,1} for all i, j����

- ��

pj(2 - ) � xijij

� xij |S| - 1i, j �s

for all ����S ��V.�

- remove the degree constraints!

M. Held and R.M. Karp, The traveling salesman problem and minimum spanning trees, Operations Research 18 (1970) 1138-1162.

------, The traveling salesman problem and minimum spanning trees: Part II, Mathematical Programming 1 (1971) 6-25.

Anbil and Barahona (1998) showed how subgradient methods could be extended to yield a solution to the linear program which is "almost" feasible and "almost" optimal.

Essentially, they add a weighted average of the (infeasible) primal solutions produced by a subgradient method.

this followed earlier work on approximate "almost feasible" by Plotkin, Shmoys, and Tardos; Grigoriadis and Khachiyan

The Volume Algorithm

F. Barahona and R. Anbil, The volume algorithm: producing primal solutions with a subgradient method,Mathematical Programming, Ser. A 87, 2000, 385-399.

The Volume Algorithm - producing good primalsolutions.

Lagrangian Methods and The Volume Algorithm (continued)

#

Rows#

Columns#

ElementsPrimal-

DualInterior

DualSimplex

Volume plusDual

Simplex

2504 53226 553148 1341 3747 320

2991 46450 502338 1984 6928 923

4810 95933 1009283 4165 25917 2576

Lin Kernighan Algorithm 1973

Given a feasible solution to a Traveling Salesman problem, improve it by local exchanges

Local search:alternatively removes and adds edges to a tour until a

better tour is found or it gives up.

Start from a random tour

Chained Lin Kernighan (Martin and Otto 1996) - random 4 - Interchange

oo

oo

oo

oo

D.S. Johnson, J.L. Bentley, L.A. McGeoch and E.E. Rothberg 1997

L-K produces tours 2% over opt.Chained L-K produces tours 1% over opt.

Can be implemented to run very quickly 10,000 node Euclidean TSP on average workstation - 9.7 seconds

L-K developed into local search activity - Simulated Annealing, Tabu Search, Genetic Algorithms, ...

Optimization = Separation 1980, 1981

The Ellipsoid Algorithm (L.G. Khachiyan 1979, 1980) operated by testing the centre x of an ellipsoid for membership in a polyhedron P. If x is not in P, this is shown by producing a violated defining inequality. Then the ellipsoid is updated. The entire process runs in polynomial time.

Theorem (Karp and Papadimitriou 1980, Padberg and Rao 1981, Groetschel, Lovasz, Schrijver 1981): Optimization and Separation are polynomially equivalent

Applications:

1. Submodular function minimization (Groetschel, Lovasz, Schrijver 1981)

2. Maximum weight stable sets in perfect graphs (Groetschel, Lovasz, Schrijver 1981) **

3. Separation for odd cuts (Padberg, Rao 1982)

4. Separation for matroid polyhedra (Cunningham 1984)

Let G be a graph whose nodes are letters in an alphabet and where adjacency means that letters can be confused. The maximum number of 1 letter messages that can be sent without confusion is ����(G).

Let ����(Gk ) denote the maximum number of k-letter messages which can be sent without danger of confusion. (Confusion occurs if two words are identical or confusable in all positions.)

The Shannon capacity of the pentagon is ����5 1979

_

L. Lovasz, On the Shannon capacity of a graph, IEEE Transactions on Information Theory IT-25 (1979) 1-7.

����(G) is the stability number - the maximum number of pairwise nonadjacent nodes.

C5 - the pentagon

�(C5) = 2

a

b

cd

e

a

b

cd

e�(C52) = 5

aabccedbed

The Shannon Capacity, �(G),is defined by

�(G) = sup ����(Gk) = lim ����(Gk). ����������������������

�_k

�_k

k k �� 8

Lovasz(1979) introduced the concept of an orthonormal representation of a graph G=(V,E) : a set of |V| vectors ui such that ||ui||=1 for all i and ui

Tuj = 0 for all pairs i,j of nonadjacent vertices.

TH(G) = { x � : x � 0 and x satisfies the orthonormal representation constraints for all c � }. IR

nIRV

Note: TH(G) is not polyhedral

����(G) = max{1Tx : x ���� TH(G)}.

Orthonormal representation constraint:

Let u be an orthonormal representation of G=(V,E). For any c ���� n with ||c|| = 1, the incidence vector x of any stable set S satisfies

IR

� (cTui)2xi � 1.i � V

Theorem: (Lovasz 1979): �(G) � �(G)

����_

Theorem (Lovasz 1979): ��(C5) � 5.

����_5 �� �(C5) ����(C5) � 5.

����_

and where did it lead?

plus, properties of ����(G) and how to compute and estimate for a general graph G.

Groetschel, Lovasz, Schrijver (1981, 1988):

�(G, w) = max{wTx : x � TH(G)}.

Theorem: TH(G) is a polytope if and only if G is a perfect graph.

Theorem: TH(G) = STAB(G) if and only if G is perfect.

Theorem: The weak optimization problem for TH(G) can be solved in polynolmial time.

Corollary: Both the weighted stable set problem and weighted clique problem for perfect graphs can be solved in polynomial time.

but we still cannot recognize a perfect graph in polynomial time.

.878 Approximation Algorithm for the max cut problem 1994

Given an undirected graph G=(V,E) with edge weights we: e � E, partition V into sets V1 and V2 so that the weights of the edges between the two sets is maximized.

NP-hard, Sahni and Gonzalez (1976) gave a 1/2 approximation algorithm.

M.X. Goemans and D.P. Williamson, 0.878-approximation algorithm for MAX-CUT and MAX-2SAT, Proceedings of the 26th Annual ACM Symposium on the Theory of Computing (1994) 422-431.

Formulated the problem as a quadratic {+1, -1} problem;

Relaxed it to a semidefinite programming problem;

Solved polynomially, using GLS 81, 87.

Used randomized rounding to get good expected solution,

Derandomized

1. Euler's Theorem 17362. Max-flow Min-cut Theorem 19563. Edmonds' matching algorithm & polyhedron 19654. Edmonds' matroid intersection 19655. Cook's Theorem 19716. Dantzig Fulkerson and Johnson 49 cities 1954

Held & Karp relaxation of the TSP 1970, 717. Lin Kernighan local search for the TSP 19738. Optimization = Separation 1981 9. Lovasz's Shannon Capacity of pentagon 197910. Goemans Williamson .878 approx for MAX CUT

1994

Top Ten ListTop Ten List

a reference ...