Whistler Workshop June 2010 1

Integer Polytope Sampling and Network Tomography

Bertrand HaasEdoardo Airoldi

Harvard University06/21/2010

Whistler Workshop June 2010 2

Plan

• Motivation: Network Tomography + 2 more applications

• The Solution Polytope• Finding a first vertex of the polytope• Finding all the vertices of the polytope• Integer Polytope Sampler 1, 2 and 3 and

comparisons• From sample point to integer sample point• Efficiency

Whistler Workshop June 2010 3

Network Tomography• Computer-Router networks• Routes between pairs of end-

nodes predetermined• Routers record traffic• Problem: Infer traffic between

pairs of end-nodes• Network matrix A (incidence)– Column <-> pair of end-nodes– Row <-> edge– A[i,j] = 1 if edge i incident to route

of pair j. A[i,j] = 0 otherwise

• Network Matrix of full rank -> eliminate some rows (flow conservation at each router)• No loop -> number of solutions is finite

Whistler Workshop June 2010 4

1-Router Networks: Special Interest

• m x n network matrix A (here m = 7, n = 16)• Problem: Given link count b, solve A x = b• Underdetermined problem -> sample integer non-negative

solutions with a distribution (uniform,…)• R-solution space = Flat (dim. n-m) ∧ non-neg. orthant• # integer solutions finite -> convex polytope

Whistler Workshop June 2010 5

2 more applications

• Sampling from contingency tables reduces to Network Tomography on a 1-router network.

• Sampling (directed) networks with given degree vector reduces to a {0,1}-contingency table problem– Intersection of a hypercube with a Flat (TUM)

Whistler Workshop June 2010 6

Total UniModularity• Definition: Matrix is TUM <-> all its sub-

determinants are -1, 0 or 1• Theorem (): Network matrices are TUM• Theorem (): Polytopes arising from a TUM

matrices are integral (vertices are integral).• New Problem:

1. Find vertices of the solution polytope.2. Sample integer points of the polytope.

Whistler Workshop June 2010 7

Hermite Normal Decomposition• Hermite Normal Form of integer matrix A: B = A Q– Where B is integral, lower triangular, B[i,j] ≤ B[i,k], k < i– Q is unimodular

• A is TUM -> A = [A1|A2], where A1 unimodular

• Columns of Q2 generates null-space

Whistler Workshop June 2010 8

On previous 1-Router Network

A =

Q =

(with columns reshuffledto a nicer form)

Q1 Q2

Whistler Workshop June 2010 9

The First Vertex of the Solution Polytope

• Solving A x = b <-> A Q (Q-1 x) = b <-> B x’ = b <-> x’ = (b,w)T where x’ = Q-1 x• x = Q x’ <-> x = (b’|0)T + Q2 w where b’ = A1

-1 b

• Vertices of solution polytope are intersections of the solution-flat (dim. n-m) with m-coordinate planes.• -> Vertices have at most m non-zero coordinates

• So if x(1) = Q(b,0)T = Q1b ≥ 0 then x(1) = v0 is a first vertex of the solution polytope.

Whistler Workshop June 2010 10

From the 1st Vertex to Neighbor Vertices

• Lemma: each column of Q2 has at least one -1

• Let Kj = {k:Q2[k,j] = -1} and i = argmin b’k over all k in Kj

• Then vj = v0 + b’i Q2[:,j], for all j = 1,…,n-m

At most m non-negativecoordinates

Whistler Workshop June 2010 11

From an integer solution to a non-neg. one

• By similar operation (“pivoting”) we move from one one integer solution to a less negative one.

Permute

Whistler Workshop June 2010 12

The Geometry of Pivoting

• Example: n=6, m=2 -> hyperplane of coordinates become lines in the flat. Orthants are polyhedra.

• 1st solut. Q1 b = x(1) -> x(2) -> x(3) -> x(4) = v0 1st vertex

Whistler Workshop June 2010 13

Polytope Sampler 1• Associate a coefficient αi to each of the r vertices of the

polytope (of dimension n).• Decompose the polytope into simplexes:• P = S1 S∨ 2 … S∨ ∨ r-n

• Associate weights pi to each Si (with Σpi=1)• for example pi = Vol(Si)/Vol(P)

• Draw a Si with probability pi

• Let α0, α1,…, αn be the coefficients associated to Si’s vertices v0, v1,…,vn and let V = [v1-v0 ,…, vn-v0]

• Draw a vector Y ~ Dir(α0, α1, …, αn)

• Scale and translate: X = v0 + V Y

Whistler Workshop June 2010 14

Polytope Sampler 2

• Theorem (): A polytope P of dim. n with r vertices is the projection of a simplex S of dim. r-1

• Associate coefficients α0, α1,…, αr to P’s vertices• Lift P to a simplex S of dim. r-1.• Draw a vector Y ~ Dir(α0, α1,…, αr)

• Project onto P: X = projP(Y)

Whistler Workshop June 2010 15

Polytope Sampler 3• Let di = Σvi,j, d0 = max(di), and vi,0 = d0-di • So we lift P to P’ in the plane Σxi = d0 in Rn+1

• Define the monomials• mi = x0^(vi,0) x1^(vi,1) … xn^(vi,n)

• so deg(mi) = d0 for all i

• Here: m0 = x0, m1 = x1, m2 = x2

• Associate αi to each vertex vi • For example all αi = 1

• Set Φ(x) = Σ αi mi vi / Σ αi mi • Theorem: Φ isom.: (R>0)n int(P’)

• Here Φ(x) = (x0,x1,x2)T/Σxi

• Draw X ~ dist(R>0)n & set Y = Φ(X) • for example Xi ~ Gamma(ai)

• We get Y ~ Dir(a1,a2,a3)

Example:P = triangle

Whistler Workshop June 2010 16

Polytope Samplers Comparisons• Sampler 1 (Simplicial Decomposition):

• (+) Perfect uniform distribution• (+) Possible finer decomposition (interior points)• (-) Simplicial decomposition not straightforward and not unique

• Sampler 2 (Projection of a Dirichlet dist.)• (+) Easy• (-) Lifting to simplex not unique• (-) only approximate uniform distribution

• Sampler 3 (map Φ: (R>0)n int(P’))• (+) Direct method; Generalization of Dirichlet• (-) Computations might be costly (with high degree)• (-) Distribution difficult to control (Jacobian not simple)

Whistler Workshop June 2010 17

From a sampled point to an integer point

• Rounding off the coordinates possibly gets out of the solution flat.

• Project onto the n-m coordinate plane x1 = x2 =…= xm = 0, round off, project back to solution flat.

• Still possible errors and bias on the boundary.

Whistler Workshop June 2010 18

Efficiency

• Compare with stat-CS methods– EM algorithm (J. Cao at al.), Metropolis Hasting-

Gibbs (C. Tabaldi, M. West)• On examples where Solution Polytope =

Simplex:– At least one order of magnitude faster– Much more accurate (no error recorded)– Scalability?