Dimension reduction for finite trees in L1

University of Washington

James R. Lee

École Normale Supérieure

Arnaud De Mesmay

Mohammad Moharrami

0 1

0 0

0

0

01

1 1

1

1 10

0 1

0 0

0

0

01

1 1

1

1 10

dimension reduction in Lp

Given an n-point subset X µ Rd, find a mapping

kf (x) ¡ f (y)k1 =

F : X ! Rk

such that for all x, y 2 X,kx ¡ ykp · kF (x) ¡ F (y)kp · D ¢kx ¡ ykp

k = target dimensionD = distortion

n = size of X

Dimension reduction as “geometric information theory”

the case p=2

When p=2, the Johnson-Lindenstrauss transform gives, for everyn-point subset X µ Rd and " > 0,

kf (x) ¡ f (y)k1 =

k = O³

logn"2

´D · 1+ "

Applications to…- Statistics over data streams- Nearest-neighbor search- Compressed sensing- Quantum information theory- Machine learning

dimension reduction in L1

Natural to consider is p=1.

History:

k = target dimensionD = distortion

n = size of X

- Caratheodory’s theorem yields D=1 and k = ¡n2¢

- [Schechtman’87, Bourgain-Lindenstrauss-Milman’89, Talagrand’90] Linear mappings (sampling + reweighting) yield

k = O³

n log n"2

´D · 1+" and

- [Batson-Spielman-Srivastava’09, Newman-Rabinovich’10] Sparsification techniques yield

k = O ¡ n"2

¢D · 1+" and

the Brinkman-Charikar lower bound

[Brinkman-Charikar’03]: There are n-point subsets such that distortion D requires

k ¸ n (1=D 2)

Very technical argument based on LP-duality.[L-Naor’04]: One-page argument based on uniform convexity.

[Brinkman-Karagiozova-L 07] Lower bound tight for these spaces

more lower bounds

[Andoni-Charikar-Neiman-Nguyen’11]: There are n-point subsets such that distortion 1+" requires

k ¸ n1¡ O(1=log(1="))

[Regev’11]: Simple, elegant, information-theoretic proof of both the Brinkman-Charikar and ACNN lower bounds.

Low-dimensional embedding ) encoding scheme

4

the simplest of L1 objects

A tree metric is a graph theoretic tree T=(V, E) together withnon-negative lengths on the edgeslen : E ! [0;1 )

Easy to embed isometrically into RE equipped with the L1 norm.

dimension reduction for trees in L1

Charikar and Sahai (2002) showed that for trees one can achievek = O

³log3 n

"2

´D · 1+ "

A. Gupta improved this to k = O³

log2 n"2

´

In 2003 in Princeton with Gupta and Talwar, we asked:

Is possible?k = O(logn)D = O(1)

even for complete binary trees?

dimension reduction for trees in L1

Theorem: For every n-point tree metric, one can achievek = O ¡ 1

"4 log¡ 1"¢¢¢logn

D · 1+ "and

(Can get for “symmetric” trees.)k = O³

logn"2

´

Complete binary tree using local lemmaSchulman’s tree codesComplete binary tree using re-randomizationExtension to general trees

dimension reduction for the complete binary tree

0 1

0 0

0

0

01

1 1

1

1 1 1

001 110

Every edge gets B bits ) target dimension = B log2nChoose edge labels uniformly at random.Nodes at tree distance have probability to get labels with hamming distance

(logn) n¡ O(B )

(logn)

dimension reduction for the complete binary tree

0 1

0 0

0

0

01

1 1

1

1 1 1

001 110

Every edge gets B bits ) target dimension = B log2nChoose edge labels uniformly at random.

Siblings have probability 2-B to have the same label, yetthere are n/2 of them.

Lovász Local Lemma

Pairs at distance L have probability to be “good”1¡ 2¡ (L )

Number of dependent “distance L” events is2O(L )

LLL + sum over levels ) good embedding

Schulman’s tree codes

- LLL argument difficult to extend to arbitrary trees. - Same as construction of Schulman’96: Tree codes for interactive communication

0 1

0 0

0

0

01

1 1

1

1 1 1

001 110

re-randomization

0 1

0 0

0

0

01

1 1

1

1 10

0 1

0 0

0

0

01

1 1

1

1 10

0 1

For every level on the right, exchange 0’s and 1’s with probability half(independently for each level)

Random isometry:

re-randomization

0 1

0 0

0

0

01

1 1

1

1 10

0 1

0 0

0

0

01

1 1

1

1 10

0 1

Pairs at distance L have probability to be “good”1¡ 2¡ (L )

Number of pairs at distance L is 2O(L )

extension to general trees

How many coordinates to change per edge, and bywhat magnitude?

Unfortunately, the general case is technical (paper is 50 pages)Obstacles:

General trees do not have O(log n) depthUse “topological depth” of Matousek.

Multi-scale entropy functional

open problems

Coding/dimension reduction:Extend/make explicit the connection between L1 dimensionreduction and information theory.

Close the gap: For distortion 10, is the right target dimension

n or n0:01 ?Other Lp norms: Nothing non-trivial is known forp =2 f1;2;1 g