1

A Network Positioning System for the Internet

T. S. Eugene Ng and Hui ZhangUSENIX ’04

Presented By: Imranul Hoque

2

Problem

• Network distance = Round Trip Time

Given two Internet hosts, can we accurately estimate the network distance between them

without sending any RTT probes between them?

3

Motivation

• Example:– Overlay networks, closest server selection

• Why not explicit measurement?

Assign coordinates to hosts to enable latency prediction

(X1, Y1) (X2, Y2)

(X3, Y3)

4

Challenges

• Accuracy• Stability• Adaptability• Scalability

5

Timeline

GNP (Ng)INFOCOM02

PIC (Costa)ICDCS04

Vivaldi (Dabek)SIGCOMM04

PCoord (Lehman)NCA04

NPS (Ng)USENIX04

Virtual Landmarks (Tang)IMC03

Big-Bang (Shavitt)INFOCOM03

LightHouse (Pias)IPTPS03

Stable-Accurate Coordinate (Ledlie)ICDCS06

Coordinate in Wild (Ledlie)NSDI07

Adversarial Network (Zage)CCS07

Secure Internet Coordinate (Kaafar)SIGCOMM07

2002

2003

2004

2006

2007

6

Global Network Positioning

Step 1: Measure inter-landmark distance

Landmark 2 Landmark 3

Landmark 1

Global Network Positioning

Step 2: Send measured distance to coordinator

Landmark 2 Landmark 3

Landmark 1

6

Global Network Positioning

Landmark 2 (x2, y2) Landmark 3 (x3, y3)

Landmark 1 (x1, y1)

Step 3: Coordinator computes coordinates by minimizing the overall discrepancy between measured distances and computed distances

2

ij

ijijijij

jiijij

ddd

dd

dd

ˆˆ, where

,ˆ,minimize

6

Global Network Positioning

Landmark 2 (x2, y2) Landmark 3 (x3, y3)

Landmark 1 (x1, y1)

Step 4: Each ordinary host measures its distance to the landmarks

Host 1

6

Global Network Positioning

Landmark 2 (x2, y2) Landmark 3 (x3, y3)

Landmark 1 (x1, y1)

Step 5: Ordinary host computes its coordinates relative to the landmarks by minimizing the overall

discrepancy between measured distance and computed distance

Host 1 (xh, yh)

6

Problems of GNP

• Centralized coordinator– Solution: distributed calculation

• Fixed set of landmarks– Solution: any set of existing nodes can be

reference points

Layer 0

Layer 1

Layer 0 Layer 0

Layer 2

Layer 3

Layer 2

7

Network Positioning System

Landmark 1 Landmark 2 Landmark 3

Membership Server

Layer 0

8

Network Positioning System

Landmark 1 Landmark 2 Landmark 3

Membership Server

Host h1

Layer 0

8

Network Positioning System

Landmark 1 Landmark 2 Landmark 3

Membership Server

Host h1

I need some reference points

Layer 0

8

Network Positioning System

Landmark 1 Landmark 2 Landmark 3

Membership Server

Host h1

L1, L2, L3

Layer 0

8

Network Positioning System

Landmark 1 Landmark 2 Landmark 3

Membership Server

Layer 0

Layer 1

Host h1

Ref: [L1, L2, L3]Layer = maxLayer(L1, L2, L3) + 1

8

Network Positioning System

Landmark 1 Landmark 2 Landmark 3

Membership Server

Host h1

Ref: [L1, L2, L3]

Layer 0

PING

Layer 1

8

Network Positioning System

Landmark 1 Landmark 2 Landmark 3

Membership Server

Host h1

Ref: [L1, L2, L3]

Layer 0

Coord(Li), RTT(h1, Li)Layer 1

8

Network Positioning System

Landmark 1 Landmark 2 Landmark 3

Membership Server

Host h1

Ref: [L1, L2, L3]

Layer 0

Calculate coordinate using RTT(h1, Li) & Coord(Li)

Layer 1

What if the RTTs are not correct?What if Li was also calculating its coordinate?

8

Network Positioning System

Landmark 1 Landmark 2 Landmark 3

Membership Server

Host h1

Ref: [L1, L2, L3]

Layer 0

3. Calculate coordinate (using min RTT to each host)

Layer 1

Repeats the steps until in 3 consecutive iterations node moves < 1msec

1. PING

2. Coord(Li), RTT(h1, Li)

8

Network Positioning System

Landmark 1 Landmark 2 Landmark 3

Membership Server

Host h1

Ref: [L1, L2, L3]

Layer 0

3. Calculate coordinate (using min RTT to each host)

Layer 1

Period: 12 hours (hosts), 3 hours (landmarks)

1. PING

2. Coord(Li), RTT(h1, Li)

8

Issues in NPS

• How many layers?– Accuracy vs. Scalability (NPS uses 3 layers)

• Handling malicious nodes– Calculate fitting error to each reference point– Reject reference point with largest fitting error if the error

is significantly large• Sudden change in topology– Reference points can trigger reposition

• Effect of reference point selection– Random, close, hybrid [PIC: Costa04]

9

PlanetLab Experiment

• Duration: 20 hours• 127 nodes: 15 well distributed nodes are

landmarks [PCoord: Lehman04]• 8 dimensional coordinates (why?)• Landmarks started 5 minutes prior to ordinary

hosts• 2 layers: landmarks and ordinary hosts• Accuracy metric: relative error

|actual – predicted| / min(actual, predicted)

10

Accuracy of NPS

Among landmarks, end

Among ordinary hosts, end

11

Controlled Experiment

• Objective:– Accuracy under RTT dynamics– Effect of topology change– Effect of maliciousness

• Simulation result from 1044 node random topology• 8 dimensional coordinates• 20 landmarks, 1024 hosts • 2 layers, 3 layers• Accuracy metric: 90 percentile relative error

12

Effect of Topology Change

Topology replaced

With triggered update

Without triggered update

13

Discussion

• Stable and accurate coordinates [Ledlie06]– Data from PlanetLab: 2 random nodes

RTT varies a lot over time

14

Discussion (2)

• Network Coordinates in the Wild [Ledlie07]– Higher relative error– More triangle inequality violation– Higher churn

Behavior in real network is completely different from behavior under PlanetLab experiments

15

Discussion (3)

• Effect of maliciousness– Few nodes lying huge vs. many nodes lying small– Use a random set of verifiers to verify that relative

error is below a certain threshold

Very easy to distort the whole coordinate space

16

Questions?

17

31

Downhill Simplex Method

• Simplex = N+1 vertices in N dimensions• Moves:– Reflection– Reflection and Expansion– Contraction– Multiple Contraction

32

Downhill Simplex Method

• (a) is highest (c) is lowest

33

Reflection

34

Reflection and Expansion

• (d) is less than (c)

35

Contraction

• (d) is higher than second highest: (b)

36

Multiple Contraction

• (e) is still the highest among: (b), (c), (e)