Complex System Approach to Internet Measurements
Transcript of Complex System Approach to Internet Measurements
![Page 1: Complex System Approach to Internet Measurements](https://reader030.fdocuments.in/reader030/viewer/2022041007/624f14b1e8b3f7355d2145f3/html5/thumbnails/1.jpg)
Complex System Approach to Internet Measurements
The EVERGROW Measurement and Virtual Observatory Programs
ETOMIC DIMES
![Page 2: Complex System Approach to Internet Measurements](https://reader030.fdocuments.in/reader030/viewer/2022041007/624f14b1e8b3f7355d2145f3/html5/thumbnails/2.jpg)
��������������� ���������� ������� � ��������������� ��� ���������� ��������
Measure Internet as a Complex System
• Internet is complex:
1. Complex network topology
2. Complex traffic on the links of the network
![Page 3: Complex System Approach to Internet Measurements](https://reader030.fdocuments.in/reader030/viewer/2022041007/624f14b1e8b3f7355d2145f3/html5/thumbnails/3.jpg)
��������������� ���������� ������� � ��������������� ��� ���������� ��������
Measure Internet as a Complex System
• Internet is complex:
1. Complex network topology
2. Complex traffic on the links of the network
![Page 4: Complex System Approach to Internet Measurements](https://reader030.fdocuments.in/reader030/viewer/2022041007/624f14b1e8b3f7355d2145f3/html5/thumbnails/4.jpg)
Erd s-Rényi model (1960)
- Democratic
- Random
������������������� ������
p=1/6N=10 �k� ~ 1.5 Poisson distribution
Connect withprobability p
![Page 5: Complex System Approach to Internet Measurements](https://reader030.fdocuments.in/reader030/viewer/2022041007/624f14b1e8b3f7355d2145f3/html5/thumbnails/5.jpg)
Expected
P(k) ~ k-γ
Found
������������� ��
��� ���������� ��
R. Albert, H. Jeong, A-L Barabási
![Page 6: Complex System Approach to Internet Measurements](https://reader030.fdocuments.in/reader030/viewer/2022041007/624f14b1e8b3f7355d2145f3/html5/thumbnails/6.jpg)
��������������� ���������� ������� � ��������������� ��� ���������� ��������
The Internet Structure
routers
![Page 7: Complex System Approach to Internet Measurements](https://reader030.fdocuments.in/reader030/viewer/2022041007/624f14b1e8b3f7355d2145f3/html5/thumbnails/7.jpg)
��������������� ���������� ������� � ��������������� ��� ���������� ��������
The Internet Structure
The AS graph
![Page 8: Complex System Approach to Internet Measurements](https://reader030.fdocuments.in/reader030/viewer/2022041007/624f14b1e8b3f7355d2145f3/html5/thumbnails/8.jpg)
The Structure of the Internet Graph
[Wax88] E-R graph [FFF99] power law• Dramatically changed the way
researchers view the Internet
• Become the standard simulation topology
k
Pr(k)
<k>
0 2 4 6 8 10 120
2
4
6
8
10
12
14
log(degree)
log(
Pr(
degr
ee))
DIMES+BGP (Feb 05)
![Page 9: Complex System Approach to Internet Measurements](https://reader030.fdocuments.in/reader030/viewer/2022041007/624f14b1e8b3f7355d2145f3/html5/thumbnails/9.jpg)
��������������� ���������� ������� � ��������������� ��� ���������� ��������
What is EVERGROW DIMES?
• Distributed Internet measurement and monitoring– Based on agents downloaded by volunteers
from www.netdimes.org
• Aims– Static Internet maps at all granularity level
• connectivity, delay, loss, bandwidth, jitter, ….
– Tracking the Internet evolution in time– Correlating social/ecomonic/political evolution
with the Internet evolution
![Page 10: Complex System Approach to Internet Measurements](https://reader030.fdocuments.in/reader030/viewer/2022041007/624f14b1e8b3f7355d2145f3/html5/thumbnails/10.jpg)
��������������� ���������� ������� � ��������������� ��� ���������� ��������
How to get AS level data? (1)
• BGP listeners– Listen to as many as possible BGP announcers– RouteViews at University of Oregon: close to 70– Problems:
• What about non BGP connections:– static inter-domain connections
• Non broadcasted BGP entries• Many BGP links are filtered• biased
![Page 11: Complex System Approach to Internet Measurements](https://reader030.fdocuments.in/reader030/viewer/2022041007/624f14b1e8b3f7355d2145f3/html5/thumbnails/11.jpg)
��������������� ���������� ������� � ��������������� ��� ���������� ��������
How to get AS level data? (2)
• Active traceroute from a few pointstoo many links are missed
measurements traffic is too dense
![Page 12: Complex System Approach to Internet Measurements](https://reader030.fdocuments.in/reader030/viewer/2022041007/624f14b1e8b3f7355d2145f3/html5/thumbnails/12.jpg)
��������������� ���������� ������� � ��������������� ��� ���������� ��������
• Active traceroute from a few pointstoo many links are missed
measurements traffic is too dense
How to get AS level data? (2)
![Page 13: Complex System Approach to Internet Measurements](https://reader030.fdocuments.in/reader030/viewer/2022041007/624f14b1e8b3f7355d2145f3/html5/thumbnails/13.jpg)
��������������� ���������� ������� � ��������������� ��� ���������� ��������
• Active traceroute from a few pointstoo many links are missed
measurements traffic is too dense
How to get AS level data? (2)
![Page 14: Complex System Approach to Internet Measurements](https://reader030.fdocuments.in/reader030/viewer/2022041007/624f14b1e8b3f7355d2145f3/html5/thumbnails/14.jpg)
��������������� ���������� ������� � ��������������� ��� ���������� ��������
• Active traceroute from a few pointstoo many links are missed
measurements traffic is too dense
• Increase the number– Equipment & management costs are too high
– See NIMI
How to get AS level data? (2)
![Page 15: Complex System Approach to Internet Measurements](https://reader030.fdocuments.in/reader030/viewer/2022041007/624f14b1e8b3f7355d2145f3/html5/thumbnails/15.jpg)
��������������� ���������� ������� � ��������������� ��� ���������� ��������
Our Solution
• Let the network measure itself• Persuade people to download our
measurement DIMES agent• Carry on experiments to measure and map
the Internet and track its growth.• Target:
– 1,000s of users in first year– ������s of users in three years
![Page 16: Complex System Approach to Internet Measurements](https://reader030.fdocuments.in/reader030/viewer/2022041007/624f14b1e8b3f7355d2145f3/html5/thumbnails/16.jpg)
Agent screenshot
![Page 17: Complex System Approach to Internet Measurements](https://reader030.fdocuments.in/reader030/viewer/2022041007/624f14b1e8b3f7355d2145f3/html5/thumbnails/17.jpg)
��������������� ���������� ������� � ��������������� ��� ���������� ��������
Current Status
• About 830 agents– 50 countries
– All continents
– Over 150 ASes
– More than 140 are active daily
• Over 150,000 measurements a day
![Page 18: Complex System Approach to Internet Measurements](https://reader030.fdocuments.in/reader030/viewer/2022041007/624f14b1e8b3f7355d2145f3/html5/thumbnails/18.jpg)
Agents by CountryArgentina
Australia
Austria
Belgium
Brazil
Canada
China
Croatia
Czech Republic
Denmark
Egypt
Estonia
Finland
France
Germany
Greece
Hong Kong
Hungary
India
Israel
Italy
Japan
Korea (Republic of )
Latvia
Lithuania
Luxembourg
Malaysia
N/A
Namibia
Netherlands
USA
Israel
UK
Sweden
Ger.
Den .
Hun.
Fr.
Ita ly
Can.
Mar 2005
![Page 19: Complex System Approach to Internet Measurements](https://reader030.fdocuments.in/reader030/viewer/2022041007/624f14b1e8b3f7355d2145f3/html5/thumbnails/19.jpg)
DIMES Agents in Europe
![Page 20: Complex System Approach to Internet Measurements](https://reader030.fdocuments.in/reader030/viewer/2022041007/624f14b1e8b3f7355d2145f3/html5/thumbnails/20.jpg)
Iberia AS Map
![Page 21: Complex System Approach to Internet Measurements](https://reader030.fdocuments.in/reader030/viewer/2022041007/624f14b1e8b3f7355d2145f3/html5/thumbnails/21.jpg)
Hungary AS Map
![Page 22: Complex System Approach to Internet Measurements](https://reader030.fdocuments.in/reader030/viewer/2022041007/624f14b1e8b3f7355d2145f3/html5/thumbnails/22.jpg)
AS map on March 1st
BGP
• ~19,500 nodes
• ~48,400 edges
• <k> = 4.96
DIMES
• ~14,700 nodes
• ~38,750 edges
• <k> = 5.34
38,600 edges �So what has DIMES discovered?
9,800 edges
![Page 23: Complex System Approach to Internet Measurements](https://reader030.fdocuments.in/reader030/viewer/2022041007/624f14b1e8b3f7355d2145f3/html5/thumbnails/23.jpg)
AS map on March 1st
BGP
• ~19,500 nodes
• ~48,400 edges
• <k> = 4.96
DIMES
• ~14,700 nodes
• ~38,750 edges
• <k> = 5.34
38,600 edges 27,400 in both maps
11,300 new edges9,800 edges +
59,700 edges <k> > 6.12
![Page 24: Complex System Approach to Internet Measurements](https://reader030.fdocuments.in/reader030/viewer/2022041007/624f14b1e8b3f7355d2145f3/html5/thumbnails/24.jpg)
![Page 25: Complex System Approach to Internet Measurements](https://reader030.fdocuments.in/reader030/viewer/2022041007/624f14b1e8b3f7355d2145f3/html5/thumbnails/25.jpg)
��������������� ���������� ������� � ��������������� ��� ���������� ��������
DIMES Future
• DIMES as a leading research tool
– Data will be available to all
– Easy to run distributed experiments• Fast deploy cycle
– Easy to add new measurement capabilities
• Plug-ins to improve applications– P2P communication
– Web download (in process)
![Page 26: Complex System Approach to Internet Measurements](https://reader030.fdocuments.in/reader030/viewer/2022041007/624f14b1e8b3f7355d2145f3/html5/thumbnails/26.jpg)
Measure Internet as a Complex System
• Internet is complex:
1. Complex network topology
2. Complex traffic on the links of the network
![Page 27: Complex System Approach to Internet Measurements](https://reader030.fdocuments.in/reader030/viewer/2022041007/624f14b1e8b3f7355d2145f3/html5/thumbnails/27.jpg)
��������������� ���������� ������� � ��������������� ��� ���������� ��������
What is ETOMIC ?
• Evergrow Traffic Observatory MeasurementInfrastruCturewww.etomic.org
• Measurement stationsdistributed in Europe, capableto carry out precise, global time synchronizedmeasurements
• Central management system, to carry uploadmeasurement code, reservemeasurement time, tocollect the results (no need to hack)
• high precision to inject super accurate flashesofpacket trains into thenetwork
![Page 28: Complex System Approach to Internet Measurements](https://reader030.fdocuments.in/reader030/viewer/2022041007/624f14b1e8b3f7355d2145f3/html5/thumbnails/28.jpg)
Fukuda &Takayasu 1999
Router-to-router congestion propagation
![Page 29: Complex System Approach to Internet Measurements](https://reader030.fdocuments.in/reader030/viewer/2022041007/624f14b1e8b3f7355d2145f3/html5/thumbnails/29.jpg)
Simulation results: 10 queues, 10 TCP
![Page 30: Complex System Approach to Internet Measurements](https://reader030.fdocuments.in/reader030/viewer/2022041007/624f14b1e8b3f7355d2145f3/html5/thumbnails/30.jpg)
50 queues, 1 TCP/queuedeffect propagation
![Page 31: Complex System Approach to Internet Measurements](https://reader030.fdocuments.in/reader030/viewer/2022041007/624f14b1e8b3f7355d2145f3/html5/thumbnails/31.jpg)
��������������� ���������� ������� � ��������������� ��� ���������� ��������
Interacting traffic flows
• Traffic flows crossing the same bottleneck can inherit Long RangeDependence from each other
![Page 32: Complex System Approach to Internet Measurements](https://reader030.fdocuments.in/reader030/viewer/2022041007/624f14b1e8b3f7355d2145f3/html5/thumbnails/32.jpg)
Partsof themeasurement infrastructure
WANWAN
IBM BladeCenter
SWITCH/LAN
ETOM
RS 422 MAX. 100M
1000BaseTX for DAG
1000BaseTX LAN
1000BaseTX LAN
GPS
PCPC
![Page 33: Complex System Approach to Internet Measurements](https://reader030.fdocuments.in/reader030/viewer/2022041007/624f14b1e8b3f7355d2145f3/html5/thumbnails/33.jpg)
��������������� ���������� ������� � ��������������� ��� ���������� ��������
The server PC of thestation• Intel Pentium4 2.6GHz• Intel S875WP1-E (Winterpack)
single motherboard, 800MHz FSB, dual ECC mem
• Kingston 1GB (2*512MB) ECC DDRAM 400 MHz
• 1 Gbit/s + 100Mbit/s Intel integrated NIC
• 2 channel integrated Serial ATA RAID
• 200Gb Serial ATA 7200 RPM 8MB cache Seagate Barracuda Winchester
![Page 34: Complex System Approach to Internet Measurements](https://reader030.fdocuments.in/reader030/viewer/2022041007/624f14b1e8b3f7355d2145f3/html5/thumbnails/34.jpg)
��������������� ���������� ������� � ��������������� ��� ���������� ��������
The GPS receiver module
• Garmin 35HVS GPS reciver
• 1 us PPS signal
• RS 232 – RS 422 converter – max 100m cable
GPSRS232RS232-->422>422
RS422RS422-->232>232
Serial portSerial port
DAG PPSDAG PPS
PCPC
![Page 35: Complex System Approach to Internet Measurements](https://reader030.fdocuments.in/reader030/viewer/2022041007/624f14b1e8b3f7355d2145f3/html5/thumbnails/35.jpg)
��������������� ���������� ������� � ��������������� ��� ���������� ��������
The EndaceDAG 3.6GE card
• PCI bus – 32 bit 33 Mhz
• Single port full packet captureat 10/100/1000 Mbit/s speed
• precise timestamping
![Page 36: Complex System Approach to Internet Measurements](https://reader030.fdocuments.in/reader030/viewer/2022041007/624f14b1e8b3f7355d2145f3/html5/thumbnails/36.jpg)
��������������� ���������� ������� � ��������������� ��� ���������� ��������
Burst of Patterned Traffic Generator
• intended to send IPv4 Packets within Ethernet frames at full-rate 10/100/1000 Mbit/s using precisely defined frame sizes, frame contents and inter-frame delays
• Automatic ID increment and timestamps
• target applications: available bandwidth measurements (packet pair, quartet), network tomography, new fast transmit layer development
![Page 37: Complex System Approach to Internet Measurements](https://reader030.fdocuments.in/reader030/viewer/2022041007/624f14b1e8b3f7355d2145f3/html5/thumbnails/37.jpg)
��������������� ���������� ������� � ��������������� ��� ���������� ��������
Measurement scenarios
• Passive– SPAN port, splitter, HUB or tap needed
• Active:– NIC to NIC
– NIC to DAG
– DAG to DAG
![Page 38: Complex System Approach to Internet Measurements](https://reader030.fdocuments.in/reader030/viewer/2022041007/624f14b1e8b3f7355d2145f3/html5/thumbnails/38.jpg)
ETOMIC stations in Europe
![Page 39: Complex System Approach to Internet Measurements](https://reader030.fdocuments.in/reader030/viewer/2022041007/624f14b1e8b3f7355d2145f3/html5/thumbnails/39.jpg)
Evergrow Traffic ObservatoryManagement System (ETOMS)
![Page 40: Complex System Approach to Internet Measurements](https://reader030.fdocuments.in/reader030/viewer/2022041007/624f14b1e8b3f7355d2145f3/html5/thumbnails/40.jpg)
Your measurement is just a click away!
![Page 41: Complex System Approach to Internet Measurements](https://reader030.fdocuments.in/reader030/viewer/2022041007/624f14b1e8b3f7355d2145f3/html5/thumbnails/41.jpg)
What can wemeasure?
• High precision propagation delay of IP packets
![Page 42: Complex System Approach to Internet Measurements](https://reader030.fdocuments.in/reader030/viewer/2022041007/624f14b1e8b3f7355d2145f3/html5/thumbnails/42.jpg)
The ETOMIC overlay
![Page 43: Complex System Approach to Internet Measurements](https://reader030.fdocuments.in/reader030/viewer/2022041007/624f14b1e8b3f7355d2145f3/html5/thumbnails/43.jpg)
SICSStockholm, Sweden
TeliaSoneraStockholm, Sweden
ELTEBudapest, Hungary
157.181.141.9
UNIVBudapest, Hungary
62.40.96.58
62.40.96.62
HUJIJerusalem, Israel
UniNavarraPamplona, Spain
EricssonStockholm, Sweden
0.0005 ms
0.012 ms
0.0921 ms
0.0085 ms
0.0754 ms
0.028 ms
0.0078 ms
0.086 ms
0.1617 ms
Network tomography
![Page 44: Complex System Approach to Internet Measurements](https://reader030.fdocuments.in/reader030/viewer/2022041007/624f14b1e8b3f7355d2145f3/html5/thumbnails/44.jpg)
Plans for the future3-15 months
• ETOMS opens in June2005 for thegeneral public
• ETOMIC – ETOMIC network tomograpy (15 X 15)
• DIMES – ETOMIC network tomography (1000 X 15)
• a livemap of theEuropean network traffic from theautumn of 2005
• public databasebuilding
• network oriented databasebuilding
![Page 45: Complex System Approach to Internet Measurements](https://reader030.fdocuments.in/reader030/viewer/2022041007/624f14b1e8b3f7355d2145f3/html5/thumbnails/45.jpg)
��������������� ���������� ������� � ��������������� ��� ���������� ��������
Possiblecollaborations with IPS MoMe
• Evergrow can contribute to the IPS MoMe dataset• Evergrow can collaborate on data standards• Evergrow can collaborateon analysis methods
• Jointly lobbying at GÉANT
• You can download DIMES from www.netdimes.org• You can host our ETOMIC station in your lab• You can design your activemeasurement on ETOMS
involving thestation of your lab• Wecan host your stations as well
![Page 46: Complex System Approach to Internet Measurements](https://reader030.fdocuments.in/reader030/viewer/2022041007/624f14b1e8b3f7355d2145f3/html5/thumbnails/46.jpg)
EVERGROW needs you!
Join our effort! Deploy an ETOMIC box! Download a DIMES agent! Do your own measurement!
www.netdimes.org www.etomic.org
![Page 47: Complex System Approach to Internet Measurements](https://reader030.fdocuments.in/reader030/viewer/2022041007/624f14b1e8b3f7355d2145f3/html5/thumbnails/47.jpg)
��������������� ���������� ������� � ��������������� ��� ���������� ��������
Speed of light in cableJ
![Page 48: Complex System Approach to Internet Measurements](https://reader030.fdocuments.in/reader030/viewer/2022041007/624f14b1e8b3f7355d2145f3/html5/thumbnails/48.jpg)
��������������� ���������� ������� � ��������������� ��� ���������� ��������
Evergrow BladeCenters