perfSONAR MDM and TCP buffers - GÉANT · PDF fileperfSONAR MDM and TCP buffers Domenico...
Transcript of perfSONAR MDM and TCP buffers - GÉANT · PDF fileperfSONAR MDM and TCP buffers Domenico...
![Page 1: perfSONAR MDM and TCP buffers - GÉANT · PDF fileperfSONAR MDM and TCP buffers Domenico Vicinanza DANTE, Cambridge, UK domenico.vicinanza@dante.net EGI Technical Forum 2013, Madrid,](https://reader034.fdocuments.in/reader034/viewer/2022051718/5a71c0f67f8b9abb538d0e3f/html5/thumbnails/1.jpg)
perfSONAR MDM and TCP buffers
Domenico Vicinanza DANTE, Cambridge, UK
EGI Technical Forum 2013, Madrid, Spain
![Page 2: perfSONAR MDM and TCP buffers - GÉANT · PDF fileperfSONAR MDM and TCP buffers Domenico Vicinanza DANTE, Cambridge, UK domenico.vicinanza@dante.net EGI Technical Forum 2013, Madrid,](https://reader034.fdocuments.in/reader034/viewer/2022051718/5a71c0f67f8b9abb538d0e3f/html5/thumbnails/2.jpg)
2 Connect | Communicate | Collaborate
perfSONAR MDM
! Multi-domain monitoring service ! Based on monitoring probes installed in the network ! Based on a standard (perfSONAR) protocol ! Hundreds of deployments around the world ! Web interface!
![Page 3: perfSONAR MDM and TCP buffers - GÉANT · PDF fileperfSONAR MDM and TCP buffers Domenico Vicinanza DANTE, Cambridge, UK domenico.vicinanza@dante.net EGI Technical Forum 2013, Madrid,](https://reader034.fdocuments.in/reader034/viewer/2022051718/5a71c0f67f8b9abb538d0e3f/html5/thumbnails/3.jpg)
3 Connect | Communicate | Collaborate
perfSONAR in a nutshell
! Performance focused Service Oriented Network monitoring Architecture
! International collaboration for network monitoring ! Contributors are GÉANT, Internet2, ESnet, and
RNP ! Two main implementations committed to
interoperate: ! perfSONAR MDM within GÉANT:
http://perfsonar.geant.net ! perfSONAR PS within I2/ESnet:
http://psps.perfsonar.net/ ! Open OGF protocol to exchange data ! Web-service based ! Design goals: flexibility, extensibility, openness,
and decentralization.
![Page 4: perfSONAR MDM and TCP buffers - GÉANT · PDF fileperfSONAR MDM and TCP buffers Domenico Vicinanza DANTE, Cambridge, UK domenico.vicinanza@dante.net EGI Technical Forum 2013, Madrid,](https://reader034.fdocuments.in/reader034/viewer/2022051718/5a71c0f67f8b9abb538d0e3f/html5/thumbnails/4.jpg)
4 Connect | Communicate | Collaborate
How does perfSONAR work?
perfSONAR MP
perfSONAR MP
Available Bandwidth One Way Delay Jitter IP Route Tracing
Web UI
![Page 5: perfSONAR MDM and TCP buffers - GÉANT · PDF fileperfSONAR MDM and TCP buffers Domenico Vicinanza DANTE, Cambridge, UK domenico.vicinanza@dante.net EGI Technical Forum 2013, Madrid,](https://reader034.fdocuments.in/reader034/viewer/2022051718/5a71c0f67f8b9abb538d0e3f/html5/thumbnails/5.jpg)
5 Connect | Communicate | Collaborate
perfSONAR web user interface
![Page 6: perfSONAR MDM and TCP buffers - GÉANT · PDF fileperfSONAR MDM and TCP buffers Domenico Vicinanza DANTE, Cambridge, UK domenico.vicinanza@dante.net EGI Technical Forum 2013, Madrid,](https://reader034.fdocuments.in/reader034/viewer/2022051718/5a71c0f67f8b9abb538d0e3f/html5/thumbnails/6.jpg)
6 Connect | Communicate | Collaborate
Using perfSONAR UI to run TCP measurements
! Left Panel on perfSONAR UI ! Click on “Make available
throughput measurement”
![Page 7: perfSONAR MDM and TCP buffers - GÉANT · PDF fileperfSONAR MDM and TCP buffers Domenico Vicinanza DANTE, Cambridge, UK domenico.vicinanza@dante.net EGI Technical Forum 2013, Madrid,](https://reader034.fdocuments.in/reader034/viewer/2022051718/5a71c0f67f8b9abb538d0e3f/html5/thumbnails/7.jpg)
7 Connect | Communicate | Collaborate
London-Frankfurt 1Kbyte buffer
![Page 8: perfSONAR MDM and TCP buffers - GÉANT · PDF fileperfSONAR MDM and TCP buffers Domenico Vicinanza DANTE, Cambridge, UK domenico.vicinanza@dante.net EGI Technical Forum 2013, Madrid,](https://reader034.fdocuments.in/reader034/viewer/2022051718/5a71c0f67f8b9abb538d0e3f/html5/thumbnails/8.jpg)
8 Connect | Communicate | Collaborate
First measurement: Geneva-Frankfurt
570Km 4.5ms
![Page 9: perfSONAR MDM and TCP buffers - GÉANT · PDF fileperfSONAR MDM and TCP buffers Domenico Vicinanza DANTE, Cambridge, UK domenico.vicinanza@dante.net EGI Technical Forum 2013, Madrid,](https://reader034.fdocuments.in/reader034/viewer/2022051718/5a71c0f67f8b9abb538d0e3f/html5/thumbnails/9.jpg)
9 Connect | Communicate | Collaborate
Geneva-Frankfurt 1Kbyte buffer
1K=1024Byte
Bandwidth<2Mb/s!!
![Page 10: perfSONAR MDM and TCP buffers - GÉANT · PDF fileperfSONAR MDM and TCP buffers Domenico Vicinanza DANTE, Cambridge, UK domenico.vicinanza@dante.net EGI Technical Forum 2013, Madrid,](https://reader034.fdocuments.in/reader034/viewer/2022051718/5a71c0f67f8b9abb538d0e3f/html5/thumbnails/10.jpg)
10 Connect | Communicate | Collaborate
What happens with TCP buffer=1KB?
Geneva Frankfurt Delay: 4.5ms
1024 Byte ! 1024Bytes sent every 4.5ms ! Max Throughput≤(TCP buffer)/Delay
! 𝑀𝑎𝑥 𝑇ℎ𝑟𝑜𝑢𝑔ℎ𝑝𝑢𝑡≤ 1024∙8/4.5∙ 10↑−3 ~1.8 Mbit/s ! Bandwidth limited by the buffer!
![Page 11: perfSONAR MDM and TCP buffers - GÉANT · PDF fileperfSONAR MDM and TCP buffers Domenico Vicinanza DANTE, Cambridge, UK domenico.vicinanza@dante.net EGI Technical Forum 2013, Madrid,](https://reader034.fdocuments.in/reader034/viewer/2022051718/5a71c0f67f8b9abb538d0e3f/html5/thumbnails/11.jpg)
11 Connect | Communicate | Collaborate
Optimal TCP Window size: Geneva-Frankfurt
! RTT: 9ms ! 1Gb/s network interface Calculating Bandwidth Delay Product ! BDP=9∙ 10↑−3 ∙1∙10↑9 =9∙10↑6 =9𝑀𝑏𝑖𝑡=1.07𝑀𝐵𝑦𝑡𝑒 ! Optimal TCP window = BDP
! TCP window = 1MByte for Geneva-Frankfurt
![Page 12: perfSONAR MDM and TCP buffers - GÉANT · PDF fileperfSONAR MDM and TCP buffers Domenico Vicinanza DANTE, Cambridge, UK domenico.vicinanza@dante.net EGI Technical Forum 2013, Madrid,](https://reader034.fdocuments.in/reader034/viewer/2022051718/5a71c0f67f8b9abb538d0e3f/html5/thumbnails/12.jpg)
12 Connect | Communicate | Collaborate
Geneva-Frankfurt 1Mbyte TCP Buffer
1M=1048756Byte
Bandwidth ~0.9 Gb/s!!
![Page 13: perfSONAR MDM and TCP buffers - GÉANT · PDF fileperfSONAR MDM and TCP buffers Domenico Vicinanza DANTE, Cambridge, UK domenico.vicinanza@dante.net EGI Technical Forum 2013, Madrid,](https://reader034.fdocuments.in/reader034/viewer/2022051718/5a71c0f67f8b9abb538d0e3f/html5/thumbnails/13.jpg)
13 Connect | Communicate | Collaborate
Second measurement: London – New York
5570Km 50ms
![Page 14: perfSONAR MDM and TCP buffers - GÉANT · PDF fileperfSONAR MDM and TCP buffers Domenico Vicinanza DANTE, Cambridge, UK domenico.vicinanza@dante.net EGI Technical Forum 2013, Madrid,](https://reader034.fdocuments.in/reader034/viewer/2022051718/5a71c0f67f8b9abb538d0e3f/html5/thumbnails/14.jpg)
14 Connect | Communicate | Collaborate
London-New York 1Kbyte Buffer
1K=1024Byte
Bandwidth<200Kb/s!!
![Page 15: perfSONAR MDM and TCP buffers - GÉANT · PDF fileperfSONAR MDM and TCP buffers Domenico Vicinanza DANTE, Cambridge, UK domenico.vicinanza@dante.net EGI Technical Forum 2013, Madrid,](https://reader034.fdocuments.in/reader034/viewer/2022051718/5a71c0f67f8b9abb538d0e3f/html5/thumbnails/15.jpg)
15 Connect | Communicate | Collaborate
Optimal TCP Window size: London-New York
! RTT: 100ms ! 1Gb/s network interface Calculating Bandwidth Delay Product ! BDP=100∙ 10↑−3 ∙1∙10↑9 =100∙10↑6 =12𝑀𝐵𝑦𝑡𝑒 ! Optimal TCP window = BDP
! TCP window = 12MByte for London-New York
![Page 16: perfSONAR MDM and TCP buffers - GÉANT · PDF fileperfSONAR MDM and TCP buffers Domenico Vicinanza DANTE, Cambridge, UK domenico.vicinanza@dante.net EGI Technical Forum 2013, Madrid,](https://reader034.fdocuments.in/reader034/viewer/2022051718/5a71c0f67f8b9abb538d0e3f/html5/thumbnails/16.jpg)
16 Connect | Communicate | Collaborate
London-New York 12Mbyte Buffer
12M=12582912Byte
Bandwidth ~0.95Gb/s!!
![Page 17: perfSONAR MDM and TCP buffers - GÉANT · PDF fileperfSONAR MDM and TCP buffers Domenico Vicinanza DANTE, Cambridge, UK domenico.vicinanza@dante.net EGI Technical Forum 2013, Madrid,](https://reader034.fdocuments.in/reader034/viewer/2022051718/5a71c0f67f8b9abb538d0e3f/html5/thumbnails/17.jpg)
17 Connect | Communicate | Collaborate
Live demo: Madrid-Tallin
27ms
![Page 18: perfSONAR MDM and TCP buffers - GÉANT · PDF fileperfSONAR MDM and TCP buffers Domenico Vicinanza DANTE, Cambridge, UK domenico.vicinanza@dante.net EGI Technical Forum 2013, Madrid,](https://reader034.fdocuments.in/reader034/viewer/2022051718/5a71c0f67f8b9abb538d0e3f/html5/thumbnails/18.jpg)
18 Connect | Communicate | Collaborate
Optimal TCP Window size: Madrid-Tallinn
! RTT: 54ms ! 1Gb/s network interface Calculating Bandwidth Delay Product ! BDP=54∙ 10↑−3 ∙1∙10↑9 =54∙10↑6 =54𝑀𝑏𝑖𝑡=6.43𝑀𝐵𝑦𝑡𝑒 ! Optimal TCP window = BDP
! TCP window = 6.43MByte for Madrid-Tallin
![Page 19: perfSONAR MDM and TCP buffers - GÉANT · PDF fileperfSONAR MDM and TCP buffers Domenico Vicinanza DANTE, Cambridge, UK domenico.vicinanza@dante.net EGI Technical Forum 2013, Madrid,](https://reader034.fdocuments.in/reader034/viewer/2022051718/5a71c0f67f8b9abb538d0e3f/html5/thumbnails/19.jpg)
19 Connect | Communicate | Collaborate
Let’s try on the perfSONAR UI!
! We’ll set: ! Source: GEANT Madrid ! Destination: GEANT Tallinn ! TCP Window size to 6.43MByte=6750000 Byte
![Page 20: perfSONAR MDM and TCP buffers - GÉANT · PDF fileperfSONAR MDM and TCP buffers Domenico Vicinanza DANTE, Cambridge, UK domenico.vicinanza@dante.net EGI Technical Forum 2013, Madrid,](https://reader034.fdocuments.in/reader034/viewer/2022051718/5a71c0f67f8b9abb538d0e3f/html5/thumbnails/20.jpg)
20 Connect | Communicate | Collaborate
Madrid-Tallinn Test with optimal TCP window
6.43MByte=6750000 Byte
![Page 21: perfSONAR MDM and TCP buffers - GÉANT · PDF fileperfSONAR MDM and TCP buffers Domenico Vicinanza DANTE, Cambridge, UK domenico.vicinanza@dante.net EGI Technical Forum 2013, Madrid,](https://reader034.fdocuments.in/reader034/viewer/2022051718/5a71c0f67f8b9abb538d0e3f/html5/thumbnails/21.jpg)
21 Connect | Communicate | Collaborate
Madrid-Tallinn
! Sometimes we have the possibility to check the buffer set in kernel ! Max receiver (Tallinn) buffer: 67108864 bytes = 64MBytes
! If we set TCP buffers> 67108864 bytes ! there will be packet loss!
! Let’s try!
![Page 22: perfSONAR MDM and TCP buffers - GÉANT · PDF fileperfSONAR MDM and TCP buffers Domenico Vicinanza DANTE, Cambridge, UK domenico.vicinanza@dante.net EGI Technical Forum 2013, Madrid,](https://reader034.fdocuments.in/reader034/viewer/2022051718/5a71c0f67f8b9abb538d0e3f/html5/thumbnails/22.jpg)
22 Connect | Communicate | Collaborate
TCP Window too large à Packet drops
TCP window size: 160MByte Max buffer (Kernel): 64MByte
Bandwidth drop due to packet loss <160Mb/s!!
![Page 23: perfSONAR MDM and TCP buffers - GÉANT · PDF fileperfSONAR MDM and TCP buffers Domenico Vicinanza DANTE, Cambridge, UK domenico.vicinanza@dante.net EGI Technical Forum 2013, Madrid,](https://reader034.fdocuments.in/reader034/viewer/2022051718/5a71c0f67f8b9abb538d0e3f/html5/thumbnails/23.jpg)
23 Connect | Communicate | Collaborate
Conclusions
! Buffers are critical in TCP communication ! Not too small
! But not too big! ! Particular care with long-haul, high-bandwidth network
! Long Fat Networks, LFN ! Buffers can be the limiting factor ! perfSONAR MDM
! Add interactive, on-demand testing capability ! Particularly useful for TCP debugging
! Measurement strategy ! Calculate the BDP ! Set the TCP Window size=BDP
![Page 24: perfSONAR MDM and TCP buffers - GÉANT · PDF fileperfSONAR MDM and TCP buffers Domenico Vicinanza DANTE, Cambridge, UK domenico.vicinanza@dante.net EGI Technical Forum 2013, Madrid,](https://reader034.fdocuments.in/reader034/viewer/2022051718/5a71c0f67f8b9abb538d0e3f/html5/thumbnails/24.jpg)
24 Connect | Communicate | Collaborate
www.geant.net
www.twitter.com/GEANTnews | www.facebook.com/GEANTnetwork | www.youtube.com/GEANTtv
Connect | Communicate | Collaborate
Thank you!