Datacenter Wide-areaEnterprise LOAD-BALANCER Client Servers.
-
Upload
hortense-lloyd -
Category
Documents
-
view
219 -
download
1
Transcript of Datacenter Wide-areaEnterprise LOAD-BALANCER Client Servers.
![Page 1: Datacenter Wide-areaEnterprise LOAD-BALANCER Client Servers.](https://reader031.fdocuments.in/reader031/viewer/2022032105/56649d8e5503460f94a770a7/html5/thumbnails/1.jpg)
Should a load-balancer choose the path
as well as the server?
Nikhil HandigolStanford University
Joint work with Nick McKeown and Ramesh Johari
![Page 2: Datacenter Wide-areaEnterprise LOAD-BALANCER Client Servers.](https://reader031.fdocuments.in/reader031/viewer/2022032105/56649d8e5503460f94a770a7/html5/thumbnails/2.jpg)
Datacenter
Wide-area Enterprise
![Page 3: Datacenter Wide-areaEnterprise LOAD-BALANCER Client Servers.](https://reader031.fdocuments.in/reader031/viewer/2022032105/56649d8e5503460f94a770a7/html5/thumbnails/3.jpg)
Can’t choose path :’(
LOAD-BALANCER
Client
Servers
![Page 4: Datacenter Wide-areaEnterprise LOAD-BALANCER Client Servers.](https://reader031.fdocuments.in/reader031/viewer/2022032105/56649d8e5503460f94a770a7/html5/thumbnails/4.jpg)
Outline and goals
A new architecture for distributed load-balancing joint (server, path) selection
Demonstrate a nation-wide prototype
Interesting preliminary results
![Page 5: Datacenter Wide-areaEnterprise LOAD-BALANCER Client Servers.](https://reader031.fdocuments.in/reader031/viewer/2022032105/56649d8e5503460f94a770a7/html5/thumbnails/5.jpg)
I’m here to ask for your help!
![Page 6: Datacenter Wide-areaEnterprise LOAD-BALANCER Client Servers.](https://reader031.fdocuments.in/reader031/viewer/2022032105/56649d8e5503460f94a770a7/html5/thumbnails/6.jpg)
Data Path (Hardware)
Control Path
OpenFlow
OpenFlow Controller
OpenFlow Protocol (SSL)
Control Path
![Page 7: Datacenter Wide-areaEnterprise LOAD-BALANCER Client Servers.](https://reader031.fdocuments.in/reader031/viewer/2022032105/56649d8e5503460f94a770a7/html5/thumbnails/7.jpg)
Custom Hardware
Custom Hardware
Custom Hardware
Custom Hardware
Custom Hardware
OS
OS
OS
OS
OS
Network OS
Feature
Feature
Software Defined Networking
Feature Feature
Feature Feature
Feature Feature
Feature Feature
Feature Feature
7
![Page 8: Datacenter Wide-areaEnterprise LOAD-BALANCER Client Servers.](https://reader031.fdocuments.in/reader031/viewer/2022032105/56649d8e5503460f94a770a7/html5/thumbnails/8.jpg)
Load Balancing is just
Smart Routing
![Page 9: Datacenter Wide-areaEnterprise LOAD-BALANCER Client Servers.](https://reader031.fdocuments.in/reader031/viewer/2022032105/56649d8e5503460f94a770a7/html5/thumbnails/9.jpg)
Custom Hardware
Custom Hardware
Custom Hardware
Custom Hardware
Custom Hardware
Network OS
Load-balancing logic
Load-balancing as a network primitive
Load-balancing decision
Load-balancing decision
Load-balancing decision
Load-balancing decision
Load-balancing decision
9
![Page 10: Datacenter Wide-areaEnterprise LOAD-BALANCER Client Servers.](https://reader031.fdocuments.in/reader031/viewer/2022032105/56649d8e5503460f94a770a7/html5/thumbnails/10.jpg)
Aster*x Controller
![Page 11: Datacenter Wide-areaEnterprise LOAD-BALANCER Client Servers.](https://reader031.fdocuments.in/reader031/viewer/2022032105/56649d8e5503460f94a770a7/html5/thumbnails/11.jpg)
![Page 12: Datacenter Wide-areaEnterprise LOAD-BALANCER Client Servers.](https://reader031.fdocuments.in/reader031/viewer/2022032105/56649d8e5503460f94a770a7/html5/thumbnails/12.jpg)
http://www.openflow.org/videos
![Page 13: Datacenter Wide-areaEnterprise LOAD-BALANCER Client Servers.](https://reader031.fdocuments.in/reader031/viewer/2022032105/56649d8e5503460f94a770a7/html5/thumbnails/13.jpg)
So far…
A new architecture for distributed load-balancing joint (server, path) selection
Aster*x – a nation-wide prototype
Promising results that joint (server, path) selection might have great benefits
![Page 14: Datacenter Wide-areaEnterprise LOAD-BALANCER Client Servers.](https://reader031.fdocuments.in/reader031/viewer/2022032105/56649d8e5503460f94a770a7/html5/thumbnails/14.jpg)
What next?
![Page 15: Datacenter Wide-areaEnterprise LOAD-BALANCER Client Servers.](https://reader031.fdocuments.in/reader031/viewer/2022032105/56649d8e5503460f94a770a7/html5/thumbnails/15.jpg)
How big is the pie?
Characterizing and quantifying the performance of joint (server, path) selection
![Page 16: Datacenter Wide-areaEnterprise LOAD-BALANCER Client Servers.](https://reader031.fdocuments.in/reader031/viewer/2022032105/56649d8e5503460f94a770a7/html5/thumbnails/16.jpg)
Load-balancing Controller
MININET-RT
![Page 17: Datacenter Wide-areaEnterprise LOAD-BALANCER Client Servers.](https://reader031.fdocuments.in/reader031/viewer/2022032105/56649d8e5503460f94a770a7/html5/thumbnails/17.jpg)
Load-balancing Controller
![Page 18: Datacenter Wide-areaEnterprise LOAD-BALANCER Client Servers.](https://reader031.fdocuments.in/reader031/viewer/2022032105/56649d8e5503460f94a770a7/html5/thumbnails/18.jpg)
Clients CDNISP
Model
![Page 19: Datacenter Wide-areaEnterprise LOAD-BALANCER Client Servers.](https://reader031.fdocuments.in/reader031/viewer/2022032105/56649d8e5503460f94a770a7/html5/thumbnails/19.jpg)
ParametersTopology
Intra-AS topologies
BRITE (2000 topologies)
CAIDA (1000 topologies)
Rocketfuel (~100 topos.)
20-50 nodesUniform link capacity
![Page 20: Datacenter Wide-areaEnterprise LOAD-BALANCER Client Servers.](https://reader031.fdocuments.in/reader031/viewer/2022032105/56649d8e5503460f94a770a7/html5/thumbnails/20.jpg)
ParametersServers
5-10 serversRandom placement
ServiceSimple HTTP serviceServing 1 MB fileAdditional server-side computation
![Page 21: Datacenter Wide-areaEnterprise LOAD-BALANCER Client Servers.](https://reader031.fdocuments.in/reader031/viewer/2022032105/56649d8e5503460f94a770a7/html5/thumbnails/21.jpg)
ParametersClients
3-5 client locationsRandom placement
Request patternPoisson process
Mean rate: 5-10 req/sec
![Page 22: Datacenter Wide-areaEnterprise LOAD-BALANCER Client Servers.](https://reader031.fdocuments.in/reader031/viewer/2022032105/56649d8e5503460f94a770a7/html5/thumbnails/22.jpg)
Load-balancing strategies?
![Page 23: Datacenter Wide-areaEnterprise LOAD-BALANCER Client Servers.](https://reader031.fdocuments.in/reader031/viewer/2022032105/56649d8e5503460f94a770a7/html5/thumbnails/23.jpg)
Simple but suboptimal
Complex but optimal
Design space
Disjoint-Shortest-Path
Joint
Disjoint-Traffic-Engineering
![Page 24: Datacenter Wide-areaEnterprise LOAD-BALANCER Client Servers.](https://reader031.fdocuments.in/reader031/viewer/2022032105/56649d8e5503460f94a770a7/html5/thumbnails/24.jpg)
Anatomy of a request-response
Client Load-Balancer ServerR
esp
onse
Tim
e
Deliver
Retrieve
Choose
Request
Response 1st byte
Response last byte
Last byte ack
![Page 25: Datacenter Wide-areaEnterprise LOAD-BALANCER Client Servers.](https://reader031.fdocuments.in/reader031/viewer/2022032105/56649d8e5503460f94a770a7/html5/thumbnails/25.jpg)
Disjoint-Shortest-Path
CDN selects the least loaded server
Load = retrieve + deliver
ISP independently selects the shortest path
![Page 26: Datacenter Wide-areaEnterprise LOAD-BALANCER Client Servers.](https://reader031.fdocuments.in/reader031/viewer/2022032105/56649d8e5503460f94a770a7/html5/thumbnails/26.jpg)
Disjoint-Traffic-Engineering
CDN selects the least loaded server
Load = retrieve + deliver
ISP independently selects path to minimize max load
Max bandwidth headroom
![Page 27: Datacenter Wide-areaEnterprise LOAD-BALANCER Client Servers.](https://reader031.fdocuments.in/reader031/viewer/2022032105/56649d8e5503460f94a770a7/html5/thumbnails/27.jpg)
Joint
Single controller jointly selects the best (server, path) pair
Total latency = retrieve + estimated deliver
![Page 28: Datacenter Wide-areaEnterprise LOAD-BALANCER Client Servers.](https://reader031.fdocuments.in/reader031/viewer/2022032105/56649d8e5503460f94a770a7/html5/thumbnails/28.jpg)
Disjoint-Shortest-Path vs Joint
Disjoint-Shortest-Path performs ~2x worse than Joint
![Page 29: Datacenter Wide-areaEnterprise LOAD-BALANCER Client Servers.](https://reader031.fdocuments.in/reader031/viewer/2022032105/56649d8e5503460f94a770a7/html5/thumbnails/29.jpg)
Disjoint-Traffic-Engg. vs Joint
Disjoint-Traffic-Engineering performs almost as well as Joint
![Page 30: Datacenter Wide-areaEnterprise LOAD-BALANCER Client Servers.](https://reader031.fdocuments.in/reader031/viewer/2022032105/56649d8e5503460f94a770a7/html5/thumbnails/30.jpg)
Is Disjoint truly disjoint?Client Load-Balancer Server
Resp
onse
Tim
e
Deliver
Retrieve
Choose
Request
Response 1st byte
Response last byte
Last byte ack
Server response time contains network information
![Page 31: Datacenter Wide-areaEnterprise LOAD-BALANCER Client Servers.](https://reader031.fdocuments.in/reader031/viewer/2022032105/56649d8e5503460f94a770a7/html5/thumbnails/31.jpg)
The bottleneck effect
A single bottleneck resource along the path determines the performance.
![Page 32: Datacenter Wide-areaEnterprise LOAD-BALANCER Client Servers.](https://reader031.fdocuments.in/reader031/viewer/2022032105/56649d8e5503460f94a770a7/html5/thumbnails/32.jpg)
Clients CDNISP
The CDN-ISP game
![Page 33: Datacenter Wide-areaEnterprise LOAD-BALANCER Client Servers.](https://reader031.fdocuments.in/reader031/viewer/2022032105/56649d8e5503460f94a770a7/html5/thumbnails/33.jpg)
The CDN-ISP game
System load monotonically decreases
Both push system in the same direction
![Page 34: Datacenter Wide-areaEnterprise LOAD-BALANCER Client Servers.](https://reader031.fdocuments.in/reader031/viewer/2022032105/56649d8e5503460f94a770a7/html5/thumbnails/34.jpg)
Summary of observations
Disjoint-SP is ~2x worse than Joint
Disjoint-TE performs almost as well as Joint (despite decoupling of server selection
and traffic engineering)
Game theoretic analysis supports the empirical observation
![Page 35: Datacenter Wide-areaEnterprise LOAD-BALANCER Client Servers.](https://reader031.fdocuments.in/reader031/viewer/2022032105/56649d8e5503460f94a770a7/html5/thumbnails/35.jpg)
How we could collaborate
Netflix video - ~30% Internet traffic
Important to efficiently utilize the available resources
I want to apply my research work to Netflix’s service “How can we jointly optimize (server, path) selection to
achieve near-optimal performance?”
How can we work together on this?
![Page 36: Datacenter Wide-areaEnterprise LOAD-BALANCER Client Servers.](https://reader031.fdocuments.in/reader031/viewer/2022032105/56649d8e5503460f94a770a7/html5/thumbnails/36.jpg)
Can you share video streaming data? How can I model the “Netflix network”? Topology? B/W? Where is the bottleneck? Servers? Network? Where are the servers located? How many? What is the client request pattern? What is the video stream size distribution? Duration?
Bandwidth? How do you choose a server for a given request? How do you choose a path for a given request?
Questions – Video Streaming
![Page 37: Datacenter Wide-areaEnterprise LOAD-BALANCER Client Servers.](https://reader031.fdocuments.in/reader031/viewer/2022032105/56649d8e5503460f94a770a7/html5/thumbnails/37.jpg)
Can you share video streaming data? Cost structure – What is the cost
model? Why do you outsource video streaming to CDNs?
How do you deal with non-streaming part of the service (UI)?
Questions – Video Streaming
![Page 38: Datacenter Wide-areaEnterprise LOAD-BALANCER Client Servers.](https://reader031.fdocuments.in/reader031/viewer/2022032105/56649d8e5503460f94a770a7/html5/thumbnails/38.jpg)
Questions – AWS Deployment
Can we work together to characterize the AWS deployment? E.g., Size of deployment, incoming request pattern, inter-VM traffic
Are there web-level SLAs? Does AWS pose challenges?
What are the scaling bottlenecks? CPU? Network? Other?
![Page 39: Datacenter Wide-areaEnterprise LOAD-BALANCER Client Servers.](https://reader031.fdocuments.in/reader031/viewer/2022032105/56649d8e5503460f94a770a7/html5/thumbnails/39.jpg)
Let’s chat more!
![Page 40: Datacenter Wide-areaEnterprise LOAD-BALANCER Client Servers.](https://reader031.fdocuments.in/reader031/viewer/2022032105/56649d8e5503460f94a770a7/html5/thumbnails/40.jpg)
Conclusion
A new architecture for distributed load-balancing joint (server, path) selection
Aster*x - a nation-wide prototype
Interesting preliminary results
Future – application to streaming media services
![Page 41: Datacenter Wide-areaEnterprise LOAD-BALANCER Client Servers.](https://reader031.fdocuments.in/reader031/viewer/2022032105/56649d8e5503460f94a770a7/html5/thumbnails/41.jpg)
Extra slides…
![Page 42: Datacenter Wide-areaEnterprise LOAD-BALANCER Client Servers.](https://reader031.fdocuments.in/reader031/viewer/2022032105/56649d8e5503460f94a770a7/html5/thumbnails/42.jpg)
Questions – AWS Deployment
Can you share Netflix AWS deployment data? How many VMs? What size? What is the service structure? How many tiers of
services? Do you have any SLAs to meet? Any problems there? Would joint VM placement + routing help? What is the avg. NIC/CPU utilization on the VMs? Is the network ever a bottleneck? Do you do any MapReduce-style computation?
![Page 43: Datacenter Wide-areaEnterprise LOAD-BALANCER Client Servers.](https://reader031.fdocuments.in/reader031/viewer/2022032105/56649d8e5503460f94a770a7/html5/thumbnails/43.jpg)
Sample topologies
BRITE CAIDA