Scalable Label Assignment in Data Center Networks
description
Transcript of Scalable Label Assignment in Data Center Networks
![Page 1: Scalable Label Assignment in Data Center Networks](https://reader036.fdocuments.in/reader036/viewer/2022081513/568165b1550346895dd8a57b/html5/thumbnails/1.jpg)
Scalable Label Assignment in Data
Center Networks
With: Radhika Niranjan Mysore, Malveeka Tewari, Ying Zhang (Ericsson Research),
Keith Marzullo, Amin Vahdat
Meg Walraed-SullivanUniversity of California, San Diego
![Page 2: Scalable Label Assignment in Data Center Networks](https://reader036.fdocuments.in/reader036/viewer/2022081513/568165b1550346895dd8a57b/html5/thumbnails/2.jpg)
Group of entities that want to communicate◦ Need a way to refer to one another
Historically, a common problem
◦ E.g. laptop has two labels (MAC address, IP address)
Labeling in data center networks is unique
Labeling in Distributed Networks
◦ Phone system◦ Snail mail◦ Internet
◦ Wireless networks
2
![Page 3: Scalable Label Assignment in Data Center Networks](https://reader036.fdocuments.in/reader036/viewer/2022081513/568165b1550346895dd8a57b/html5/thumbnails/3.jpg)
Interconnect of switches connecting hosts Massive in scale: 10k switches, 100k hosts,
millions of VMs
Data Center Network Size
3
![Page 4: Scalable Label Assignment in Data Center Networks](https://reader036.fdocuments.in/reader036/viewer/2022081513/568165b1550346895dd8a57b/html5/thumbnails/4.jpg)
Designed with regular, symmetric structure◦ Often multi-rooted trees (e.g. fat tree)
Data Center Network Structure
Reality doesn’t always match the blueprint◦ Components and partitions are added/removed◦ Links/switches/hosts fail and recover◦ Cables are connected incorrectly
4
![Page 5: Scalable Label Assignment in Data Center Networks](https://reader036.fdocuments.in/reader036/viewer/2022081513/568165b1550346895dd8a57b/html5/thumbnails/5.jpg)
What gets labeled in a data center network?◦ Switch ports◦ Host NICs◦ Virtual machines at hosts◦ Etc.
Labels in Data Center Networks
5
![Page 6: Scalable Label Assignment in Data Center Networks](https://reader036.fdocuments.in/reader036/viewer/2022081513/568165b1550346895dd8a57b/html5/thumbnails/6.jpg)
Flat Addressing◦ E.g. MAC Addresses (Layer 2)UniqueAutomatic✗Scalability:
Switches have limited forwarding entries (say, 10k) # Labels in forwarding tables = # Nodes
Data Center Labeling Techniques
6
![Page 7: Scalable Label Assignment in Data Center Networks](https://reader036.fdocuments.in/reader036/viewer/2022081513/568165b1550346895dd8a57b/html5/thumbnails/7.jpg)
Hierarchical Addressing◦ E.g. IP Addresses (Layer 3) with DHCPScalable forwarding state
# Labels in forwarding tables < # Nodes
✗Relies on manual configuration: Unrealistic at scale
Data Center Labeling Techniques
7
![Page 8: Scalable Label Assignment in Data Center Networks](https://reader036.fdocuments.in/reader036/viewer/2022081513/568165b1550346895dd8a57b/html5/thumbnails/8.jpg)
PortLand’s LDP: Location Discovery Protocol DAC: Data center Address Configuration
Manual configuration via blueprints Rely on centralized control
◦ Cannot directly connect controller to all nodes◦ Requires separate out-of-band control network or
flooding techniques
Combining L2 and L3 Benefits
8
PortLand: A Scalable Fault-Tolerance Layer 2 Data Center Network Fabric. Niranjan Mysore et al. SIGCOMM 2009Generic and Automatic Address Configuration for Data Center Networks. Chen et al. SIGCOMM 2010
![Page 9: Scalable Label Assignment in Data Center Networks](https://reader036.fdocuments.in/reader036/viewer/2022081513/568165b1550346895dd8a57b/html5/thumbnails/9.jpg)
Scalability vs. Management
Network Size
Labe
l Ass
ignm
ent
Man
agem
ent O
verh
ead
Ethernet
IP
Target location
Hardware Limit:Need Labels < Nodes
Flat Labels Structured Labels
Automation
9
![Page 10: Scalable Label Assignment in Data Center Networks](https://reader036.fdocuments.in/reader036/viewer/2022081513/568165b1550346895dd8a57b/html5/thumbnails/10.jpg)
Cost of Automation Less management means more automation Structured labels encode topology∴Labels change with topology dynamics
Network Size
Man
agem
ent O
verh
ead
Ethernet
IP
Target
10
![Page 11: Scalable Label Assignment in Data Center Networks](https://reader036.fdocuments.in/reader036/viewer/2022081513/568165b1550346895dd8a57b/html5/thumbnails/11.jpg)
ALIAS Overview ALIAS: topology discovery and label
assignment in hierarchical networks
Approach: Automatic, decentralized assignment of hierarchical labels
Benefits:◦ Scalability (structured labels, shared label prefixes)◦ Low management overhead (automation)◦ No out-of-band control network (decentralized)
11
![Page 12: Scalable Label Assignment in Data Center Networks](https://reader036.fdocuments.in/reader036/viewer/2022081513/568165b1550346895dd8a57b/html5/thumbnails/12.jpg)
Systems (Implementation/Evaluation)
ALIAS Evolution
Theory (Proof/Protocol Derivation)
ALIAS: Scalable, Decentralized Label Assignment for Data Centers. M. Walraed-Sullivan, R. Niranjan Mysore, M. Tewari, Y. Zhang, K. Marzullo, A. Vahdat. SOCC 2011
Brief Announcement: A Randomized Algorithm for Label Assignment in Dynamic Networks. M. Walraed-Sullivan, R. Niranjan Mysore, K. Marzullo, A. Vahdat. DISC 2011
ALIAS: topology discovery and label assignment in hierarchical networks
12
![Page 13: Scalable Label Assignment in Data Center Networks](https://reader036.fdocuments.in/reader036/viewer/2022081513/568165b1550346895dd8a57b/html5/thumbnails/13.jpg)
Multi-rooted trees◦ Multi-stage switch fabric connecting hosts◦ Indirect hierarchy◦ May allow peer links
Labels ultimately used for communication◦ Multiple paths between nodes
Data Center Network Topologies
13
![Page 14: Scalable Label Assignment in Data Center Networks](https://reader036.fdocuments.in/reader036/viewer/2022081513/568165b1550346895dd8a57b/html5/thumbnails/14.jpg)
Switches and hosts have labels◦ Labels encode (shortest physical) paths from the root
of the hierarchy to a switch/host◦ Each switch/host may have multiple labels◦ Labels encode location and expose path multiplicity
ALIAS Labels
h’s Labelsa d g hb e g hb f g hc f g h
a d gb e gb f gc f g
g’s Labelsb
d e
g
f
ca
h14
![Page 15: Scalable Label Assignment in Data Center Networks](https://reader036.fdocuments.in/reader036/viewer/2022081513/568165b1550346895dd8a57b/html5/thumbnails/15.jpg)
Hierarchical routing leverages this info◦ Push packets upward, downward path is explicit
Communication over ALIAS Labels
h’s Labelsa d g hb e g hb f g hc f g h
a d gb e gb f gc f g
g’s Labelsb
d e
g
f
ca
h15
![Page 16: Scalable Label Assignment in Data Center Networks](https://reader036.fdocuments.in/reader036/viewer/2022081513/568165b1550346895dd8a57b/html5/thumbnails/16.jpg)
Continuously1 Overlay appropriate hierarchy on network fabric2 Group sets of related switches into hypernodes3 Assign coordinates to switches4 Combine coordinates to form labels
Periodic state exchange between immediate neighbors
Distributed Protocol Overview
16
![Page 17: Scalable Label Assignment in Data Center Networks](https://reader036.fdocuments.in/reader036/viewer/2022081513/568165b1550346895dd8a57b/html5/thumbnails/17.jpg)
Switches are at levels 1 through n Hosts are at level 0
Step 1. Overlay Hierarchy
Only requires 1 host to beginLevel 0
Level 1
Level 2
Level 3
17
![Page 18: Scalable Label Assignment in Data Center Networks](https://reader036.fdocuments.in/reader036/viewer/2022081513/568165b1550346895dd8a57b/html5/thumbnails/18.jpg)
Continuously1 Overlay appropriate hierarchy on network fabric2 Group sets of related switches into hypernodes3 Assign coordinates to switches4 Combine coordinates to form labels
Distributed Protocol Overview
18
![Page 19: Scalable Label Assignment in Data Center Networks](https://reader036.fdocuments.in/reader036/viewer/2022081513/568165b1550346895dd8a57b/html5/thumbnails/19.jpg)
Labels encode paths from a root to a host◦ Multiple paths lead to multiple labels per host
Aggregate for label compaction◦ Locate switches that reach same hosts
Step 2. Discover Hypernodes
Level 1
Level 2
Level 3
Level 4
(hosts omitted for space)
19
![Page 20: Scalable Label Assignment in Data Center Networks](https://reader036.fdocuments.in/reader036/viewer/2022081513/568165b1550346895dd8a57b/html5/thumbnails/20.jpg)
Step 2. Discover Hypernodes
Hypernode (HN):Maximal set of switches that connect to same HNs below
(via any member)
Level 1
Level 2
Level 3
Level 4Hypernode members are
indistinguishable on downward path from root
Base Case: Each Level 1 switch
is in its own hypernode
20
![Page 21: Scalable Label Assignment in Data Center Networks](https://reader036.fdocuments.in/reader036/viewer/2022081513/568165b1550346895dd8a57b/html5/thumbnails/21.jpg)
Continuously1 Overlay appropriate hierarchy on network fabric2 Group sets of related switches into hypernodes3 Assign coordinates to switches4 Combine coordinates to form labels
Distributed Protocol Overview
21
![Page 22: Scalable Label Assignment in Data Center Networks](https://reader036.fdocuments.in/reader036/viewer/2022081513/568165b1550346895dd8a57b/html5/thumbnails/22.jpg)
Coordinates combine to make up labels Labels used to route downwards
Step 3. Assign Coordinates
22
Switches in a HN share a coordinate
HN’s with a parent in common need distinct coordinates
![Page 23: Scalable Label Assignment in Data Center Networks](https://reader036.fdocuments.in/reader036/viewer/2022081513/568165b1550346895dd8a57b/html5/thumbnails/23.jpg)
Step 3. Assign Coordinates
23
choosers
deciders
Can we make this problem simpler?
Switches in a HN share a coordinate
HN’s with a parent in common need distinct coordinates
![Page 24: Scalable Label Assignment in Data Center Networks](https://reader036.fdocuments.in/reader036/viewer/2022081513/568165b1550346895dd8a57b/html5/thumbnails/24.jpg)
To assign coordinates to hypernodes:a. Define abstraction
(choosers/deciders)b.Design solution for abstractionc. Apply solution throughout multi-
rooted tree
Step 3. Assign Coordinates
24
choosers
deciders
![Page 25: Scalable Label Assignment in Data Center Networks](https://reader036.fdocuments.in/reader036/viewer/2022081513/568165b1550346895dd8a57b/html5/thumbnails/25.jpg)
Label Selection Problem (LSP)◦ Chooser processes connected to Decider
processes◦ In a bipartite graph
Step 3. Assign Coordinatesa. Decider/Chooser abstraction
d2 d3d1 d4
c1 c2 c3 c4 c5 c6 Choosers(hypernodes)
deciders(parent
switches)
25
![Page 26: Scalable Label Assignment in Data Center Networks](https://reader036.fdocuments.in/reader036/viewer/2022081513/568165b1550346895dd8a57b/html5/thumbnails/26.jpg)
Label Selection Problem Goals:◦ All choosers eventually select coordinates◦ Choosers sharing a decider have distinct
coordinates
Step 3. Assign Coordinates
d2 d3d1 d4
c1 c2 c3 c4 c5 c6 choosers
deciders
x y z y
q
z
z
x
Multiple instances of LSP
Per-instance coordinates
y z
26
a. Decider/Chooser abstraction
![Page 27: Scalable Label Assignment in Data Center Networks](https://reader036.fdocuments.in/reader036/viewer/2022081513/568165b1550346895dd8a57b/html5/thumbnails/27.jpg)
Label Selection Problem (LSP)◦ Difficulty: connections can change over time
Step 3. Assign Coordinates
d2 d3d1 d4
c1 c2 c3 c4 c5 c6
x y z y
q
z
z
xz
r27
a. Decider/Chooser abstraction
![Page 28: Scalable Label Assignment in Data Center Networks](https://reader036.fdocuments.in/reader036/viewer/2022081513/568165b1550346895dd8a57b/html5/thumbnails/28.jpg)
Decider/Chooser Protocol (DCP)◦ Distributed algorithm that implements LSP◦ Las-Vegas style randomized algorithm
Probabilistically fast, guaranteed to be correct
◦ Practical: Low message overhead, quick convergence
◦ Reacts quickly and locally to topology dynamics Transient startup conditions Miswirings Failure/recovery, connectivity changes
Step 3. Assign Coordinatesb. Design Solution for Abstraction
28
![Page 29: Scalable Label Assignment in Data Center Networks](https://reader036.fdocuments.in/reader036/viewer/2022081513/568165b1550346895dd8a57b/html5/thumbnails/29.jpg)
c2:y?c1:x? c2:y?c1:x?
Algorithm:◦ Choosers select coordinates randomly and send
to deciders◦ Deciders reply with [yes] or [no+hints]◦ One no reselect, All yeses finished
Step 3. Assign Coordinatesb. Design Solution for Abstraction
d2d1
c1 c2
c1:c2:
c1:c2:
c1: xc2: y
c1: xc2: y
yes yesyesyes
Coord: x Coord: y
29
![Page 30: Scalable Label Assignment in Data Center Networks](https://reader036.fdocuments.in/reader036/viewer/2022081513/568165b1550346895dd8a57b/html5/thumbnails/30.jpg)
Hypernodes are choosers for their coordinates
Switches are deciders for neighbors below
Step 3. Assign Coordinatesc. Apply DCP through Hierarchy
30
2 choosers
3 deciders 2 choosers
1 decider
3 choosers
3 deciders
![Page 31: Scalable Label Assignment in Data Center Networks](https://reader036.fdocuments.in/reader036/viewer/2022081513/568165b1550346895dd8a57b/html5/thumbnails/31.jpg)
DCP assigns level 1 coordinates
Step 3. Assign Coordinates
3 choosers
3 deciders
31
c. Apply DCP through Hierarchy
![Page 32: Scalable Label Assignment in Data Center Networks](https://reader036.fdocuments.in/reader036/viewer/2022081513/568165b1550346895dd8a57b/html5/thumbnails/32.jpg)
DCP for upper levels:◦ HN switches cooperate (per-parent restrictions)◦ Not directly connected
Step 3. Assign Coordinates
2 choosers
3 deciders
32
c. Apply DCP through Hierarchy
Communicate via shared L1 switch
“Distributed-Chooser DCP”
![Page 33: Scalable Label Assignment in Data Center Networks](https://reader036.fdocuments.in/reader036/viewer/2022081513/568165b1550346895dd8a57b/html5/thumbnails/33.jpg)
Continuously1 Overlay appropriate hierarchy on network fabric2 Group related switches into hypernodes3 Assign per-hypernode coordinates4 Combine coordinates to form labels
Distributed Protocol Overview
33
![Page 34: Scalable Label Assignment in Data Center Networks](https://reader036.fdocuments.in/reader036/viewer/2022081513/568165b1550346895dd8a57b/html5/thumbnails/34.jpg)
Concatenate coordinates from root downward
Step 4. Assign Labels
(For clarity, assume labels same across instances of
LSP)
34
![Page 35: Scalable Label Assignment in Data Center Networks](https://reader036.fdocuments.in/reader036/viewer/2022081513/568165b1550346895dd8a57b/html5/thumbnails/35.jpg)
Hypernodes create clusters of hosts that share label prefixes
Step 4. Assign Labels
35
![Page 36: Scalable Label Assignment in Data Center Networks](https://reader036.fdocuments.in/reader036/viewer/2022081513/568165b1550346895dd8a57b/html5/thumbnails/36.jpg)
Topology changes may cause paths to change
Which causes labels to change Evaluation:
◦ Quick convergence ◦ Localized effects
Relabeling
36
![Page 37: Scalable Label Assignment in Data Center Networks](https://reader036.fdocuments.in/reader036/viewer/2022081513/568165b1550346895dd8a57b/html5/thumbnails/37.jpg)
Many overlying communication protocols◦ Hierarchical-style forwarding makes most sense
E.g. MAC address rewriting◦ At sender’s ingress switch: dest. MAC ALIAS label◦ At recipient’s egress switch: ALIAS labeldest. MAC◦ Up*/down* forwarding (AutoNet, SOSP91)◦ Proxy ARP for resolution
E.g. encapsulation, tunneling
Using ALIAS labels
37
![Page 38: Scalable Label Assignment in Data Center Networks](https://reader036.fdocuments.in/reader036/viewer/2022081513/568165b1550346895dd8a57b/html5/thumbnails/38.jpg)
“Standard” systems approach◦ Implementation, experimentation, deployment
Theoretical approach◦ Proof, formalization, verification via model
checking
Goal: ◦ Verify correctness, feasibility◦ Assess scalability
Evaluation Methodology
38
![Page 39: Scalable Label Assignment in Data Center Networks](https://reader036.fdocuments.in/reader036/viewer/2022081513/568165b1550346895dd8a57b/html5/thumbnails/39.jpg)
Does ALIAS assign labels correctly? Do labels enable scalable communication?
✓Implemented in Mace (www.macesystems.org)✓Used Mace Model Checker to verify
Label assignment: levels, hypernodes, coordinates Sample overlying communication: pairs of nodes can
communicate when physically connected
✓Ported to small testbed with existing communication protocol for realistic evaluation
Evaluation: Correctness
39
![Page 40: Scalable Label Assignment in Data Center Networks](https://reader036.fdocuments.in/reader036/viewer/2022081513/568165b1550346895dd8a57b/html5/thumbnails/40.jpg)
Does DCP solve the Label Selection Problem?
✓Proof that DCP implements LSP✓Implemented in Mace and model checked all
versions of DCP
Is LSP a reasonable abstraction?
✓Formal protocol derivation from basic DCPALIAS
Evaluation: Correctness
40
![Page 41: Scalable Label Assignment in Data Center Networks](https://reader036.fdocuments.in/reader036/viewer/2022081513/568165b1550346895dd8a57b/html5/thumbnails/41.jpg)
Is overhead (storage, control) acceptable?
✓Resource requirements of algorithm Memory: ~KBs for 10k host network Control overhead: agility/overhead tradeoff
✓Memory usage on testbed deployment (<150B)
Evaluation: Feasibility
41
Ports/Switch Hosts Cycle (ms) Control Overhead (Mbps, %10G link)
64 65k 100 31.5 (0.3%)500 6.29 (0.06%)
128 524k 1000 25.16 (0.25%)2000 12.58 (0.12%)
![Page 42: Scalable Label Assignment in Data Center Networks](https://reader036.fdocuments.in/reader036/viewer/2022081513/568165b1550346895dd8a57b/html5/thumbnails/42.jpg)
Is the protocol practical in convergence time?
✓DCP: Used Mace simulator to verify that “probabilistically fast” is quite fast in practice
✓Measured convergence on tested deployment On startup After failure (speed and locality)
✓Used Mace model checker to verify locality of failure reactions for larger networks
Evaluation: Feasibility
42
![Page 43: Scalable Label Assignment in Data Center Networks](https://reader036.fdocuments.in/reader036/viewer/2022081513/568165b1550346895dd8a57b/html5/thumbnails/43.jpg)
Does ALIAS scale to data center sizes?
✓Used Mace model checker to verify labels and communication for larger networks than testbed
✓Wrote simulation code to analyze network behavior for enormous networks
Evaluation: Scalability
43
![Page 44: Scalable Label Assignment in Data Center Networks](https://reader036.fdocuments.in/reader036/viewer/2022081513/568165b1550346895dd8a57b/html5/thumbnails/44.jpg)
Result: Small Forwarding StateTopology
ALIAS Forwarding
Table EntriesLevels Ports % Fully Provisioned Servers
3
32100
8,19245
80 26250 17320 86
64100
65,65390
80 102850 65320 291
4 32100
131,07246
80 127850 207920 2415
5 16100
65,65323
80 49250 88620 1108
44
e.g. MACe.g. IP,
LDP/DAC
![Page 45: Scalable Label Assignment in Data Center Networks](https://reader036.fdocuments.in/reader036/viewer/2022081513/568165b1550346895dd8a57b/html5/thumbnails/45.jpg)
Scale and complexity of data center networks make labeling problem unique
ALIAS enables scalable data center communication by:◦ Using a distributed approach◦ Leveraging hierarchy to form topologically
significant labels◦ Eliminating manual configuration
Conclusion
45
![Page 46: Scalable Label Assignment in Data Center Networks](https://reader036.fdocuments.in/reader036/viewer/2022081513/568165b1550346895dd8a57b/html5/thumbnails/46.jpg)
46
Convergence of DCP
![Page 47: Scalable Label Assignment in Data Center Networks](https://reader036.fdocuments.in/reader036/viewer/2022081513/568165b1550346895dd8a57b/html5/thumbnails/47.jpg)
Convergence vs. Coord. Domain
0 1 2 3 4
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
d=4d=8
d=16d=32
d=64d=128
m=4d=4d=8d=16d=32d=64d=128
k
P(k,
4,d)
47
![Page 48: Scalable Label Assignment in Data Center Networks](https://reader036.fdocuments.in/reader036/viewer/2022081513/568165b1550346895dd8a57b/html5/thumbnails/48.jpg)
Convergence vs. Coord. Domain
0 1 2 3 4 5 6 7 8
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
d=8d=16
d=32d=64
d=128
m=8d=8d=16d=32d=64d=128
k
P(k,
8,d)
48
![Page 49: Scalable Label Assignment in Data Center Networks](https://reader036.fdocuments.in/reader036/viewer/2022081513/568165b1550346895dd8a57b/html5/thumbnails/49.jpg)
Convergence vs. Coord. Domain
0 2 4 6 810 12 14 16
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
m=16
d=16d=32d=64d=128
k
P(k,
16,d
)
49
![Page 50: Scalable Label Assignment in Data Center Networks](https://reader036.fdocuments.in/reader036/viewer/2022081513/568165b1550346895dd8a57b/html5/thumbnails/50.jpg)
Convergence vs. Coord. Domain
50